Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 670700

Summary: app-crypt/veracrypt should filter -mno-aes flags (fails to compile, error: inlining failed in call to ‘always_inline’ ‘_mm_aesenc_si128’: target specific option mismatch)
Product: Gentoo Linux Reporter: Oleh <moonlapse81>
Component: Current packagesAssignee: Göktürk Yüksek <gokturk>
Status: RESOLVED FIXED    
Severity: normal CC: ago, sam, soap
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
URL: https://github.com/veracrypt/VeraCrypt/issues/892
Whiteboard:
Package list:
Runtime testing required: ---

Description Oleh 2018-11-08 17:59:39 UTC
As in summary, veracrypt has AES cpu instruction detection by brief look at Cpu.h in sources. However, there is no support for enable/disable for AES relying on detection. In turn, users, who manually adjust their CFLAGS and set -mno-aes due to their cpu's such as celeron or pentium with cut-off instructions will face failure. Ebuild should filter "-mno-aes" and it should be god for those.

Reproducible: Always

Steps to Reproduce:
1. CFLAGS="$CFLAGS -mno-aes" emerge veracrypt
2.
3.
Actual Results:  
x86_64-pc-linux-gnu-gcc -MMD -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGE_FILES -I/var/tmp/portage/app-crypt/veracrypt-1.22/work/VeraCrypt-VeraCrypt_1.22/src -I/var/tmp/portage/app-crypt/veracrypt-1.22/work/VeraCrypt-VeraCrypt_1.22/src/Crypto -I/var/tmp/portage/app-crypt/veracrypt-1.22/work/VeraCrypt-VeraCrypt_1.22/src/PKCS11 -O2 -fno-strict-aliasing   -D TC_ARCH_X64 -DTC_UNIX -DTC_LINUX -fdata-sections -ffunction-sections -fpie -Wall -msse2 -maes -mssse3 -mssse3 -msse4.1 -march=ivybridge -O2 -pipe -mno-aes -mno-rdrnd -c ../Crypto/cpu.c -o ../Crypto/cpu.o
In file included from ../Crypto/cpu.h:175:0,
                 from ../Crypto/cpu.c:3:
../Crypto/cpu.c: In function ‘TryAESNI’:
/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.1/include/wmmintrin.h:61:1: error: inlining failed in call to always_inline ‘_mm_aesenc_si128’: target specific option mismatch
 _mm_aesenc_si128 (__m128i __X, __m128i __Y)
 ^~~~~~~~~~~~~~~~
../Crypto/cpu.c:245:14: note: called from here
   ciphered = _mm_aesenc_si128(block, subkey);
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../Crypto/cpu.h:175:0,
                 from ../Crypto/cpu.c:3:
/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.1/include/wmmintrin.h:61:1: error: inlining failed in call to always_inline ‘_mm_aesenc_si128’: target specific option mismatch
 _mm_aesenc_si128 (__m128i __X, __m128i __Y)
 ^~~~~~~~~~~~~~~~
../Crypto/cpu.c:245:14: note: called from here
   ciphered = _mm_aesenc_si128(block, subkey);
Comment 1 Matt Turner gentoo-dev 2018-11-11 18:50:17 UTC
I'm not the maintainer, so I'm just offering an opinion, etc. I've told everyone who will listen to not set -mno-flags for exactly this reason.

I think if the user specified -march=native instead of -march=ivybridge they would get what they want (no AES) without explicitly disabling anything. I suppose if they're building binpkgs that won't work though.

Maybe cflag filtering is the best solution.
Comment 2 Oleh 2018-11-12 07:07:52 UTC
Users can do whatever they want in CFLAGS (because it's not forbidden). It's a remote report for such exact case where -mno-aes is set by someone. Due to fact veracrypt build system has no ability to control AES enable/disable at build time that can be feed into portage, filtering -mno-aes would be the workaround for such CFLAGS custom choices.
Comment 3 Göktürk Yüksek archtester gentoo-dev 2018-11-14 00:03:37 UTC
(In reply to Oleg from comment #0)
> As in summary, veracrypt has AES cpu instruction detection by brief look at
> Cpu.h in sources. However, there is no support for enable/disable for AES
> relying on detection. In turn, users, who manually adjust their CFLAGS and
> set -mno-aes due to their cpu's such as celeron or pentium with cut-off
> instructions will face failure. Ebuild should filter "-mno-aes" and it
> should be god for those.
> 
> Reproducible: Always
> 
> Steps to Reproduce:
> 1. CFLAGS="$CFLAGS -mno-aes" emerge veracrypt
> 2.
> 3.
> Actual Results:  
> x86_64-pc-linux-gnu-gcc -MMD -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
> -D_LARGE_FILES
> -I/var/tmp/portage/app-crypt/veracrypt-1.22/work/VeraCrypt-VeraCrypt_1.22/
> src
> -I/var/tmp/portage/app-crypt/veracrypt-1.22/work/VeraCrypt-VeraCrypt_1.22/
> src/Crypto
> -I/var/tmp/portage/app-crypt/veracrypt-1.22/work/VeraCrypt-VeraCrypt_1.22/
> src/PKCS11 -O2 -fno-strict-aliasing   -D TC_ARCH_X64 -DTC_UNIX -DTC_LINUX
> -fdata-sections -ffunction-sections -fpie -Wall -msse2 -maes -mssse3 -mssse3
> -msse4.1 -march=ivybridge -O2 -pipe -mno-aes -mno-rdrnd -c ../Crypto/cpu.c
> -o ../Crypto/cpu.o
> In file included from ../Crypto/cpu.h:175:0,
>                  from ../Crypto/cpu.c:3:
> ../Crypto/cpu.c: In function ‘TryAESNI’:
> /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.1/include/wmmintrin.h:61:1: error:
> inlining failed in call to always_inline ‘_mm_aesenc_si128’: target specific
> option mismatch
>  _mm_aesenc_si128 (__m128i __X, __m128i __Y)
>  ^~~~~~~~~~~~~~~~
> ../Crypto/cpu.c:245:14: note: called from here
>    ciphered = _mm_aesenc_si128(block, subkey);
>               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> In file included from ../Crypto/cpu.h:175:0,
>                  from ../Crypto/cpu.c:3:
> /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.1/include/wmmintrin.h:61:1: error:
> inlining failed in call to always_inline ‘_mm_aesenc_si128’: target specific
> option mismatch
>  _mm_aesenc_si128 (__m128i __X, __m128i __Y)
>  ^~~~~~~~~~~~~~~~
> ../Crypto/cpu.c:245:14: note: called from here
>    ciphered = _mm_aesenc_si128(block, subkey);

Can you try adding '-DCRYPTOPP_DISABLE_AESNI' to your CFLAGS and see if it works?
Comment 4 Oleh 2018-11-15 18:13:11 UTC
Compiling sha512-x64-nayuki.S
x86_64-pc-linux-gnu-gcc -MMD -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGE_FILES -I/var/tmp/portage/app-crypt/veracrypt-1.23/work/VeraCrypt-VeraCrypt_1.23/src -I/var/tmp/portage/app-crypt/veracrypt-1.23/work/VeraCrypt-VeraCrypt_1.23/src/Crypto -I/var/tmp/portage/app-crypt/veracrypt-1.23/work/VeraCrypt-VeraCrypt_1.23/src/PKCS11 -O2 -fno-strict-aliasing   -D TC_ARCH_X64 -DTC_UNIX -DTC_LINUX -fdata-sections -ffunction-sections -fpie -Wall -msse2 -maes -mssse3 -mssse3 -msse4.1 -DCRYPTOPP_DISABLE_AESNI -c ../Crypto/sha512-x64-nayuki.S -o ../Crypto/sha512-x64-nayuki.o
Assembling sha256_avx1_x64.asm
yasm -D __GNUC__ -D __YASM__ -f elf64 -D __BITS__=64 -o ../Crypto/sha256_avx1_x64.o ../Crypto/sha256_avx1_x64.asm
Assembling sha256_avx2_x64.asm
yasm -D __GNUC__ -D __YASM__ -f elf64 -D __BITS__=64 -o ../Crypto/sha256_avx2_x64.o ../Crypto/sha256_avx2_x64.asm
Assembling sha256_sse4_x64.asm
yasm -D __GNUC__ -D __YASM__ -f elf64 -D __BITS__=64 -o ../Crypto/sha256_sse4_x64.o ../Crypto/sha256_sse4_x64.asm
Assembling sha512_avx1_x64.asm
yasm -D __GNUC__ -D __YASM__ -f elf64 -D __BITS__=64 -o ../Crypto/sha512_avx1_x64.o ../Crypto/sha512_avx1_x64.asm
Assembling sha512_avx2_x64.asm
yasm -D __GNUC__ -D __YASM__ -f elf64 -D __BITS__=64 -o ../Crypto/sha512_avx2_x64.o ../Crypto/sha512_avx2_x64.asm
Assembling sha512_sse4_x64.asm
yasm -D __GNUC__ -D __YASM__ -f elf64 -D __BITS__=64 -o ../Crypto/sha512_sse4_x64.o ../Crypto/sha512_sse4_x64.asm
Compiling Aeskey.c
x86_64-pc-linux-gnu-gcc -MMD -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGE_FILES -I/var/tmp/portage/app-crypt/veracrypt-1.23/work/VeraCrypt-VeraCrypt_1.23/src -I/var/tmp/portage/app-crypt/veracrypt-1.23/work/VeraCrypt-VeraCrypt_1.23/src/Crypto -I/var/tmp/portage/app-crypt/veracrypt-1.23/work/VeraCrypt-VeraCrypt_1.23/src/PKCS11 -O2 -fno-strict-aliasing   -D TC_ARCH_X64 -DTC_UNIX -DTC_LINUX -fdata-sections -ffunction-sections -fpie -Wall -msse2 -maes -mssse3 -mssse3 -msse4.1 -DCRYPTOPP_DISABLE_AESNI -c ../Crypto/Aeskey.c -o ../Crypto/Aeskey.o
Compiling Aestab.c
x86_64-pc-linux-gnu-gcc -MMD -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGE_FILES -I/var/tmp/portage/app-crypt/veracrypt-1.23/work/VeraCrypt-VeraCrypt_1.23/src -I/var/tmp/portage/app-crypt/veracrypt-1.23/work/VeraCrypt-VeraCrypt_1.23/src/Crypto -I/var/tmp/portage/app-crypt/veracrypt-1.23/work/VeraCrypt-VeraCrypt_1.23/src/PKCS11 -O2 -fno-strict-aliasing   -D TC_ARCH_X64 -DTC_UNIX -DTC_LINUX -fdata-sections -ffunction-sections -fpie -Wall -msse2 -maes -mssse3 -mssse3 -msse4.1 -DCRYPTOPP_DISABLE_AESNI -c ../Crypto/Aestab.c -o ../Crypto/Aestab.o
Compiling cpu.c
x86_64-pc-linux-gnu-gcc -MMD -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGE_FILES -I/var/tmp/portage/app-crypt/veracrypt-1.23/work/VeraCrypt-VeraCrypt_1.23/src -I/var/tmp/portage/app-crypt/veracrypt-1.23/work/VeraCrypt-VeraCrypt_1.23/src/Crypto -I/var/tmp/portage/app-crypt/veracrypt-1.23/work/VeraCrypt-VeraCrypt_1.23/src/PKCS11 -O2 -fno-strict-aliasing   -D TC_ARCH_X64 -DTC_UNIX -DTC_LINUX -fdata-sections -ffunction-sections -fpie -Wall -msse2 -maes -mssse3 -mssse3 -msse4.1 -DCRYPTOPP_DISABLE_AESNI -c ../Crypto/cpu.c -o ../Crypto/cpu.o
^[[01m^[[K../Crypto/cpu.c:^[[m^[[K In function ‘^[[01m^[[KTryAESNI^[[m^[[K’:
^[[01m^[[K../Crypto/cpu.c:245:14:^[[m^[[K ^[[01;35m^[[Kwarning: ^[[m^[[Kimplicit declaration of function ‘^[[01m^[[K_mm_aesenc_si128^[[m^[[K’; did you mean ‘^[[01m^[[K_mm_and_si128^[[m^[[K’? [^[[01;35m^[[K-Wimplicit-function-declaration^[[m^[[K]
   ciphered = ^[[01;35m^[[K_mm_aesenc_si128^[[m^[[K(block, subkey);
              ^[[01;35m^[[K^~~~~~~~~~~~~~~~^[[m^[[K
              ^[[32m^[[K_mm_and_si128^[[m^[[K
^[[01m^[[K../Crypto/cpu.c:245:12:^[[m^[[K ^[[01;31m^[[Kerror: ^[[m^[[Kincompatible types when assigning to type ‘^[[01m^[[K__m128i {aka __vector(2) long long int}^[[m^[[K’ from type ‘^[[01m^[[Kint^[[m^[[K’
   ciphered ^[[01;31m^[[K=^[[m^[[K _mm_aesenc_si128(block, subkey);
            ^[[01;31m^[[K^^[[m^[[K
make[1]: *** [/var/tmp/portage/app-crypt/veracrypt-1.23/work/VeraCrypt-VeraCrypt_1.23/src/Build/Include/Makefile.inc:21: ../Crypto/cpu.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory '/var/tmp/portage/app-crypt/veracrypt-1.23/work/VeraCrypt-VeraCrypt_1.23/src/Volume'
make: *** [Makefile:391: all] Error 2
Comment 5 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2021-08-02 04:02:20 UTC
*** Bug 806133 has been marked as a duplicate of this bug. ***
Comment 6 Göktürk Yüksek archtester gentoo-dev 2022-02-13 07:38:46 UTC
I did go through the source code again and I couldn't see a clear way to honor '-mno-aes'. I asked upstream if they support it.
Comment 7 Göktürk Yüksek archtester gentoo-dev 2023-10-28 22:20:14 UTC
Please try 1.25.9 (https://gitweb.gentoo.org/repo/gentoo.git/commit/app-crypt/veracrypt/veracrypt-1.25.9.ebuild?id=0e6ddb7525688d55595ad746c47adb88f98c1b0a) and re-open if the issue still exists.