ceph built against 5.6.4, after updating crypto++ to 5.6.5: /usr/bin/ceph-mon: symbol lookup error: /usr/bin/ceph-mon: undefined symbol: _ZNK8CryptoPP8Rijndael3Dec21AdvancedProcessBlocksEPKhS3_Phmj $ c++filt _ZNK8CryptoPP8Rijndael3Dec21AdvancedProcessBlocksEPKhS3_Phmj CryptoPP::Rijndael::Dec::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const
Not sure if this could be related https://github.com/weidai11/cryptopp/issues/70 ? Crypto++ 5.6.3 cleared a number of issues uncovered by analysis and testing tools. To maintain compatibility with Crypto++ 5.6.2, MAINTAIN_BACKWARDS_COMPATIBILITY_562 was introduced. It allowed us to clear most of the findings while maintaining compatibility. If MAINTAIN_BACKWARDS_COMPATIBILITY_562 was not defined, then all the issues were mitigated. Also https://github.com/weidai11/cryptopp/issues/283 seems interesting: Link failure with mismatched library and program capabilities #283 .. "The reason for the failure is different defines guard AdvancedProcessBlocks for the encryption and decryption transformations. When -march=x86-64, CRYPTOPP_BOOL_AESNI_INTRINSICS_AVAILABLE is not available."
For the record: 18:21 <@aballier> K_F: 2nd link is intersting 18:22 <@aballier> should be noted im building binpkgs on skylake for core2duo cpus 18:22 <@K_F> now, _that_ might be interesting in this case, indeed 18:22 <@aballier> so it may be 'CRYPTOPP_BOOL_AESNI_INTRINSICS_AVAILABLE' is switched between them 18:23 <@K_F> could you check? 18:28 <@aballier> K_F: it should be ok, unless this #if is new in 5.6.5 which broke abi 18:28 <@aballier> K_F: http://pastebin.ca/3732390 18:29 <@aballier> K_F: what matters for the #if is if gcc defines __AES__ i think, which depends on -march 18:29 <@aballier> so headers are the same, changing compile flags with 5.6.5 breaks abi 18:30 <@aballier> funny upstream 18:31 <@K_F> aballier: please comment on bug so alonbl gets it pastebin contents: $ cat foo.cc #include <cryptopp/config.h> #if CRYPTOPP_BOOL_AESNI_INTRINSICS_AVAILABLE #error "aes" #endif $ g++ -c foo.cc $ $ g++ -march=haswell -c foo.cc foo.cc:4:2: error: #error "aes" #error "aes" ^
I can add some USE flags including +asm cpu_flags_x86_sse2 -CRYPTOPP_DISABLE_SSE2 cpu_flags_x86_sse3 -CRYPTOPP_DISABLE_SSE3 cpu_flags_x86_sse4_1 -CRYPTOPP_DISABLE_SSE4 cpu_flags_x86_aes -CRYPTOPP_DISABLE_AESNI cpu_flags_x86_avx -CRYPTOPP_DISABLE_AVX cpu_flags_arm_neon CRYPTOPP_BOOL_NEON_INTRINSICS_AVAILABLE=0|1 cpu_flags_arm_crc32 CRYPTOPP_BOOL_ARM_CRC32_INTRINSICS_AVAILABLE=0|1 cpu_flags_arm_crypto CRYPTOPP_BOOL_ARM_CRYPTO_INTRINSICS_AVAILABLE=0|1 For the arm we do not actually have formal cpu flags. However, as far as I see I must modify the config.h to make this permanent, which I do not like at all! Recently I touched based with upstream, I will consult what they think. Thanks!
(In reply to Alexis Ballier from comment #0) > ceph built against 5.6.4, after updating crypto++ to 5.6.5: > > /usr/bin/ceph-mon: symbol lookup error: /usr/bin/ceph-mon: undefined symbol: > _ZNK8CryptoPP8Rijndael3Dec21AdvancedProcessBlocksEPKhS3_Phmj > > $ c++filt _ZNK8CryptoPP8Rijndael3Dec21AdvancedProcessBlocksEPKhS3_Phmj > CryptoPP::Rijndael::Dec::AdvancedProcessBlocks(unsigned char const*, > unsigned char const*, unsigned char*, unsigned long, unsigned int) const Arg... I believe the commit that changed things was "Remove library supplied aesenc, aesdec and friends (Issue 206)", https://github.com/weidai11/cryptopp/commit/fb6a11ff08b9277e9f7a2fecae83027d3e0e60df.
(In reply to Kristian Fiskerstrand from comment #1) > Not sure if this could be related Here's the one you want... "Link failure with mismatched library and program capabilities", https://github.com/weidai11/cryptopp/issues/283.
(In reply to Alon Bar-Lev from comment #3) > I can add some USE flags including > +asm > cpu_flags_x86_sse2 -CRYPTOPP_DISABLE_SSE2 > cpu_flags_x86_sse3 -CRYPTOPP_DISABLE_SSE3 > cpu_flags_x86_sse4_1 -CRYPTOPP_DISABLE_SSE4 > cpu_flags_x86_aes -CRYPTOPP_DISABLE_AESNI > cpu_flags_x86_avx -CRYPTOPP_DISABLE_AVX > cpu_flags_arm_neon CRYPTOPP_BOOL_NEON_INTRINSICS_AVAILABLE=0|1 > cpu_flags_arm_crc32 CRYPTOPP_BOOL_ARM_CRC32_INTRINSICS_AVAILABLE=0|1 > cpu_flags_arm_crypto CRYPTOPP_BOOL_ARM_CRYPTO_INTRINSICS_AVAILABLE=0|1 > > For the arm we do not actually have formal cpu flags. > > However, as far as I see I must modify the config.h to make this permanent, > which I do not like at all! > > Recently I touched based with upstream, I will consult what they think. Thanks Alon. Some of the back story... The library used to supply inline functions with ASM mirroring the intrinsics provided by GCC. If we encountered a down level GCC then we would fall back to our definitions. Compilers around that time were GCC 4.6 through GCC 4.8. It worked well for years but it started showing some gaps when we added Clang support and enhanced testing. For example, Clang is its own little fiefdom even though it claims to be GCC by setting certain preprocessor macros. It took us months to get proper Clang support cut-in. And when testing debug builds and C++11, C++11 and C++17, we encountered a number of issues like https://github.com/weidai11/cryptopp/issues/206. We felt the way to move forward was to remove our mirrors, and add -maes -mpclmul -mrdrnd -mrdseed to CXXFLAGS when needed. Effectively, this combines SSE2 with AESNI, RDRAND and RDSSED without -march=native tainting. You are safe to enable AESNI, RDRAND and RDSSED because their code paths are selected based on a runtime check. You still need to tell GCC to build for SSE2. I believe you do that with -march=i686 -mmmx -msse -msse2. If your target is i386 rather than i686, I suppose you can avoid all the flags. SSE2 is part of amd64 core instruction set, so I don't believe you need to do anything special. You can use -march=x86-64 for amd64, and we use it on over-capable machines when simulating downlevel machines. I'm guessing you are already doing similar under Gentoo's build system. We test the configuration on an old PIII machine, and old XEON server and an old Core2 Duo machine in our test script here: https://github.com/weidai11/cryptopp/blob/master/cryptest.sh#L4618. That is, we take an old machine and we enable AESNI, RDRAND and RDSEED to ensure SIGILLs are avoided. The runtime check for the CPU features occurs here: https://github.com/weidai11/cryptopp/blob/master/cpu.cpp#L221. Once the cpu feature flags are set they are used like in https://github.com/weidai11/cryptopp/blob/master/rijndael.cpp#L229. Once the library is compiled with AESNI, it does not matter what user machines offer or what user programs do. rijndael.o will contain the symbol, and the code paths will be available.
(In reply to Jeffrey Walton from comment #6) > Once the library is compiled with AESNI, it does not matter what user > machines offer or what user programs do. rijndael.o will contain the symbol, > and the code paths will be available. I am unsure I follow. If library is not compiled with AESNI, however, a program that uses it is built in -march that does support AESNI, what will happen? As far as I understand, this will fail as the config.h will expect AESNI to be available in library.
(In reply to Alon Bar-Lev from comment #7) > (In reply to Jeffrey Walton from comment #6) > > Once the library is compiled with AESNI, it does not matter what user > > machines offer or what user programs do. rijndael.o will contain the symbol, > > and the code paths will be available. > > I am unsure I follow. > If library is not compiled with AESNI, however, a program that uses it is > built in -march that does support AESNI, what will happen? This type of bug report will happen. The semi-reduced case is https://github.com/weidai11/cryptopp/issues/283. The semi-reduced case builds the library with -march=x86-64 (SSE2 only), and builds the test program with -march=native (SSE4, AENI, AVX, BMI, ADX, etc) (assuming a capable machine). To avoid this type of bug report, and as a distro, you have to ensure you are _always_ making AESNI available when you build and package the library. The way to do it with GCC is the -maes option. The preprocessor macro you are citing, CRYPTOPP_BOOL_AESNI_INTRINSICS_AVAILABLE, is tied to GCC's __AES__ preprocessor macro. All you need to do to get both of them defined is to use GCC's -maes option. Also see https://github.com/weidai11/cryptopp/blob/master/config.h#L497. Once the library is built with AESNI, then CryptoPP::Rijndael::Dec::AdvancedProcessBlocks will be available in the static library and/or shared object. If a program wants to link to it, it will be available. Don't worry if the machine lacks AESNI, RDRAND, etc. The library will do the right thing at runtime. The same _cannot_ be said of SSE3, SSSE3, SSE4, and friends because GCC will regularly emit those instructions. You can go the other way, too. You can categorically disable AESNI. But we think its better to categorically enable it because AESNI is a big performance boost and it is guarded at runtime. ----- > As far as I understand, this will fail as the config.h will expect AESNI to be available in library. OK, this is the light at the end of the tunnel. Once rijndael.cpp is compiled into rijndael.o, then the symbol CryptoPP::Rijndael::Dec::AdvancedProcessBlocks will be available. It does not matter what config.h thinks. ----- You feel this pain due to the way the library guarded Rijndael::Dec::AdvancedProcessBlocks. Hindsight is always 20/20. We feel the pain, too. Rijndal is so entangled that we have not added native AES support for VIA chips or ARMv8. I think we are at the point we can make changes without fear of breaking something because we have test in place to verify code generation and AES operations. ----- Ironically, I did not make the changes necessary to clear the Crypto++ 283 issue because I did not want to break ABI compatibility. No good deed goes unpunished...
Hi, Thanks for the detailed explanation, however, it is the upstream role to provide a stable working version, we (as any other distribution) cannot teak each package and understand the internals that may change between versions. You should decide if you want to perform runtime optimization or build time. If you use runtime then you must enable everything required at built time to be able to build all kind of optimizations so that at runtime you choose what to enable. If you use build time then you must freeze whatever optimizations used at build time so that runtime will use no more than that level. Also, if you provide an option, such as disabling ASM during build time, you must also make this permanent, by either adding stubs or making sure they are persistent if they are part of the interface. In any case a dynamically detection of machine features should not be part of library consumer interface. In your case this probably mean that the config.h that is used for building differs from the config.h that can be used by other programs as consumer unless somehow you make sure that this will work in all cases and be able to be affected by the build time options. Thanks!
> Thanks for the detailed explanation, however, it is the upstream role to > provide a stable working version, we (as any other distribution) cannot teak > each package and understand the internals that may change between versions. OK, thanks. Sorry we could not be of more help.
(In reply to Alexis Ballier from comment #0) > ceph built against 5.6.4, after updating crypto++ to 5.6.5: > > /usr/bin/ceph-mon: symbol lookup error: /usr/bin/ceph-mon: undefined symbol: > _ZNK8CryptoPP8Rijndael3Dec21AdvancedProcessBlocksEPKhS3_Phmj > > $ c++filt _ZNK8CryptoPP8Rijndael3Dec21AdvancedProcessBlocksEPKhS3_Phmj > CryptoPP::Rijndael::Dec::AdvancedProcessBlocks(unsigned char const*, > unsigned char const*, unsigned char*, unsigned long, unsigned int) const This was cleared at Commit 733a073d6554. Also see https://github.com/weidai11/cryptopp/commit/733a073d65548848aabc39a45b5addb0e01b68fe. My apologies for the break.
Upstream brakes ABI/API nothing we can do.
