Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 322713 - dev-libs/crypto++ problems (net-p2p/amule-2.2.6 crash)
Summary: dev-libs/crypto++ problems (net-p2p/amule-2.2.6 crash)
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: x86 Linux
: High normal with 1 vote (vote)
Assignee: Crypto team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-06-04 07:10 UTC by tekwyzrd
Modified: 2011-12-28 07:02 UTC (History)
12 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
With this, gcc 4.5 can build amule 2.2.6 (amule-2.2.6-gcc45.patch,483 bytes, patch)
2010-09-10 13:54 UTC, Joel
Details | Diff
Patch to build without assembler code (assembler-amule.patch,698 bytes, patch)
2010-10-14 17:01 UTC, Dominique Michel
Details | Diff
patch to incorporate assembler-amule.patch into the ebuild (assembler-amule.patch,698 bytes, patch)
2010-10-14 17:04 UTC, Dominique Michel
Details | Diff
Sorry, this is the correct file for the ebuild (crypto++-5.6.1-r1.patch,282 bytes, patch)
2010-10-14 17:17 UTC, Dominique Michel
Details | Diff
Patch from upstream to fix infinite recursion in deciding whether to use portable code, SSE2 assembly, or AESNI in Rijndael.cpp (crypto++-5.6.1-rijndael.patch,578 bytes, patch)
2011-12-27 15:37 UTC, .:deadhead:.
Details | Diff
Patch to update crypto++ ebuild to include rijndael.patch (crypto++-5.6.1-r2.ebuild.patch,440 bytes, patch)
2011-12-27 15:41 UTC, .:deadhead:.
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description tekwyzrd 2010-06-04 07:10:39 UTC
aMule 2.2.6 fails to complete startup due to crypto++ problem and requires intervcention by user to kill (ctrl=c if started in konsole or via system monitor if started using .desktop file

Reproducible: Always

Steps to Reproduce:
1.emerge crypto++
2.emerge amule
3.start amule

Actual Results:  
amule present in system monitor but doesn't complete startup and no gui appears

Expected Results:  
amule should start and gui appear.

re-emerged current gentoo crypto++-5.6.0-r1 and amule 2.2.6

gdb /usr/bin/amule

warning: Can not parse XML syscalls information; XML support was disabled at compile time.
GNU gdb (Gentoo 7.0 p1) 7.0
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.gentoo.org/>...
Reading symbols from /usr/bin/amule...Reading symbols from /usr/lib/debug/usr/bin/amule.debug...done.
(no debugging symbols found)...done.
(gdb) run
Starting program: /usr/bin/amule 
[Thread debugging using libthread_db enabled]
^C
Program received signal SIGINT, Interrupt.
0xb7d99a31 in CryptoPP::X86_SHA256_HashBlocks(unsigned int*, unsigned int const*, unsigned int) () from /usr/lib/libcrypto++.so.0
(gdb) bt
#0  0xb7d99a31 in CryptoPP::X86_SHA256_HashBlocks(unsigned int*, unsigned int const*, unsigned int) () from /usr/lib/libcrypto++.so.0
Cannot access memory at address 0x9cfec676
(gdb) bt full
#0  0xb7d99a31 in CryptoPP::X86_SHA256_HashBlocks(unsigned int*, unsigned int const*, unsigned int) () from /usr/lib/libcrypto++.so.0
No symbol table info available.
Cannot access memory at address 0x9cfec676
(gdb) thread apply all bt

Thread 1 (Thread 0xb4ee4720 (LWP 6714)):
#0  0xb7d99a31 in CryptoPP::X86_SHA256_HashBlocks(unsigned int*, unsigned int const*, unsigned int) () from /usr/lib/libcrypto++.so.0
Cannot access memory at address 0x9cfec676
(gdb) kill
Kill the program being debugged? (y or n) y
(gdb)
Comment 1 Joel 2010-09-10 13:54:15 UTC
Created attachment 246709 [details, diff]
With this, gcc 4.5 can build amule 2.2.6
Comment 2 Kamen Dokov 2010-09-29 19:57:54 UTC
I still can not run amule:

Sep 29 22:52:40 [kernel] [ 2832.814008] amulegui[25131]: segfault at 7fff9778bff8 ip 00007fcbafd93e5f sp 00007fff9778bfc0 error 6 in libcrypto++.so.0.0.0[7fcbafab4000+3d8000]
Comment 3 blackdream 2010-10-01 03:01:09 UTC
(In reply to comment #2)
> I still can not run amule:
> 
> Sep 29 22:52:40 [kernel] [ 2832.814008] amulegui[25131]: segfault at
> 7fff9778bff8 ip 00007fcbafd93e5f sp 00007fff9778bfc0 error 6 in
> libcrypto++.so.0.0.0[7fcbafab4000+3d8000]
> 

Same error here. ~amd64
Comment 4 Patrizio Bassi 2010-10-02 17:01:15 UTC
after upgrading to new crypto++-5.6.1
i have a seg fault on startup

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff725c993 in CryptoPP::BlockTransformation::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()
   from /usr/lib/libcrypto++.so.0
(gdb) bt
#0  0x00007ffff725c993 in CryptoPP::BlockTransformation::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()
   from /usr/lib/libcrypto++.so.0
#1  0x00007ffff735b0db in CryptoPP::Rijndael::Enc::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()
   from /usr/lib/libcrypto++.so.0
#2  0x00007ffff735b113 in CryptoPP::Rijndael::Enc::ProcessAndXorBlock(unsigned char const*, unsigned char const*, unsigned char*) const () from /usr/lib/libcrypto++.so.0
#3  0x00007ffff725cab9 in CryptoPP::BlockTransformation::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()
   from /usr/lib/libcrypto++.so.0
#4  0x00007ffff735b0db in CryptoPP::Rijndael::Enc::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()
   from /usr/lib/libcrypto++.so.0
#5  0x00007ffff735b113 in CryptoPP::Rijndael::Enc::ProcessAndXorBlock(unsigned char const*, unsigned char const*, unsigned char*) const () from /usr/lib/libcrypto++.so.0
#6  0x00007ffff725cab9 in CryptoPP::BlockTransformation::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()
   from /usr/lib/libcrypto++.so.0
#7  0x00007ffff735b0db in CryptoPP::Rijndael::Enc::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()
   from /usr/lib/libcrypto++.so.0
#8  0x00007ffff735b113 in CryptoPP::Rijndael::Enc::ProcessAndXorBlock(unsigned char const*, unsigned char const*, unsigned char*) const () from /usr/lib/libcrypto++.so.0
#9  0x00007ffff725cab9 in CryptoPP::BlockTransformation::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()
   from /usr/lib/libcrypto++.so.0
#10 0x00007ffff735b0db in CryptoPP::Rijndael::Enc::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()              
   from /usr/lib/libcrypto++.so.0                                                                                                                                                       
#11 0x00007ffff735b113 in CryptoPP::Rijndael::Enc::ProcessAndXorBlock(unsigned char const*, unsigned char const*, unsigned char*) const () from /usr/lib/libcrypto++.so.0               
#12 0x00007ffff725cab9 in CryptoPP::BlockTransformation::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()        
   from /usr/lib/libcrypto++.so.0                                                                                                                                                       
#13 0x00007ffff735b0db in CryptoPP::Rijndael::Enc::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()              
   from /usr/lib/libcrypto++.so.0                                                                                                                                                       
#14 0x00007ffff735b113 in CryptoPP::Rijndael::Enc::ProcessAndXorBlock(unsigned char const*, unsigned char const*, unsigned char*) const () from /usr/lib/libcrypto++.so.0               
#15 0x00007ffff725cab9 in CryptoPP::BlockTransformation::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()        
   from /usr/lib/libcrypto++.so.0                                                                                                                                                       
#16 0x00007ffff735b0db in CryptoPP::Rijndael::Enc::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()              
   from /usr/lib/libcrypto++.so.0                                                                                                                                                       
#17 0x00007ffff735b113 in CryptoPP::Rijndael::Enc::ProcessAndXorBlock(unsigned char const*, unsigned char const*, unsigned char*) const () from /usr/lib/libcrypto++.so.0
#18 0x00007ffff725cab9 in CryptoPP::BlockTransformation::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()
   from /usr/lib/libcrypto++.so.0
#19 0x00007ffff735b0db in CryptoPP::Rijndael::Enc::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()
   from /usr/lib/libcrypto++.so.0
#20 0x00007ffff735b113 in CryptoPP::Rijndael::Enc::ProcessAndXorBlock(unsigned char const*, unsigned char const*, unsigned char*) const () from /usr/lib/libcrypto++.so.0
#21 0x00007ffff725cab9 in CryptoPP::BlockTransformation::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()
   from /usr/lib/libcrypto++.so.0
#22 0x00007ffff735b0db in CryptoPP::Rijndael::Enc::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()
   from /usr/lib/libcrypto++.so.0
#23 0x00007ffff735b113 in CryptoPP::Rijndael::Enc::ProcessAndXorBlock(unsigned char const*, unsigned char const*, unsigned char*) const () from /usr/lib/libcrypto++.so.0
#24 0x00007ffff725cab9 in CryptoPP::BlockTransformation::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()
   from /usr/lib/libcrypto++.so.0
#25 0x00007ffff735b0db in CryptoPP::Rijndael::Enc::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()
   from /usr/lib/libcrypto++.so.0
#26 0x00007ffff735b113 in CryptoPP::Rijndael::Enc::ProcessAndXorBlock(unsigned char const*, unsigned char const*, unsigned char*) const () from /usr/lib/libcrypto++.so.0
#27 0x00007ffff725cab9 in CryptoPP::BlockTransformation::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()
   from /usr/lib/libcrypto++.so.0
#28 0x00007ffff735b0db in CryptoPP::Rijndael::Enc::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()
   from /usr/lib/libcrypto++.so.0
#29 0x00007ffff735b113 in CryptoPP::Rijndael::Enc::ProcessAndXorBlock(unsigned char const*, unsigned char const*, unsigned char*) const () from /usr/lib/libcrypto++.so.0
#30 0x00007ffff725cab9 in CryptoPP::BlockTransformation::AdvancedProcessBlocks(unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) const ()
Comment 5 Patrizio Bassi 2010-10-03 08:51:00 UTC
after unmerging 5.6.1, remerging it, and remerging amule it works back ....
Comment 6 tekwyzrd 2010-10-03 17:57:23 UTC
After removing amule 2.2.6 and crypto++-5.6.0-r1 I emerged crypto++-5.6.1 and re-emerged. For the first time in about six months I am now able to start amule.
Comment 7 Dominique Michel 2010-10-14 16:59:44 UTC
To unmerge crypto++ and remerge it and amule didn't helped into my system. One solution for me is to mask all recent crypto++ vesrions and use crypto++-5.5.2-r1.

As amule-2.2.6-gcc45.patch didn't work for some users, I didn't try it.
Following the clue on http://forum.amule.org/index.php?topic=18222.0
I patched config.h in order to build CryptoPP without Assembler code.

And it work on my ~amd64 system with
[ebuild   R   ] dev-libs/crypto++-5.6.1-r1  0 kB [1]
[ebuild   R   ] net-p2p/amule-10313  USE="daemon geoip gtk nls remote stats unicode upnp (-aqua) -debug (-kdeenablefinal) -plasma -xchat" 0 kB [1]

This is with the svn version of amule, but the problem and the solution are exactly the same with the version in portage for me.
Comment 8 Dominique Michel 2010-10-14 17:01:01 UTC
Created attachment 250585 [details, diff]
Patch to build without assembler code
Comment 9 Dominique Michel 2010-10-14 17:04:14 UTC
Created attachment 250587 [details, diff]
patch to incorporate assembler-amule.patch into the ebuild
Comment 10 Dominique Michel 2010-10-14 17:17:25 UTC
Created attachment 250589 [details, diff]
Sorry, this is the correct file for the ebuild
Comment 11 Dominique Michel 2010-10-21 15:56:43 UTC
I try to report this bug upstream, but they want a test program with the report, so I gave up (I don't have the knowledge for that).
Comment 12 Lubomir Krajcovic 2010-11-22 20:35:45 UTC
(In reply to comment #11)
> I try to report this bug upstream, but they want a test program with the
> report, so I gave up (I don't have the knowledge for that).
> 

None of this worked for me. Tried your patch, as well as this much more ellegant (does the same, but in more generic way):
--- crypto++-5.6.1.ebuild       2010-11-22 20:24:56.970000010 +0100
+++ crypto++-5.6.1-r1.ebuild    2010-11-22 20:18:04.340000010 +0100
@@ -30,6 +30,7 @@
        replace-flags -O? -O1
        filter-flags -fomit-frame-pointer
-       use amd64 && append-flags -DCRYPTOPP_DISABLE_ASM
+       use amd64 && append-flags -DCRYPTOPP_DISABLE_ASM -DCRYPTOPP_DISABLE_SSE2
        emake -f GNUmakefile CXX="$(tc-getCXX)" CXXFLAGS="${CXXFLAGS}" \
                LIBDIR="$(get_libdir)" || die "emake failed"
 }

With this patch crypto++ 5.6.1 passes the ebuild test suite, but amule still fails (even after rebuild or unmerge+emerge).

What worked for me was going back to crypto++-5.6.0-r1 (masking >=dev-libs/crypto++-5.6.1) and rebuilding crypto++ and amule (in this order).

So the proper way is to fix the amule-2.2.6 ebuild (disable crypto++ >= 5.6.1
Comment 13 Lubomir Krajcovic 2010-11-22 20:40:27 UTC
(In reply to comment #11)
> I try to report this bug upstream, but they want a test program with the
> report, so I gave up (I don't have the knowledge for that).
> 

btw.: to run the embedded test program (with report) emerge crypto++ with "test" feature on - and send them the output:

linux-box # FEATURES="test" emerge -1v crypto++
Comment 14 Dominique Michel 2010-11-22 23:10:17 UTC
(In reply to comment #12)
> (In reply to comment #11)
> > I try to report this bug upstream, but they want a test program with the
> > report, so I gave up (I don't have the knowledge for that).
> > 
> 
> None of this worked for me. Tried your patch, as well as this much more
> ellegant (does the same, but in more generic way):
> --- crypto++-5.6.1.ebuild       2010-11-22 20:24:56.970000010 +0100
> +++ crypto++-5.6.1-r1.ebuild    2010-11-22 20:18:04.340000010 +0100
> @@ -30,6 +30,7 @@
>         replace-flags -O? -O1
>         filter-flags -fomit-frame-pointer
> -       use amd64 && append-flags -DCRYPTOPP_DISABLE_ASM
> +       use amd64 && append-flags -DCRYPTOPP_DISABLE_ASM
> -DCRYPTOPP_DISABLE_SSE2
>         emake -f GNUmakefile CXX="$(tc-getCXX)" CXXFLAGS="${CXXFLAGS}" \
>                 LIBDIR="$(get_libdir)" || die "emake failed"
>  }
> 
> With this patch crypto++ 5.6.1 passes the ebuild test suite, but amule still
> fails (even after rebuild or unmerge+emerge).
> 
> What worked for me was going back to crypto++-5.6.0-r1 (masking
> >=dev-libs/crypto++-5.6.1) and rebuilding crypto++ and amule (in this order).
> 
> So the proper way is to fix the amule-2.2.6 ebuild (disable crypto++ >= 5.6.1
> 

This is not a fix but a workaround. The bug is into libcrypto with the inlined assembler code.

And more, the only one portage version that work with amule is 5.5.2-r1 for me, so which libcrypto version will you use?
Comment 15 Dominique Michel 2010-11-22 23:11:02 UTC
(In reply to comment #13)
> (In reply to comment #11)
> > I try to report this bug upstream, but they want a test program with the
> > report, so I gave up (I don't have the knowledge for that).
> > 
> 
> btw.: to run the embedded test program (with report) emerge crypto++ with
> "test" feature on - and send them the output:
> 
> linux-box # FEATURES="test" emerge -1v crypto++

Done : https://sourceforge.net/apps/trac/cryptopp/ticket/6
> 

Comment 16 Zooko O'Whielacronx 2010-12-19 18:18:57 UTC
Could you please tell me what version of binutils you had installed when you built libcrypto++? There was a bug in binutils 2.20 which was fixed in binutils 2.20.1, which might explain this.
Comment 17 Zooko O'Whielacronx 2010-12-19 18:19:48 UTC
The bug I mean was tracked here:

http://sourceware.org/bugzilla/show_bug.cgi?id=10856

And its impact on various other projects was collected here:

https://bugs.launchpad.net/ubuntu/+source/binutils/+bug/461303
Comment 18 mondrillo 2011-01-02 11:01:07 UTC
eix amule && eix crypto++

[I] net-p2p/amule
     Available versions:  2.2.6 {daemon debug geoip gtk nls remote stats unicode upnp}
     Installed versions:  2.2.6(11:35:15 02/01/11)(debug geoip gtk nls stats unicode -daemon -remote -upnp)
     Homepage:            http://www.amule.org/
     Description:         aMule, the all-platform eMule p2p client

[I] dev-libs/crypto++
     Available versions:  5.5.2-r1 (~)5.6.0 5.6.0-r1 [m](~)5.6.1 {sse3}
     Installed versions:  5.6.0-r1(11:18:39 02/01/11)
     Homepage:            http://cryptopp.com
     Description:         Crypto++ is a C++ class library of cryptographic schemes

After one month, I've masked crypto++-5.6.1 and apply amule-2.2.6-gcc45.patch in a portage overlay now I've have amule runnig now on my amd64 machine.

Some usefull url http://en.gentoo-wiki.com/wiki/Writing_Ebuilds

Comment 19 Zooko O'Whielacronx 2011-01-02 13:45:26 UTC
(In reply to comment #18)
> 
> [I] dev-libs/crypto++
>      Available versions:  5.5.2-r1 (~)5.6.0 5.6.0-r1 [m](~)5.6.1 {sse3}
>      Installed versions:  5.6.0-r1(11:18:39 02/01/11)
...
> After one month, I've masked crypto++-5.6.1 and apply amule-2.2.6-gcc45.patch
> in a portage overlay now I've have amule runnig now on my amd64 machine.

mondrillo:

I don't actually have anything to do with the Gentoo project--I don't use it and I'm not a significant contributor of patches to it--but I found this while searching for issues related to Crypto++, which is a library that I rely on a lot. (I contribute to Tahoe-LAFS and pycryptopp projects.)

Could you please tell me what version of binutils you are using? If all of the people who are experiencing this issue are using binutils 2.20 then I can assume that this issue is caused by that bug in binutils (see #c17). If you are using a different version of binutils then this may be a previously unknown bug in Crypto++ which requires investigation.

Thanks!

Does the "I've masked crypto++-5.6.1" and the versions quoted above mean that you have this issue with crypto++-5.6.1 but you do not have this issue with crypto++-5.6.0?
Comment 20 Patrick Lauer gentoo-dev 2011-01-05 10:18:46 UTC
+  05 Jan 2011; Patrick Lauer <patrick@gentoo.org> amule-2.2.6.ebuild:
+  Temporary fix - amule segfaults on startup with certain crypto++ versions bug
+  #322713

The failure seems to be limited to 5.6.1 only. I will try to narrow it down some more, for now I've adapted the dependencies until we get a proper fix.
Comment 21 Zooko O'Whielacronx 2011-01-05 13:35:12 UTC
Dear Patrick Lauer:

Could you tell me what version of binutils you're using?


Comment 22 Patrick Lauer gentoo-dev 2011-01-05 13:50:45 UTC
(In reply to comment #21)
> Dear Patrick Lauer:
> 
> Could you tell me what version of binutils you're using?
> 

[ebuild   R   ] sys-devel/binutils-2.21  USE="nls -multislot -multitarget -test -vanilla" 18,313 kB
and it's an amd64 install
Comment 23 Zooko O'Whielacronx 2011-01-05 16:39:44 UTC
(In reply to comment #22)
> [ebuild   R   ] sys-devel/binutils-2.21  USE="nls -multislot -multitarget -test
> -vanilla" 18,313 kB
> and it's an amd64 install

Okay since it is binutils-2.21 then it is definitely not https://bugs.launchpad.net/ubuntu/+source/binutils/+bug/461303 . Could you please do the following:

1. wget http://cryptopp.com/cryptopp561.zip
2. sha1sum on that file should yield 31dbb456c21f50865218c57b7eaf4c955a222ba1
3. unzip cryptopp561.zip
4. make
5. ./cryptest.exe v

And report the result? The make and the cryptest.exe v should succeed, report success, and exit with exit code 0. If they don't then we've narrowed down the bug to being in the combination of upstream Crypto++ source and your build environment.
Comment 24 Patrick Lauer gentoo-dev 2011-01-05 17:15:04 UTC
(In reply to comment #23)

> please do the following:
> 
> 1. wget http://cryptopp.com/cryptopp561.zip
> 2. sha1sum on that file should yield 31dbb456c21f50865218c57b7eaf4c955a222ba1
> 3. unzip cryptopp561.zip
> 4. make
> 5. ./cryptest.exe v
> 
> And report the result? 

All tests passed!

Test ended at Wed Jan  5 17:13:55 2011
Seed used was: 1294247634

> The make and the cryptest.exe v should succeed, report
> success, and exit with exit code 0. If they don't then we've narrowed down the
> bug to being in the combination of upstream Crypto++ source and your build
> environment.
> 

Comment 25 Zooko O'Whielacronx 2011-01-05 18:22:26 UTC
(In reply to comment #24)
> All tests passed!

Okay, this is progress! So something about the emerge causes the test failure to appear when it doesn't appear in the upstream version. Are there any patches applied to the Crypto++ source code itself by the Gentoo emerge? If so, maybe we could try applying those patches and then using the upstream Crypto++ make and ./cryptest.exe v to see if those patches make the difference.

If there are not any patches to the source itself then the difference has to be in the way the emerge build differs from the upstream make build. If that is the case, could you please show me a transcript of what g++ commands are issued by each of those two build systems?

Thanks!
Comment 26 Patrick Lauer gentoo-dev 2011-01-05 18:38:29 UTC
http://sourcestest.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/dev-libs/crypto%2B%2B/files/

there's two patches applied, the -sunos patch looks harmless, but the fix_build_system patch seems to invoke libtool and changes around a few things. I would suspect that as the difference in behaviour.
Comment 27 Patrick Lauer gentoo-dev 2011-01-05 18:49:25 UTC
Testing MessageDigest algorithm SHA-1.
make: *** [test] Segmentation fault
emake failed
 * Crypto++ self-tests failed.

So that's the difference. I'll see if I can narrow it done even more.
Comment 28 Patrick Lauer gentoo-dev 2011-01-05 18:55:05 UTC
So it appears the problem is completely triggered on the crypto++ side, the fix-build-system patch causes miscompilation (possibly limited to new binutils versions) that cause reproducable segfaults in the sha1 tests.
This may be limited to amd64 only and seems to be limited to crypto++ 5.6.1 only.
Comment 29 Zooko O'Whielacronx 2011-01-05 19:37:03 UTC
(In reply to comment #28)
> So it appears the problem is completely triggered on the crypto++ side, the
> fix-build-system patch causes miscompilation

Nice work. Can you give me a URL to view the fix-build-system patch?
Comment 30 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2011-01-05 21:38:12 UTC
(In reply to comment #23)

You should test with the same CFLAGS/CXXFLAGS/LDFLAGS as when using ebuild.
Using of libtool shouldn't cause any problems.
Comment 31 Nick Bowler 2011-01-06 02:24:25 UTC
Building upstream crypto++ with -DCRYPTOPP_DISABLE_ASM added to CXXFLAGS causes the same segfault in the SHA-1 test.  The ebuild adds this on amd64 for some reason which is not documented.
Comment 32 Zooko O'Whielacronx 2011-01-06 06:04:12 UTC
(In reply to comment #31)
> Building upstream crypto++ with -DCRYPTOPP_DISABLE_ASM added to CXXFLAGS causes
> the same segfault in the SHA-1 test.  The ebuild adds this on amd64 for some
> reason which is not documented.

Wait, is that right -- *adding* CRYPTOPP_DISABLE_ASM makes the segfault happen? Is this reproducible? That's surprising that the asm version would pass tests but the C++ version would segfault.
Comment 33 Patrick Lauer gentoo-dev 2011-01-06 08:44:34 UTC
(In reply to comment #32)
> (In reply to comment #31)
> > Building upstream crypto++ with -DCRYPTOPP_DISABLE_ASM added to CXXFLAGS causes
> > the same segfault in the SHA-1 test.  The ebuild adds this on amd64 for some
> > reason which is not documented.
> 
> Wait, is that right -- *adding* CRYPTOPP_DISABLE_ASM makes the segfault happen?

Yes, I've isolated it down to that. Seems to be independent of the patching I suspected.
> Is this reproducible? That's surprising that the asm version would pass tests
> but the C++ version would segfault.
> 
The ebuild also forces CFLAGS="-O1" - that might be part of it. I'll test and report back :)
Comment 34 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2011-01-06 17:29:58 UTC
The original report was about a problem in dev-libs/crypto++-5.6.0* on x86.
The last comments are about a problem in dev-libs/crypto++-5.6.1* only on amd64, which was caused by -DCRYPTOPP_DISABLE_ASM (bug #308335).

Raúl Porcel: Do you still need -DCRYPTOPP_DISABLE_ASM on amd64?
Comment 35 Raúl Porcel (RETIRED) gentoo-dev 2011-01-06 18:48:54 UTC
(In reply to comment #34)
> The original report was about a problem in dev-libs/crypto++-5.6.0* on x86.
> The last comments are about a problem in dev-libs/crypto++-5.6.1* only on
> amd64, which was caused by -DCRYPTOPP_DISABLE_ASM (bug #308335).
> 
> Raúl Porcel: Do you still need -DCRYPTOPP_DISABLE_ASM on amd64?
> 

without it the tests on crypto++-5.6.1* eat a lot of memory...
Comment 36 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2011-01-06 19:04:55 UTC
(In reply to comment #35)
> (In reply to comment #34)
> > Raúl Porcel: Do you still need -DCRYPTOPP_DISABLE_ASM on amd64?
> 
> without it the tests on crypto++-5.6.1* eat a lot of memory...

For some people, this problem occurs *with* -DCRYPTOPP_DISABLE_ASM.
I suggest to introduce "asm" USE flag, which will control addition of -DCRYPTOPP_DISABLE_ASM.
Comment 37 Patrizio Bassi 2011-01-06 19:32:10 UTC
i don't think the scope of any use flag is "please make this package work or not"

personally i'm on amd64, and i have no problems with it now, after a full
amule and crypto++ unmerge, plus a remerge from scratch, because i felt something wrong was going with the build process.

just give a try for this.


Comment 38 Zooko O'Whielacronx 2011-01-06 21:07:37 UTC
I am not interested in figuring out what flags make the package work. I'm interested in figuring out what makes the package fail, because that indicates a bug and possibly a security hole that the Crypto++ maintainers (or possibly the Gentoo or gcc or binutils maintainers) need to understand and deal with.

Oh! I was able to reproduce it on Ubuntu 10.04 simply by adding CRYPTOPP_DISABLE_ASM! This doesn't cause it to segfault every time, but it did cause it to segfault once just now. Great! Now I can submit a bug report to Crypto++ and/or Ubuntu and/or gcc and/or binutils without bothering you folks about it any more. :-)
Comment 39 Zooko O'Whielacronx 2011-01-06 23:41:26 UTC
Related issue tickets on other trackers: http://tahoe-lafs.org/trac/pycryptopp/ticket/66 and https://sourceforge.net/apps/trac/cryptopp/ticket/6 .
Comment 40 Zooko O'Whielacronx 2011-01-07 06:12:28 UTC
This is fixed in upstream Crypto++ trunk. This is fixed in upstream Crypto++ trunk, see https://sourceforge.net/apps/trac/cryptopp/ticket/6 . I suggest that you work-around the bug by *not* adding -DCRYPTOPP_DISABLE_ASM. :-)
Comment 41 Patrizio Bassi 2011-01-07 12:15:21 UTC
why not leave the asm code enabled and apply the upstream patch?
it's really small and not intrusive
Comment 42 Raúl Porcel (RETIRED) gentoo-dev 2011-01-07 14:40:03 UTC
Ah, nevermind, i was wrong. Removing the CRYPTO ASM thing makes the tests pass. So yeah, this can be removed now. Sorry for the confusion before
Comment 43 Dane Smith (RETIRED) gentoo-dev 2011-01-08 04:13:24 UTC
Should be fixed by crypto++-5.6.1-r1

*crypto++-5.6.1-r1 (08 Jan 2011)

  08 Jan 2011; Dane Smith <c1pher@gentoo.org> +crypto++-5.6.1-r1.ebuild:
  Remove no longer needed -DCRYPTOPP_DISABLE_ASM on amd64.
  *Fix crypto++ issues on amd64 wrt amule as noted in bug 322713.
  *Fix issues with tests on amd64 as noted in bug 343373.
  There is an upstream patch to fix these issues with DCRYPTOPP_DISABLE_ASM
  at
  http://sourceforge.net/apps/trac/cryptopp/ticket/6 but as we no longer
  use it,I didn't include it.

I tested amule and didn't run into any issues. Feel free to reopen if I missed anything. Thanks for the help everyone!
Comment 44 Patrizio Bassi 2011-01-08 10:48:05 UTC
dane, please remove the limitation from amule ebuild

dev-libs/crypto++:0

  (dev-libs/crypto++-5.6.1-r1, ebuild scheduled for merge) conflicts with
    <=dev-libs/crypto++-5.6.1 required by (net-p2p/amule-2.2.6, installed)
Comment 45 Dane Smith (RETIRED) gentoo-dev 2011-01-10 20:36:37 UTC
net-p2p,
I don't mind removing the restriction from the amule ebuild, but I'd like an explicit ok from someone if possible.

Thanks!
Comment 46 Raúl Porcel (RETIRED) gentoo-dev 2011-01-10 20:47:55 UTC
(In reply to comment #45)
> net-p2p,
> I don't mind removing the restriction from the amule ebuild, but I'd like an
> explicit ok from someone if possible.
> 
> Thanks!
> 

explicit ok :D

You shouldn't ask since the depend was done by a non-member of the net-p2p team :)
Comment 47 Dane Smith (RETIRED) gentoo-dev 2011-01-10 21:40:41 UTC
Fixed by Patrick in CVS. Reclosing.
Comment 48 .:deadhead:. 2011-12-27 15:31:20 UTC
Trying amule-2.3.1 I was having problems similar to the one in comment #4 .
Looking around I've found that the patch from the cryptopp devel is needed :

"Turns out the code for deciding whether to use portable code, SSE2 assembly, or AESNI in Rijndael.cpp introduced an infinite recursion when on x64, assembly disabled, and no AESNI."

http://sourceforge.net/apps/trac/cryptopp/ticket/6

Adding this patch to dev-libs/crypto++-5.6.1-r1 I've been able to make works amule-2.3.1. This will not brake amule-2.2.6 either (I've just recompiled it, no core dumps).

Can you please review and commit the following patches and bump 5.6.1-r2?
Comment 49 .:deadhead:. 2011-12-27 15:37:22 UTC
Created attachment 297091 [details, diff]
Patch from upstream to fix infinite recursion in deciding whether to use portable code, SSE2 assembly, or AESNI in Rijndael.cpp

http://sourceforge.net/apps/trac/cryptopp/ticket/6#comment:3
Comment 50 .:deadhead:. 2011-12-27 15:41:11 UTC
Created attachment 297095 [details, diff]
Patch to update crypto++ ebuild to include rijndael.patch
Comment 51 Patrick Lauer gentoo-dev 2011-12-28 07:02:29 UTC
+  28 Dec 2011; Patrick Lauer <patrick@gentoo.org> +crypto++-5.6.1-r2.ebuild,
+  +files/crypto++-5.6.1-rijndael.patch:
+  Fixing #322713 again, this time with upstream patch

Many thanks for working on this, I hope this is the last time we fix it :)