I'm creating this bug to bring attention to the poor 64bit performance of openssl. On the same system running 32bit userlands, the performance can be up to 85% better depending on the test itself. The current openssl-0.9.7i ebuild parses out the optimization flags that portage hands it. That is likely the performance problem; however, when optimization flags are added back in, openssl and libs compile but fail some tests fail. Example: ../util/shlib_wrap.sh ./rmdtest error calculating RIPEMD160 on '' got 659fb160f84022ed7c331d3c1e8d147a3ed139e5 instead of 9c1185a5c5e9fc54612808977ee8f548b2258d31 error calculating RIPEMD160 on 'a' got 856f10b43434c0f8822e4ae4e4e389f8171aa9bc instead of 0bdc9d2d256b3ee9daae347be6f4dc835a467ffe error calculating RIPEMD160 on 'abc' got 653b9ced70cededc68b52e47ba9da5d3680f7b1b instead of 8eb208f7e05d987a9b044a8e98c6b087f15a0bfc error calculating RIPEMD160 on 'message digest' got 0cbbe4907a1defac787c6ce1ee9db1f1106ae880 instead of 5d0689ef49d2fae572b881b123a85ffa21595f36 error calculating RIPEMD160 on 'abcdefghijklmnopqrstuvwxyz' got fd47125391ab3e03a4d4397461c67b73734e133d instead of f71c27109c692c1b56bbdceb5b9d2865b3708dbc error calculating RIPEMD160 on 'abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq' got f04401d823cba764448058945f5233575f219ac4 instead of 12a053384a9c0c88e405a06c27dcf49ada62eb2b error calculating RIPEMD160 on 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789' got 335f7ec5b303aab010cef3ea64064ed85ecd62c8 instead of b0e20b6e3116640286ed3a87a5713079b21f5189 error calculating RIPEMD160 on '12345678901234567890123456789012345678901234567890123456789012345678901234567890' got 698d7a91819c622cad5210bc38bf650a7c546d05 instead of 9b752e45573d4b39f4dbd3323cab82bf63326bfb make[1]: *** [test_rmd] Error 8 make[1]: Leaving directory `/var/tmp/portage/openssl-0.9.7i/work/openssl-0.9.7i/test' make: *** [tests] Error 2 !!! ERROR: dev-libs/openssl-0.9.7i failed. In our (ppc64) discussions on this thus far, we believe there may be some issues with the toolchain and/or openssl contributing to this problem. But we will likely need help from those who know openssl better, so we can work backwards from the test failure. If anyone has any openssl upstream contacts, we would readily accept their help and even provide access to a machine to help debug. emerge info follows: merganser openssl # emerge --info Portage 2.0.54 (default-linux/ppc/ppc64/2006.0/64bit-userland, gcc-3.4.4, glibc-2.3.4.20041102-r2, 2.6.12.3 ppc64) ================================================================= System uname: 2.6.12.3 ppc64 PPC970, altivec supported Gentoo Base System version 1.6.14 dev-lang/python: 2.3.5-r2, 2.4.2 sys-apps/sandbox: 1.2.12 sys-devel/autoconf: 2.13, 2.59-r7 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1 sys-devel/binutils: 2.16.1 sys-devel/libtool: 1.5.22 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="ppc64" AUTOCLEAN="yes" CBUILD="powerpc64-unknown-linux-gnu" CFLAGS="-O2 -pipe" CHOST="powerpc64-unknown-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/eselect/compiler /etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig distlocks sandbox sfperms strict" GENTOO_MIRRORS="http://gentoo.chem.wisc.edu/gentoo/" MAKEOPTS="-j4" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://tux.rchland.ibm.com/gentoo-portage" USE="ppc64 X alsa altivec apache2 apm arts audiofile berkdb bitmap-fonts bzip2 cli cscope ctype cups dba dri eds emboss encode esd expat fastbuild foomaticdb force-cgi-redirect fortran ftp gd gdbm gif gnome gpm gstreamer gtk gtk2 ibm imlib ipv6 jpeg kde lcms libg++ libwww mad memlimit mikmod mng motif mp3 mpeg ncurses nls nptl ogg opengl oss pam pcre pdflib perl png posix python qt quicktime readline sdl session simplexml soap sockets spell spl ssl tcpd tiff tokenizer truetype truetype-fonts type1-fonts udev unicode vorbis xml xml2 xmms xsl xv zlib userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS
adding ppc64 team
i'm pretty sure it's the ppc64 toolchain
Created attachment 86968 [details, diff] Update the ebuild for openssl-0.9.7i to give large speed improvement. O.k. Two more sets of data points. Here is a patch that gives a large improvement in speed, but it kinda overlooks the problem. The second is that the problem with -O2 (or greater) is crypto/ripemd. This is the rmd160 hash algorithm. I haven't seen this used much, but I may just not be observant enough. With this algorithm removed then all the tests pass with full (well -O2) optimisation. I don't see any reason why they should fail at -Os or -O3 though since they pass all the tests before the ripemd ones. Need more information than this?
I agree the problem is probably the toolchain. Further details: 1) All but crypto/ripemd compiles, tests, and (as far as I've seen) runs fine at -O2. This implies to mean that there is no systemic problem with the openssl library. 2) The whole library compiles, tests and runs fine (again, as far as I've seen) at -O. Together with (1) this suggests that there is a probably an optimisation bug in gcc (-3.4.6). 3) That there are similar problems with crypto/ripemd without optimisation flags suggests to me that the problem is indeed gcc. From what I've read gcc is not always that reliable without any -O flags. So, apart from having a sticking plaster to cover the problem (using the previous patch to set -O) the next step probably seems to be getting some compiler guru to look at the crypto/ripemd case. This is a relatively small test case. The rest of the library appears to be reliable and trustworthy so this example should demonstrate the compiler issue. I'm not up on powerpc assembly so can't push this much further forward.
you'd prob be better off doing tests with gcc-4.1.x on ppc64 and it'd be a ton of help to rip down the crypto/ripemd stuff to a simpler test case for a compiler guru to review
I compiled once using an unmodified ebuild and no test failed. then I applied the patch attached in the bug, but left out the 'replace-flags' parts. It compiled just fine using CFLAGS="-O2 -pipe -mtune=power5 -mcpu=power5" and passed all tests. The tests run quicker than without optimazion. This is gcc-4.1.0-r1.
gcc 4.1 is stable. guess this is fixed then. marking as FIXED. please reopen if I misunderstood the situation.
