Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 442568 - dev-libs/libgcrypt-1.5.0-r2 - aes-ni segfaults
Summary: dev-libs/libgcrypt-1.5.0-r2 - aes-ni segfaults
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Library (show other bugs)
Hardware: All Linux
: Normal normal with 1 vote (vote)
Assignee: Crypto team [DISABLED]
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-11-10 09:15 UTC by Toralf Förster
Modified: 2012-12-04 04:17 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build log (app-crypt:gnupg-2.0.19:20121110-090814.log.gz,23.88 KB, text/plain)
2012-11-10 09:15 UTC, Toralf Förster
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Toralf Förster gentoo-dev 2012-11-10 09:15:24 UTC
Created attachment 329074 [details]
build log

Unfortunately it segfaults too at the command line :

tfoerste@n22 ~ $ gpg --armor --recipient 'Herr Grendelsen' --encrypt --sign text 
gpg: enabled debug flags: memstat trust extprog assuan

You need a passphrase to unlock the secret key for
user: "Toralf Förster <toralf.foerster@gmx.de>"
1024-bit DSA key, ID 7DB69DA3, created 2004-08-14

gpg: DBG: connection to agent established

gpg: signal Segmentation fault caught ... exiting
Segmentation fault




n22 /usr/local/portage/sci-misc/boinc # emerge --info gnupg
Portage 2.1.11.9 (default/linux/x86/10.0/desktop/kde, gcc-4.6.3, glibc-2.15-r3, 3.6.6 i686)
=================================================================
                        System Settings
=================================================================
System uname: Linux-3.6.6-i686-Intel-R-_Core-TM-_i5-2540M_CPU_@_2.60GHz-with-gentoo-2.1
Timestamp of tree: Sat, 10 Nov 2012 08:15:01 +0000
ccache version 3.1.7 [disabled]
app-shells/bash:          4.2_p37
dev-java/java-config:     2.1.11-r3
dev-lang/python:          2.7.3-r2, 3.2.3
dev-util/ccache:          3.1.7
dev-util/cmake:           2.8.9
dev-util/pkgconfig:       0.27.1
sys-apps/baselayout:      2.1-r1
sys-apps/openrc:          0.10.5
sys-apps/sandbox:         2.5
sys-devel/autoconf:       2.13, 2.68
sys-devel/automake:       1.9.6-r3, 1.11.6
sys-devel/binutils:       2.22-r1
sys-devel/gcc:            4.6.3
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4-r1
sys-devel/make:           3.82-r3
sys-kernel/linux-headers: 3.4 (virtual/os-headers)
sys-libs/glibc:           2.15-r3
Repositories: gentoo toralf
ACCEPT_KEYWORDS="x86"
ACCEPT_LICENSE="*"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=native -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.4/ext-active/ /etc/php/cgi-php5.4/ext-active/ /etc/php/cli-php5.4/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -march=native -pipe"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--autounmask=n --keep-going=y --nospinner --tree --deep --quiet-build"
FCFLAGS="-O2 -march=i686 -pipe"
FEATURES="assume-digests binpkg-logs compress-build-logs config-protect-if-modified distlocks ebuild-locks fixlafiles news parallel-fetch parse-eapi-ebuild-head protect-owned sandbox sfperms strict test test-fail-continue unknown-features-warn unmerge-logs unmerge-orphans userfetch"
FFLAGS="-O2 -march=i686 -pipe"
GENTOO_MIRRORS="http://mirror.leaseweb.com/gentoo/ http://gentoo.mirror.dkm.cz/pub/gentoo/ http://ftp.snt.utwente.nl/pub/os/linux/gentoo http://gentoo.mneisen.org/ http://www.gtlib.gatech.edu/pub/gentoo"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="en en_GB"
MAKEOPTS="-j5"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://linux.rz.ruhr-uni-bochum.de/gentoo-portage"
USE="X a52 aac acl acpi alsa apache2 berkdb bittorrent branding bzip2 cairo cdda cdr classic cli consolekit corefonts cracklib crypt css cups cxx dbus declarative dri dts dvb dvd dvdr emboss encode exif fam fastbuild ffmpeg firefox flac fontconfig fortran g3dvl gdbm gif git gmp gphoto2 gpm gtk hyphenation iconv ipv6 java jpeg kde kipi kqemu kvm lcms ldap libnotify lirc logrotate mad mbox mmx mng modules mp3 mp4 mpeg mudflap mysql mysqli ncurses nls nptl nsplugin ntfs ogg opengl openmp pam pango pcre pdf phonon plasma png policykit ppds pppd qemu qt3support qt4 rdesktop readline sdl session sna spell sse sse2 sse4_1 ssl ssse3 startup-notification svg tcpd thinkpad threads tiff tk transparent-proxy truetype udev udisks unicode upower usb uxa v4l vaapi vim-syntax vlc vnc vorbis wxwidgets x264 x86 xa xcb xcomposite xinerama xml xpm xscreensaver xv xvid xvmc zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en en_GB" LIRC_DEVICES="devinput inputlirc" PHP_TARGETS="php5-4" PYTHON_TARGETS="python3_2 python2_7" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby19" USERLAND="GNU" VIDEO_CARDS="intel" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON

=================================================================
                        Package Settings
=================================================================

app-crypt/gnupg-2.0.19 was built with the following:
USE="bzip2 (consolekit) ldap nls (policykit) readline test usb -adns -doc (-selinux) -smartcard -static"
CFLAGS="-O2 -march=native -pipe -g -ggdb"
CXXFLAGS="-O2 -march=native -pipe -g -ggdb"
Comment 1 Toralf Förster gentoo-dev 2012-11-10 09:38:34 UTC
Well, it seems to be rather a problem with libgrypt - update from 1.4.x to 1.5 ?? :

tfoerste@n22 ~ $ cat devel/gdb_pgp.config                                                                                                   
set args --armor --recipient Grendelsen --encrypt --sign text                                                                               

          
       tfoerste@n22 ~ $ rm -f text.sc;  gdb -x ~/devel/gdb_pgp.config gpg
GNU gdb (Gentoo 7.4.1 p2) 7.4.1
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.gentoo.org/>...
Reading symbols from /usr/bin/gpg...Reading symbols from /usr/lib/debug/usr/bin/gpg2.debug...done.
done.
(gdb) run
Starting program: /usr/bin/gpg --armor --recipient Grendelsen --encrypt --sign text
warning: Could not load shared library symbols for linux-gate.so.1.
Do you need "set solib-search-path" or "set sysroot"?
gpg: enabled debug flags: memstat trust extprog assuan

You need a passphrase to unlock the secret key for
user: "Toralf Förster <toralf.foerster@gmx.de>"
1024-bit DSA key, ID 7DB69DA3, created 2004-08-14

gpg: DBG: connection to agent established
File `text.asc' exists. Overwrite? (y/N) y

Program received signal SIGSEGV, Segmentation fault.
0xb7f42fdd in do_aesni_enc_aligned (
    a=0xb7f7c8f8 "\001K\257\"x\246\235\063\035Q\200\020\066C\351\232gC\303\321Q\232\264\362͚x\253\t\245\021\275]\036\362\r\316ּ\274\022\023\032\307\305G\210\252\b\016\225\027\353\026wq\232\317r\200\206\004", <incomplete sequence \343>, b=0xbfffe520 "", ctx=0xbfffe334) at rijndael.c:710
710     rijndael.c: No such file or directory.
(gdb) bt
#0  0xb7f42fdd in do_aesni_enc_aligned (
    a=0xb7f7c8f8 "\001K\257\"x\246\235\063\035Q\200\020\066C\351\232gC\303\321Q\232\264\362͚x\253\t\245\021\275]\036\362\r\316ּ\274\022\023\032\307\305G\210\252\b\016\225\027\353\026wq\232\317r\200\206\004", <incomplete sequence \343>, b=0xbfffe520 "", ctx=0xbfffe334) at rijndael.c:710
#1  do_aesni (ctx=0xbfffe334, decrypt_flag=0, bx=0xbfffe520 "", 
    ax=0xb7f7c8f8 "\001K\257\"x\246\235\063\035Q\200\020\066C\351\232gC\303\321Q\232\264\362͚x\253\t\245\021\275]\036\362\r\316ּ\274\022\023\032\307\305G\210\252\b\016\225\027\353\026wq\232\317r\200\206\004", <incomplete sequence \343>) at rijndael.c:1132
#2  0xb7f4329e in rijndael_encrypt (
    a=0xb7f7c8f8 "\001K\257\"x\246\235\063\035Q\200\020\066C\351\232gC\303\321Q\232\264\362͚x\253\t\245\021\275]\036\362\r\316ּ\274\022\023\032\307\305G\210\252\b\016\225\027\353\026wq\232\317r\200\206\004", <incomplete sequence \343>, b=0xbfffe520 "", context=0xbfffe334) at rijndael.c:1155
#3  rijndael_encrypt (context=0xbfffe334, b=0xbfffe520 "", 
    a=0xb7f7c8f8 "\001K\257\"x\246\235\063\035Q\200\020\066C\351\232gC\303\321Q\232\264\362͚x\253\t\245\021\275]\036\362\r\316ּ\274\022\023\032\307\305G\210\252\b\016\225\027\353\026wq\232\317r\200\206\004", <incomplete sequence \343>) at rijndael.c:1138
#4  0xb7f4389c in selftest_basic_128 () at rijndael.c:1660
#5  0xb7f4340f in selftest () at rijndael.c:1749
#6  do_setkey (keylen=32, 
    key=0xb7fd2064 "\377\002\204Ϻ\213ʢ̊\275\347\343q\305\301\226\252\225a\260\271#7\211\333\001", 
    ctx=0xb7fd2880) at rijndael.c:209
#7  rijndael_setkey (context=0xb7fd2880, 
    key=0xb7fd2064 "\377\002\204Ϻ\213ʢ̊\275\347\343q\305\301\226\252\225a\260\271#7\211\333\001", keylen=32)
    at rijndael.c:444
#8  0xb7f27517 in cipher_setkey (c=0xb7fd2800, key=<optimized out>, keylen=32) at cipher.c:900
#9  0xb7f1d644 in gcry_cipher_setkey (hd=0xb7fd2800, key=0xb7fd2064, keylen=32) at visibility.c:521
#10 0x08062057 in make_session_key (dek=0xb7fd2050) at seskey.c:53
#11 0x0808785e in encrypt_filter (opaque=0xbfffe7a8, control=4, a=0x81114e0, buf=0x8118c28 "", 
    ret_len=0xbfffe6dc) at encode.c:761
#12 0x080bf218 in iobuf_flush (a=0x8110ab8) at iobuf.c:1916
#13 iobuf_flush (a=0x8110ab8) at iobuf.c:1888
#14 0x080bf4d8 in iobuf_push_filter2 (a=0x8110ab8, f=0x8057c00 <compress_filter>, ov=0xbfffe7e4, rel_ov=0)
    at iobuf.c:1613
#15 0x08058237 in push_compress_filter (out=0x8110ab8, zfx=0xbfffe7e4, algo=2) at compress.c:320
#16 0x080895ec in sign_file (filenames=0x80fdfe0, detached=0, locusr=0x0, encryptflag=1, remusr=0x80fdeb8, 
    outfile=0x0) at sign.c:945
#17 0x08053872 in main (argc=1, argv=0xbfffec3c) at gpg.c:3513
(gdb) quit
A debugging session is active.                                                                                  
                                                                                                                
        Inferior 1 [process 16027] will be killed.                                                              
                                                                                                                             
Quit anyway? (y or n) y
Comment 2 Toralf Förster gentoo-dev 2012-11-10 09:39:37 UTC
FWIW :

tfoerste@n22 ~ $ zgrep AES /proc/config.gz 
# CONFIG_SND_MAESTRO3 is not set
CONFIG_CRYPTO_AES=m
CONFIG_CRYPTO_AES_586=m
CONFIG_CRYPTO_AES_NI_INTEL=m
Comment 3 Toralf Förster gentoo-dev 2012-11-10 14:24:34 UTC
I simply downgraded to 1.4.6 and gpg works fine again (and kgpg & friends too).

Maybe this bug is related to bug #441918 ?
Comment 4 Toralf Förster gentoo-dev 2012-11-10 21:36:24 UTC
I bisected the root cause : 
https://bugs.g10code.com/gnupg/issue1452

A work around till a fix would be to configure with "--disable-aesni-support" (at least) at my system.
Comment 5 Blu3 2012-11-14 20:37:38 UTC
https://plus.google.com/107251663111083327808/posts/R661S9gwUMo

new laptop, new cpu, new bug found.  libgcrypt employs assembler that doesn't work with i7 cpus for AES-NI cipher.

workaround:

--- libgcrypt-1.5.0-r2.ebuild~  2012-09-23 18:31:08.000000000 -0400
+++ libgcrypt-1.5.0-r2.ebuild   2012-10-15 14:37:38.817144821 -0400
@@-37,6 +37,7 @@
                --disable-dependency-tracking \
                --enable-noexecstack \
                --disable-O-flag-munging \
+               --disable-aesni-support \
                $(use_enable static-libs static)
 }
Comment 6 Joakim Tjernlund 2012-11-15 17:38:58 UTC
Got bitten by this too. Found the problem keyschedule is not
16 bytes aligned on x86(amd64 appears to get it rigth by pure luck).
Came up with this little patch which fixes the problem for me:

--- libgcrypt-1.5.0/cipher/rijndael.c.org	2012-11-15 18:00:25.140266907 +0100
+++ libgcrypt-1.5.0/cipher/rijndael.c	2012-11-15 18:04:35.293685269 +0100
@@ -104,7 +104,7 @@
   union
   {
     PROPERLY_ALIGNED_TYPE dummy;
-    byte keyschedule[MAXROUNDS+1][4][4];
+    byte keyschedule[MAXROUNDS+1][4][4] __attribute__ ((aligned (16)));
 #ifdef USE_PADLOCK
     /* The key as passed to the padlock engine.  It is only used if
        the padlock engine is used (USE_PADLOCK, below).  */
@@ -114,7 +114,7 @@
   union
   {
     PROPERLY_ALIGNED_TYPE dummy;
-    byte keyschedule[MAXROUNDS+1][4][4];
+    byte keyschedule[MAXROUNDS+1][4][4] __attribute__ ((aligned (16)));
   } u2;
   int rounds;               /* Key-length-dependent number of rounds.  */
   int decryption_prepared;  /* The decryption key schedule is available.  */
Comment 7 Toralf Förster gentoo-dev 2012-11-15 18:41:57 UTC
(In reply to comment #6)
> Got bitten by this too. Found the problem keyschedule is not
> 16 bytes aligned on x86(amd64 appears to get it rigth by pure luck).
> Came up with this little patch which fixes the problem for me:

/me too - all tests passed fine and no SEGV while using gpg or kgpg
Comment 8 Joakim Tjernlund 2012-11-15 20:11:24 UTC
hmm, could be enough with 8 byte alignmnet too. I just assumed it should be the
same alignment as everything else in this file.
Can't test this ATM but feel free :)
Comment 9 Joakim Tjernlund 2012-11-15 20:14:16 UTC
BTW, could the maintainer add a call to epatch_user in
the next rev of this ebuild?
Makes it very easy to test new patches by adding them in /etc/portage/patches
Comment 10 Toralf Förster gentoo-dev 2012-11-15 20:46:02 UTC
(In reply to comment #8)
8 byte alignment is sufficient enough here

(In reply to comment #9)
> BTW, could the maintainer add a call to epatch_user in
> the next rev of this ebuild?
> Makes it very easy to test new patches by adding them in /etc/portage/patches
there's bug #442630 opened for that
Comment 11 Joakim Tjernlund 2012-11-19 16:44:33 UTC
Did some more investigation and I think may patch may be overkill.
Tried this instead:
--- libgcrypt-1.5.0/cipher/rijndael.c.org	2012-11-19 17:34:32.569495918 +0100
+++ libgcrypt-1.5.0/cipher/rijndael.c	2012-11-19 17:35:13.481725341 +0100
@@ -1613,7 +1613,7 @@
 static const char*
 selftest_basic_128 (void)
 {
-  RIJNDAEL_context ctx;
+  RIJNDAEL_context ctx ATTR_ALIGNED_16;
   unsigned char scratch[16];
 
   /* The test vectors are from the AES supplied ones; more or less
@@ -1671,7 +1671,7 @@
 static const char*
 selftest_basic_192 (void)
 {
-  RIJNDAEL_context ctx;
+  RIJNDAEL_context ctx ATTR_ALIGNED_16;
   unsigned char scratch[16];
 
   static unsigned char plaintext_192[16] =
@@ -1707,7 +1707,7 @@
 static const char*
 selftest_basic_256 (void)
 {
-  RIJNDAEL_context ctx;
+  RIJNDAEL_context ctx ATTR_ALIGNED_16;
   unsigned char scratch[16];
 
   static unsigned char plaintext_256[16] =

This makes sure the RIJNDAEL_context is 16 bytes aligned for the self tests and
this also fixes the problem for me.

Where are the Crypto Team? Could we have a new ebuild please?
Comment 12 Joakim Tjernlund 2012-11-19 16:52:31 UTC
Forgot to mention, I am using stable gentoo gcc:
gcc (Gentoo 4.5.4 p1.0, pie-0.4.7) 4.5.4
Copyright (C) 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

make.conf
CFLAGS="-O2 -march=native -g"
CHOST="i686-pc-linux-gnu"
Comment 13 Joakim Tjernlund 2012-11-19 17:29:05 UTC
Interesting read about gcc stack alignment:
 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40838

I got the impression one cannot trust gcc to aligne the stack at 16
bytes in many cases.
Comment 14 Joakim Tjernlund 2012-11-19 17:38:39 UTC
Perhaps this is a better patch? The you won't have to annotate
all uses of this structure.

--- libgcrypt-1.5.0/cipher/rijndael.c.org	2012-11-19 18:33:32.277273547 +0100
+++ libgcrypt-1.5.0/cipher/rijndael.c	2012-11-19 18:34:10.141484494 +0100
@@ -124,7 +124,7 @@
 #ifdef USE_AESNI
   int use_aesni;            /* AES-NI shall be used.  */
 #endif /*USE_AESNI*/
-} RIJNDAEL_context;
+} RIJNDAEL_context ATTR_ALIGNED_16;
 
 /* Macros defining alias for the keyschedules.  */
 #define keyschenc  u1.keyschedule
Comment 15 Toralf Förster gentoo-dev 2012-11-19 18:40:06 UTC
(In reply to comment #14)
> Perhaps this is a better patch? The you won't have to annotate
> all uses of this structure.
$> ebuild ... test
passes fine with this test
Comment 16 Joakim Tjernlund 2012-11-19 21:45:46 UTC
(In reply to comment #15)
> (In reply to comment #14)
> > Perhaps this is a better patch? The you won't have to annotate
> > all uses of this structure.
> $> ebuild ... test
> passes fine with this test

Good, then we are two with a working gcrypt :)

Any idea how to wake the Crypto Team?
Mayby there isn't one anymore?
Comment 17 Diego Elio Pettenò (RETIRED) gentoo-dev 2012-11-19 21:48:20 UTC
It's on my todo list :(

I'll try to patch it in soon, promised.

(I need to check if my server has aesni).
Comment 18 Toralf Förster gentoo-dev 2012-11-19 21:54:00 UTC
just FWIW upstream is at least informed :

https://bugs.g10code.com/gnupg/issue1452
Comment 19 Diego Elio Pettenò (RETIRED) gentoo-dev 2012-11-19 21:54:37 UTC
Yes I've seen the back and forth on the mailing list, I was hoping for a new release but I don't count on it just yet.
Comment 20 Joakim Tjernlund 2012-11-20 07:34:25 UTC
(In reply to comment #18)
> just FWIW upstream is at least informed :
> 
> https://bugs.g10code.com/gnupg/issue1452

Maybe you should mention the latest findings?
Not sure if upstream are following this bug.
Comment 21 Tim Harder gentoo-dev 2012-12-04 04:17:52 UTC
Fixed in libgcrypt-1.5.0-r4.