Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 163803 - sys-fs/cryptsetup-luks - all versions segfault
Summary: sys-fs/cryptsetup-luks - all versions segfault
Status: RESOLVED TEST-REQUEST
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Benjamin Smee (strerror) (RETIRED)
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-01-25 19:41 UTC by Doug Goldstein (RETIRED)
Modified: 2010-12-31 03:00 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge --info for both systems (emerge--info.txt,7.25 KB, text/plain)
2007-01-28 10:10 UTC, Martin Lindquist
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Doug Goldstein (RETIRED) gentoo-dev 2007-01-25 19:41:39 UTC
cryptsetup luksFormat /dev/mapper/vg-data is the command I'm running. Once I say, yes. It segfaults.

It's a LVM2 partition.

It segfaults in pbkdf.c in the sigvtalarm() function. foo=26.

Unfortunately gdb does not want to attach to this and provide me any more useful data.

# emerge --info
Portage 2.1.1-r2 (hardened/amd64/multilib, gcc-3.4.6, glibc-2.3.6-r5, 2.6.17-hardened-r1 x86_64)
=================================================================
System uname: 2.6.17-hardened-r1 x86_64 AMD Athlon(tm) 64 X2 Dual Core Processor 4200+
Gentoo Base System release 1.12.6
Last Sync: Wed, 24 Jan 2007 14:00:01 +0000
ccache version 2.3 [enabled]
app-admin/eselect-compiler: [Not Present]
dev-java/java-config: [Not Present]
dev-lang/python:     2.4.3-r4
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     2.3
dev-util/confcache:  [Not Present]
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.61
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10
sys-devel/binutils:  2.16.1-r3
sys-devel/gcc-config: 1.3.14
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.11-r2
ACCEPT_KEYWORDS="amd64"
AUTOCLEAN="yes"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=athlon64"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/revdep-rebuild /etc/terminfo"
CXXFLAGS="-O2 -pipe -march=athlon64"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig buildpkg ccache distlocks metadata-transfer sandbox sfperms strict"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="amd64 apache2 bash-completion berkdb crypt dlloader elibc_glibc hardened justify kernel_linux ldap ncurses nptl nptlonly pam pic python readline samba sse2 ssl tcpd userland_GNU vhosts xorg zlib"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY
Comment 1 Doug Goldstein (RETIRED) gentoo-dev 2007-01-25 19:48:27 UTC
grsec: denied resource overstep by requesting 4096 for RLIMIT_CORE against limit 0 for /bin/cryptsetup[cryptsetup:31574] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[bash:17789] uid/euid:0/0 gid/egid:0/0
cryptsetup[30923] general protection rip:40b7f6 rsp:711b92a97fa8 error:0
grsec: signal 11 sent to /bin/cryptsetup[cryptsetup:30923] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[bash:17789] uid/euid:0/0 gid/egid:0/0

Well that's why it's segfaulting.
Comment 2 Doug Goldstein (RETIRED) gentoo-dev 2007-01-25 19:51:05 UTC
wtf am I saying.. that shows nothing except the attempt to dump a core when it's turned off.
Comment 3 Jakub Moc (RETIRED) gentoo-dev 2007-01-25 21:16:48 UTC
http://www.gentoo.org/proj/en/qa/backtraces.xml
Comment 4 Doug Goldstein (RETIRED) gentoo-dev 2007-01-25 21:31:20 UTC
Jakub... EAD... and read the bug report.

Core was generated by `cryptsetup luksFormat /dev/vg/data'.
Program terminated with signal 11, Segmentation fault.
#0  0x000000000040b82e in sigvtalarm (foo=26) at pbkdf.c:87

No freaking difference then what I posted to the bug.
Comment 5 Doug Goldstein (RETIRED) gentoo-dev 2007-01-25 21:49:27 UTC
The bug exists with all gcc-config profiles available on my machine. However if I compile it on a plain amd64 machine that has gcc 4.1.1 and copy the binary over it works.
Comment 6 Kevin F. Quinn (RETIRED) gentoo-dev 2007-01-25 23:11:04 UTC
If you're having trouble debugging on a hardened systems, possible reasons include:
1) PaX - do '/sbin/paxctl -m /bin/cryptsetup' before running it, otherwise gdb can't modify the executing process (to set breakpoints etc)
2) PIE - use gdb-6.3-r5, it's the only version that supports PIEs (otherwise you can't get meaningful backtraces).
3) Also, try using the pre-stripped executable from the working directory instead of the one installed on the system as that may not be able to find its source (and build with -ggdb2)

(put hardened@ back on CC:, since if the problem occurs on hardened but not on vanilla it's probably something we should keep an eye on :)
Comment 7 solar (RETIRED) gentoo-dev 2007-01-25 23:56:32 UTC
Removing hardened. He confirmed the problem happens any and everytime when not using a gcc-4.x based compiler. Even happens when using vanilla specs.
Comment 8 Benjamin Smee (strerror) (RETIRED) gentoo-dev 2007-01-26 15:02:32 UTC
this is working for me on both hardened and normal, i use it on both and in each case use lvm2. Can anyone else replicate this?
Comment 9 Doug Goldstein (RETIRED) gentoo-dev 2007-01-26 23:40:20 UTC
Every single hardened amd64 box I own does this. Which is 4.
Comment 10 Doug Goldstein (RETIRED) gentoo-dev 2007-01-26 23:41:34 UTC
Kevin:

I'll do what you suggested. I had only done #2 and #3. Didn't think about #1.
Comment 11 Martin Lindquist 2007-01-28 10:09:23 UTC
I'll just add my experience to the cauldron:

I installed cryptsetup-luks-1.0.3-r2 on my hardened amd64 system in august (2006) and encrypted a 105GB SATA disk partition - no problems. In the middle of january (2007) I installed cryptsetup-luks-1.0.3-r2 on my hardened x86 system and encrypted three (1x120GB, 2x80GB) PATA disk partitions - again, no problems. Yesterday I emptied the partition on the SATA disk in the amd64 box and created two new partitions in its place. When attempting to encrypt these partitions using 'cryptsetup luksFormat /dev/sda{9,10}' it segfaults after getting confirmation. The package has not been rebuilt or upgraded since last time it worked on either system. PaX flags on the binary have not been altered on either system.
Comment 12 Martin Lindquist 2007-01-28 10:10:13 UTC
Created attachment 108360 [details]
emerge --info for both systems
Comment 13 Martin Lindquist 2007-02-04 18:41:12 UTC
I went through my files and investigated my previous entanglement with cryptsetup-luks on the hardened amd64 box and realized I used a static x86_64 binary of cryptsetup 1.0.3 to encrypt the partition back in august, so I'm guessing the emerged version segfaulted on me back then too. The static binary was downloaded from the LUKS website and works without problems, so I'm guessing this is an issue with a Gentoo patch somewhere.

Sorry for the misinformation in my previous post.
Comment 14 PaX Team 2007-04-24 11:25:03 UTC
can you guys post a *non*-working binary? i looked at the code and it's fine (well, there seems to be an assumption in that __PBKDF2_global_j will be initialized, that is, PBKDF2_HMAC_SHA1 will be called before the first SIGVTALRM is delivered, but i guess that always happens within 1sec), so it's probably some compiler/toolchain issue that produced something bad. also when you post a backtrace, post 'i r', 'x/8i $pc' and 'x/8x $sp' as well please.
Comment 15 Markus 2007-07-04 16:25:00 UTC
Upstream in version 1.0.5 they fixed some segfaults, tried it?
(I made an ebuild in bug 183407)
Comment 16 Benjamin Smee (strerror) (RETIRED) gentoo-dev 2007-09-09 21:13:48 UTC
please retry with the latest version of cryptsetup and let me know if pain persists.
Comment 17 Georg Weiss 2007-11-02 19:25:10 UTC
(In reply to comment #16)
> please retry with the latest version of cryptsetup and let me know if pain
> persists.
> 

i tried "sys-fs/cryptsetup-1.0.5-r1" and still got sig 11

--8<--
cryptsetup[17771] general protection rip:40bb96 rsp:704ea643e838 error:0
grsec: From 192.168.1.1: signal 11 sent to /sbin/cryptsetup[cryptsetup:17771] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[bash:8767] uid/euid:0/0 gid/egid:0/0
--8<--

emerge --info
--8<--
Portage 2.1.3.16 (selinux/2007.0/x86/hardened, gcc-3.4.6, glibc-2.6.1-r0, 2.6.23-hardened-r1europa-gw01 x86_64)
=================================================================
System uname: 2.6.23-hardened-r1europa-gw01 x86_64 AMD Athlon(tm) 64 Processor 3000+
Timestamp of tree: Fri, 02 Nov 2007 00:20:01 +0000
distcc 2.18.3 x86_64-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
app-shells/bash:     3.2_p17
dev-lang/python:     2.4.4-r6
dev-python/pycrypto: 2.0.1-r6
sys-apps/baselayout: 1.12.9-r2
sys-apps/sandbox:    1.2.18.1-r2
sys-devel/autoconf:  2.13, 2.61-r1
sys-devel/automake:  1.7.9-r1, 1.10
sys-devel/binutils:  2.18-r1
sys-devel/gcc-config: 1.3.16
sys-devel/libtool:   1.5.24
virtual/os-headers:  2.6.22-r2
ACCEPT_KEYWORDS="x86"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -march=athlon64 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/udev/rules.d"
CXXFLAGS="-O2 -march=athlon64 -pipe"
DISTDIR="/local/portage/distfiles"
FEATURES="distlocks loadpolicy metadata-transfer sandbox selinux sesandbox sfperms strict unmerge-orphans userfetch userpriv"
GENTOO_MIRRORS="ftp://gentoo-mirror.jupiter.biduda.org/ http://mirror.manitu.net/gentoo http://pandemonium.tiscali.de/pub/gentoo/ http://mirrors.sec.informatik.tu-darmstadt.de/gentoo/ http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/ ftp://sunsite.informatik.rwth-aachen.de/pub/Linux/gentoo ftp://ftp.tu-clausthal.de/pub/linux/gentoo/ ftp://linux.rz.ruhr-uni-bochum.de/gentoo-mirror/"
LANG="de_DE.UTF-8"
LC_ALL="de_DE.UTF-8"
MAKEOPTS="-j3"
PKGDIR="/local/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --filter=H_**/files/digest-*"
PORTAGE_TMPDIR="/local/portage/build"
PORTDIR="/local/portage/tree"
PORTDIR_OVERLAY="/local/portage/layman/sunrise /local/portage/overlay"
SYNC="rsync://localhost/gentoo-portage"
USE="acpi bash-completion bzip2 caps crypt hardened imap ipv6 ldap logrotate maildir minimal mmx nls nptl nptlonly pam pic pie readline samba selinux sse sse2 ssl unicode usb utf8 x86 zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1         emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m       maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" MISDN_CARDS="hfcpci" USERLAND="GNU" VIDEO_CARDS="apm ark chips cirrus cyrix dummy fbdev glint i128 i740 i810 imstt   mach64 mga neomagic nsc nv r128 radeon rendition s3 s3virge savage   siliconmotion sis sisusb tdfx tga trident tseng v4l vesa vga via vmware     voodoo"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LDFLAGS, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
--8<--
Comment 18 Georg Weiss 2007-11-02 19:58:58 UTC
i had problems with uploading a non-working binary. so grab it here:
<http://www.georgweiss.de/gentoo/cryptsetup.bz2>
this is from cryptsetup-1.0.5-r1 ebuild (amd64 box from comment #17)

i just tested cryptsetup-luks-1.0.4-r3 on a x86 hardened system. luksFormat worked there.
Comment 19 Jens Pranaitis 2007-11-23 15:53:29 UTC
I'm experiencing the same error on hardened amd64. However for me the binary works when compiling with USE="dynamic". 

Portage 2.1.3.19 (hardened/amd64/multilib, gcc-3.4.6, glibc-2.6.1-r0, 2.6.22-hardened-r8 x86_64)
=================================================================
System uname: 2.6.22-hardened-r8 x86_64 AMD Athlon(tm) 64 X2 Dual Core Processor 5600+
Timestamp of tree: Wed, 21 Nov 2007 16:46:01 +0000
app-shells/bash:     3.2_p17
dev-java/java-config: 1.3.7, 2.0.33-r1
dev-lang/python:     2.4.4-r6
dev-python/pycrypto: 2.0.1-r6
sys-apps/baselayout: 1.12.9-r2
sys-apps/sandbox:    1.2.18.1-r2
sys-devel/autoconf:  2.61-r1
sys-devel/automake:  1.5, 1.7.9-r1, 1.9.6-r2, 1.10
sys-devel/binutils:  2.18-r1
sys-devel/gcc-config: 1.3.16
sys-devel/libtool:   1.5.24
virtual/os-headers:  2.6.22-r2
ACCEPT_KEYWORDS="amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=athlon64 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /opt/openfire/resources/security/"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/terminfo /etc/udev/rules.d"
CXXFLAGS="-march=athlon64 -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks metadata-transfer noinfo parallel-fetch sandbox sfperms strict unmerge-orphans userfetch"
GENTOO_MIRRORS="http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/ http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/"
LC_ALL="en_US.UTF-8"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --filter=H_**/files/digest-*"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/portage/local/local"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="7zip aim amd64 authdaemond bash-completion berkdb bitmap-fonts bzip2 caps cgi chroot cli cracklib crypt cscope ctype curl curlwrappers dynamicplugin encode expat fam fastcgi flatfile ftp gd gdbm geoip gif gnutls gpgme hardened iconv idn imagemagick imap iproute2 ipv6 ithreads jpeg jpeg2k kqemu libg++ libwww lighttpd logrotate maildir mailwrapper mime mng mudflap mysql ncurses nls nptl nptlonly ntlm offensive ogg pam pcre pdf perl php pic png pop python readline reflection rrdtool ruby sasl session slang smime smtp snmp socks5 spell spl sse sse2 ssl svg sysfs tcpd theora threads tidy tiff tordns truetype truetype-fonts type1-fonts unicode userlocales vhosts vim x264 xml xvid zip zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x  ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3       trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" ELIBC="glibc" INPUT_DEVICES="mouse keyboard" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="apm ark chips cirrus cyrix dummy fbdev glint i128 i810 mach64       mga neomagic nv r128 radeon rendition s3 s3virge savage siliconmotion sis       sisusb tdfx tga trident tseng v4l vesa vga via vmware voodoo"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LDFLAGS, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 20 PaX Team 2007-11-24 16:33:52 UTC
(In reply to comment #18)
> i had problems with uploading a non-working binary. so grab it here:
> <http://www.georgweiss.de/gentoo/cryptsetup.bz2>
> this is from cryptsetup-1.0.5-r1 ebuild (amd64 box from comment #17)
> 
> i just tested cryptsetup-luks-1.0.4-r3 on a x86 hardened system. luksFormat
> worked there.

what is the exact cmdline that triggers the crash?
Comment 21 Vítor Brandão (noisebleed) 2009-09-08 13:21:35 UTC
USE="dynamic" works for me too.