Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 144687 - Segfault in cryptsetup-luks
Summary: Segfault in cryptsetup-luks
Status: RESOLVED INVALID
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: AMD64 Linux
: High normal
Assignee: Benjamin Smee (strerror) (RETIRED)
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-08-21 13:03 UTC by Stefan Buller
Modified: 2010-11-02 19:21 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Buller 2006-08-21 13:03:26 UTC
Cryptsetup version: sys-fs/cryptsetup-luks-1.0.3-r2
Command: cryptsetup -y -s 256 luksFormat /dev/loop1 encrypted
And, what was output:
---------------------
WARNING!
========
This will overwrite data on /dev/loop/2 irrevocably.

Are you sure? (Type uppercase yes): YES
Segmentation fault
---------------------

Rather than figuring out the problem though, I just went over to http://luks.endorphin.org/ and downloaded a precompiled binary for x86_64 (http://luks.endorphin.org/precompiled/cryptsetup-1.0.3-x86_64-pc-linux-gnu-static.bz2), which runs just fine.

Sorry, if this bugreport sucks - it's my first one.

# emerge --info
Portage 2.1-r2 (default-linux/amd64/2006.0, gcc-3.4.6, glibc-2.3.6-r4, 2.6.17-gentoo-r4 x86_64)
=================================================================
System uname: 2.6.17-gentoo-r4 x86_64 Intel(R) Pentium(R) D CPU 3.40GHz
Gentoo Base System version 1.12.4
app-admin/eselect-compiler: [Not Present]
dev-lang/python:     2.4.3-r1
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     [Not Present]
dev-util/confcache:  [Not Present]
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.59-r7
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2
sys-devel/binutils:  2.16.1-r3
sys-devel/gcc-config: 1.3.13-r3
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.11-r2
ACCEPT_KEYWORDS="amd64"
AUTOCLEAN="yes"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -fomit-frame-pointer -march=nocona"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/revdep-rebuild /etc/terminfo"
CXXFLAGS="-O2 -pipe -fomit-frame-pointer -march=nocona"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks metadata-transfer sandbox sfperms strict"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="amd64 async automount avi bash-completion berkdb bitmap-fonts cgi chroot cli crypt curl curlwrappers dlloader doc dri eds emboss encode fam fastcgi fortran fuse gif gnutls gpm hardenedphp imlib iproute2 ipv6 isdnlog john jpeg lighttpd logrotate lzw lzw-tiff mp3 mpeg multiuser mysql ncurses nls nptl pam pcre pdf pdflib perl php png pppd python quicktime readline reflection ruby samba screen sdl session spell spl ssl swat syslog tcpd threads tidy tiff truetype-fonts type1-fonts udev usb vim xattr xpm xv zlib elibc_glibc input_devices_keyboard input_devices_mouse input_devices_evdev kernel_linux userland_GNU video_cards_apm video_cards_ark video_cards_ati video_cards_chips video_cards_cirrus video_cards_cyrix video_cards_dummy video_cards_fbdev video_cards_glint video_cards_i128 video_cards_i810 video_cards_mga video_cards_neomagic video_cards_nv video_cards_rendition video_cards_s3 video_cards_s3virge video_cards_savage video_cards_siliconmotion video_cards_sis video_cards_sisusb video_cards_tdfx video_cards_tga video_cards_trident video_cards_tseng video_cards_v4l video_cards_vesa video_cards_vga video_cards_via video_cards_vmware video_cards_voodoo"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY
Comment 1 solar (RETIRED) gentoo-dev 2006-08-21 21:28:33 UTC
Thanks for reporting this bug. 

Normally an app that segfaults is not a security problem. If it could cause a kernel panic or was privilege escalation then that would be another story.

Usually we try to check who a package belongs to vs assigning it to whoever. 
(security in this case) 

You can find out who maintains a given package by checking the meatadata.xml directly or using tools. I use the following tool http://dev.gentoo.org/~solar/portage_misc/epkginfo 

which tells us.

user $ epkginfo sys-fs/cryptsetup-luks

Package: sys-fs/cryptsetup-luks
Herd: base-system
Maintainer: strerror@gentoo.org
Location: /var/cvsroot/gentoo-x86/sys-fs/cryptsetup-luks
Keywords: cryptsetup-luks-1.0.1-r1:  amd64 sparc hppa 
Keywords: cryptsetup-luks-1.0.1-r2:  ~x86 ~ppc ~sh ~ppc64 ~arm 
Keywords: cryptsetup-luks-1.0.3-r2:  ppc ~amd64 s390 ~mips ~sparc ppc64 arm sh ia64 ~alpha x86 
Comment 2 Stefan Buller 2006-08-21 21:42:18 UTC
> Normally an app that segfaults is not a security problem. If it could cause a
> kernel panic or was privilege escalation then that would be another story.
> 
> Usually we try to check who a package belongs to vs assigning it to whoever. 
> (security in this case) 

Thanks for clarifying that, I was unsure what the security category was for.
Comment 3 Benjamin Smee (strerror) (RETIRED) gentoo-dev 2006-10-12 04:19:17 UTC
I can't replicate this problem at all. What is your losetup command?
Comment 4 adr 2006-10-18 05:14:22 UTC
Here the same with sys-fs/cryptsetup-luks-1.0.3-r2 (not on AMD64)...

# cryptsetup -y -c blowfish-cbc-essiv:sha256 -s 448 luksFormat /dev/hdc1

WARNING!
========
This will overwrite data on /dev/hdc1 irrevocably.

Are you sure? (Type uppercase yes): YES
Enter LUKS passphrase:
Verify passphrase:
Command successful.
# cryptsetup luksOpen /dev/hdc1 home
Enter LUKS passphrase:
key slot 0 unlocked.
Segmentation fault
# 

It did work, untill I did emerge -e for a gcc upgrade last weekend...

Maybe a clue: in the boot output on screen, cryptfs shows this error for a previously made luks partition:
...
device-mapper: /dev/hda6 too small for target
...

# emerge --info
Portage 2.1.1 (default-linux/x86/2006.1/desktop, gcc-4.1.1, glibc-2.4-r3, 2.6.18-gentoo-r1 i686)
=================================================================
System uname: 2.6.18-gentoo-r1 i686 AMD-K7(tm) Processor
Gentoo Base System version 1.12.5
Last Sync: Mon, 16 Oct 2006 08:30:02 +0000
app-admin/eselect-compiler: [Not Present]
dev-java/java-config: 1.3.7, 2.0.30
dev-lang/python:     2.4.3-r4
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     [Not Present]
dev-util/confcache:  [Not Present]
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.59-r7
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2
sys-devel/binutils:  2.16.1-r3
sys-devel/gcc-config: 1.3.13-r4
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.17-r1
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=i686 -mtune=athlon -fomit-frame-pointer -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/lib/fax /usr/share/X11/xkb /usr/share/config /usr/share/keymaps /var/spool/fax/etc"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/gconf /etc/java-config/vms/ /etc/revdep-rebuild /etc/terminfo"
CXXFLAGS="-O2 -march=i686 -mtune=athlon -fomit-frame-pointer -pipe"
DISTDIR="/mnt/data/portage/distfiles"
FEATURES="autoconfig distlocks fixpackages metadata-transfer sandbox sfperms strict"
GENTOO_MIRRORS="http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/ http://ftp.snt.utwente.nl/pub/os/linux/gentoo http://ftp.belnet.be/mirror/rsync.gentoo.org/gentoo/ ftp://mirror.scarlet-internet.nl/pub/gentoo"
LINGUAS="nl en"
MAKEOPTS="-j2"
PKGDIR="/mnt/data/portage/packages"
PORTAGE_RSYNC_EXTRA_OPTS="--exclude-from=/etc/portage/rsync_excludes"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_TMPDIR="/mnt/data"
PORTDIR="/usr/poortage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="x86 3dnow 3dnowext X alsa arts berkdb bitmap-fonts cairo cdr cli crypt cups dbus dlloader dri dvd eds elibc_glibc emboss encode fam flac fortran gdbm gif gpm gstreamer hal input_devices_keyboard input_devices_mouse ipv6 isdnlog jpeg kde kdeenablefinal kernel_linux ldap libg++ linguas_en linguas_nl mad mmap mmx mmxext mp3 mpeg ncurses nls nptl nptlonly ogg opengl pam pcre perl pic png ppds pppd python qt3 qt4 quicktime readline reflection rtc sdl session spell spl ssl tcpd truetype truetype-fonts type1-fonts udev unicode userland_GNU video_cards_r128 vorbis win32codecs xine xml xorg xv zlib"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, PORTDIR_OVERLAY
Comment 5 adr 2006-10-18 06:37:17 UTC
(In addition to comment #4)
> Maybe a clue: in the boot output on screen, cryptfs shows this error for a
> previously made luks partition:
> ...
> device-mapper: /dev/hda6 too small for target
> ...

Sorry, it is the Gentoo Livecd 2006.1 (cryptsetup-1.0.3) that talks about device-mapper...

livecd root # cryptsetup luksOpen /dev/hda6 test
Enter LUKS passphrase:
key slot 0 unlocked.
device-mapper: device /dev/hda6 too small for target
device-mapper: crypt: Device lookup failed
device-mapper: error adding target to table
device-mapper: device doesn't appear to be in the dev hash table.
Segmentation fault

Normal system boot with cryptfs:
 * Setting up dm-crypt mappings ...
 * dm-crypt map swap1 ...
  [ ok ]
 *   Running pre_mount commands for swap1 ...
  [ ok ]
 * dm-crypt map test ...
 * /bin/cryptsetup -c blowfish-cbc-essiv:sha256 -s 448 luksOpen /dev/hda6 test
key slot 0 unlocked.
/lib/rcscripts/addons/dm-crypt-start.sh: line 9:  1786 Segmentation fault      /bin/cryptsetup ${options} luksOpen ${source} ${target} >/dev/console </dev/console
 * failure running cryptsetup-luks
  [ !! ]
 * Failed to setup dm-crypt devices
  [ !! ]
Comment 6 adr 2006-10-18 06:59:49 UTC
(In reply to comment #4)
> 
> # cryptsetup -y -c blowfish-cbc-essiv:sha256 -s 448 luksFormat /dev/hdc1
> 

???!
Strangely I can still open an old floppy...

# cryptsetup luksOpen /dev/fd0 floppy
Enter LUKS passphrase:
key slot 0 unlocked.
Command successful.
Comment 7 adr 2006-10-20 00:25:29 UTC
Syslogger:
...
Oct 20 09:09:40 [kernel] device-mapper: table: device /dev/hda6 too small for target
Oct 20 09:09:40 [kernel] device-mapper: table: 254:1: crypt: Device lookup failed
Oct 20 09:09:40 [kernel] device-mapper: ioctl: error adding target to table
Oct 20 09:09:40 [kernel] device-mapper: ioctl: device doesn't appear to be in the dev hash table.
...
Comment 8 adr 2006-10-27 03:20:16 UTC
cryptsetup-1.0-3-i686-pc-linux-gnu-static.bz2 - Precompiled binary, version 1.0.3 for x86 from http://luks.endorphin.org/dm-crypt

...works for me, even with the Gentoo cryptfs init script (as long as I don't enter a wrong passphrase). Temp solution to get things working again...
Comment 9 Benjamin Smee (strerror) (RETIRED) gentoo-dev 2006-10-27 03:30:19 UTC
(In reply to comment #4)
> Here the same with sys-fs/cryptsetup-luks-1.0.3-r2 (not on AMD64)...
> 
> # cryptsetup -y -c blowfish-cbc-essiv:sha256 -s 448 luksFormat /dev/hdc1

I can replicate the problem with this command, but only when the keylength is 448, otherwise it works fine. I'll look into it in more detail but I'm not convinced that this is a problem introduced by any patches we do but perhaps by the different libs that we link to compared to the ones used by Clemens.
Comment 10 Benjamin Smee (strerror) (RETIRED) gentoo-dev 2006-10-27 03:33:54 UTC
to confirm if you use a keysize of 256 do you have any problems?
Comment 11 adr 2006-10-27 06:30:25 UTC
(In reply to comment #10)
> to confirm if you use a keysize of 256 do you have any problems?
> 

hmm, you're right. no segfault creating/opening with key size 256...

# '/bin/cryptsetup.bak' -y -c blowfish-cbc-essiv:sha256 -s 256 luksFormat /dev/hdc1
...
# '/bin/cryptsetup.bak' luksOpen /dev/hdc1 test
...
Comment 12 Benjamin Smee (strerror) (RETIRED) gentoo-dev 2007-01-08 15:25:03 UTC
closing this as i think there is a requirement on the nature of the keysize (ie supported sizes only).
Comment 13 adr 2007-01-08 17:20:17 UTC
OK. That it worked it the past, doesn't mean that it was supported... But it still works in the newest version of Clemens. The kernel supports Blowfish up to 448 bits in length...
Comment 14 adr 2007-01-08 18:59:02 UTC
(In reply to comment #13)
> OK. That it worked it the past, doesn't mean that it was supported... But it
> still works in the newest version of Clemens. The kernel supports Blowfish up
> to 448 bits in length...
> 

...the binary 1.0.3, not the source 1.0.4 of clemens. So it might got something to do with differnt libraries on Gentoo (as you said earlier).