Bug 138579

Summary:	tpm-emulator-0.3 has linking trouble with hardened libgmp
Product:	Gentoo Linux	Reporter:	Daniel Black (RETIRED) <dragonheart>
Component:	[OLD] Unspecified	Assignee:	Daniel Black (RETIRED) <dragonheart>
Status:	VERIFIED WONTFIX
Severity:	normal	CC:	pageexec, solar
Priority:	High
Version:	unspecified
Hardware:	All
OS:	Linux
URL:	https://developer.berlios.de/bugs/?func=detailbug&bug_id=8055&group_id=2491
Whiteboard:
Package list:		Runtime testing required:	---

Description Daniel Black (RETIRED) gentoo-dev

2006-06-30 06:30:26 UTC

I think ./crypto/libgmp.a has hardened ssp stuff in it that gets linked to the kernel modules. Part of the pecularity with userspace stuff in a kernel module I guess.

$ ebuild tpm-emulator-0.3.ebuild install
>>> Creating Manifest for /home/dan/gentoo/gentoo-x86/app-crypt/tpm-emulator
>>> checking ebuild checksums ;-)
>>> checking auxfile checksums ;-)
>>> checking miscfile checksums ;-)
>>> checking tpm_emulator-0.3.tar.gz ;-)
 * Determining the location of the kernel source code
 * Found kernel source directory:
 *     /usr/src/linux
 * Found sources for kernel version:
 *     2.6.16-gentoo-r9
>>> Checking tpm_emulator-0.3.tar.gz's mtime...
>>> /home/dan/gentoo/gentoo-x86/app-crypt/tpm-emulator/tpm-emulator-0.3.ebuild has been updated; recreating WORKDIR...
>>> Unpacking source...
>>> Unpacking tpm_emulator-0.3.tar.gz to /var/tmp/portage/tpm-emulator-0.3/work
>>> Source unpacked.
>>> Compiling source in /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3 ...
 * Preparing tpm_emulator module
test -r ./crypto/gmp.h || cat /usr/include/gmp.h | \
        sed -e "s/\(__GMP_DECLSPEC [^e].*\);/\1 __attribute__ ((regparm(0)));/" | \
        sed -e "s/^int$/int __attribute__ ((regparm(0)))/" | \
        sed -e "s/^void$/void __attribute__ ((regparm(0)))/" | \
        sed -e "s/^size_t$/size_t __attribute__ ((regparm(0)))/" | \
        sed -e "s/^mp_limb_t$/mp_limb_t __attribute__ ((regparm(0)))/" | \
        sed -e "s/^__GMP_EXTERN_INLINE void$/__GMP_EXTERN_INLINE void __attribute__ ((regparm(0)))/" | \
        sed -e "s/^unsigned long$/unsigned long __attribute__ ((regparm(0)))/" | \
        sed -e "s/\(.* (\*__gmp_allocate_func) .*\);/\1 __attribute__ ((regparm(0)));/" | \
        sed -e "s/\(.* (\*__gmp_reallocate_func) .*\);/\1 __attribute__ ((regparm(0)));/" | \
        sed -e "s/\(.* (\*__gmp_free_func) .*\);/\1 __attribute__ ((regparm(0)));/" \
        > ./crypto/gmp.h
test -f ./crypto/libgmp.a || ln -s /usr/lib/libgmp.a ./crypto/libgmp.a
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/./linux_module.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/crypto/gmp_kernel_wrapper.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/crypto/hmac.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/crypto/rc4.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/crypto/rsa.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/crypto/sha1.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_audit.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_authorization.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_capability.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_cmd_handler.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_context.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_counter.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_credentials.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_crypto.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_daa.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_data.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_delegation.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_deprecated.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_error.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_eviction.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_gpio.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_handles.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_identity.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_integrity.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_maintenance.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_management.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_marshalling.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_migration.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_nv_storage.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_owner.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_startup.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_storage.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_testing.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_ticks.o
  CC [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm/tpm_transport.o
  LD [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm_emulator.o
  Building modules, stage 2.
  MODPOST
*** Warning: "__guard" [/var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm_emulator.ko] undefined!
*** Warning: "__stack_smash_handler" [/var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm_emulator.ko] undefined!
  CC      /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm_emulator.mod.o
  LD [M]  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm_emulator.ko
>>> Source compiled.
bzip2: Output file environment.bz2 already exists.
>>> Test phase [none]: app-crypt/tpm-emulator-0.3

>>> Install tpm-emulator-0.3 into /var/tmp/portage/tpm-emulator-0.3/image/ category app-crypt
 * Installing tpm_emulator module

!!! ERROR: app-crypt/tpm-emulator-0.3 failed.
Call stack:
  ebuild.sh, line 1545:   Called dyn_install
  ebuild.sh, line 1019:   Called src_install
  tpm-emulator-0.3.ebuild, line 31:   Called die

!!! cannot have gmp compiled with hardened flags
!!! If you need support, post the topmost build error, and the call stack if relevant.

$ scanelf -s  __guard  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm_emulator.ko
 TYPE   SYM FILE
ET_REL __guard /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm_emulator.ko
$ scanelf -s  __stack_smash_handler  /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm_emulator.ko
 TYPE   SYM FILE
ET_REL __stack_smash_handler /var/tmp/portage/tpm-emulator-0.3/work/tpm_emulator-0.3/tpm_emulator.ko

Comment 1 Daniel Black (RETIRED) gentoo-dev

2006-06-30 06:32:12 UTC

$ emerge --info
Portage 2.1.1_pre1-r5 (!/home/dan/gentoo/gentoo-x86/profiles/hardened/x86/2.6, gcc-4.1.0/hardened, glibc-2.3.6-r4, 2.6.16-gentoo-r9 i686)
=================================================================
System uname: 2.6.16-gentoo-r9 i686 AMD Athlon(tm)
Gentoo Base System version 1.12.1
distcc 2.18.3 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
ccache version 2.3 [disabled]
dev-lang/python:     2.3.5-r2, 2.4.3-r1
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     2.3
dev-util/confcache:  [Not Present]
sys-apps/sandbox:    1.2.18.1
sys-devel/autoconf:  2.13, 2.59-r7
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2
sys-devel/binutils:  2.17
sys-devel/gcc-config: 2.0.0_rc1
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.11-r5
ACCEPT_KEYWORDS="x86 ~x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=athlon-xp -O2 -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/lib/X11/xkb /usr/lib/mozilla/defaults/pref /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/"
CONFIG_PROTECT_MASK="/etc/env.d /etc/eselect/compiler /etc/gconf /etc/revdep-rebuild /etc/terminfo"
CXXFLAGS="-march=athlon-xp -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig collision-protect cvs digest keepwork metadata-transfer nostrip sandbox sfperms sign strict stricter test userpriv usersandbox verify-rdepend"
GENTOO_MIRRORS=""
LDFLAGS=" -Wl,--as-needed"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/home/dan/gentoo/gentoo-x86"
SYNC="rsync://rsync.au.gentoo.org/gentoo-portage"
USE="3dnow 3dnowex X aac acpi alsa avi bash-completion berkdb bitmap-fonts bzip2 caps crypt cups curl divx4linux dlloader ecc encode ethereal extensions fastcgi fla flac fortran gd gmp gnutls gphoto2 gstreamer gtk gtk2 hardened hpn i8x0 ilbc imagemagick imlib ipv6 javacomm jpeg kde kdeenablefinal kerberos libgda lzo mbox mmx mp3 mpeg multitarget mysql nptl nptlonly ntlm ogg oggvorbis openal opengl operanom2 pam perl php pic png postgres ppds python qt quicktime readline samba sdl sguil slp smime socks5 sox spell sse ssl tiff true-type truetype type1-fonts usb userlocales vhosts vorbis x86 xine xinerama xml xml2 xorg zlib elibc_glibc input_devices_mouse input_devices_keyboard kernel_linux userland_GNU"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, PORTAGE_RSYNC_EXTRA_OPTS

Comment 2 PaX Team 2006-06-30 08:20:30 UTC

hmm, i thought hardened had some provision for compiling kernel modules so that ssp would be disabled... solar?

Comment 3 solar (RETIRED) gentoo-dev

2006-06-30 09:36:47 UTC

There is no such thing as a hardened-gcc-4.x compiler so whatever dragonheart is 
encountering is a result of old libs leaking into his non-hardened gcc-4.x setup.
Also yes hardened prevents ssp from being enabled on kernel code using a 
{!D__KERNEL__: macro

The root problem here has to be that the module is stupid and is linking 
with userland code when it should not.

Comment 4 PaX Team 2006-06-30 11:23:14 UTC

(In reply to comment #3)
> The root problem here has to be that the module is stupid and is linking 
> with userland code when it should not.

yeah, on a second look i agree, this is an upstream issue, using userland code in the kernel is a big no-no (think of potential issues like PIC, TLS, and here, ssp). might be safer to simply mask this package altogether until the code is properly fixed.

Comment 5 Daniel Black (RETIRED) gentoo-dev

2006-07-02 19:40:40 UTC

reported upstream as per URL. Thanks people. I didn't think of the PIC or TLS problems (because I don't understand them enough).

I've done a check for SSP in the ebuild. I probably won't mask this as it may work for most people. I'll look at fixing the problem when I do 0.4 (just released) by recompiling the libgmp build with -D__KERNEL__ (if upstream haven't fixed it).

Comment 6 Daniel Black (RETIRED) gentoo-dev

2006-07-02 19:41:28 UTC

opps ment to close as UPSTREAM.