I'm having trouble creating a 32 bit chroot using hardened kernels. I've tried both a CK6 + PaX kernel I brewed myself and a hardened-dev-sources with no modifications. icebox livecd # linux32 chroot base/ /bin/bash: error while loading shared libraries: /lib/libc.so.6: cannot apply additional memory protection after relocation: Permission denied icebox livecd # uname -a Linux icebox 2.6.7-hardened-r8 #1 Sat Oct 2 21:19:11 EDT 2004 x86_64 AMD Athlon(tm) 64 Processor 2800+ AuthenticAMD GNU/Linux Above you can see the error I'm getting, as well as the kernel I'm running. I've echoed 1 into /proc/sys/fs/pax/softmode and 0 into aslr, so PaX is likely not the culprit. The system is a stage 1 tarball for x86 from 2003.1, and it doesn't break until I've merged the new glibc, *If* glibc merges. My make.conf is attached, and I'm using /usr/portage/profiles/hardened/x86/2.6 for my /etc/make.profile with the bootstrap-cascade.sh script. You should be able to reproduce my chroot environment from that information.
Created attachment 40963 [details] make.conf my make.conf
The make.conf there uses -mtune. I started with -mcpu, and then after merging gcc (~x86 gives me gcc 3.4.2) I changed it to -mtune. You'll need to fix CFLAGS to use -mcpu if you start off with gcc <3.4
"I've merged the new glibc" <-- which glibc? emerge info please.
I would have given emerge --info HAD EMERGE WORKED. Broken glibc, broken everything. I'm rebuilding the chroot-- again-- using stage1-x86-2004.2.tar.bz2 this time. As per my process (I'm not using a hardened stage), I'm using the profile I noted in the bug report, and first merging in a new portage, bison, and gcc before rebuilding: emerge --oneshot --nodeps portage bison gcc When that's done I'll give you the emerge --info, as well as the glibc version that wants to install. This is also the first chroot built *completely* under a system using an official kernel, though the bug report was made with a chroot done with proper 'linux32 chroot' after switching to an official kernel. I don't see how this matters, but . . . removing all possible failure points.
Portage 2.0.51_rc7 (hardened/x86/2.6, gcc-3.4.2, glibc-2.3.3.20040420-r0, 2.6.7-hardened-r8 i686) ================================================================= System uname: 2.6.7-hardened-r8 i686 AMD Athlon(tm) 64 Processor 2800+ Gentoo Base System version 1.4.16 Autoconf: Automake: Binutils: sys-devel/binutils-2.14.90.0.8-r1 Headers: sys-kernel/linux-headers-2.4.21-r1 Libtools: ACCEPT_KEYWORDS="x86 ~x86" AUTOCLEAN="yes" CFLAGS="-mcpu=i686 -O2 -pipe -fno-stack-protector-all -fstack-protector -mfpmath=387" CHOST="i686-pc-linux-gnu" COMPILER="" CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3.2/share/config /usr/kde/3.3/share/config:/usr/kde/3.3/env:/usr/kde/3.3/shutdown /usr/kde/3/share/config /usr/lib/mozilla/defaults/pref /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-mcpu=i686 -O2 -pipe -fno-stack-protector-all -fstack-protector -mfpmath=387" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs buildpkg distlocks sandbox userpriv usersandbox" GENTOO_MIRRORS="http://gentoo.osuosl.org http://distro.ibiblio.org/pub/Linux/distributions/gentoo" MAKEOPTS="-j2" PKGDIR="/usr/portage//packages/x86/" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage/" PORTDIR_OVERLAY="" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="3ds X aalib acpi alsa apm arts avi berkdb bitmap-fonts bootsplash cdr composite crypt cups dmx dri dvd esd f77 flac gcj gif gimpprint gnome gstreamer gtk gtk2 gtkhtml hardened ipv6 ithreads java jbig jpeg justify lcms ldap lzw-tiff mad mikmod mmap mng mozilla mpeg multilib nls nptl objc offensive oggvorbis openal opengl oss pam perl pic pie png ppds quicktime readline samba sdl speex spell ssl svga tcltk tcpd theora threads tiff truetype unstable-meta usb videos wmf x86 xchatdccserver xml xml2 xmms xprint xv zlib" icebox portage # emerge -pv --nodeps linux26-headers binutils glibc These are the packages that I would merge, in order: Calculating dependencies ...done! [ebuild N ] sys-kernel/linux26-headers-2.6.8.1 -build 34,793 kB [ebuild U ] sys-devel/binutils-2.15.90.0.1.1-r3 [2.14.90.0.8-r1] -bootstrap -build* -debug -multitarget +nls* 10,874 kB [ebuild U ] sys-libs/glibc-2.3.4.20040808 [2.3.3.20040420] -build* -debug -erandom +hardened* -makecheck +multilib* +nls* +nptl* +pic* -userlocales 15,372 kB Total size of downloads: 61,039 kB After merging linux26-headers and binutils, I'll attempt bootstrap, which will naturally merge in glibc.
* Applying linux26-headers-2.6.8.1-strict-ansi-fix.patch... [ ok ] * Applying linux26-headers-2.6.8.1-appCompat.patch... [ ok ]>>> Source unpacked. HOSTCC scripts/basic/fixdep scripts/basic/fixdep: error while loading shared libraries: cannot apply additional memory protection after relocation: Permission denied make[1]: *** [scripts/basic/fixdep] Error 127 make: *** [scripts_basic] Error 2 !!! ERROR: sys-kernel/linux26-headers-2.6.8.1 failed. !!! Function src_compile, Line 65, Exitcode 2 That's as far as I got. Funny thing is, that's just a remerge of the headers, as this snip from my emerge --info (this is the part that's changed since comment #5) shows: Binutils: sys-devel/binutils-2.15.90.0.1.1-r3 Headers: sys-kernel/linux26-headers-2.6.8.1 Well, I'm done. Have fun.
Created attachment 41118 [details] configuration for hardened-dev-sources-2.6.7-r8 My configuration
I don't think I'll be much help to solve this bug. But could you atleast post the output of. readelf -l `which linux32` readelf -l `which chroot` readelf -l /lib64/libc.so.6 Then the output of the /lib/libc.so.6 within the 32bit chroot.
ayanami gcc # linux32 chroot /mnt/tmp/mnt/gentoo-x86-2004.2-chroot/ ayanami / # file /bin/bash /bin/bash: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.0.0, dynamically linked (uses shared libs), stripped ayanami gcc # echo "0" > /proc/sys/fs/pax/softmode ayanami gcc # linux32 chroot /mnt/tmp/mnt/gentoo-x86-2004.2-chroot/ ayanami / # file /bin/bash /bin/bash: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.0.0, dynamically linked (uses shared libs), stripped works for me. what -doesnt- work is compiling gcc multilib... do we have a seperate bug for that?
an strace on the a.out that fails during configure when multilib is enabled: execve("/a.out", ["/a.out"], [/* 45 vars */]) = 0 brk(0) = 0x92c1180 open("/etc/ld.so.preload", O_RDONLY) = -1 ENOENT (No such file or directory) open("/etc/ld.so.cache", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=43714, ...}) = 0 old_mmap(NULL, 43714, PROT_READ, MAP_PRIVATE, 3, 0) = 0x4146f000 close(3) = 0 open("/lib/libc.so.6", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0000U\1\000"..., 512) = 512 old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4147a000 fstat64(3, {st_mode=S_IFREG|0755, st_size=1210656, ...}) = 0 old_mmap(NULL, 1140996, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x4147b000 old_mmap(0x4158c000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x110000) = 0x4158c000 old_mmap(0x4158f000, 10500, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x4158f000 close(3) = 0 mprotect(0x92bf000, 4096, PROT_READ) = -1 EACCES (Permission denied) writev(2, [{"/a.out", 6}, {": ", 2}, {"error while loading shared libra"..., 36}, {": ", 2}, {"", 0}, {"", 0}, {"cannot apply additional memory p"..., 58}, {": ", 2}, {"Permission denied", 17}, {"\n", 1}], 10/a.out: error while loading shared libraries: cannot apply additional memory protection after relocation: Permission denied ) = 124 exit_group(127) = ?
readelf -e output: ELF Header: Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00 Class: ELF32 Data: 2's complement, little endian Version: 1 (current) OS/ABI: UNIX - System V ABI Version: 0 Type: DYN (Shared object file) Machine: Intel 80386 Version: 0x1 Entry point address: 0x620 Start of program headers: 52 (bytes into file) Start of section headers: 4824 (bytes into file) Flags: 0x0 Size of this header: 52 (bytes) Size of program headers: 32 (bytes) Number of program headers: 9 Size of section headers: 40 (bytes) Number of section headers: 27 Section header string table index: 24 Section Headers: [Nr] Name Type Addr Off Size ES Flg Lk Inf Al [ 0] NULL 00000000 000000 000000 00 0 0 0 [ 1] .interp PROGBITS 00000154 000154 000013 00 A 0 0 1 [ 2] .note.ABI-tag NOTE 00000168 000168 000020 00 A 0 0 4 [ 3] .hash HASH 00000188 000188 0000b8 04 A 4 0 4 [ 4] .dynsym DYNSYM 00000240 000240 0001b0 10 A 5 c 4 [ 5] .dynstr STRTAB 000003f0 0003f0 0000f9 00 A 0 0 1 [ 6] .gnu.version VERSYM 000004ea 0004ea 000036 02 A 4 0 2 [ 7] .gnu.version_r VERNEED 00000520 000520 000040 00 A 5 1 4 [ 8] .rel.dyn REL 00000560 000560 000038 08 A 4 0 4 [ 9] .rel.plt REL 00000598 000598 000020 08 A 4 b 4 [10] .init PROGBITS 000005b8 0005b8 000017 00 AX 0 0 4 [11] .plt PROGBITS 000005d0 0005d0 000050 04 AX 0 0 4 [12] .text PROGBITS 00000620 000620 000224 00 AX 0 0 16 [13] .fini PROGBITS 00000844 000844 00001a 00 AX 0 0 4 [14] .rodata PROGBITS 00000860 000860 00000d 00 A 0 0 4 [15] .eh_frame PROGBITS 00000870 000870 000004 00 A 0 0 4 [16] .ctors PROGBITS 00001ee4 000ee4 000008 00 WA 0 0 4 [17] .dtors PROGBITS 00001eec 000eec 000008 00 WA 0 0 4 [18] .jcr PROGBITS 00001ef4 000ef4 000004 00 WA 0 0 4 [19] .dynamic DYNAMIC 00001ef8 000ef8 0000d8 08 WA 5 0 4 [20] .got PROGBITS 00001fd0 000fd0 000030 04 WA 0 0 4 [21] .data PROGBITS 00002000 001000 00000c 00 WA 0 0 4 [22] .bss NOBITS 0000200c 00100c 000004 00 WA 0 0 4 [23] .comment PROGBITS 00000000 00100c 0001fb 00 0 0 1 [24] .shstrtab STRTAB 00000000 001207 0000ce 00 0 0 1 [25] .symtab SYMTAB 00000000 001710 000460 10 26 2b 4 [26] .strtab STRTAB 00000000 001b70 000269 00 0 0 1Key to Flags: W (write), A (alloc), X (execute), M (merge), S (strings) I (info), L (link order), G (group), x (unknown) O (extra OS processing required) o (OS specific), p (processor specific) Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align PHDR 0x000034 0x00000034 0x00000034 0x00120 0x00120 R E 0x4 INTERP 0x000154 0x00000154 0x00000154 0x00013 0x00013 R 0x1 [Requesting program interpreter: /lib/ld-linux.so.2] LOAD 0x000000 0x00000000 0x00000000 0x00874 0x00874 R E 0x1000 LOAD 0x000ee4 0x00001ee4 0x00001ee4 0x00128 0x0012c RW 0x1000 DYNAMIC 0x000ef8 0x00001ef8 0x00001ef8 0x000d8 0x000d8 RW 0x4 NOTE 0x000168 0x00000168 0x00000168 0x00020 0x00020 R 0x4 STACK 0x000000 0x00000000 0x00000000 0x00000 0x00000 RW 0x4 LOOS+474e552 0x000ee4 0x00001ee4 0x00001ee4 0x0011c 0x0011c R 0x1 PAX_FLAGS 0x000000 0x00000000 0x00000000 0x00000 0x00000 0x4 Section to Segment mapping: Segment Sections... 00 01 .interp 02 .interp .note.ABI-tag .hash .dynsym .dynstr .gnu.version .gnu.version_r .rel.dyn .rel.plt .init .plt .text .fini .rodata .eh_frame 03 .ctors .dtors .jcr .dynamic .got .data .bss 04 .dynamic 05 .note.ABI-tag 06 07 .ctors .dtors .jcr .dynamic .got 08
Created attachment 41136 [details] the a.out for the test that fails
<pipacs> ok, the test app works fine under plain i386 <Lv> ha <pipacs> so it's either an amd64 specific toolchain/relro problem (unlikely since it shouldn't work here then) <pipacs> or something in the kernel welp. that makes this problem officially out of my league.
<pipacs> oh fuck <pipacs> i see what it is <pipacs> arch/x86_64/ia32/sys_ia32.c <pipacs> sys32_mprotect <pipacs> if (prot & PROT_READ) <pipacs> prot |= vm_force_exec32; <pipacs> that's something like the new 'read implies exec' personality <pipacs> except under pax the relro region is not executable nor can it be made executable <pipacs> so this force_exec32 screws me up damn pipacs is quick. :)
<pipacs> i forgot about 32 bit mode heh :) so booting with noexec32=all,force is the temporary workaround, lemme comment out all offending lines from that file and submit a patch. <pipacs> actually 'all' or 'force' should both be ok <pipacs> err, rather you need both <pipacs> damn complex shit, no wonder they removed it from 2.6.9
Created attachment 41139 [details, diff] fix 32bit on amd64 with pax
ayanami lv # ./broke.a.out ayanami lv # so yeah. that should fix compiling multilib gcc as well as the linux32 problem. John, mind doing some additional testing?
That's what, a patch to apply to the kernel? . . . I'll do that when I'm awake.
Closing bug as UPSTREAM. Please direct further questions/problems regarding 32bit handling on 64bit kernels in relation to this bug to the PaX Team.
this bug isnt fixed in hardened-dev-sources. is there a chance this simple patch can be included? either that or pipacs needs more poking. ;p
This would definitely be a great addition to hardened-dev-sources.....
just an update: this patch will be included upstream if pipacs doesnt have success with his 2.6.9 port this weekend. so yeah, there -should- be no problem fixing this bug in hardened-dev-sources...
alright, the patch is added to h-d-s-2.6.7-r11. i hope nobody kicks my butt for doing an h-d-s release ^^;
