| Summary: | [ICE/4.6] sys-apps/sandbox build segfaults on hardened systems | ||
|---|---|---|---|
| Product: | Portage Development | Reporter: | J.C. Wren <jcwren> |
| Component: | Sandbox | Assignee: | Gentoo Toolchain Maintainers <toolchain> |
| Status: | RESOLVED DUPLICATE | ||
| Severity: | normal | CC: | alexanderyt, anarchy, clemente.aguiar, hardened, josef64, liviu.anghel, lorand.kelemen, maris.gis, mjo, penguin, sandbox, seventnsec, whitehatcheck, wtt6, zerochaos |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| See Also: |
https://bugs.gentoo.org/show_bug.cgi?id=301299 https://bugs.gentoo.org/show_bug.cgi?id=572092 |
||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | |||
| Bug Blocks: | 472624 | ||
| Attachments: | build.log from emerge -q sandbox | ||
|
Description
J.C. Wren
2012-07-09 17:17:38 UTC
Created attachment 317720 [details]
build.log from emerge -q sandbox
i added pre-compiled headers support to sandbox-2.6, so perhaps that's why things are crashing I'm also seeing this on one of my machines; commenting for the CC. Can you check what your pax flags are on cc1 with paxctl? Yes of course, here you are: [...] coventry ~ # paxctl -v /usr/libexec/gcc/i686-pc-linux-gnu/4.6.3/cc1 PaX control v0.7 Copyright 2004,2005,2006,2007,2009,2010,2011,2012 PaX Team <pageexec@freemail.hu> - PaX flags: -------x-e-r [/usr/libexec/gcc/i686-pc-linux-gnu/4.6.3/cc1] RANDEXEC is disabled EMUTRAMP is disabled RANDMMAP is disabled [...] Anything in the logs? I'm not able to reproduce on an amd64 (~amd64) with SELinux profile enabled (Portage 2.1.11.5 (hardened/linux/amd64/no-multilib/selinux, gcc-4.6.3, glibc-2.15-r2, 3.2.11-hardened x86_64)) J.C. & Arnim, are those systems both x86 systems? Yes, Intel(R) Pentium(R) 4 for me. Oooh, a midair collision :) That's a first for me. I don't know what changed between July 9th and July 11th, but it compiled successfully. The only emerge updates between those two dates where: sys-kernel/linux-headers-3.4-r1 sys-devel/automake-1.12.2 sys-kernel/hardened-sources-3.4.4-r2 app-admin/eselect-php-0.6.7 Yes, it's x86. To be precise: coventry ~ # uname -a Linux <snipped.out.hostname> 3.1.1-hardened-r1 #1 SMP Thu Nov 24 10:00:32 CET 2011 i686 Intel(R) Xeon(TM) CPU 2.66GHz GenuineIntel GNU/Linux It's a Pentium-4, Northwood core. It compiles fine in my chroot jasmin / # emerge --info Portage 2.1.11.7 (hardened/linux/x86, gcc-4.6.3, glibc-2.15-r2, 3.4.4-hardened-r2 i686) ================================================================= System uname: Linux-3.4.4-hardened-r2-i686-Intel-R-_Xeon-R-_CPU_E5420_@_2.50GHz-with-gentoo-2.1 Timestamp of tree: Mon, 16 Jul 2012 02:45:01 +0000 app-shells/bash: 4.2_p36 dev-lang/python: 2.6.5-r3, 2.7.3-r2, 3.1.4-r3, 3.2.3-r1 dev-util/cmake: 2.8.7-r5 dev-util/pkgconfig: 0.27 sys-apps/baselayout: 2.1-r1 sys-apps/openrc: 0.10.5 sys-apps/sandbox: 2.6 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.11.1, 1.12.2 sys-devel/binutils: 2.22-r1 sys-devel/gcc: 4.4.5, 4.5.3-r2, 4.6.3 sys-devel/gcc-config: 1.7.3 sys-devel/libtool: 2.4.2 sys-devel/make: 3.82-r3 sys-kernel/linux-headers: 3.4-r1 (virtual/os-headers) sys-libs/glibc: 2.15-r2 Repositories: gentoo ACCEPT_KEYWORDS="x86 ~x86" ACCEPT_LICENSE="*" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -pipe -march=i686" CHOST="i686-pc-linux-gnu" CXXFLAGS="-O2 -pipe -march=i686" LDFLAGS="-Wl,-O1 -Wl,--as-needed" Perhaps if you rollback your linux-headers to 3.4 (instead of 3.4-r1) you may be able to reproduce it. That's the only thing that looks like a likely candidate of the 4 portage entries that changed between the time it wouldn't compile to the time it would. (In reply to comment #12) > Perhaps if you rollback your linux-headers to 3.4 (instead of 3.4-r1) you > may be able to reproduce it. That's the only thing that looks like a likely > candidate of the 4 portage entries that changed between the time it wouldn't > compile to the time it would. Even if i rollback to 3.4 i can't get sandbox to fail. Well, I can't explain it. As I said, only those 4 packages changed between a failed and succesful emerge of sandbox. The other similarly configured machine compiled it too, so there must be some common denominator. As a user, I can consider this closed since it's emerged successfully. But it'd sure be nice to know why. Rolling back headers to 3.4 doesn't solve this for me. It still breaks at the same point: [...] CC sb_memory.lo CC debug.lo i686-pc-linux-gnu-gcc: internal compiler error: Segmentation fault (program cc1) [...] There is nothing in the logs (no errors/warnings), save for the above. (In reply to comment #15) > Rolling back headers to 3.4 doesn't solve this for me. It still breaks at > the same point: > > [...] > CC sb_memory.lo > CC debug.lo > > i686-pc-linux-gnu-gcc: internal compiler error: Segmentation fault (program > cc1) > [...] > > There is nothing in the logs (no errors/warnings), save for the above. Could you go to build directory and 1) run make V=1, 2) copy failing command and run with additional flag: -save-temps 3) attach output to this bug (filename.i) 4) if given file use precompiled headers (headers.*.gch) please repeat 1-3) for headers.h, you may need to compress this one. Also attach your emerge --info please. I'm getting very weird results with that: [...] coventry build-default # make V=1 ... libtool: compile: i686-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I../../sandbox-2.6/libsbutil -I.. -I../../sandbox-2.6 -I../../sandbox-2.6/libsbutil/include -DETCDIR=\"/etc\" -DLIBSANDBOX_PATH=\"/usr/lib\" -DSANDBOX_BASHRC_PATH=\"/usr/share/sandbox\" -D_REENTRANT -march=native -O2 -pipe -fomit-frame-pointer -Wall -Winvalid-pch -fdata-sections -ffunction-sections -MT debug.lo -MD -MP -MF .deps/debug.Tpo -c ../../sandbox-2.6/libsbutil/src/debug.c -fPIC -DPIC -o .libs/debug.o i686-pc-linux-gnu-gcc: internal compiler error: Segmentation fault (program cc1) [...] However, when I copy that command in the src/ dir, I get no error: [...] coventry src # /bin/sh libtool --tag=CC --mode=compile i686-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I../../sandbox-2.6/libsbutil -I.. -I../../sandbox-2.6 -I../../sandbox-2.6/libsbutil/include -DETCDIR="\"/etc\"" -DLIBSANDBOX_PATH="\"/usr/lib\"" -DSANDBOX_BASHRC_PATH="\"/usr/share/sandbox\"" -D_REENTRANT -march=native -O2 -pipe -fomit-frame-pointer -Wall -Winvalid-pch -fdata-sections -ffunction-sections -MT debug.lo -MD -MP -MF .deps/debug.Tpo -c -o debug.lo `test -f 'src/debug.c' || echo '../../sandbox-2.6/libsbutil/'`src/debug.c libtool: compile: i686-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I../../sandbox-2.6/libsbutil -I.. -I../../sandbox-2.6 -I../../sandbox-2.6/libsbutil/include -DETCDIR=\"/etc\" -DLIBSANDBOX_PATH=\"/usr/lib\" -DSANDBOX_BASHRC_PATH=\"/usr/share/sandbox\" -D_REENTRANT -march=native -O2 -pipe -fomit-frame-pointer -Wall -Winvalid-pch -fdata-sections -ffunction-sections -MT debug.lo -MD -MP -MF .deps/debug.Tpo -c ../../sandbox-2.6/libsbutil/src/debug.c -fPIC -DPIC -o .libs/debug.o libtool: compile: i686-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I../../sandbox-2.6/libsbutil -I.. -I../../sandbox-2.6 -I../../sandbox-2.6/libsbutil/include -DETCDIR=\"/etc\" -DLIBSANDBOX_PATH=\"/usr/lib\" -DSANDBOX_BASHRC_PATH=\"/usr/share/sandbox\" -D_REENTRANT -march=native -O2 -pipe -fomit-frame-pointer -Wall -Winvalid-pch -fdata-sections -ffunction-sections -MT debug.lo -MD -MP -MF .deps/debug.Tpo -c ../../sandbox-2.6/libsbutil/src/debug.c -o debug.o >/dev/null 2>&1 coventry src # [...] After that, it errors with the same compiler segfault on `environ.c' again (when I make V=1). The output of debug.i is 10498 lines, do you still need me to paste that or offer it online in some fashion? Can someone else confirm my findings of the above? *** Bug 428782 has been marked as a duplicate of this bug. *** Will this bug be fixed? It's still broken in my current environment. I'm also seeing this on my machine (hardened,i686, Intel T2300) It's been there for quite some time now (since sandbox-2.6) as far as I can remember. Someone told me to report this after they couldn't help me in a forum. Can provide build.log emerge --info. Other than that I#m just a noob - sorry (In reply to comment #21) workaround from the Forum[german] http://www.gentooforum.de/artikel/20912/failed-to-emerge-sys-apps-sandbox-2-6-mal-wieder.html is: build sandbox-2.6 with a non-hardened Kernel the next release will have a configure flag which should let you disable pch (since it's just a build-time optimization) http://git.overlays.gentoo.org/gitweb/?p=proj/sandbox.git;a=commitdiff;h=f2500f5954611d110ac18e9990f42d5a915f8101 (In reply to SpanKY from comment #23) > the next release will have a configure flag which should let you disable pch > (since it's just a build-time optimization) > > http://git.overlays.gentoo.org/gitweb/?p=proj/sandbox.git;a=commitdiff; > h=f2500f5954611d110ac18e9990f42d5a915f8101 7 months and still no release, what is the expected time frame on this update? *** Bug 474880 has been marked as a duplicate of this bug. *** *** Bug 474958 has been marked as a duplicate of this bug. *** So it looks like we have multiple duplicate bugs that land us to this page. Any updates to this issue? My bug was reported here: https://bugs.gentoo.org/show_bug.cgi?id=474880 this isn't a bug in sandbox. gcc (when hardening is enabled) is broken. disabling pch in sandbox doesn't fix gcc. (In reply to SpanKY from comment #28) > this isn't a bug in sandbox. gcc (when hardening is enabled) is broken. > disabling pch in sandbox doesn't fix gcc. its not gcc, but PaX's randmmap, so even a vanilla gcc with a pax enabled kernel would hit this. if we pax-mark cc1 and cc1plus -r, then randmmap is disable and it works. i just tested to convince myself. @those hitting this do the following: 1) Check that your running a pax hardened kernel and where the kernel is looking for the markings: zcat /proc/config.gz | grep _PAX_FLAGS 2) If you are using the traditional PT_PAX then check paxctl -v /usr/libexec/gcc/x86_64-pc-linux-gnu/4.6.3/cc1* and make sure RANDMMAP is disabled. 3) If you are using the new and sexy XATTR_PAX_FLAGS, then paxctl-ng -v /usr/libexec/gcc/x86_64-pc-linux-gnu/4.6.3/cc1* Having said that, I'm not a great fan of pch. I'm not sure what the gain is on small projects. I'd prefer to leave -r off cc1 and cc1plus and not deal with pch, but that's not the world we live in. I'm still running across this problem, but no PAX enabled on my end...
cat /usr/src/linux/.config | grep _PAX_FLAGS
# CONFIG_PAX_PT_PAX_FLAGS is not set
# CONFIG_PAX_XATTR_PAX_FLAGS is not set
Tried paxctl just for kicks:
- PaX flags: -------x-e-r [/usr/libexec/gcc/i686-pc-linux-gnu/4.6.3/cc1]
RANDEXEC is disabled
EMUTRAMP is disabled
RANDMMAP is disabled
- PaX flags: -------x-e-r [/usr/libexec/gcc/i686-pc-linux-gnu/4.6.3/cc1plus]
RANDEXEC is disabled
EMUTRAMP is disabled
RANDMMAP is disabled
Current gcc set:
[20] i686-pc-linux-gnu-4.6.3 *
Result...
i686-pc-linux-gnu-gcc: internal compiler error: Segmentation fault (program cc1)
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://bugs.gentoo.org/> for instructions.
make[2]: *** [debug.lo] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
emake failed
* ERROR: sys-apps/sandbox-2.6-r1 failed (compile phase):
* (no error message)
Thanks!
(In reply to lou from comment #30) > I'm still running across this problem, but no PAX enabled on my end... > > cat /usr/src/linux/.config | grep _PAX_FLAGS > # CONFIG_PAX_PT_PAX_FLAGS is not set > # CONFIG_PAX_XATTR_PAX_FLAGS is not set > > Tried paxctl just for kicks: > > - PaX flags: -------x-e-r [/usr/libexec/gcc/i686-pc-linux-gnu/4.6.3/cc1] > RANDEXEC is disabled > EMUTRAMP is disabled > RANDMMAP is disabled > - PaX flags: -------x-e-r [/usr/libexec/gcc/i686-pc-linux-gnu/4.6.3/cc1plus] > RANDEXEC is disabled > EMUTRAMP is disabled > RANDMMAP is disabled > > Current gcc set: > [20] i686-pc-linux-gnu-4.6.3 * > > Result... > > i686-pc-linux-gnu-gcc: internal compiler error: Segmentation fault (program > cc1) > Please submit a full bug report, > with preprocessed source if appropriate. > See <http://bugs.gentoo.org/> for instructions. > make[2]: *** [debug.lo] Error 1 > make[2]: *** Waiting for unfinished jobs.... > make[1]: *** [all-recursive] Error 1 > make: *** [all] Error 2 > emake failed > * ERROR: sys-apps/sandbox-2.6-r1 failed (compile phase): > * (no error message) > > > Thanks! I'm not sure you don't have pax there, just not the markings. Try vanilla-sources or gentoo-sources. I bet you the problem will go away :) Really? I have hardened-sources with no PAX markings, and I need to change kernel? (In reply to lou from comment #32) > Really? I have hardened-sources with no PAX markings, and I need to change > kernel? No pax markins doesn't mean pax is not enforced. (In reply to Anthony Basile from comment #29) i'm not familiar with pch internals. i guess the issue is that pch is compiled and then expected to get mmap-ed back at the same address ? and if it doesn't (because of say randmmap), it goes boom ? (In reply to SpanKY from comment #34) > (In reply to Anthony Basile from comment #29) > > i'm not familiar with pch internals. i guess the issue is that pch is > compiled and then expected to get mmap-ed back at the same address ? and if > it doesn't (because of say randmmap), it goes boom ? bug #301299 comment 31 The only way around i see is to add use flag for pch. If that don't work we need to get a bt on the error. (In reply to SpanKY from comment #34) > (In reply to Anthony Basile from comment #29) > > i'm not familiar with pch internals. i guess the issue is that pch is > compiled and then expected to get mmap-ed back at the same address ? and if > it doesn't (because of say randmmap), it goes boom ? basically yes I have come across this same bug, although compilation breaks at a different point: --- CC sb_memory.lo CC debug.lo CC string.lo CC file.lo CC config.lo CC dynbuf.lo x86_64-pc-linux-gnu-gcc: internal compiler error: Segmentation fault (program cc1) --- I have three boxes with hardened sources. Sandbox does not emerge in one of the boxes, but it successfully compiles in the others. The three boxes have similar PAX configuration, so I wonder if this is really PAX related. These are the three boxes. I get the bug in the first box: 1 Linux 3.9.5-hardened #4 SMP Fri Jul 19 14:59:10 CEST 2013 x86_64 AMD Opteron(tm) Processor 4180 AuthenticAMD GNU/Linux CFLAGS="-march=native -O2 -pipe" sandbox-2.5 emerges correctly sandbox-2.6.r1 fails to emerge 2 Linux 3.9.5-hardened #1 SMP Thu Jul 11 21:19:21 CEST 2013 x86_64 Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz GenuineIntel GNU/Linux CFLAGS="-march=core2 -O2 -pipe" 3 Linux 3.9.5-hardened_GNT #6 SMP Fri Jul 19 21:20:29 CEST 2013 x86_64 Six-Core AMD Opteron(tm) Processor 2431 AuthenticAMD GNU/Linux CFLAGS="-march=native -O2 -pipe" The three boxes have this same PAX configuration: zcat /proc/config.gz | grep PAX CONFIG_PAX_USERCOPY_SLABS=y CONFIG_PAX=y # CONFIG_PAX_SOFTMODE is not set # CONFIG_PAX_PT_PAX_FLAGS is not set # CONFIG_PAX_XATTR_PAX_FLAGS is not set CONFIG_PAX_NO_ACL_FLAGS=y # CONFIG_PAX_HAVE_ACL_FLAGS is not set # CONFIG_PAX_HOOK_ACL_FLAGS is not set CONFIG_PAX_NOEXEC=y CONFIG_PAX_PAGEEXEC=y CONFIG_PAX_EMUTRAMP=y CONFIG_PAX_MPROTECT=y # CONFIG_PAX_MPROTECT_COMPAT is not set # CONFIG_PAX_ELFRELOCS is not set # CONFIG_PAX_KERNEXEC is not set CONFIG_PAX_KERNEXEC_PLUGIN_METHOD="" CONFIG_PAX_ASLR=y # CONFIG_PAX_RANDKSTACK is not set CONFIG_PAX_RANDUSTACK=y CONFIG_PAX_RANDMMAP=y # CONFIG_PAX_MEMORY_SANITIZE is not set # CONFIG_PAX_MEMORY_STACKLEAK is not set # CONFIG_PAX_MEMORY_STRUCTLEAK is not set # CONFIG_PAX_MEMORY_UDEREF is not set # CONFIG_PAX_REFCOUNT is not set # CONFIG_PAX_USERCOPY is not set # CONFIG_PAX_SIZE_OVERFLOW is not set # CONFIG_PAX_LATENT_ENTROPY is not set Same gcc version: gcc-config -l [1] x86_64-pc-linux-gnu-4.6.3 * [2] x86_64-pc-linux-gnu-4.6.3-hardenednopie [3] x86_64-pc-linux-gnu-4.6.3-hardenednopiessp [4] x86_64-pc-linux-gnu-4.6.3-hardenednossp [5] x86_64-pc-linux-gnu-4.6.3-vanilla That bug happened to me and it seems that Anthony is quite right. My system had PAX enabled but my kernel was not reading the ELF flags nor the FS flags so basically, the flags were always ignored. I fixed my kernel to read XATTR_FLAGS and then updated my pax flags using `migrate-pax -m` and that solved the problem. (In reply to Mounir Lamouri from comment #38) > That bug happened to me and it seems that Anthony is quite right. My system > had PAX enabled but my kernel was not reading the ELF flags nor the FS flags > so basically, the flags were always ignored. I fixed my kernel to read > XATTR_FLAGS and then updated my pax flags using `migrate-pax -m` and that > solved the problem. You are right! Enabling CONFIG_PAX_XATTR_PAX_FLAGS allows sandbox to emerge. # zcat /proc/config.gz | grep XATTR_PAX CONFIG_PAX_XATTR_PAX_FLAGS=y Thanks! I was running into the exact same problem as everyone else, with cc1 segfault on file.o. So I noticed that I didn't have CONFIG_PAX_XATTR_PAX_FLAGS set in my kernel, and set that and recompiled. I then did 'migrate-pax -m' as recommended in comment #38 but this was not enough to make it work. I had to do paxctl-ng -p -e -m -r -s /usr/libexec/gcc/i486-pc-linux-gnu/4.6.4/cc1* to get sandbox to compile. I removed -p, -m, and -s afterwards, to put it back the way it was (just -e and -r). I have to say it's very frustrating running into this kind of thing and people automatically say that you have faulty hardware. That said, thanks to everyone who helped with solutions to this problem. Seems like this is more of a stopgap though than a real solution, FWIW. I had this same issue pop up today when building a test setup on a vm and doing the "emerge world -uDN," but I was showing a segfault with cc1 at debug.lo. What I had done exactly, was build a semi-usable kernel that will get the system loaded while I tinker the rest of it into place. It is important to note that I did not do my homework and enabled: CONFIG_PAX_PT_PAX_FLAGS=y at this point, where had I read: http://www.gentoo.org/proj/en/hardened/pax-migrate-xattr.xml?style=printable, I would have seen that PT_PAX is depricated. After booting into the new system, I saw SELinux working, and I proceeded to relabel the filesystem and get the rest of the system set up. Of course I mixed variables and modified the features in grsec and inadvertently broke the flags. I believe this is where the problem was since I added: CONFIG_PAX_XATTR_PAX_FLAGS=y After running: "migrate-pax -m" and setting CONFIG_PAX_PT_PAX_FLAGS=n, it worked. So, this is killing infra's build box for amd64 and x86 right now. When the hardened box tries to build a default stage we get an ice which the kernel then freaks out and locks the portage account which obviously ends the autobuilds. Personally, I would just remove the pre-compiled headers entirely as the milliseconds saved on the build hardly seem worth it. But I'd be perfectly happy with a use flag disabled per default. Commit message: Disable pch logic for now http://sources.gentoo.org/sys-apps/sandbox/files/sandbox-2.6-no-pch.patch?rev=1.1 http://sources.gentoo.org/sys-apps/sandbox/sandbox-2.6-r1.ebuild?r1=1.14&r2=1.15 Close? (In reply to Magnus Granberg from comment #44) > Close? I haven't seen this issue in a long time, so +1 from me. the issue went away because we disabled pch usage in the sandbox-2.6-r1, not because anything was actually fixed *** This bug has been marked as a duplicate of bug 301299 *** |
