| Summary: | dev-util/ninja: sigsegv when compiling on hardened amd64 machine with gcc-6.x | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Joshua Kinard <kumba> |
| Component: | Current packages | Assignee: | Chromium Project <chromium> |
| Status: | RESOLVED DUPLICATE | ||
| Severity: | normal | CC: | aos, hardened, toolchain |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | AMD64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
|
Description
Joshua Kinard
2016-12-30 22:14:29 UTC
# emerge --info Portage 2.3.3 (python 2.7.12-final-0, hardened/linux/amd64, gcc-6.3.0, glibc-2.24, 4.8.14 x86_64) ================================================================= System uname: Linux-4.8.14-x86_64-Intel-R-_Core-TM-_i7_CPU_970_@_3.20GHz-with-gentoo-2.3 KiB Mem: 16351384 total, 10293896 free KiB Swap: 4209024 total, 4209024 free Timestamp of repository gentoo: Fri, 30 Dec 2016 15:45:01 +0000 sh bash 4.4_p5-r1 ld GNU ld (Gentoo 2.27 p1.0) 2.27 distcc 3.2rc1 x86_64-pc-linux-gnu [disabled] ccache version 3.3.3 [disabled] app-shells/bash: 4.4_p5-r1::gentoo dev-java/java-config: 2.2.0-r3::gentoo dev-lang/perl: 5.24.1_rc4::gentoo dev-lang/python: 2.7.12::gentoo, 3.4.5::gentoo, 3.5.2::gentoo dev-util/ccache: 3.3.3::gentoo dev-util/cmake: 3.7.1::gentoo dev-util/pkgconfig: 0.29.1::gentoo sys-apps/baselayout: 2.3::gentoo sys-apps/openrc: 0.23::gentoo sys-apps/sandbox: 2.11-r3::gentoo sys-devel/autoconf: 2.13::gentoo, 2.69-r2::gentoo sys-devel/automake: 1.11.6-r2::gentoo, 1.12.6-r1::gentoo, 1.13.4-r1::gentoo, 1.14.1-r1::gentoo, 1.15-r2::gentoo sys-devel/binutils: 2.26.1::gentoo, 2.27::gentoo sys-devel/gcc: 6.2.0-r1::gentoo, 6.3.0::gentoo sys-devel/gcc-config: 1.8-r1::gentoo sys-devel/libtool: 2.4.6-r2::gentoo sys-devel/make: 4.2.1::gentoo sys-kernel/linux-headers: 4.9::gentoo (virtual/os-headers) sys-libs/glibc: 2.24::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://gentoo.cs.uni.edu/gentoo-portage priority: -1000 local location: /usr/portage/local masters: gentoo priority: 0 ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -march=corei7 -mtune=corei7 -mfpmath=sse -msse -msse2 -mssse3 -msse4.1 -msse4.2 -mcx16 -msahf -mcrc32 -maccumulate-outgoing-args -fforce-addr -fmodulo-sched -fivopts -ftree-loop-im -ftree-loop-linear -ftree-loop-ivcanon -fgcse-after-reload -fgcse-lm -fgcse-sm -fgcse-las -floop-interchange -ftree-loop-distribution -floop-strip-mine -floop-block -ftree-vectorize -flto=8 -fuse-linker-plugin" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/init.d /etc/pam.d /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-O2 -march=corei7 -mtune=corei7 -mfpmath=sse -msse -msse2 -mssse3 -msse4.1 -msse4.2 -mcx16 -msahf -mcrc32 -maccumulate-outgoing-args -fforce-addr -fmodulo-sched -fivopts -ftree-loop-im -ftree-loop-linear -ftree-loop-ivcanon -fgcse-after-reload -fgcse-lm -fgcse-sm -fgcse-las -floop-interchange -ftree-loop-distribution -floop-strip-mine -floop-block -ftree-vectorize -flto=8 -fuse-linker-plugin" DISTDIR="/usr/portage/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs candy cgroup config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms sign splitdebug unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS=" http://gentoo.osuosl.org/ http://mirror.usu.edu/mirrors/gentoo/ http://mirror.csclub.uwaterloo.ca/gentoo-distfiles/ http://gentoo.arcticnetwork.ca/ " INSTALL_MASK="/usr/lib/systemd /etc/systemd" LANG="en_US.iso885915" LDFLAGS="-O2 -march=corei7 -mtune=corei7 -mfpmath=sse -msse -msse2 -mssse3 -msse4.1 -msse4.2 -mcx16 -msahf -mcrc32 -maccumulate-outgoing-args -fforce-addr -fmodulo-sched -fivopts -ftree-loop-im -ftree-loop-linear -ftree-loop-ivcanon -fgcse-after-reload -fgcse-lm -fgcse-sm -fgcse-las -floop-interchange -ftree-loop-distribution -floop-strip-mine -floop-block -ftree-vectorize -flto=8 -fuse-linker-plugin -Wl,-z,now -Wl,-z,relro" MAKEOPTS="-j14" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/ramfs" USE="amd64 apache2 berkdb bzip2 cli cracklib curl cxx dbus dri encode gd gdbm graphite hardened inotify ipv6 justify lto mmx mmxext modules multilib ncurses nptl nptlonly offensive openmp pam pax_kernel pcre pic pie readline real rtc seccomp session sqlite sse sse2 ssl ssp tcpd unicode urandom xattr xml xtpax zlib" ABI_X86="64 32" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_4 python3_5" QEMU_SOFTMMU_TARGETS="mips mips64 mipsel mips64el" RUBY_TARGETS="ruby21" USERLAND="GNU" VIDEO_CARDS="vesa vga fbdev" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON Have you tried using simpler CXXFLAGS? (In reply to Mike Gilbert from comment #2) > Have you tried using simpler CXXFLAGS? The issue is reproducible outside of Portage: tar zfx /usr/portage/distfiles/ninja-1.7.2.tar.gz cd ninja-1.7.2/ ./configure.py --bootstrap --verbose --debug <boom after several seconds> Ah. Well, I cannot reproduce this on a non-hardened profile. could you try to build glibc with disable-bind-now? or use a older glibc 2.23? we have seen some segfault and build probs with 2.24 and pie (In reply to Magnus Granberg from comment #5) > could you try to build glibc with disable-bind-now? > or use a older glibc 2.23? > we have seen some segfault and build probs with 2.24 and pie Is "disable-bind-now" a USE flag? quse doesn't indicate that exists. I'm in the process of rebuilding my userland after a forced hardware upgrade. Somehow, I incidentally fixed GDB to actually emit a more complete backtrace, which is revealing some additional information: # gdb ninja GNU gdb (Gentoo 7.12.1 vanilla) 7.12.1 Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <https://bugs.gentoo.org/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from ninja...done. (gdb) run Starting program: /usr/src/ninja-1.7.2/ninja warning: Cannot call inferior functions, Linux kernel PaX protection forbids return to non-executable pages! [0/27] INLINE build/browse_py.hninja: ../sysdeps/unix/sysv/linux/spawni.c:360: __spawnix: Assertion `ec >= 0' failed. Program received signal SIGABRT, Aborted. 0x000003d52ecaff0f in raise () from /lib64/libc.so.6 (gdb) bt #0 0x000003d52ecaff0f in raise () from /lib64/libc.so.6 #1 0x000003d52ecb140a in abort () from /lib64/libc.so.6 #2 0x000003d52eca8936 in __assert_fail_base () from /lib64/libc.so.6 #3 0x000003d52eca89f1 in __assert_fail () from /lib64/libc.so.6 #4 0x000003d52ed5b7ea in __spawnix () from /lib64/libc.so.6 #5 0x000003d52ed5bd5a in __spawni () from /lib64/libc.so.6 #6 0x000003d52ed5b49a in posix_spawn () from /lib64/libc.so.6 #7 0x0000002862990c95 in Subprocess::Start (this=0x2862df3e20, set=0x2862df3350, command="\"./src/inline.sh\" kBrowsePy < src/browse.py > build/browse_py.h") at ./src/subprocess-posix.cc:101 #8 0x00000028629914d9 in SubprocessSet::Add (this=0x2862df3350, command="\"./src/inline.sh\" kBrowsePy < src/browse.py > build/browse_py.h", use_console=false) at ./src/subprocess-posix.cc:210 #9 0x0000002862931d7a in RealCommandRunner::StartCommand (this=0x2862df3340, edge=0x2862dedf70) at ./src/build.cc:540 #10 0x0000002862932ebe in Builder::StartEdge (this=0x3ef385453f0, edge=0x2862dedf70, err=0x3ef385454e0) at ./src/build.cc:753 #11 0x00000028629328ed in Builder::Build (this=0x3ef385453f0, err=0x3ef385454e0) at ./src/build.cc:665 #12 0x0000002862917041 in (anonymous namespace)::NinjaMain::RunBuild (this=0x3ef38545630, argc=0, argv=0x3ef38545a50) at ./src/ninja.cc:951 #13 0x0000002862917c95 in (anonymous namespace)::real_main (argc=0, argv=0x3ef38545a50) at ./src/ninja.cc:1143 #14 0x0000002862917d7a in main (argc=1, argv=0x3ef38545a48) at ./src/ninja.cc:1172 (gdb) This is after manually compiling ninja per my notes in Comment #3. Still leaves segfault messages in dmesg, though: [22296.267615] ninja[18737]: segfault at 3a62b775e50 ip 000003a62a9b2804 sp 000003a62b775e50 error 6 in libc-2.25.so[3a62a8d3000+197000] [22331.247998] ninja[18743]: segfault at 358a9b60e50 ip 00000358a8dae804 sp 00000358a9b60e50 error 6 in libc-2.25.so[358a8ccf000+197000] Looks like "ec" means "error code from child. Source patch that added this assert is: https://sourceware.org/ml/libc-alpha/2016-09/msg00360.html One of the follow-up messages suggests the more meaningful variable name that doesn't appear to have been taken into context. So it looks like on a hardened system, ninja is tripping up this assert in glibc. Not sure now if that means it's really a glibc bug, ninja bug, or hardened bug. Can you rebuild glibc and put -fstack-check=no in the cflags? (In reply to Joshua Kinard from comment #7) > Looks like "ec" means "error code from child. It would be nice to have more detailed info from the child process, which is failing early in __spawni_child(). (In reply to Magnus Granberg from comment #8) > Can you rebuild glibc and put -fstack-check=no in the cflags? I was able to reproduce the issue in a container with a current amd64-hardened stage3 tarball + gcc-6.3.0 + glibc-2.24-r1. Rebuilding glibc-2.24-r1 with -fstack-check=no seems to make the problem go away. Thanks for the triage help. Sounds like this is a bug in glibc or gcc. (In reply to Mike Gilbert from comment #10) > (In reply to Magnus Granberg from comment #8) > > Can you rebuild glibc and put -fstack-check=no in the cflags? > > I was able to reproduce the issue in a container with a current > amd64-hardened stage3 tarball + gcc-6.3.0 + glibc-2.24-r1. > > Rebuilding glibc-2.24-r1 with -fstack-check=no seems to make the problem go > away. Stack checking caused an issue on one of the kernels for my SGI machines by causing it to panic in hardcoded assembler for a spin lock. I had a funny feeling it might be tied to this, but didn't think to rebuild glibc with -fno-stack-check. (In reply to Mike Gilbert from comment #10) > (In reply to Magnus Granberg from comment #8) > > Can you rebuild glibc and put -fstack-check=no in the cflags? > > I was able to reproduce the issue in a container with a current > amd64-hardened stage3 tarball + gcc-6.3.0 + glibc-2.24-r1. > > Rebuilding glibc-2.24-r1 with -fstack-check=no seems to make the problem go > away. I was about to state that I cannot reproduce this success, as I was using -fno-stack-check. but, on a whim, switched to -fstack-check=no and that works. So it looks like flags-o-matic needs a small update to handle the -fno version of this flag, as both -fno-stack-check and -fstack-check=no are equivalent. *** This bug has been marked as a duplicate of bug 608788 *** *** Bug 620944 has been marked as a duplicate of this bug. *** |