My chromium always immediately segfaults when I try to start it, before its window becomes visible: Sep 10 21:57:17 lap kernel: NetworkChangeNo[9859]: segfault at fffffc1f00d2d450 ip 00000010a674cb82 sp 000003e0ff2e8760 error 5 in chrome[10a04fb000+7e9d000] Addresses vary from crash to crash (perhaps because I have address space randomization turned on), but its always "NetworkChangeNo". Differing from bug 593452 and bug 591938, this is with USE +suid +tcmalloc: [ebuild R ] www-client/chromium-54.0.2840.16::gentoo USE="cups (gn) gnome gnome-keyring (pic) proprietary-codecs suid system-ffmpeg tcmalloc -custom-cflags (-gtk3) -hangouts -kerberos (-neon) -pulseaudio (-selinux) {-test} -widevine" ~: emerge --info Portage 2.3.0 (python 2.7.12-final-0, default/linux/amd64/13.0/no-multilib, gcc-5.4.0, glibc-2.23-r2, 4.7.3-hardened x86_64) ================================================================= System uname: Linux-4.7.3-hardened-x86_64-Intel-R-_Core-TM-_i7-3940XM_CPU_@_3.00GHz-with-gentoo-2.3 KiB Mem: 32902628 total, 18107424 free KiB Swap: 0 total, 0 free Timestamp of repository gentoo: Sat, 10 Sep 2016 15:15:01 +0000 sh bash 4.3_p46 ld GNU ld (Gentoo 2.26.1 p1.0) 2.26.1 app-shells/bash: 4.3_p46::gentoo dev-java/java-config: 2.2.0-r3::gentoo dev-lang/perl: 5.24.0-r1::gentoo dev-lang/python: 2.7.12::gentoo, 3.5.2::gentoo dev-util/cmake: 3.6.2::gentoo dev-util/pkgconfig: 0.29.1::gentoo sys-apps/baselayout: 2.3::gentoo sys-apps/openrc: 0.21.7::gentoo sys-apps/sandbox: 2.10-r2::gentoo sys-devel/autoconf: 2.13::gentoo, 2.69-r2::gentoo sys-devel/automake: 1.14.1-r1::gentoo, 1.15-r2::gentoo sys-devel/binutils: 2.26.1::gentoo sys-devel/gcc: 5.4.0::gentoo sys-devel/gcc-config: 1.8-r1::gentoo sys-devel/libtool: 2.4.6-r2::gentoo sys-devel/make: 4.2.1::gentoo sys-kernel/linux-headers: 4.7::gentoo (virtual/os-headers) sys-libs/glibc: 2.23-r2::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://rsync.de.gentoo.org/gentoo-portage priority: -1000 ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="* -@EULA dlj-1.1 AdobeFlash-11.x Oracle-BCLA-JavaSE google-chrome googleearth Vivaldi FraunhoferFDK" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -mtune=native -flto -fuse-linker-plugin -O3 -fomit-frame-pointer -fweb -ftracer -fivopts -frename-registers -maccumulate-outgoing-args -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=native -mtune=native -flto -fuse-linker-plugin -O3 -fomit-frame-pointer -fweb -ftracer -fivopts -frename-registers -maccumulate-outgoing-args -pipe" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--quiet-build --quiet-fail --with-bdeps=y" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs collision-protect config-protect-if-modified distlocks ebuild-locks fixlafiles keeptemp keepwork merge-sync news noclean parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://linux.rz.rub.de/download/gentoo-mirror http://ftp.spline.inf.fu-berlin.de/mirrors/gentoo http://distfiles.gentoo.org" LANG="en_DE.iso885915" LC_ALL="en_DE.iso885915" LDFLAGS="-march=native -mtune=native -flto -fuse-linker-plugin -O3 -fomit-frame-pointer -fweb -ftracer -fivopts -frename-registers -maccumulate-outgoing-args -pipe" MAKEOPTS="-j8" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/portage" USE="64bit X a52 aac adobe-cff alsa amd64 apng applet archive ass avx bzip2 cairo cdda cdparanoia clang cli clipboard clockdiff contrast cups curl cxx dbus dconf demosaic detex devfs-compat dga divx dns dot dri dri3 drm dts dvd dvdnav dvdr dvi dvipdfm egl encode epspdf exif expat extra faad fdk ffmpeg fftw flac fontconfig foomaticdb fts3 g3dvl gallium gbm gdk-pixbuf gegl gif gimp glamor gles gles1 gles2 glib glibc-omitfp gmp graphics gs gstreamer gtk gtk2 gudev harfbuzz hpack-tools hpn htmlreport http http2 hwdb iconv icu imagemagick inotify javascript jbig jemalloc jemalloc3 jit jpeg jpeg2k kpathsea lasi latex latex3 lcdfilter lcms leaps_timezone lensfun libkms libnotify libopts libressl libsamplerate libwww lightning llvm llvm-gcc llvm-shared-libs lz4 lzma lzo mad matroska metric midi minizip mmap mms mmx mmxext mng modern-top modules mp3 mpeg mpfr mta mudflap multicall natspec ncat ncurses ndiff nping nptl nscd nsplugin offensive ogg oldnet opencl opengl openmax openmp openvg opus orc pam pango pax_kernel pcap pcre pdf pic plugins png policykit postproc postscript ppds proprietary-codecs pstricks pth ptpax quicktime r600-llvm-compiler rar raw readline realmedia right_timezone rle rpc rtc rule_generator sanitize scanner schroedinger scope seccomp secure-delete session smp sndfile sound sqlite sqlite3 sse sse2 sse3 sse4 sse4_1 sse4_2 ssh ssl ssse3 svc svg symlink sync-plugin-portage system-cairo system-ffmpeg system-harfbuzz system-icu system-jpeg system-jsoncpp system-libevent system-libvpx system-sqlite systemd t1lib texi2html theora threads thunar tiff tools tracepath tremor truetype udev udisks unicode unlock-notify unwind usb user-session utils v4l v4l2 vaapi vdpau vim-with-x vorbis vpx webkit2 webp wext wifi wmf wmp wxwidgets x264 x265 xa xattr xcb xcomposite xetex xkb xmp xorg xpm xrandr xtpax xulrunner xv xvid xvmc zenmap zip zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" CURL_SSL="libressl" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard evdev synaptics" KERNEL="linux" L10N="en" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer pdfimport" LINGUAS="en" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python3_5 python2_7" RUBY_TARGETS="ruby22" SANE_BACKENDS="epson" USERLAND="GNU" VIDEO_CARDS="radeon radeonsi amdgpu" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
If you can get a backtrace from the NetworkChangeNo process, that would be helpful.
(In reply to Mike Gilbert from comment #1) > If you can get a backtrace from the NetworkChangeNo process, that would be > helpful. That's what gdb says for the core: #0 0x0000000915da7b82 in __start_google_malloc () #1 0x0000000910651e97 in operator new(unsigned long, std::nothrow_t const&) () #2 0x000000091069b99d in void std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::_M_emplace_back_aux<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () #3 0x0000000911f073b9 in ?? () #4 0x0000000911f0a40a in ?? () #5 0x00000009121ad74b in ?? () #6 0x00000009121ac5f6 in ?? () #7 0x0000000912167040 in ?? () #8 0x0000000911f594f5 in ?? () #9 0x0000000911f5561d in ?? () #10 0x0000036998958434 in start_thread (arg=0x36981529700) at pthread_create.c:333 #11 0x000003698e8933fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
Works on Gentoo Hardened w/o segfaults. But is not useful. Lags and freezes. Dont sure to open a new bug.
(In reply to reagentoo from comment #3) > Works on Gentoo Hardened w/o segfaults. But is not useful. Lags and freezes. > Dont sure to open a new bug. I am on a gentoo hardened kernel (Pax + Grsec), but it segfaults for me. What are the differences between my hardened and your hardened? Moreover, the segfault does not seem to be related to the hardened kernel, at least not directly: It is really a segfault, not a kill by Pax due to some Pax protection violation.
(In reply to Klaus Kusche from comment #4) > I am on a gentoo hardened kernel (Pax + Grsec), but it segfaults for me. > What are the differences between my hardened and your hardened? Yes. I have gentoo hardened Pax + Grsec (with mprotect enabled). My chromuim--54.0.2840.16 started under firejail works, but with freezes and LAGS. Older versions of the chromuim works normal. I will test to launch 54 without firejail on other machine later. [ebuild R ] www-client/chromium-54.0.2840.16 [53.0.2785.101] USE="(gn) hangouts (pic) proprietary-codecs suid system-ffmpeg tcmalloc -cups -custom-cflags -gnome -gnome-keyring (-gtk3) -kerberos (-neon) (-pulseaudio) (-selinux) {-test} -widevine" L10N="ru -am -ar -bg -bn -ca -cs -da -de -el -en-GB -es -es-419 -et -fa -fi -fil -fr -gu -he -hi -hr -hu -id -it -ja -kn -ko -lt -lv -ml -mr -ms -nb -nl -pl -pt-BR -pt-PT -ro -sk -sl -sr -sv -sw -ta -te -th -tr -uk -vi -zh-CN -zh-TW" [ebuild R ] www-plugins/chrome-binary-plugins-54.0.2840.16_beta [53.0.2785.101] USE="widevine" [ebuild R ] media-libs/libvpx-1.6.0 USE="svc threads -doc -postproc -static-libs {-test}" CPU_FLAGS_X86="avx avx2 mmx sse sse2 sse3 sse4_1 ssse3"
(In reply to reagentoo from comment #5) > (In reply to Klaus Kusche from comment #4) > > I am on a gentoo hardened kernel (Pax + Grsec), but it segfaults for me. > > What are the differences between my hardened and your hardened? > > Yes. I have gentoo hardened Pax + Grsec (with mprotect enabled). My > chromuim--54.0.2840.16 started under firejail works, but with freezes and > LAGS. Older versions of the chromuim works normal. I will test to launch 54 > without firejail on other machine later. The one segfaulting here was 54.0.2840.14. mprotect (and randomization) are also enabled here, but are not the reason for the segfault. No firejail here, no chrome-binary-plugins here, hangouts off, cups on, gnome and gnome-keyring on here as forced by .14. 53.0.2785.* works fine here, but with -gn -gnome -gnome-keyring, so the gnome part could be the problem. Are you using the same toolchain? (gcc 5.4.0 here) > [ebuild R ] www-client/chromium-54.0.2840.16 [53.0.2785.101] USE="(gn) > hangouts (pic) proprietary-codecs suid system-ffmpeg tcmalloc -cups > -custom-cflags -gnome -gnome-keyring (-gtk3) -kerberos (-neon) (-pulseaudio) > (-selinux) {-test} -widevine" L10N="ru -am -ar -bg -bn -ca -cs -da -de -el > -en-GB -es -es-419 -et -fa -fi -fil -fr -gu -he -hi -hr -hu -id -it -ja -kn > -ko -lt -lv -ml -mr -ms -nb -nl -pl -pt-BR -pt-PT -ro -sk -sl -sr -sv -sw > -ta -te -th -tr -uk -vi -zh-CN -zh-TW" > > > [ebuild R ] www-plugins/chrome-binary-plugins-54.0.2840.16_beta > [53.0.2785.101] USE="widevine" > [ebuild R ] media-libs/libvpx-1.6.0 USE="svc threads -doc -postproc > -static-libs {-test}" CPU_FLAGS_X86="avx avx2 mmx sse sse2 sse3 sse4_1 ssse3"
(In reply to Klaus Kusche from comment #6) > Are you using the same toolchain? (gcc 5.4.0 here) Yes. sys-devel/gcc-5.4.0. > so the gnome part could be the problem. gnome flag disabled.
this is valid for regular (non-hardened) as well. and with chromium-55* too. [ 5527.167911] NetworkChangeNo[15308]: segfault at ffffc3657d4884d6 ip 0000564dd9eb2512 sp 00007fb2acea67e0 error 5 in chrome[564dd3bea000+7f36000]
(gdb) run Starting program: /usr/lib64/chromium-browser/chrome [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". [New Thread 0x7fffe34b6700 (LWP 16131)] [New Thread 0x7fffe2cb5700 (LWP 16137)] [New Thread 0x7fffe2184700 (LWP 16138)] [New Thread 0x7fffe1983700 (LWP 16139)] Thread 4 "NetworkChangeNo" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffe2184700 (LWP 16138)] 0x000055555b81c512 in __start_google_malloc () (gdb)
(gdb) run Starting program: /usr/lib64/chromium-browser/chrome [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". [New Thread 0x7fffe34b6700 (LWP 16131)] [New Thread 0x7fffe2cb5700 (LWP 16137)] [New Thread 0x7fffe2184700 (LWP 16138)] [New Thread 0x7fffe1983700 (LWP 16139)] Thread 4 "NetworkChangeNo" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffe2184700 (LWP 16138)] 0x000055555b81c512 in __start_google_malloc () (gdb) bt #0 0x000055555b81c512 in __start_google_malloc () #1 0x00005555560549d7 in operator new(unsigned long, std::nothrow_t const&) () #2 0x00005555560ddfed in void std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::_M_emplace_back_aux<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () #3 0x00005555579c3b29 in ?? () #4 0x00005555579c6b8a in ?? () #5 0x0000555557f0915b in ?? () #6 0x0000555557f08006 in ?? () #7 0x0000555557ec2990 in ?? () #8 0x0000555557a15d45 in ?? () #9 0x0000555557a11e5d in ?? () #10 0x00007ffff7bc4865 in start_thread () from /lib64/libpthread.so.0 #11 0x00007fffee640fdd in clone () from /lib64/libc.so.6 (gdb)
above gdb is for chromium-55.0.2853.0
[ebuild R #] www-client/chromium-55.0.2853.0::gentoo USE="custom-cflags (gn) hangouts (pic) proprietary-codecs suid system-ffmpeg tcmalloc -cups -gnome -gnome-keyring (-gtk3) -kerberos (-neon) -pulseaudio (-selinux) {-test} -widevine" L10N="ru uk -am -ar -bg -bn -ca -cs -da -de -el -en-GB -es -es-419 -et -fa -fi -fil -fr -gu -he -hi -hr -hu -id -it -ja -kn -ko -lt -lv -ml -mr -ms -nb -nl -pl -pt-BR -pt-PT -ro -sk -sl -sr -sv -sw -ta -te -th -tr -vi -zh-CN -zh-TW" 0 KiB
(In reply to Oleg from comment #12) > [ebuild R #] www-client/chromium-55.0.2853.0::gentoo USE="custom-cflags > (gn) hangouts (pic) proprietary-codecs suid system-ffmpeg tcmalloc -cups > -gnome -gnome-keyring (-gtk3) -kerberos (-neon) -pulseaudio (-selinux) > {-test} -widevine" L10N="ru uk -am -ar -bg -bn -ca -cs -da -de -el -en-GB > -es -es-419 -et -fa -fi -fil -fr -gu -he -hi -hr -hu -id -it -ja -kn -ko -lt > -lv -ml -mr -ms -nb -nl -pl -pt-BR -pt-PT -ro -sk -sl -sr -sv -sw -ta -te > -th -tr -vi -zh-CN -zh-TW" 0 KiB Ok, so it's not related to hardened, and it's not related to the gnome or cups USE flags. I think the next things to try would be -tcmalloc and/or -suid. As the sigsegv hits in some malloc function, tcmalloc would perhaps be the first guess.
I am also seeing this with 54.* and 55.*, both with gcc 5.4 and 6.2. The segfault backtrace is the same as above and disabling tcmalloc will indeed make chromium start normally again -- at least with 55.*, haven't tried 54.*, but I don't think it will make any difference. The binary chrome packages work just fine, though.
Same error here, not on hardened. Currently rebuilding 54.x with -tcmalloc enforced. +tcmalloc is the default.
Given the backtrace, it likely is related to tcmalloc. However, I do not experience any segfault with tcmalloc enabled.
(In reply to Mike Gilbert from comment #16) > Given the backtrace, it likely is related to tcmalloc. However, I do not > experience any segfault with tcmalloc enabled. Works fine here but some extensions (Adblock Plus and Tampermonkey) acted up on re-launch. Clearing cache and restarting worked.
So, after debugging this most of the day (with each re-compile taking around an hour), here my surprising results, which will most likely also fix bug #591938: The problem is the use of the system libevent. If one uses libevent 2.0.22-r2 together with tcmalloc, chromium >=54 will crash on start. This situation will arise even if you are on ~arch but have firefox installed with system-libevent set and thus get a blocker on >=libevent-2.1 ... and probably decided to forgo installation of 2.1 for a while. ;-) If you have libevent 2.1.5-r4 installed, chromium will start fine but behave very erratically, from pages that stall and eventually crash the tab ("aw snap...") to other pages just working absolutely fine. If, on the other hand, you use the bundled libevent, everything is peachy again, and there are no problems whatsoever and chromium-54 is as stable as it has ever been (with tcmalloc enabled). While I was looking through the sources, I also checked what libevent version chromium bundles: 1.4.15. There haven't been any changes to the sources between 53 and 55. But they changed the build from a source_set to a static lib with 54. And the bundled libevent is at least built for gn. I haven't had the time to dive deeper into why this happens and what the root cause is, but at least we have it nailed down to libevent and can workaround it by, for now, only using the bundled libevent. My suggestion at this point: The introduction of a masked and unset system-libevent use flag. That way, the p.mask of chromium can be lifted again.
One more thought: This might also be of relevance for Gentoo's Qt project since qtwebengine is based on chromium and I have no idea what upstream/downstream is building it with.
(In reply to Matthias Dahl from comment #18) I think you just saved me a lot of work. Thank you very much! I'll try to update the ebuilds tonight.
rebuilding of latest 55.0.2859.0 with bundled libevent does not fix the segfault. It is the same: NetworkChangeNo[11213]: segfault at ffffd5d5811c2082 ip 0000558444759512 sp 00007fd0ed55d7e0 error 5 in chrome[55843e491000+7f36000]
ai don't think libevent is to blame. gdb bcktrace deals with malloc rather. so the tcmalloc is likely real problem. haven't had any debug yet
(In reply to Oleg from comment #22) The backtrace is missing symbol information for most of the stack; we have no way of knowing what other code is involved there.
(In reply to Mike Gilbert from comment #20) > (In reply to Matthias Dahl from comment #18) > > I think you just saved me a lot of work. Thank you very much! You are very welcome. I meant to further debug this (wouldn't be the first time getting my hands dirty on the chromium intestinals to fix a bug) but this whole bug triangulation cost quite a bit of time and I needed to do other things as well. (In reply to Oleg from comment #21) > rebuilding of latest 55.0.2859.0 with bundled libevent does not fix the > segfault. It is the same: > NetworkChangeNo[11213]: segfault at ffffd5d5811c2082 ip 0000558444759512 sp > 00007fd0ed55d7e0 error 5 in chrome[55843e491000+7f36000] That is rather strange since I tested quite broadly. Just to make sure, I am building 55.0.2859.0 now w/ debug symbols and see if I can reproduce this. If I can, we finally have a valid bt and can work from there. I will report back later...
Sorry, but 55.0.2859.0 is stable here w/ the current ebuild that uses the bundled libevent. I have yet to see any crash and/or strange behavior at all. In fact, I am writing this from said version, listening to Google Play Music and have quite a few tabs open. Is there anything out of the ordinary on your end? Could you please post an emerge --info? And you are absolutely sure you tried 55.0.2859.0 with the current ebuild?
*** Bug 591938 has been marked as a duplicate of this bug. ***
This is more a workaround than fix - just used bundled libevent for now. A fix would be way more involved - there is a TODO in the ebuild.
