Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 908979 - www-client/firefox segfault with elf-hack
Summary: www-client/firefox segfault with elf-hack
Status: RESOLVED UPSTREAM
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Mozilla Gentoo Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-06-21 22:28 UTC by stefan11111
Modified: 2023-07-18 03:04 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
whatsap web strace (strace_firefox,8.57 KB, text/plain)
2023-07-04 18:29 UTC, stefan11111
Details
FF 115.0 coredumpctl debug (coredumpctl_debug.log,11.78 KB, text/x-log)
2023-07-09 00:34 UTC, Ivan
Details

Note You need to log in before you can comment on or make changes to this bug.
Description stefan11111 2023-06-21 22:28:16 UTC
This has been an issue for quite some time, but I only recently pinpointed the cause.
https://forums.gentoo.org/viewtopic-t-1163310.html
https://forums.gentoo.org/viewtopic-t-1163906.html

Librewolf and firefox both build and run fine when built with clang.
This happens because of this part of the ebuild:

mozconfig_add_options_ac 'elf-hack is broken when using Clang' --disable-elf-hack

However, as the message implies, this configure option isn't set when librewolf and firefox are built with gcc. This causes segfaults when visiting some sites, one of them being whatsapp web. The result of this is that the tabs crash when visiting these sites.
Here is the output from dmesg:

[369568.560857] MediaPD~oder #2[28093]: segfault at 7f1905fff000 ip 00007f190536a23a sp 00007f1903ffa560 error 6 in libmozavcodec.so[7f1905348000+15c000] likely on CPU 2 (core 2, socket 0)
[369568.560868] Code: 8b 44 24 68 b9 40 02 00 00 29 d9 48 63 db 49 8d 84 06 d0 16 00 00 48 63 c9 48 01 d8 48 c1 e1 02 49 8d 54 85 00 31 c0 48 89 d7 <f3> aa 41 8b 85 60 04 00 00 41 8b 95 68 04 00 00 89 c1 f7 d9 41 39

The way I fixed this for librewolf is by modifying the ebuild:

# diff -u librewolf-114.0.1_p2.ebuild.bak librewolf-114.0.1_p2.ebuild
--- librewolf-114.0.1_p2.ebuild.bak     2023-06-22 01:18:09.520842971 +0300
+++ librewolf-114.0.1_p2.ebuild 2023-06-21 21:41:54.138977068 +0300
@@ -1049,6 +1049,8 @@
                fi
        fi

+       mozconfig_add_options_ac 'elf-hack is broken' --disable-elf-hack
+
        if use clang ; then
                # https://bugzilla.mozilla.org/show_bug.cgi?id=1482204
                # https://bugzilla.mozilla.org/show_bug.cgi?id=1483822
@@ -1062,9 +1064,9 @@
                        disable_elf_hack=yes
                fi

-               if [[ -n ${disable_elf_hack} ]] ; then
-                       mozconfig_add_options_ac 'elf-hack is broken when using Clang' --disable-elf-hack
-               fi
+#              if [[ -n ${disable_elf_hack} ]] ; then
+#                      mozconfig_add_options_ac 'elf-hack is broken when using Clang' --disable-elf-hack
+#              fi
        elif tc-is-gcc ; then
                if ver_test $(gcc-fullversion) -ge 10 ; then
                        einfo "Forcing -fno-tree-loop-vectorize to workaround GCC bug, see bug 758446 ..."

Where should I report this so that is also gets fixed for librewolf?
Comment 1 stefan11111 2023-06-22 08:33:11 UTC
Seems like this was opened on gitlab:
https://gitlab.com/librewolf-community/browser/gentoo/-/issues/61

Can someone here tell them about this bug?
I can't make an account on gitlab without providing some payment info for verification, but I don't have that.
Comment 2 Joonas Niilola gentoo-dev 2023-06-22 13:10:15 UTC
(In reply to stefan11111 from comment #1)
> Seems like this was opened on gitlab:
> https://gitlab.com/librewolf-community/browser/gentoo/-/issues/61
> 
> Can someone here tell them about this bug?
> I can't make an account on gitlab without providing some payment info for
> verification, but I don't have that.

Can you log in via Github then? IIRC Gitlab allows that. 

In the end they sync changes from ::gentoo's firefox ebuilds to that librewolf overlay so when it gets fixed in ::gentoo shouldn't take long until librewolf is also fixed. 

elf-hack is pretty nice feature and some could say the only reason to build with gcc. I'll have to search around if this is a known issue, or something unique to your system?
Comment 3 stefan11111 2023-06-22 13:26:59 UTC
(In reply to Joonas Niilola from comment #2)
> (In reply to stefan11111 from comment #1)
> > Seems like this was opened on gitlab:
> > https://gitlab.com/librewolf-community/browser/gentoo/-/issues/61
> > 
> > Can someone here tell them about this bug?
> > I can't make an account on gitlab without providing some payment info for
> > verification, but I don't have that.
> 
> Can you log in via Github then? IIRC Gitlab allows that. 

No, it still asks me for payment details.

> In the end they sync changes from ::gentoo's firefox ebuilds to that
> librewolf overlay so when it gets fixed in ::gentoo shouldn't take long
> until librewolf is also fixed. 

It that case, I can just wait.

> elf-hack is pretty nice feature and some could say the only reason to build
> with gcc. I'll have to search around if this is a known issue, or something
> unique to your system?

I can wait.
I first found this issue with librewolf 112, and also tested with firefox 113.
The former was broken and the latter was working. I haven't tried librewolf 113, but I think it works.
For now I added a patched ebuild to my overlay.
Comment 4 Joonas Niilola gentoo-dev 2023-06-22 15:23:01 UTC
Btw what's your configuration like? Are you using mold with gcc? And possibly -lto -pgo?

I figure elf-hack with mold linker is broken too, but mold can only be used with gcc when -lto -pgo is specified.
Comment 5 stefan11111 2023-06-22 18:13:48 UTC
(In reply to Joonas Niilola from comment #4)
> Btw what's your configuration like? Are you using mold with gcc? And
> possibly -lto -pgo?
> 
> I figure elf-hack with mold linker is broken too, but mold can only be used
> with gcc when -lto -pgo is specified.

I use regular ld, with lto and pgo.

[ebuild   R   ] www-client/librewolf-114.0.1_p2  USE="X eme-free gmp-autoupdate hardened jumbo-build lto openh264 pgo pulseaudio system-av1 system-harfbuzz system-icu system-jpeg system-libevent system-libvpx system-png system-webp -clang
-dbus -debug -geckodriver -hwaccel -jack -libproxy -screencast (-selinux) -sndio (-system-python-libs) -telemetry -valgrind -wayland -wifi" L10N="-ach -af -an -ar -ast -az -be -bg -bn -br -bs -ca -ca-valencia -cak -cs -cy -da -de -dsb -el
-en-CA -en-GB -eo -es-AR -es-CL -es-ES -es-MX -et -eu -fa -ff -fi -fr -fur -fy -ga -gd -gl -gn -gu -he -hi -hr -hsb -hu -hy -ia -id -is -it -ja -ka -kab -kk -km -kn -ko -lij -lt -lv -mk -mr -ms -my -nb -ne -nl -nn -oc -pa -pl -pt-BR -pt-PT -rm -ro -ru -sc -sco -si -sk -sl -son -sq -sr -sv -szl -ta -te -th -tl -tr -trs -uk -ur -uz -vi -xh -zh-CN -zh-TW"


Portage 3.0.49 (python 3.11.4-final-0, default/linux/amd64/17.1, gcc-13, glibc-2.37-r3, 6.3.8-gentoo x86_64)
=================================================================
System uname: Linux-6.3.8-gentoo-x86_64-Intel-R-_Core-TM-_i5-7400_CPU_@_3.00GHz-with-glibc2.37
KiB Mem:     8123296 total,   1395368 free
KiB Swap:   16777212 total,  16777212 free
Timestamp of repository gentoo: Thu, 22 Jun 2023 00:00:01 +0000
Head commit of repository gentoo: cc4916c484d7cc796bed82a857f02f4311b0258c
Head commit of repository gentoo-static: b220537760d6a06378c2142e216fc6644183ce19

Timestamp of repository libressl: Wed, 21 Jun 2023 16:31:56 +0000
Head commit of repository libressl: 454e8c1d337e625437db0127c9803599a7f0e93d

Timestamp of repository pg_overlay: Wed, 21 Jun 2023 10:31:53 +0000
Head commit of repository pg_overlay: 52d131d0dec598e7c76c26fd477e455c62a80775

Timestamp of repository steam-overlay: Sat, 17 Jun 2023 10:17:28 +0000
Head commit of repository steam-overlay: 6d66cb6aaaeceafa8b4df65009bc5f851e6e7b01

Head commit of repository stefan_overlay: cb7a3c90cd3e80388954dcf8f27e2f1f1d43412f

Head commit of repository librewolf: b6e7ff8733a4c21a1a5541690713328ba79f89f3

sh bash 5.2_p15-r3
ld GNU ld (Gentoo 2.40 p5) 2.40.0
app-misc/pax-utils:        1.3.7::gentoo
app-shells/bash:           5.2_p15-r3::gentoo
dev-lang/perl:             5.36.1-r2::gentoo
dev-lang/python:           3.11.4::libressl
dev-lang/rust:             1.69.0-r1::libressl
dev-util/cmake:            3.26.4-r1::gentoo
dev-util/meson:            1.1.1::gentoo
sys-apps/baselayout:       2.13-r1::gentoo
sys-apps/openrc:           0.47.1::gentoo
sys-apps/sandbox:          2.32::gentoo
sys-devel/autoconf:        2.13-r8::gentoo, 2.71-r6::gentoo
sys-devel/automake:        1.16.5-r1::gentoo
sys-devel/binutils:        2.40-r5::gentoo
sys-devel/binutils-config: 5.5::gentoo
sys-devel/clang:           16.0.6::pg_overlay
sys-devel/gcc:             13.1.1_p20230527::gentoo
sys-devel/gcc-config:      2.11::gentoo
sys-devel/libtool:         2.4.7-r1::gentoo
sys-devel/llvm:            16.0.6::gentoo
sys-devel/make:            4.4.1-r1::gentoo
sys-kernel/linux-headers:  6.3::gentoo (virtual/os-headers)
sys-libs/glibc:            2.37-r3::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000
    volatile: False
    sync-rsync-verify-jobs: 1
    sync-rsync-verify-max-age: 24
    sync-rsync-extra-opts: 
    sync-rsync-verify-metamanifest: yes

gentoo-static
    location: /var/db/repos/gentoo-static
    sync-type: git
    sync-uri: https://github.com/NeddySeagoon/gentoo-static.git
    masters: gentoo
    volatile: False

libressl
    location: /var/db/repos/libressl
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/libressl.git
    masters: gentoo
    volatile: False

pg_overlay
    location: /var/db/repos/pg_overlay
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/pg_overlay.git
    masters: gentoo
    volatile: False

steam-overlay
    location: /var/db/repos/steam-overlay
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/steam-overlay.git
    masters: gentoo
    volatile: False

stefan_overlay
    location: /var/db/repos/stefan_overlay
    sync-type: git
    sync-uri: https://github.com/stefan11111/stefan_overlay.git
    masters: gentoo
    volatile: False

librewolf
    location: /var/db/repos/librewolf
    sync-type: git
    sync-uri: https://gitlab.com/librewolf-community/browser/gentoo.git
    masters: gentoo
    priority: 50
    volatile: False

ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=native -flto=4"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe -march=native -flto=4"
DISTDIR="/var/cache/distfiles"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-O2 -pipe -march=native -flto=4"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
FFLAGS="-O2 -pipe -march=native -flto=4"
GENTOO_MIRRORS="http://ftp.romnet.org/gentoo/ http://mirrors.xservers.ro/gentoo/"
INSTALL_MASK="/etc/systemd /lib/systemd /usr/lib/systemd /usr/lib/modules-load.d *udev* /usr/lib/tmpfiles.d *tmpfiles* /var/lib/dbus /usr/bin/gdbus /lib/udev /usr/share/icons /usr/share/applications /usr/share/gtk-3.0/emoji"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LEX="flex"
MAKEOPTS="-j4"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
SHELL="/bin/bash"
USE="X alsa amd64 asm git grub ipv6 jit jumbo-build libressl lto man native-symlinks olde-gentoo pgo rsync-verify samba split-usr ssl test-rust threads verify-sig" ABI_X86="64" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sse sse2 sse3 sse4_1 sse4_2 ssse3" CURL_SSL="libressl" ELIBC="glibc" FFTOOLS="aviocat cws2fws ffescape ffeval ffhash fourcc2pixfmt graph2dot ismindex pktdumper qt-faststart sidxindex trasher" GRUB_PLATFORMS="efi-64" KERNEL="linux" LUA_SINGLE_TARGET="luajit" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_11" RUBY_TARGETS="ruby31" VIDEO_CARDS="nvidia"
Unset:  ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
Comment 6 stefan11111 2023-06-22 20:22:56 UTC
After asking Neddy on the forums, I opened a pr on github.
https://github.com/gentoo/gentoo/pull/31578
Comment 7 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-06-23 01:10:05 UTC
A full, untruncated backtrace would be helpful. I suspect this might be related to bug 908412 given the nature of it (it's not as simple as a "normal" SIGILL).
Comment 8 stefan11111 2023-06-23 09:31:01 UTC
(In reply to Sam James from comment #7)
> A full, untruncated backtrace would be helpful. I suspect this might be
> related to bug 908412 given the nature of it (it's not as simple as a
> "normal" SIGILL).

I will assume from now on that firefox and librewolf are interchangeable for this purpose. I have librewolf 114 built with gcc without elf-hack and firefox 114 built with gcc with elf-hack. more rebuilds will take 3 hours each.
Just tried it with firefox 114. Things work fine when run under strace.
Still broken otherwise.
Comment 9 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-06-23 09:32:22 UTC
Backtrace with it broken then please? ALso, are you using LTO for these gcc builds when it is broken with elf-hack?
Comment 10 stefan11111 2023-06-23 09:49:32 UTC
(In reply to Sam James from comment #9)
> Backtrace with it broken then please?
I ran strace firefox > backtrace, which I then realized is not the way to do this. I later removed the old backtrace file and ran strace -o backtrace firefox. I got the whatsapp web tab to crash once. I will post the backtrace, but it's 200k lines long.
ALso, are you using LTO for these gcc
> builds when it is broken with elf-hack?
Yes. I posted here the USE flags I build librewolf with. Firefox is built with the same USE flags.
Comment 11 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-06-23 09:51:33 UTC
(In reply to stefan11111 from comment #10)
> (In reply to Sam James from comment #9)
> > Backtrace with it broken then please?
> I ran strace firefox > backtrace, which I then realized is not the way to do
> this. I later removed the old backtrace file and ran strace -o backtrace
> firefox. I got the whatsapp web tab to crash once. I will post the
> backtrace, but it's 200k lines long.

A backtrace is from gdb, not strace.

See https://wiki.gentoo.org/wiki/Debugging#Per-package.

> ALso, are you using LTO for these gcc
> > builds when it is broken with elf-hack?
> Yes. I posted here the USE flags I build librewolf with. Firefox is built
> with the same USE flags.

Then it's probably the ODR issue I linked, I suppose.
Comment 12 stefan11111 2023-06-23 10:23:31 UTC
(In reply to Sam James from comment #11)
> (In reply to stefan11111 from comment #10)
> > (In reply to Sam James from comment #9)
> > > Backtrace with it broken then please?
> > I ran strace firefox > backtrace, which I then realized is not the way to do
> > this. I later removed the old backtrace file and ran strace -o backtrace
> > firefox. I got the whatsapp web tab to crash once. I will post the
> > backtrace, but it's 200k lines long.
> 
> A backtrace is from gdb, not strace.
> 
> See https://wiki.gentoo.org/wiki/Debugging#Per-package.

Will start rebuilding firefox according to the link you posted.
Gdb doesn't start firefox as-is.
$ gdb firefox
GNU gdb (Gentoo 13.2 vanilla) 13.2
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
"/usr/bin/firefox": not in executable format: file format not recognized
(gdb) q


> > ALso, are you using LTO for these gcc
> > > builds when it is broken with elf-hack?
> > Yes. I posted here the USE flags I build librewolf with. Firefox is built
> > with the same USE flags.
> 
> Then it's probably the ODR issue I linked, I suppose.

I will also try this without lto.
Comment 13 stefan11111 2023-06-23 16:50:32 UTC
(In reply to stefan11111 from comment #12)
> Will start rebuilding firefox according to the link you posted.

Killed it at 302:36.50. It was using 7GB ram and 10GB swap with MAKEOPTS="-j1".
There is no way I can build this. Even if I let it run, It would probably just get killed for running out of memory.
Comment 14 Sven B. 2023-06-23 20:07:05 UTC
> For help, type "help".
> Type "apropos word" to search for commands related to "word"...
> "/usr/bin/firefox": not in executable format: file format not recognized
> (gdb) q
> 

/usr/bin/firefox is a wrapper so try with /usr/lib64/firefox/firefox
Comment 15 stefan11111 2023-06-23 20:42:35 UTC
(In reply to Sven B. from comment #14)
> > For help, type "help".
> > Type "apropos word" to search for commands related to "word"...
> > "/usr/bin/firefox": not in executable format: file format not recognized
> > (gdb) q
> > 
> 
> /usr/bin/firefox is a wrapper so try with /usr/lib64/firefox/firefox

$ gdb /usr/lib64/firefox/firefox
GNU gdb (Gentoo 13.2 vanilla) 13.2
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/lib64/firefox/firefox...
(No debugging symbols found in /usr/lib64/firefox/firefox)
(gdb)

Still nothing pops out
Comment 16 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-06-23 21:32:10 UTC
You have to enter 'r' then 'bt'.
Comment 17 stefan11111 2023-06-23 21:49:14 UTC
Thread 4 "IPC I/O Parent" received signal SIGPIPE, Broken pipe.
[Switching to Thread 0x7fffebbc16c0 (LWP 19940)]
0x00007ffff7b8977d in sendmsg () from /lib64/libc.so.6
(gdb) bt
#0  0x00007ffff7b8977d in sendmsg () from /lib64/libc.so.6
#1  0x00007ffff21ea8a5 in ?? () from /usr/lib64/firefox/libxul.so
#2  0x00007ffff21eb184 in ?? () from /usr/lib64/firefox/libxul.so
#3  0x00007ffff2d66e2e in ?? () from /usr/lib64/firefox/libxul.so
#4  0x00007ffff2d66c3a in ?? () from /usr/lib64/firefox/libxul.so
#5  0x00007ffff249ccff in ?? () from /usr/lib64/firefox/libxul.so
#6  0x00007ffff21ec033 in ?? () from /usr/lib64/firefox/libxul.so
#7  0x00007ffff249c406 in ?? () from /usr/lib64/firefox/libxul.so
#8  0x00007fffee4ef248 in ?? () from /usr/lib64/libevent-2.1.so.7
#9  0x00007fffee4f058f in event_base_loop () from /usr/lib64/libevent-2.1.so.7
#10 0x00007ffff249ade8 in ?? () from /usr/lib64/firefox/libxul.so
#11 0x00007ffff249ad45 in ?? () from /usr/lib64/firefox/libxul.so
#12 0x00007ffff2491e6c in ?? () from /usr/lib64/firefox/libxul.so
#13 0x00007ffff2491d77 in ?? () from /usr/lib64/firefox/libxul.so
#14 0x00007ffff7b06db9 in ?? () from /lib64/libc.so.6
#15 0x00007ffff7b877bc in ?? () from /lib64/libc.so.6
Comment 18 stefan11111 2023-06-23 22:00:56 UTC
(In reply to stefan11111 from comment #17)
> Thread 4 "IPC I/O Parent" received signal SIGPIPE, Broken pipe.
> [Switching to Thread 0x7fffebbc16c0 (LWP 19940)]
> 0x00007ffff7b8977d in sendmsg () from /lib64/libc.so.6
> (gdb) bt
> #0  0x00007ffff7b8977d in sendmsg () from /lib64/libc.so.6
> #1  0x00007ffff21ea8a5 in ?? () from /usr/lib64/firefox/libxul.so
> #2  0x00007ffff21eb184 in ?? () from /usr/lib64/firefox/libxul.so
> #3  0x00007ffff2d66e2e in ?? () from /usr/lib64/firefox/libxul.so
> #4  0x00007ffff2d66c3a in ?? () from /usr/lib64/firefox/libxul.so
> #5  0x00007ffff249ccff in ?? () from /usr/lib64/firefox/libxul.so
> #6  0x00007ffff21ec033 in ?? () from /usr/lib64/firefox/libxul.so
> #7  0x00007ffff249c406 in ?? () from /usr/lib64/firefox/libxul.so
> #8  0x00007fffee4ef248 in ?? () from /usr/lib64/libevent-2.1.so.7
> #9  0x00007fffee4f058f in event_base_loop () from
> /usr/lib64/libevent-2.1.so.7
> #10 0x00007ffff249ade8 in ?? () from /usr/lib64/firefox/libxul.so
> #11 0x00007ffff249ad45 in ?? () from /usr/lib64/firefox/libxul.so
> #12 0x00007ffff2491e6c in ?? () from /usr/lib64/firefox/libxul.so
> #13 0x00007ffff2491d77 in ?? () from /usr/lib64/firefox/libxul.so
> #14 0x00007ffff7b06db9 in ?? () from /lib64/libc.so.6
> #15 0x00007ffff7b877bc in ?? () from /lib64/libc.so.6

When I get this, firefox freezes. This doesn't happen when not ran under gdb.

I also got it to segfault and just crash the tab, but gdb didn't stop. If I exit firefox myself and then press bt, it says no backtrace.
I do get some output from gdb though:

[Parent 21528, IPC I/O Parent] WARNING: process 21701 exited on signal 11: file /var/tmp/portage/www-client/firefox-114.0/work/firefox-114.0/ipc/chromium/src/base/process_util_posix.cc:264
JavaScript error: resource://gre/modules/XULStore.sys.mjs, line 60: Error: Can't find profile directory.
Comment 19 stefan11111 2023-06-23 22:07:06 UTC
Firefox with USE="-lto -pgo" just build.
Now firefox just sometimes segfaults, about half the time before I manage to sign in.
After signing is, it mostly doesn't segfault.
Comment 20 stefan11111 2023-06-23 22:10:08 UTC
Got it to freeze again under gdb.
Seems like it has some debugging symbols:

Thread 4 "IPC I/O Parent" received signal SIGPIPE, Broken pipe.
[Switching to Thread 0x7fffe96546c0 (LWP 29719)]
0x00007ffff7c6377d in sendmsg () from /lib64/libc.so.6
(gdb) bt
#0  0x00007ffff7c6377d in sendmsg () from /lib64/libc.so.6
#1  0x00007fffee794ce2 in IPC::(anonymous namespace)::corrected_sendmsg (flags=64, message=0x7fffe964f200,
socket=<optimized out>)
    at /var/tmp/portage/www-client/firefox-114.0/work/firefox-114.0/ipc/chromium/src/chrome/common/ipc_channel_posix.cc:130
#2  IPC::Channel::ChannelImpl::ProcessOutgoingMessages (this=this@entry=0x7fffc644ab80) at /var/tmp/portage/www-client/firefox-114.0/work/firefox-114.0/ipc/chromium/src/chrome/common/ipc_channel_posix.cc:687
#3  0x00007fffee795602 in IPC::Channel::ChannelImpl::Send (this=0x7fffc644ab80, message=...) at /var/tmp/portage/www-client/firefox-114.0/work/firefox-114.0/ipc/chromium/src/chrome/common/ipc_channel_posix.cc:821
#4  0x00007fffee79564f in IPC::Channel::Send (this=<optimized out>, message=...) at /var/tmp/portage/www-client/firefox-114.0/work/firefox-114.0/ipc/chromium/src/chrome/common/ipc_channel_posix.cc:1242
#5  0x00007fffee7cf454 in mozilla::ipc::NodeChannel::SendMessage (this=this@entry=0x7fffc736cf60, aMessage=...) at /var/tmp/portage/www-client/firefox-114.0/work/firefox-114.0/ipc/glue/NodeChannel.cpp:216
#6  0x00007fffee7cf6ca in mozilla::ipc::NodeChannel::Introduce (this=this@entry=0x7fffc736cf60, aIntroduction=...) at /var/tmp/portage/www-client/firefox-114.0/work/firefox-114.0/ipc/glue/NodeChannel.cpp:169
#7  0x00007fffee7d53e9 in mozilla::ipc::NodeController::OnRequestIntroduction (this=<optimized out>, aFromNode=..., aName=...) at /var/tmp/portage/www-client/firefox-114.0/work/firefox-114.0/ipc/glue/NodeController.cpp:693
#8  0x00007fffee7d687f in mozilla::ipc::NodeChannel::OnMessageReceived (this=0x7fffc9c07f10, aMessage=...)
at /var/tmp/portage/www-client/firefox-114.0/work/firefox-114.0/ipc/glue/NodeChannel.cpp:244
#9  0x00007fffee793db5 in IPC::Channel::ChannelImpl::ProcessIncomingMessages (this=0x7fffc9dc53a0) at /var/tmp/portage/www-client/firefox-114.0/work/firefox-114.0/ipc/chromium/src/chrome/common/ipc_channel_posix.cc:552
#10 0x00007fffee794b1d in IPC::Channel::ChannelImpl::OnFileCanReadWithoutBlocking (this=0x7fffc9dc53a0, fd=<optimized out>)
    at /var/tmp/portage/www-client/firefox-114.0/work/firefox-114.0/ipc/chromium/src/chrome/common/ipc_channel_posix.cc:850
#11 0x00007fffee7808fb in base::MessagePumpLibevent::OnLibeventNotification (fd=152, flags=<optimized out>, context=0x7fffc9dc53a0)
    at /var/tmp/portage/www-client/firefox-114.0/work/firefox-114.0/ipc/chromium/src/base/message_pump_libevent.cc:261
#12 0x00007fffebf82248 in ?? () from /usr/lib64/libevent-2.1.so.7
#13 0x00007fffebf8358f in event_base_loop () from /usr/lib64/libevent-2.1.so.7
#14 0x00007fffee783419 in base::MessagePumpLibevent::Run (this=0x7fffe95312c0, delegate=0x7fffe9653c90) at
/var/tmp/portage/www-client/firefox-114.0/work/firefox-114.0/ipc/chromium/src/base/message_pump_libevent.cc:374
#15 0x00007fffee782f1b in MessageLoop::RunInternal (this=0x7fffe9653c90) at /var/tmp/portage/www-client/firefox-114.0/work/firefox_build/dist/include/mozilla/RefPtr.h:319
#16 MessageLoop::RunHandler (this=0x7fffe9653c90) at /var/tmp/portage/www-client/firefox-114.0/work/firefox-114.0/ipc/chromium/src/base/message_loop.cc:362
#17 MessageLoop::Run (this=this@entry=0x7fffe9653c90) at /var/tmp/portage/www-client/firefox-114.0/work/firefox-114.0/ipc/chromium/src/base/message_loop.cc:344
#18 0x00007fffee795841 in base::Thread::ThreadMain (this=0x7fffe954f510) at /var/tmp/portage/www-client/firefox-114.0/work/firefox-114.0/ipc/chromium/src/base/thread.cc:187
#19 0x00007fffee77ecfa in ThreadFunc (closure=<optimized out>) at /var/tmp/portage/www-client/firefox-114.0/work/firefox-114.0/ipc/chromium/src/base/platform_thread_posix.cc:40
#20 0x00007ffff7be0db9 in ?? () from /lib64/libc.so.6
#21 0x00007ffff7c617bc in ?? () from /lib64/libc.so.6
Comment 21 stefan11111 2023-06-26 10:20:05 UTC
Nothing else to say?
Comment 22 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-06-26 15:07:48 UTC
(In reply to stefan11111 from comment #20)
> Got it to freeze again under gdb.
> Seems like it has some debugging symbols:
> 

Thanks, the previous bt wasn't very helpful, but this is better.

> Thread 4 "IPC I/O Parent" received signal SIGPIPE, Broken pipe.
> [Switching to Thread 0x7fffe96546c0 (LWP 29719)]
> 0x00007ffff7c6377d in sendmsg () from /lib64/libc.so.6
> (gdb) bt
> #0  0x00007ffff7c6377d in sendmsg () from /lib64/libc.so.6
> #1  0x00007fffee794ce2 in IPC::(anonymous namespace)::corrected_sendmsg
> (flags=64, message=0x7fffe964f200,
> socket=<optimized out>)

Unfortunately, this is the wrong process. We need it to grab the signal from the dying child process instead.

I see https://firefox-source-docs.mozilla.org/contributing/debugging/debugging_firefox_with_gdb.html but not sure if any of it is relevant here (not read it all).

You could try 'set follow-fork-mode child' before 'r' to see if it helps.
Comment 23 stefan11111 2023-06-26 21:20:26 UTC
> I see
> https://firefox-source-docs.mozilla.org/contributing/debugging/
> debugging_firefox_with_gdb.html but not sure if any of it is relevant here
> (not read it all).
> 
> You could try 'set follow-fork-mode child' before 'r' to see if it helps.

Doesn't work:

stefan@GentooPC ~/github/stefan_overlay $ gdb /usr/lib64/firefox/firefox
GNU gdb (Gentoo 13.2 vanilla) 13.2
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/lib64/firefox/firefox...
Reading symbols from /usr/lib/debug//usr/lib64/firefox/firefox.debug...
(gdb) set follow-fork-mode child
(gdb) r
Starting program: /usr/lib64/firefox/firefox
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[New Thread 0x7ffff77ff6c0 (LWP 28800)]
[Thread 0x7ffff77ff6c0 (LWP 28800) exited]
[Attaching after Thread 0x7ffff7a82480 (LWP 28797) fork to child process 28801]
[New inferior 2 (process 28801)]
[Detaching after fork from parent process 28797]
[Inferior 1 (process 28797) detached]
warning: Target and debugger are in different PID namespaces; thread lists and other data are likely unreli
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[Inferior 2 (process 28801) exited normally]
(gdb) console.error: ({})
JavaScript error: resource://gre/modules/XULStore.sys.mjs, line 60: Error: Can't find profile directory.
console.error: (new TypeError("str is undefined", "resource://gre/modules/components-utils/FilterExpressions.sys.mjs", 109))
console.error: (new TypeError("str is undefined", "resource://gre/modules/components-utils/FilterExpressions.sys.mjs", 109))
bt
No stack.
(gdb) JavaScript error: https://web.whatsapp.com/serviceworker.js, line 43: TypeError: Cache.put: Cache got basic response with bad status 206 while trying to add request https://web.whatsapp.com/notification_2a485d84012c106acef03b527bb54635.mp3
[Parent 28797, IPC I/O Parent] WARNING: process 28964 exited on signal 11: file /var/tmp/portage/www-client/firefox-114.0/work/firefox-114.0/ipc/chromium/src/base/process_util_posix.cc:264
bt
No stack.
(gdb)


I pressed bt twice. The second one was after the segfault.
Comment 24 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-06-29 06:56:13 UTC
Please file a bug upstream (link it here) & seek advice there on how to proceed.
Comment 25 Joonas Niilola gentoo-dev 2023-06-29 07:02:25 UTC
I don't know. I'm glad it works for you without elf-hack, but still I'm unsure whether that's a global or a "you" issue. Then again there seems to be a known case when gcc+lto is being used, even with elf-hack disabled. So I think that needs to be fixed first, and it just _may_ resolve your issue with elf-hack too, since they might be related.
Comment 26 stefan11111 2023-06-29 07:12:57 UTC
(In reply to Sam James from comment #24)
> Please file a bug upstream (link it here) & seek advice there on how to
> proceed.

I open the bug here:
https://bugzilla.mozilla.org/show_bug.cgi?id=1841001
Comment 27 stefan11111 2023-06-29 07:13:55 UTC
(In reply to Joonas Niilola from comment #25)
> I don't know. I'm glad it works for you without elf-hack, but still I'm
> unsure whether that's a global or a "you" issue. Then again there seems to
> be a known case when gcc+lto is being used, even with elf-hack disabled. So
> I think that needs to be fixed first, and it just _may_ resolve your issue
> with elf-hack too, since they might be related.

I rebuilt firefox without lto and pgo. The problem is still there.
All backtraces with debug symbols are without lto and pgo.
Comment 28 NHO 2023-07-03 06:44:10 UTC
Yeah, also suffering from random segfaults in firefox.
Comment 29 Joonas Niilola gentoo-dev 2023-07-03 07:52:20 UTC
(In reply to jy6x2b32pie9 from comment #28)
> Yeah, also suffering from random segfaults in firefox.

Does disabling elf-hack fix it for you? Does it happen with the -bin version too?
Comment 30 Joonas Niilola gentoo-dev 2023-07-04 08:55:06 UTC
https://bugzilla.mozilla.org/show_bug.cgi?id=1840931 this may be of interest. You can try the commit as a .patch file in /etc/portage/patches, but I'll include this commit in the upcoming 115.0 release.

https://hg.mozilla.org/mozilla-central/rev/51b78cf4881b -> 
https://hg.mozilla.org/mozilla-central/raw-rev/51b78cf4881b
Comment 31 NHO 2023-07-04 12:50:13 UTC
Disabling elf-hack did nothing.
Currently waiting for positive or negative result on -bin, because crash is kinda random. Haven't happened in ~2 hours.
On the other hand, 115 should be released today, waiting for it to happen.
Comment 32 Joonas Niilola gentoo-dev 2023-07-04 13:06:54 UTC
(In reply to jy6x2b32pie9 from comment #31)
> Disabling elf-hack did nothing.
> Currently waiting for positive or negative result on -bin, because crash is
> kinda random. Haven't happened in ~2 hours.
> On the other hand, 115 should be released today, waiting for it to happen.

I'm currently doing test runs for 115, if there are no problems I'm able to push the updated version tonight.
Comment 33 Larry the Git Cow gentoo-dev 2023-07-04 16:32:06 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=03ee26bcb2e7fece7f04f18f39c3cd153f06ce5d

commit 03ee26bcb2e7fece7f04f18f39c3cd153f06ce5d
Author:     Joonas Niilola <juippis@gentoo.org>
AuthorDate: 2023-07-04 16:30:32 +0000
Commit:     Joonas Niilola <juippis@gentoo.org>
CommitDate: 2023-07-04 16:32:02 +0000

    www-client/firefox: add 115.0
    
     - drop ~ppc64 keyword due to patches not applying before a rebase.
    
    Bug: https://bugs.gentoo.org/908297
    Bug: https://bugs.gentoo.org/908979
    Bug: https://bugs.gentoo.org/908412
    Bug: https://bugs.gentoo.org/908992
    Signed-off-by: Joonas Niilola <juippis@gentoo.org>

 www-client/firefox/Manifest             |  101 +++
 www-client/firefox/firefox-115.0.ebuild | 1458 +++++++++++++++++++++++++++++++
 2 files changed, 1559 insertions(+)
Comment 34 stefan11111 2023-07-04 18:05:58 UTC
added the patch.
It made the tab crash less often.

However, after testing, I realized that the segfaults don't always crash the tab and that they still happen without elf-hack, they just are more rare(about 1 it 10 refreshes). It seems I just didn't have the tab crash while using whatsapp web.

So far I've tried the following configurations:
-lto, -pgo, -clang --disable-elf-hack, patch
-lto, -pgo, -clang, patch
lto, pgo, -clang, --disable-elf-hack, nopatch
lto, pgo, -clang, nopatch
lto, pgo, clang, --disable-elf-hack, nopatch
-lto, -pgo, clang, --disable-elf-hack, nopatch

None fully eliminates crashes.
In fact, it seems like segfaults happen and are reported in dmesg every time I refresh whatsapp web, they just don't crash the tab too often.

I'll also try -lto, -pgo, clang, --disable-elf-hack, patch
Comment 35 stefan11111 2023-07-04 18:28:56 UTC
(In reply to Joonas Niilola from comment #29)
> (In reply to jy6x2b32pie9 from comment #28)
> > Yeah, also suffering from random segfaults in firefox.
> 
> Does disabling elf-hack fix it for you? Does it happen with the -bin version
> too?

Tried the -bin version and it worked.
I got the whatsapp web tab to crash under strace.
I'll post the output, maybe it helps.

This is the output to the terminal:
$ firefox
console.error: ({})
JavaScript error: resource://gre/modules/XULStore.sys.mjs, line 60: Error: Can't find profile directory.
console.error: (new TypeError("str is undefined", "resource://gre/modules/components-utils/FilterExpressions.sys.mjs", 109))
console.error: (new TypeError("str is undefined", "resource://gre/modules/components-utils/FilterExpressions.sys.mjs", 109))
[Child 10834, MediaDecoderStateMachine #1] WARNING: Decoder=7f8c2f5fb700 Decode error: NS_ERROR_DOM_MEDIA_REMOTE_DECODER_CRASHED_UTILITY_ERR (0x806e0011) - HandleRejectionError: file /var/tmp/portage/www-client/firefox-114.0/work/firefox-114.0/dom/media/MediaDecoderStateMachineBase.cpp:164
[Parent 10646, IPC I/O Parent] WARNING: process 10834 exited on signal 11: file /var/tmp/portage/www-client
[Child 10855, MediaDecoderStateMachine #1] WARNING: Decoder=7fb54f0d9200 Decode error: NS_ERROR_DOM_MEDIA_R
14.0/dom/media/MediaDecoderStateMachineBase.cpp:164
Comment 36 stefan11111 2023-07-04 18:29:45 UTC
Created attachment 865142 [details]
whatsap web strace
Comment 37 stefan11111 2023-07-04 19:13:29 UTC
(In reply to stefan11111 from comment #34)
> I'll also try -lto, -pgo, clang, --disable-elf-hack, patch
No more segfaults with this.
Comment 38 Joonas Niilola gentoo-dev 2023-07-04 19:29:44 UTC
(In reply to stefan11111 from comment #37)
> (In reply to stefan11111 from comment #34)
> > I'll also try -lto, -pgo, clang, --disable-elf-hack, patch
> No more segfaults with this.

So you are most likely affected by the gcc+lto bug. If you compile firefox with clang by enabling the 'clang' use flag, does that help?
Comment 39 Joonas Niilola gentoo-dev 2023-07-04 19:34:19 UTC
Hmm then there's this 
https://bugzilla.mozilla.org/show_bug.cgi?id=1835488

so if you're running on 6.4 kernel, might want to boot to a lower one if still available, and/or disable hwaccel for firefox to test whether it affects stability.

So many problems lately with it :\
(but again, I'm not sold it's strictly elf-hack related!)
Comment 40 stefan11111 2023-07-04 19:51:05 UTC
(In reply to Joonas Niilola from comment #38)
> (In reply to stefan11111 from comment #37)
> > (In reply to stefan11111 from comment #34)
> > > I'll also try -lto, -pgo, clang, --disable-elf-hack, patch
> > No more segfaults with this.
> 
> So you are most likely affected by the gcc+lto bug. If you compile firefox
> with clang by enabling the 'clang' use flag, does that help?

That's how I built firefox with clang.
The only thing not done via USE flags is disabling elf-hack with gcc.
Building firefox 115 right now. Will that patch also be added to librewolf?
Comment 41 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-07-04 19:56:37 UTC
We have nothing to do with librewolf.
Comment 42 stefan11111 2023-07-04 23:58:14 UTC
(In reply to stefan11111 from comment #40)
> Building firefox 115 right now. Will that patch also be added to librewolf?
Firefox 115 just built:
$ emerge -pqv firefox
[ebuild   R   ] www-client/firefox-115.0  USE="X eme-free gmp-autoupdate hardened jumbo-build lto openh264 pgo pulseaudio system-av1 system-harfbuzz system-icu system-jpeg system-libevent system-libvpx system-png system-webp -clang -dbus -debug -geckodriver -hwaccel -jack -libproxy -screencast (-selinux) -sndio (-system-python-libs) -telemetry (-valgrind) -wayland -wifi"

Segfaults seem to be gone.
Comment 43 Ivan 2023-07-05 16:42:01 UTC
I had same issue with FF 115 today. (Well, same according to my understanding)

Another random crash and journalctl had the following records:

июл 05 16:58:56 pc plasmashell[572426]: JavaScript error: resource://gre/modules/XULStore.sys.mjs, line 60: Error: Can't find profile directory.
июл 05 16:59:09 pc kernel: audit: type=1400 audit(1688565549.290:1061): apparmor="DENIED" operation="open" class="file" profile="syslog-ng" name="/dev/tty12" pid=714 comm="syslog-ng" requested_mask="ac" denied_mask="ac" fsuid=0 ouid=0
июл 05 16:59:14 pc plasmashell[737446]: JavaScript error: https://yusupovs.com/bitrix/js/colspan.js?1636224094330, line 4: TypeError: currentEl is undefined
июл 05 16:59:23 pc systemd[1]: Created slice system-systemd\x2dcoredump.slice.
июл 05 16:59:23 pc systemd[1]: Started systemd-coredump@0-737731-0.service.
июл 05 16:59:31 pc systemd-coredump[737733]: elfutils disabled, parsing ELF objects not supported
июл 05 16:59:31 pc systemd-coredump[737733]: Process 572304 (firefox) of user 1000 dumped core.
июл 05 16:59:31 pc systemd[1]: systemd-coredump@0-737731-0.service: Deactivated successfully.
июл 05 16:59:31 pc systemd[1]: systemd-coredump@0-737731-0.service: Consumed 7.401s CPU time.

Not sure if these are useful or relevant.

> USE="X clang dbus gmp-autoupdate hwaccel jumbo-build lto openh264 pulseaudio screencast system-av1 system-harfbuzz system-icu system-jpeg system-libevent system-libvpx system-png system-webp wayland -debug -eme-free -geckodriver -hardened -jack -libproxy -pgo (-selinux) -sndio (-system-python-libs) -telemetry (-valgrind) -wifi"
Comment 44 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-07-05 17:21:42 UTC
Please show us the backtrace from systemd-coredumpctl (systemctl debug 572304, then 'bt').
Comment 45 Larry the Git Cow gentoo-dev 2023-07-06 06:48:03 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=611e4dd27464b1222aef483bd9bcc78cc38be409

commit 611e4dd27464b1222aef483bd9bcc78cc38be409
Author:     Joonas Niilola <juippis@gentoo.org>
AuthorDate: 2023-07-06 06:47:15 +0000
Commit:     Joonas Niilola <juippis@gentoo.org>
CommitDate: 2023-07-06 06:47:15 +0000

    www-client/firefox: include latest skia-avx patch for 115
    
    Bug: https://bugs.gentoo.org/908979
    Bug: https://bugs.gentoo.org/909676
    Signed-off-by: Joonas Niilola <juippis@gentoo.org>

 www-client/firefox/Manifest             | 2 +-
 www-client/firefox/firefox-115.0.ebuild | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
Comment 46 NHO 2023-07-06 17:13:07 UTC
#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=11, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x00007fc699a5c00f in __pthread_kill_internal (signo=11, threadid=<optimized out>) at pthread_kill.c:78
#2  0x00007fc699a0cef2 in __GI_raise (sig=11) at ../sysdeps/posix/raise.c:26
#3  0x00007fc695238e74 in  () at /usr/lib64/firefox/libxul.so
#4  0x00007fc695b7515d in  () at /usr/lib64/firefox/libxul.so
#5  0x00007fc699a0cfa0 in <signal handler called> () at /lib64/libc.so.6
#6  util_range_add (end=1168, start=0, range=0x7fc65e8f6c80, resource=0x0) at ../mesa-23.1.3/src/gallium/auxiliary/util/u_range.h:66
#7  tc_buffer_do_flush_region (tc=<optimized out>, ttrans=ttrans@entry=0x7fc66a90e908, box=box@entry=0x7fc66a90e914) at ../mesa-23.1.3/src/gallium/auxiliary/util/u_threaded_context.c:2776
#8  0x00007fc67e5d90c9 in tc_buffer_unmap (_pipe=0x7fc66e22a000, transfer=0x7fc66a90e908) at ../mesa-23.1.3/src/gallium/auxiliary/util/u_threaded_context.c:2873
#9  0x00007fc67e5d89ad in tc_buffer_subdata (_pipe=0x7fc66e22a000, resource=0x7fc65e8f6c00, usage=<optimized out>, offset=0, size=1168, data=0x7fc657c4b800) at ../mesa-23.1.3/src/gallium/auxiliary/util/u_threaded_context.c:3044
#10 0x00007fc67e1e149d in pipe_buffer_write (data=0x7fc657c4b800, size=1168, offset=0, buf=<optimized out>, pipe=0x7fc66e22a000) at ../mesa-23.1.3/src/gallium/auxiliary/util/u_inlines.h:464
#11 bufferobj_data (obj=0x7fc66abadba0, storageFlags=259, usage=1790630816, offset=0, memObj=0x0, data=0x7fc657c4b800, size=1168, target=34962, ctx=0x7fc66e27d000) at ../mesa-23.1.3/src/mesa/main/bufferobj.c:355
#12 _mesa_bufferobj_data (ctx=ctx@entry=0x7fc66e27d000, target=target@entry=34962, size=size@entry=1168, data=0x7fc657c4b800, usage=usage@entry=35040, storageFlags=storageFlags@entry=259, obj=0x7fc66abadba0)
    at ../mesa-23.1.3/src/mesa/main/bufferobj.c:400
#13 0x00007fc67e1e1d48 in buffer_data (no_error=false, func=0x7fc67eb623ae "glBufferData", usage=35040, data=0x7fc657c4b800, size=1168, target=34962, bufObj=0x7fc66abadba0, ctx=0x7fc66e27d000)
    at ../mesa-23.1.3/src/mesa/main/bufferobj.c:2415
#14 buffer_data_error (ctx=0x7fc66e27d000, bufObj=0x7fc66abadba0, target=34962, size=1168, data=0x7fc657c4b800, usage=35040, func=0x7fc67eb623ae "glBufferData") at ../mesa-23.1.3/src/mesa/main/bufferobj.c:2441
#15 0x00007fc67e1e6e82 in _mesa_buffer_data (func=0x7fc67eb623ae "glBufferData", usage=<optimized out>, data=<optimized out>, size=<optimized out>, target=<optimized out>, bufObj=<optimized out>, ctx=<optimized out>)
    at ../mesa-23.1.3/src/mesa/main/bufferobj.c:2457
#16 _mesa_BufferData (target=<optimized out>, size=<optimized out>, data=<optimized out>, usage=<optimized out>) at ../mesa-23.1.3/src/mesa/main/bufferobj.c:2482
#17 0x00007fc696385418 in  () at /usr/lib64/firefox/libxul.so
#18 0x00007fc6963885eb in  () at /usr/lib64/firefox/libxul.so
#19 0x00007fc69639409b in  () at /usr/lib64/firefox/libxul.so
#20 0x00007fc696377fc4 in  () at /usr/lib64/firefox/libxul.so
#21 0x00007fc696376c3f in webrender::renderer::Renderer::render::hfc4b323933e8d698 () at /usr/lib64/firefox/libxul.so
#22 0x00007fc6961769bf in  () at /usr/lib64/firefox/libxul.so
#23 0x00007fc6923cb658 in  () at /usr/lib64/firefox/libxul.so
#24 0x00007fc6923cab66 in  () at /usr/lib64/firefox/libxul.so
#25 0x00007fc6923ca34a in  () at /usr/lib64/firefox/libxul.so
#26 0x00007fc6923d15ab in  () at /usr/lib64/firefox/libxul.so
#27 0x00007fc691928c7a in  () at /usr/lib64/firefox/libxul.so
#28 0x00007fc69192c9aa in  () at /usr/lib64/firefox/libxul.so
#29 0x00007fc691e4123c in  () at /usr/lib64/firefox/libxul.so
#30 0x00007fc691e00897 in  () at /usr/lib64/firefox/libxul.so
#31 0x00007fc691926256 in  () at /usr/lib64/firefox/libxul.so
#32 0x00007fc6998b1d9f in _pt_root (arg=0x7fc6736cadc0) at /var/tmp/portage/dev-libs/nspr-4.35-r2/work/nspr-4.35/nspr/pr/src/pthreads/ptthread.c:201
#33 0x00007fc699a5a279 in start_thread (arg=<optimized out>) at pthread_create.c:444
#34 0x00007fc699add4cc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
Comment 47 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-07-06 17:13:54 UTC
Sounds like Mesa is crashing for you in util_range_add.
Comment 48 Ivan 2023-07-06 21:39:00 UTC
(In reply to Sam James from comment #44)
> Please show us the backtrace from systemd-coredumpctl (systemctl debug
> 572304, then 'bt').

Thanks, I'll try:

pc /home/ivan # coredumpctl debug
           PID: 1552 (firefox)
           UID: 1000 (ivan)
           GID: 1000 (ivan)
        Signal: 4 (ILL)
     Timestamp: Fri 2023-07-07 00:26:10 MSK (8min ago)
  Command Line: /usr/lib64/firefox/firefox
    Executable: /usr/lib64/firefox/firefox
 Control Group: /user.slice/user-1000.slice/user@1000.service/app.slice/app-firefox\x20\x281\x29-bd6c66b648c0435db4a5290d7b4b5daa.scope
          Unit: user@1000.service
     User Unit: app-firefox\x20\x281\x29-bd6c66b648c0435db4a5290d7b4b5daa.scope
         Slice: user-1000.slice
     Owner UID: 1000 (ivan)
       Boot ID: b5e7071adca345a4ba1df787633fe585
    Machine ID: 55e6f0f700ce8169c061e3e555cd8709
      Hostname: pc
       Storage: /var/lib/systemd/coredump/core.firefox.1000.b5e7071adca345a4ba1df787633fe585.1552.1688678770000000.zst (present)
  Size on Disk: 96.6M
       Message: Process 1552 (firefox) of user 1000 dumped core.

GNU gdb (Gentoo 13.2 vanilla) 13.2
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/lib64/firefox/firefox...
(No debugging symbols found in /usr/lib64/firefox/firefox)

warning: Can't open file /memfd:/.glXXXXXX (deleted) during file-backed mapping note processing

warning: Can't open file /memfd:mozilla-ipc (deleted) during file-backed mapping note processing

warning: Can't open file /memfd:pulseaudio (deleted) during file-backed mapping note processing

warning: Can't open file /SYSV00000000 (deleted) during file-backed mapping note processing

warning: Can't open file /memfd:/.nvidia_drv.XXXXXX (deleted) during file-backed mapping note processing
[New LWP 1679]
[New LWP 1558]
[New LWP 1564]
[New LWP 1566]
[New LWP 1568]
[New LWP 1645]
[New LWP 1562]
[New LWP 1557]
[New LWP 1579]
[New LWP 1581]
[New LWP 1609]
[New LWP 1585]
[New LWP 1613]
[New LWP 1587]
[New LWP 1611]
[New LWP 1584]
[New LWP 1603]
[New LWP 1580]
[New LWP 1612]
[New LWP 1649]
[New LWP 1648]
[New LWP 1646]
[New LWP 2521]
[New LWP 1608]
[New LWP 1589]
[New LWP 1652]

<----------------------->

(gdb) bt
#0  0x00007fbda33d1f9c in ?? () from /usr/lib64/libc.so.6
#1  0x00007fbda3382ef2 in raise () from /usr/lib64/libc.so.6
#2  0x00007fbd9e7705a4 in ?? () from /usr/lib64/firefox/libxul.so
#3  0x00007fbd9f0cf15d in ?? () from /usr/lib64/firefox/libxul.so
#4  <signal handler called>
#5  0x00007fbd9fc4eb57 in glean_core::metrics::timing_distribution::TimingDistributionMetric::start::h561ea73730933444 () from /usr/lib64/firefox/libxul.so
#6  0x00007fbd9fbd2338 in <firefox_on_glean::private::timing_distribution::TimingDistributionMetric as glean_core::traits::timing_distribution::TimingDistribution>::start::h3a2dcf51f6a51baa () from /usr/lib64/firefox/libxul.so
#7  0x00007fbd9f8f8a0e in webrender::resource_cache::ResourceCache::block_until_all_resources_added::h3c8fdc12188b9d79 () from /usr/lib64/firefox/libxul.so
#8  0x00007fbd9f84729a in ?? () from /usr/lib64/firefox/libxul.so
#9  0x00007fbd9f8877cd in ?? () from /usr/lib64/firefox/libxul.so
#10 0x00007fbd9f896880 in ?? () from /usr/lib64/firefox/libxul.so
#11 0x00007fbd9f8900e7 in ?? () from /usr/lib64/firefox/libxul.so
#12 0x00007fbd9f70fcb7 in ?? () from /usr/lib64/firefox/libxul.so
#13 0x00007fbd9f71cd6a in ?? () from /usr/lib64/firefox/libxul.so
#14 0x00007fbd9ac6f323 in ?? () from /usr/lib64/firefox/libxul.so
#15 0x00007fbda33d0279 in ?? () from /usr/lib64/libc.so.6
#16 0x00007fbda345331c in ?? () from /usr/lib64/libc.so.6

Is that correct data?

P.S. forgot to mention, I have FF and Chrome profiles stored in RAM using www-misc/profile-sync-daemon, if that is relevant.
Comment 49 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-07-06 21:43:58 UTC
That looks OK but we need some more info. Could you rebuild Firefox with -g or ideally -ggdb3?
Comment 50 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-07-06 21:44:07 UTC
(In reply to Sam James from comment #49)
> That looks OK but we need some more info. Could you rebuild Firefox with -g
> or ideally -ggdb3?

(see https://wiki.gentoo.org/wiki/Debugging#Per-package)
Comment 51 Ivan 2023-07-09 00:33:30 UTC
(In reply to Sam James from comment #50)
> (In reply to Sam James from comment #49)
> > That looks OK but we need some more info. Could you rebuild Firefox with -g
> > or ideally -ggdb3?
> 
> (see https://wiki.gentoo.org/wiki/Debugging#Per-package)

I tried, and it finally crashed today. I will attach the file with log from 'coredumpctl debug', it's way too long.

Btw, journalctl looks like this:

июл 09 03:21:01 pc systemd[1]: Created slice system-systemd\x2dcoredump.slice.
июл 09 03:21:01 pc systemd[1]: Started systemd-coredump@0-472001-0.service.
июл 09 03:21:01 pc kwin_x11[1128]: kwin_core: XCB error: 150 (BadDamage), sequence: 27215, resource id: 14814901, major code: 142 (DAMAGE), minor code: 3 (Subtract)
июл 09 03:21:03 pc systemd-coredump[472003]: elfutils disabled, parsing ELF objects not supported
июл 09 03:21:03 pc systemd-coredump[472003]: [🡕] Process 469682 (firefox) of user 1000 dumped core.
июл 09 03:21:03 pc systemd[1]: systemd-coredump@0-472001-0.service: Deactivated successfully.
июл 09 03:21:03 pc systemd[1]: systemd-coredump@0-472001-0.service: Consumed 2.193s CPU time.
июл 09 03:21:03 pc plasmashell[470399]: Exiting due to channel error.
июл 09 03:21:03 pc plasmashell[471999]: Exiting due to channel error.
июл 09 03:21:03 pc plasmashell[471723]: Exiting due to channel error.
июл 09 03:21:03 pc plasmashell[469951]: Exiting due to channel error.
июл 09 03:21:03 pc plasmashell[471721]: Exiting due to channel error.
июл 09 03:21:03 pc plasmashell[471915]: Exiting due to channel error.
июл 09 03:21:03 pc plasmashell[471198]: Exiting due to channel error.
июл 09 03:21:03 pc plasmashell[471249]: Exiting due to channel error.
июл 09 03:21:03 pc plasmashell[469835]: Exiting due to channel error.
июл 09 03:21:03 pc plasmashell[469767]: Exiting due to channel error.
июл 09 03:21:03 pc systemd[911]: app-firefox\x20\x281\x29-b1f1513e7d3640caa0226c87a8a2fccd.scope: Consumed 2min 46.274s CPU time.
Comment 52 Ivan 2023-07-09 00:34:31 UTC
Created attachment 865290 [details]
FF 115.0 coredumpctl debug
Comment 53 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-07-09 00:54:21 UTC
Thanks!

```
Program terminated with signal SIGILL, Illegal instruction.
[...]
(gdb) bt
#0  0x00007f7118fd1f9c in ?? () from /usr/lib64/libc.so.6
#1  0x00007f7118f82ef2 in raise () from /usr/lib64/libc.so.6
#2  0x00007f711437bde4 in nsProfileLock::FatalSignalHandler (signo=4, info=0x7f70ed4f5a70, context=<optimized out>) at /tmp/portage/www-client/firefox-115.0/work/firefox-115.0/toolkit/profile/nsProfileLock.cpp:174
#3  0x00007f7114cdaa1d in WasmTrapHandler (signum=4, info=0x7f70ed4f5a70, context=0x7f70ed4f5940) at /tmp/portage/www-client/firefox-115.0/work/firefox-115.0/js/src/wasm/WasmSignalHandlers.cpp:794
#4  <signal handler called>
#5  0x00007f711585a427 in glean_core::metrics::timing_distribution::{impl#12}::clone (self=<optimized out>) at src/metrics/timing_distribution.rs:124
#6  glean_core::metrics::timing_distribution::TimingDistributionMetric::start (self=<optimized out>) at src/metrics/timing_distribution.rs:121
#7  0x00007f71157ddc08 in firefox_on_glean::private::timing_distribution::{impl#3}::start (self=0x7f7116555650 <firefox_on_glean::metrics::wr::rasterize_glyphs_time::h17d9aca0d826478c+8>) at toolkit/components/glean/api/src/private/timing_distribution.rs:167
#8  0x00007f71155042ce in wr_glyph_rasterizer::rasterizer::GlyphRasterizer::resolve_glyphs<webrender::resource_cache::{impl#12}::block_until_all_resources_added::{closure_env#0}, webrender::profiler::TransactionProfile> (self=0x7f70ed4fb2d0, profile=0x7f70f1329f28, handle=...)
    at gfx/wr/wr_glyph_rasterizer/src/rasterizer.rs:228
#9  webrender::resource_cache::ResourceCache::block_until_all_resources_added (self=0x7f70ed4fb2d0, gpu_cache=0x7f70ed4fb040, profile=0x7f70f1329f28) at gfx/wr/webrender/src/resource_cache.rs:1310
[...]
```

Your issue is likely bug 908412.
Comment 54 stefan11111 2023-07-16 17:21:10 UTC
Can confirm that this is also fixed in librewolf