When i enable -Clinker-plugin-lto i have a 20% chance to build Firefox successful. Thunderbird-128.4.4 builds fine with rust lto. I use a llvm Profile. Reproducible: Sometimes Steps to Reproduce: 1.Build firefox with rust lto Portage 3.0.66.1 (python 3.12.7-final-0, default/linux/amd64/23.0/llvm/systemd, gcc-14, glibc-2.40-r5, 6.11.10-gentoo-dist x86_64) ================================================================= System Settings ================================================================= System uname: Linux-6.11.10-gentoo-dist-x86_64-Intel-R-_Core-TM-_i9-14900K-with-glibc2.40 KiB Mem: 65661620 total, 53179916 free KiB Swap: 8388604 total, 8388604 free Timestamp of repository gentoo: Sat, 23 Nov 2024 13:50:22 +0000 Head commit of repository gentoo: 682a97acf4fefa1f53e5aa64b3b51d972ee7e88a sh bash 5.2_p37 ld GNU ld (Gentoo 2.43 p3) 2.43.1 app-misc/pax-utils: 1.3.8::gentoo app-shells/bash: 5.2_p37::gentoo dev-build/autoconf: 2.13-r8::gentoo, 2.72-r1::gentoo dev-build/automake: 1.17-r1::gentoo dev-build/cmake: 3.31.1::gentoo dev-build/libtool: 2.5.4::gentoo dev-build/make: 4.4.1-r100::gentoo dev-build/meson: 1.6.0::gentoo dev-lang/perl: 5.40.0::gentoo dev-lang/python: 3.12.7_p1::gentoo, 3.13.0::gentoo dev-lang/rust: 1.81.0-r100::gentoo, 1.82.0-r100::gentoo sys-apps/baselayout: 2.17::gentoo sys-apps/sandbox: 2.40::gentoo sys-apps/systemd: 256.7::gentoo sys-devel/binutils: 2.43-r2::gentoo sys-devel/binutils-config: 5.5.2::gentoo sys-devel/clang: 18.1.8-r6::gentoo, 19.1.4::gentoo sys-devel/gcc: 14.2.1_p20241116::gentoo sys-devel/gcc-config: 2.11::gentoo sys-devel/lld: 18.1.8::gentoo, 19.1.4::gentoo sys-devel/llvm: 18.1.8-r6::gentoo, 19.1.4::gentoo sys-kernel/linux-headers: 6.11::gentoo (virtual/os-headers) sys-libs/glibc: 2.40-r5::gentoo Repositories: gentoo location: /var/db/repos/gentoo sync-type: git sync-uri: https://github.com/gentoo-mirror/gentoo.git priority: -1000 volatile: False local_ebuilds location: /var/db/repos/local_ebuilds masters: gentoo volatile: False ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="*" ADDR2LINE="llvm-addr2line" AR="llvm-ar" AS="clang -c" CBUILD="x86_64-pc-linux-gnu" CC="clang" CFLAGS="-march=raptorlake -O2 -pipe -flto=thin -Werror=odr -Werror=strict-aliasing" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d" CPP="clang-cpp" CXX="clang++" CXXFLAGS="-march=raptorlake -O2 -pipe -flto=thin -Werror=odr -Werror=strict-aliasing" DISTDIR="/var/cache/distfiles" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" FCFLAGS="-march=raptorlake -O2 -pipe -flto=thin -Werror=odr -Werror=strict-aliasing" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync merge-wait multilib-strict network-sandbox news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-march=raptorlake -O2 -pipe -flto=thin -Werror=odr -Werror=strict-aliasing" GENTOO_MIRRORS="https://ftp-stud.hs-esslingen.de/pub/Mirrors/gentoo/" LANG="en_US.UTF-8" LD="ld.lld" LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs -Wl,--as-needed" LEX="flex" MAKEOPTS="-j38" NM="llvm-nm" OBJCOPY="llvm-objcopy" OBJDUMP="llvm-objdump" PKGDIR="/var/cache/binpkgs" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" RANLIB="llvm-ranlib" READELF="llvm-readelf" RUSTFLAGS=" -Ctarget-cpu=raptorlake" SHELL="/bin/bash" STRINGS="llvm-strings" STRIP="llvm-strip" USE="X aac acl amd64 apng branding bzip2 cet clang crypt dbus dist-kernel ffmpeg flac gdbm gif gmp gstreamer gtk gtk3 iconv ipv6 jpeg jpeg2k jpegxl libnotify libproxy libtirpc llvm-libunwind lto mp3 mpeg mpfr multilib ncurses nls ogg openmp opus pam pcre pgo png policykit pulseaudio readline seccomp ssl svg systemd test-rust theora threads tiff udev udisks unicode vorbis vpx vulkan x264 x265 xattr xvid zlib" ABI_X86="64" ADA_TARGET="gcc_12" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2 aes avx avx2 f16c fma3 pclmul popcnt rdrand sha sse3 sse4_1 sse4_2 ssse3 vpclmulqdq" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" GUILE_SINGLE_TARGET="3-0" GUILE_TARGETS="3-0" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en en-US de de-DE" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-2" POSTGRES_TARGETS="postgres16" PYTHON_SINGLE_TARGET="python3_12" PYTHON_TARGETS="python3_12" RUBY_TARGETS="ruby32" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account" Unset: ARFLAGS, ASFLAGS, CCLD, CONFIG_SHELL, CPPFLAGS, CTARGET, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, SIZE, YACC, YFLAGS ================================================================= Package Settings ================================================================= www-client/firefox-132.0.2::gentoo was built with the following: USE="X (clang) dbus gmp-autoupdate jumbo-build libproxy openh264 pgo pulseaudio system-av1 system-harfbuzz system-icu system-jpeg system-libevent system-libvpx system-png system-webp -debug -eme-free -gnome-shell -hardened -hwaccel -jack (-selinux) -sndio -telemetry (-valgrind) -wasm -wayland -wifi" ABI_X86="(64)" L10N="de -ach -af -an -ar -ast -az -be -bg -bn -br -bs -ca -ca-valencia -cak -cs -cy -da -dsb -el -en-CA -en-GB -eo -es-AR -es-CL -es-ES -es-MX -et -eu -fa -ff -fi -fr -fur -fy -ga -gd -gl -gn -gu -he -hi -hr -hsb -hu -hy -ia -id -is -it -ja -ka -kab -kk -km -kn -ko -lij -lt -lv -mk -mr -ms -my -nb -ne -nl -nn -oc -pa -pl -pt-BR -pt-PT -rm -ro -ru -sc -sco -si -sk -skr -sl -son -sq -sr -sv -szl -ta -te -th -tl -tr -trs -uk -ur -uz -vi -xh -zh-CN -zh-TW" LLVM_SLOT="18 -17 -19" CFLAGS="-march=raptorlake -pipe -Werror=strict-aliasing" CXXFLAGS="-march=raptorlake -pipe -Werror=strict-aliasing" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live config-protect-if-modified distlocks ebuild-locks fail-clean fixlafiles ipc-sandbox merge-sync merge-wait multilib-strict network-sandbox news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs -Wl,--as-needed -Wl,--undefined-version -Wl,-rpath=/usr/lib64/firefox,--enable-new-dtags"
Created attachment 911638 [details] build.log
If you ansifilter it and then compress w/ xz -9e, can you upload the full build.log? -- """ 132.0.2/work/firefox_build/instrumented/security/manager/ssl/ipcclientcerts/dynamic-library'(B[m(B[m 78[1G[K[34m 9:18.43(B[m ld.lld: error: undefined symbol: rust_eh_personality(B[m(B[m 78[1G[K[34m 9:18.43(B[m >>> referenced by compiler_builtins.da9494e747744ed7-cgu.016(B[m(B[m 78[1G[K[34m 9:18.43(B[m >>> compiler_builtins-03e8036c28516300.compiler_builtins.da9494e747744ed7-cgu.016.rcgu.o:(.data.DW.ref.rust_eh_personality+0x0) in archive ../../../x86_64-unknown-linux-gnu/release/libgkrust.a(B[m(B[m """
I should note two things: 1) It's very odd for such a failure to be non-deterministic; 2) FF's build system already adds -Clinker-plugin-lto if doing LTO with Clang.
Created attachment 911639 [details] emerge_pqv
Created attachment 911640 [details] build.log.xz
fwiw, I was able to workaround not being able to upgrade firefox for a while by taking out "-C linker-plugin-lto" from RUSTFLAGS for the firefox upgrade as well. With it, I got the same build error since 1st of October (previous successful build 10th August). I didn't build it so often to know if it might have succeeded 20% of the times.
I think this might be this: 13:58 +juippis: sam_: something funny I think I found: https://github.com/gentoo/gentoo/blob/master/profiles/base/package.use.mask#L378 (rust llvm-libunwind package.use.mask): If you try to compile firefox[lto,pgo] with " 13:59 +juippis: " 13:59 +juippis: well that's weird 13:59 +juippis: -unwindlib=libunwind 13:59 +juippis: it fails because, I guess rust doesn't have llvm-libunwind support 14:00 +juippis: and rust-bin does I guess because rust-bin worked 14:02 +juippis: on a glibc system where you primarily use clang/llvm, so the package.use.mask is still there 14:03 +juippis: I had a point I wanted to make but it escaped my mind before I got it out. But can you see this being the case here? Can you show what use flags your rust and clang-common is built with? This might be an unwind issue, so llvm-libunwind is enabled in clang-common but not in rust. rust-bin should work, as I presume upstream builds with +llvm-libunwind Oh and rust must also be built with lto itself - the toolchain to build firefox must match what was used to build rust. So either use rust-bin, or make sure your rust is built with: "llvm-libunwind lto system-llvm" and that whatever LLVM_SLOT was used to build rust, is also matched in Firefox. Simple right!
Two more things, if you recently flipped the unwind implementantion, most likely the whole toolchain must be built with the new setting before trying out Firefox. Including compiler-rt* packages. to leio: if you use gcc to build firefox, don't expect RUSTFLAGS to work at all. We recently flipped the default +clang back so people surprised by this may now be able to compile the browser. Also can't say much without a full build.log.
> Can you show what use flags your rust and clang-common is built with? This > might be an unwind issue, so llvm-libunwind is enabled in clang-common but > not in rust. It's a llvm profile sys-devel/clang-common-19.1.4 USE="cet (default-compiler-rt) (default-libcxx) (default-lld) (llvm-libunwind) -bootstrap-prefix -hardened -verify-sig" dev-lang/rust-1.81.0-r100 USE="(llvm-libunwind) lto system-llvm (-big-endian) -clippy -debug -dist -doc (-miri) -nightly (-parallel-compiler) -rust-analyzer -rust-src -rustfmt -test -verify-sig -wasm" ABI_X86="(64) -32 (-x32)" CPU_FLAGS_X86="sse2" LLVM_SLOT="(18%*) (-19%*)" LLVM_TARGETS="(X86) -AArch64 -AMDGPU -ARC -ARM -AVR -BPF -CSKY -DirectX -Hexagon -Lanai -LoongArch -M68k -MSP430 -Mips -NVPTX -PowerPC -RISCV -SPIRV -Sparc -SystemZ -VE -WebAssembly -XCore -Xtensa" rust-bin produces the same build error.
(In reply to Joonas Niilola from comment #8) > to leio: if you use gcc to build firefox, don't expect RUSTFLAGS to work at > all. We recently flipped the default +clang back so people surprised by this > may now be able to compile the browser. Also can't say much without a full > build.log. I build with clang, but do not use llvm profiles as I don't want libcxx. [ebuild R ] sys-devel/clang-common-18.1.8-r1::gentoo USE="default-compiler-rt default-lld -bootstrap-prefix (-cet) -default-libcxx (-hardened) -llvm-libunwind -verify-sig" 0 KiB Don't worry about me until I actually am up to date with my system over the holidays, though, as I might be hitting long fixed bugs somewhere, but as the workaround is the same, and having talked with sam_, he pointed this out, so I could confirm and provide extra data points perhaps.
For me things went better now with www-client/firefox-133.0.3, but it was just a single build so far, so might have hit the reported 20%. But I did see lto approach changes in the ebuild too.
