Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 770055 - www-client/firefox and thunderbird could benefit from lld/bfd parallelism
Summary: www-client/firefox and thunderbird could benefit from lld/bfd parallelism
Status: UNCONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: ARM64 Linux
: Normal enhancement (vote)
Assignee: Mozilla Gentoo Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-02-11 11:55 UTC by Cănărău Constantin
Modified: 2024-01-14 17:22 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Cănărău Constantin 2021-02-11 11:55:47 UTC
When compiled with clang, Firefox and Thunderbird final link is done by lld on single thread.
Adding append-ldflags "-Wl,--thinlto-jobs=4" to ebuild speed up compile time on Raspberry Pi 4 8GB RAM with over an hour.
It is possible that also x86 and amd64 compile time to be improved.

pi /var/db/repos # diff -u gentoo/mail-client/thunderbird/thunderbird-78.7.1-r1.ebuild pi/mail-client/thunderbird/thunderbird-78.7.1-r1.ebuild 
--- gentoo/mail-client/thunderbird/thunderbird-78.7.1-r1.ebuild	2021-02-11 00:09:17.000000000 +0200
+++ pi/mail-client/thunderbird/thunderbird-78.7.1-r1.ebuild	2021-02-11 13:40:38.317477136 +0200
@@ -705,6 +705,8 @@
 			mozconfig_add_options_ac "forcing ld=lld due to USE=clang and USE=lto" --enable-linker=lld
 
 			mozconfig_add_options_ac '+lto' --enable-lto=cross
+
+			append-ldflags "-Wl,--thinlto-jobs=4"
 		else
 			# Linking only works when using ld.gold when LTO is enabled
 			mozconfig_add_options_ac "forcing ld=gold due to USE=lto" --enable-linker=gold


pi /var/db/repos # diff -u gentoo/www-client/firefox/firefox-85.0.2-r1.ebuild  pi/www-client/firefox/firefox-85.0.2-r1.ebuild 
--- gentoo/www-client/firefox/firefox-85.0.2-r1.ebuild	2021-02-11 00:09:20.000000000 +0200
+++ pi/www-client/firefox/firefox-85.0.2-r1.ebuild	2021-02-11 10:41:47.959488162 +0200
@@ -712,6 +712,8 @@
 			mozconfig_add_options_ac "forcing ld=lld due to USE=clang and USE=lto" --enable-linker=lld
 
 			mozconfig_add_options_ac '+lto' --enable-lto=cross
+
+			append-ldflags "-Wl,--thinlto-jobs=4"
 		else
 			# ThinLTO is currently broken, see bmo#1644409
 			mozconfig_add_options_ac '+lto' --enable-lto=full

I using that change since three versions now. Pi has 4 cores but, I believe that, generically, JOBSERVERS or MAKEOPTS jobs should be used, or half of them.

genlop -t firefox
 * www-client/firefox

     Sun Jan 10 14:44:37 2021 >>> www-client/firefox-84.0.2
       merge time: 5 hours, 30 minutes and 53 seconds.

     Sun Jan 10 21:33:14 2021 >>> www-client/firefox-84.0.2
       merge time: 4 hours, 45 minutes and 14 seconds.

     Wed Jan 27 06:10:08 2021 >>> www-client/firefox-85.0
       merge time: 5 hours, 13 minutes and 25 seconds.

     Thu Jan 28 02:46:58 2021 >>> www-client/firefox-85.0
       merge time: 4 hours, 18 minutes and 24 seconds.

     Wed Feb  3 00:54:00 2021 >>> www-client/firefox-85.0-r1
       merge time: 4 hours, 20 minutes and 33 seconds.

     Fri Feb  5 02:19:32 2021 >>> www-client/firefox-85.0-r1
       merge time: 4 hours, 24 minutes and 40 seconds.

     Sun Feb  7 20:26:33 2021 >>> www-client/firefox-85.0.1
       merge time: 4 hours, 19 minutes and 55 seconds.

So it went from 5:30 hours to 4:20.

Reproducible: Always
Comment 1 Joonas Niilola gentoo-dev 2024-01-06 18:12:18 UTC
GCC and bfd can also do -lto=N

This is definitely something to look at. I wonder if there's a need to detect whether a user has set linker parallelism and only then apply it here too.
Comment 2 Joonas Niilola gentoo-dev 2024-01-06 18:14:07 UTC
The "problem" is that LTO is enabled via build system options and not via *FLAGS, I'll have to see if/how they're modifiable.
Comment 3 Cănărău Constantin 2024-01-14 17:05:18 UTC
I transformed my Pi 4 to a headless local server long time ago, so Firefox wasn't compiled on it recently.
But on Raspberry Pi 5 compiling Firefox does not have the same issue. At the final linking stage, lld use all four cores.

Sorry for delay, I put Vivaldi on Pi 5 and, somehow, get used with it. Firefox require clang, nodejs and, of course, itself - three packages with long time required for compile.

pi5 ~ # emerge -pvO firefox

These are the packages that would be merged, in order:

[ebuild   R    ] www-client/firefox-121.0.1:rapid::gentoo  USE="X clang dbus gmp-autoupdate jumbo-build lto system-av1 system-harfbuzz system-icu system-jpeg system-libevent system-libvpx system-webp -debug (-eme-free) -geckodriver (-hardened) -hwaccel -jack -libproxy -openh264 -pgo -pulseaudio -screencast (-selinux) -sndio -system-png (-system-python-libs) -telemetry (-valgrind) -wayland -wifi" CPU_FLAGS_ARM="(-neon)" L10N="ro -ach -af -an -ar -ast -az -be -bg -bn -br -bs -ca -ca-valencia -cak -cs -cy -da -de -dsb -el -en-CA -en-GB -eo -es-AR -es-CL -es-ES -es-MX -et -eu -fa -ff -fi -fr -fur -fy -ga -gd -gl -gn -gu -he -hi -hr -hsb -hu -hy -ia -id -is -it -ja -ka -kab -kk -km -kn -ko -lij -lt -lv -mk -mr -ms -my -nb -ne -nl -nn -oc -pa -pl -pt-BR -pt-PT -rm -ru -sc -sco -si -sk -sl -son -sq -sr -sv -szl -ta -te -th -tl -tr -trs -uk -ur -uz -vi -xh -zh-CN -zh-TW" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB
Comment 4 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-01-14 17:22:13 UTC
It's not clear to me why the user shouldn't just set this.