Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 689358 - {www-client/firefox,mail-client/thunderbird}[-pgo,lto]: .../temp/ccDhlmPD.ltrans1.ltrans.o:<artificial>:function sandbox::Trap::SigSys(int, siginfo_t*, ucontext*): error: undefined reference to 'SyscallAsm'
Summary: {www-client/firefox,mail-client/thunderbird}[-pgo,lto]: .../temp/ccDhlmPD.ltr...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal with 2 votes (vote)
Assignee: Mozilla Gentoo Team
URL:
Whiteboard:
Keywords:
: 689646 689712 690286 693312 (view as bug list)
Depends on:
Blocks:
 
Reported: 2019-07-06 13:38 UTC by tt_1
Modified: 2019-09-14 20:34 UTC (History)
19 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
compressed build log from firefox-68.0_b14 (firefox-68.0_b14+lto.log.gz,336.13 KB, application/gzip)
2019-07-06 13:38 UTC, tt_1
Details
output from emerge --info (emerge--info-musl,6.53 KB, text/plain)
2019-07-06 13:41 UTC, tt_1
Details
compressed build log from firefox-68.0 (firefox-68.0-glibc-fail-with-lto.gz,379.77 KB, application/gzip)
2019-07-11 09:15 UTC, tt_1
Details
output from emerge --info (glibc) (emerge-info-glibc,7.68 KB, text/plain)
2019-07-11 09:17 UTC, tt_1
Details
firefox build patch (firefox-lto.patch,555 bytes, patch)
2019-07-11 13:49 UTC, Maciej S. Szmigiero
Details | Diff
append-cppflags -flto for lto builds (0001-www-client-firefox-include-flto-in-cppflags-with-lto.patch,897 bytes, patch)
2019-09-03 21:12 UTC, Jory A. Pratt
Details | Diff
Properly fix lto gcc builds. (fix_sandbox_lto_build.patch,1023 bytes, patch)
2019-09-07 01:49 UTC, Jory A. Pratt
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description tt_1 2019-07-06 13:38:59 UTC
Created attachment 582042 [details]
compressed build log from firefox-68.0_b14

from the build log: 

12:53.62 /usr/bin/x86_64-gentoo-linux-musl-g++ -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -Wall -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wsign-compare -Wtype-limits -Wunreachable-code -Wwrite-strings -Wno-invalid-offsetof -Wc++1z-compat -Wduplicated-cond -Wimplicit-fallthrough -Wno-error=maybe-uninitialized -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=coverage-mismatch -Wno-error=free-nonheap-object -Wno-error=multistatement-macros -Wno-error=class-memaccess -Wformat -Wformat-security -Wformat-overflow=2 -fno-sized-deallocation -pipe -flifetime-dse=1 -Wno-psabi -Wno-class-memaccess -Wno-int-in-bool-context -Wno-multistatement-macros -Wno-maybe-uninitialized -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread -pipe -freorder-blocks -O2 -fomit-frame-pointer -funwind-tables  -fPIC -shared -Wl,-z,defs -Wl,--gc-sections -Wl,-h,libmozsandbox.so -o libmozsandbox.so safe_sprintf.o icu_utf.o trap.o syscall_wrappers.o Unified_cpp_sandbox_linux0.o Unified_cpp_sandbox_linux1.o Unified_cpp_sandbox_linux2.o  -flto=12 -flifetime-dse=1 -lpthread -Wl,-O1 -Wl,--as-needed -Wl,-rpath=/usr/lib/firefox,--enable-new-dtags -Wl,--compress-debug-sections=zlib -fuse-ld=gold -fstack-protector-strong -Wl,-z,noexecstack -Wl,-z,text -Wl,-z,relro -Wl,-z,nocopyreloc -Wl,-Bsymbolic-functions -Wl,--icf=safe -Wl,-rpath-link,/var/tmp/portage/www-client/firefox-68.0_beta14/work/firefox-68.0/ff/dist/bin -Wl,-rpath-link,/usr/lib   -fdiagnostics-color       -lplds4 -lplc4 -lnspr4 -lpthread -ldl -lrt
12:53.62 /var/tmp/portage/www-client/firefox-68.0_beta14/temp/ccDhlmPD.ltrans1.ltrans.o:<artificial>:function sandbox::Trap::SigSys(int, siginfo_t*, ucontext*): error: undefined reference to 'SyscallAsm'
12:53.62 collect2: error: ld returned 1 exit status

the log is from 68.0 beta, and thus from the mozilla overlay. With the release of 68.0 only days away I don't see much of a benefit from reproducing this for the soon to be replaced firefox-67.0.4. 

the full build log is attached
Comment 1 tt_1 2019-07-06 13:41:35 UTC
Created attachment 582044 [details]
output from emerge --info
Comment 2 tt_1 2019-07-06 13:44:09 UTC
this might be another hit of this gcc bug - I'm not sure about that? 

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=57703
Comment 3 Maciej S. Szmigiero 2019-07-10 14:45:52 UTC
+1 confirmed with www-client/firefox-68.0[lto] and sys-devel/gcc-9.1.0-r1.

It looks like the problem is that security/sandbox/linux/moz.build
just doesn't detect this setup as a LTO GCC one (for some reason)
and so it does not add "--param lto-partitions=1" to LDFLAGS as it should
in this case.

If I comment out this LTO GCC check in the aforementioned file:
#for f in CONFIG['OS_CXXFLAGS']:
#    if f.startswith('-flto') and CONFIG['CC_TYPE'] != 'clang':
to force "LDFLAGS += ['--param', 'lto-partitions=1']" line to execute
unconditionally (this line also needs its leading whitespace removed as this
is a Python code) that linker flag then shows in "COMPUTED_LDFLAGS" in ff/security/sandbox/linux/backend.mk file and the package builds
successfully.
Comment 4 tt_1 2019-07-10 14:48:29 UTC
Can you please provide a diff of the ebuild?
Comment 5 Maciej S. Szmigiero 2019-07-10 14:53:30 UTC
(In reply to tt_1 from comment #4)
> Can you please provide a diff of the ebuild?

I didn't edit the ebuild, just manually edited the
security/sandbox/linux/moz.build file.
Comment 6 tt_1 2019-07-10 15:05:09 UTC
I'm happy that you seem to have found the problem, but to reproduce (and confirm) your finding I need a patch against the ebuild, or the firefox source tree.
Comment 7 Torsten Kaiser 2019-07-10 15:17:49 UTC
I hit the same problem.
The cause seems to be, that this moz.build uses CONFIG['OS_CXXFLAGS'] instead COMPILE_FLAGS['OS_CXXFLAGS'] as all the other places:

thoregon /var/tmp/portage/www-client/firefox-68.0 # find . -name moz.build -exec grep -Ha OS_CXXFLAGS \{\} \;
./work/firefox-68.0/security/sandbox/linux/moz.build:for f in CONFIG['OS_CXXFLAGS']:
./work/firefox-68.0/tools/fuzzing/libfuzzer/moz.build:for flags_var in ('OS_CFLAGS', 'OS_CXXFLAGS'):
./work/firefox-68.0/toolkit/content/moz.build:for var in ('OS_CPPFLAGS', 'OS_CXXFLAGS', 'DEBUG', 'OPTIMIZE', 'FRAMEPTR'):
./work/firefox-68.0/build/clang-plugin/tests/moz.build:COMPILE_FLAGS['OS_CXXFLAGS'] = (
./work/firefox-68.0/build/clang-plugin/tests/moz.build:    [f for f in COMPILE_FLAGS.get('OS_CXXFLAGS', []) if not f.startswith('-W')] +
./work/firefox-68.0/build/unix/elfhack/moz.build:COMPILE_FLAGS['OS_CXXFLAGS'] = [
./work/firefox-68.0/build/unix/elfhack/moz.build:    f for f in COMPILE_FLAGS['OS_CXXFLAGS'] if f != '-fno-exceptions'
./work/firefox-68.0/browser/app/moz.build:    COMPILE_FLAGS['OS_CXXFLAGS'] = [
./work/firefox-68.0/browser/app/moz.build:        f for f in COMPILE_FLAGS.get('OS_CXXFLAGS', [])

I'm currently trying the following patch:

--- security/sandbox/linux/moz.build~	2019-07-10 17:01:32.000000000 +0200
+++ security/sandbox/linux/moz.build	2019-07-10 17:15:29.528079364 +0200
@@ -98,7 +98,7 @@
 # gcc lto likes to put the top level asm in syscall.cc in a different partition
 # from the function using it which breaks the build.  Work around that by
 # forcing there to be only one partition.
-for f in CONFIG['OS_CXXFLAGS']:
+for f in COMPILE_FLAGS['OS_CXXFLAGS']:
     if f.startswith('-flto') and CONFIG['CC_TYPE'] != 'clang':
         LDFLAGS += ['--param', 'lto-partitions=1']
Comment 8 Torsten Kaiser 2019-07-10 17:49:30 UTC
... that did not work.

I tried adding debug "print" into this file, that shows that CONFIG seems to be completely empty. COMPILE_FLAGS has contents, but nothing wrt. to LTO.

COMPILE_FLAGS
{u'COVERAGE': None, u'CLANG_PLUGIN': [], u'VISIBILITY': [u'-I/var/tmp/portage/www-client/firefox-68.0/work/firefox-68.0/ff/dist/system_wrappers
', u'-include', u'/var/tmp/portage/www-client/firefox-68.0/work/firefox-68.0/config/gcc_hidden.h'], u'LIBRARY_DEFINES': None, u'DSO_PIC': [u'-f
PIC'], u'OPTIMIZE': [u'-freorder-blocks', u'-O2'], u'WARNINGS_AS_ERRORS': None, u'RTL': None, u'EXTRA_INCLUDES': [u'-I/var/tmp/portage/www-clie
nt/firefox-68.0/work/firefox-68.0/ff/dist/include'], u'PROFILE_GEN_DYN_CFLAGS': None, u'OS_CXXFLAGS': [u'-Wall', u'-Wempty-body', u'-Wignored-q
ualifiers', u'-Woverloaded-virtual', u'-Wpointer-arith', u'-Wsign-compare', u'-Wtype-limits', u'-Wunreachable-code', u'-Wwrite-strings', u'-Wno
-invalid-offsetof', u'-Wc++1z-compat', u'-Wduplicated-cond', u'-Wimplicit-fallthrough', u'-Wno-error=maybe-uninitialized', u'-Wno-error=depreca
ted-declarations', u'-Wno-error=array-bounds', u'-Wno-error=coverage-mismatch', u'-Wno-error=free-nonheap-object', u'-Wno-error=multistatement-
macros', u'-Wno-error=class-memaccess', u'-Wno-error=deprecated-copy', u'-Wformat', u'-Wformat-security', u'-Wformat-overflow=2', u'-fno-sized-
deallocation', u'-pipe', u'-march=znver1', u'-O2', u'-flifetime-dse=1', u'-U_FORTIFY_SOURCE', u'-D_FORTIFY_SOURCE=2', u'-fstack-protector-strong', u'-fno-exceptions', u'-fno-strict-aliasing', u'-fno-rtti', u'-ffunction-sections', u'-fdata-sections', u'-fno-exceptions', u'-fno-math-errno', u'-pthread', u'-pipe'], u'OS_COMPILE_CXXFLAGS': [u'-DMOZILLA_CLIENT', u'-include', u'/var/tmp/portage/www-client/firefox-68.0/work/firefox-68.0/ff/mozilla-config.h'], u'DEFINES': None, u'STL': [u'-I/var/tmp/portage/www-client/firefox-68.0/work/firefox-68.0/ff/dist/stl_wrappers'], u'WARNINGS_CFLAGS': [u'-Wall', u'-Wempty-body', u'-Wignored-qualifiers', u'-Wpointer-arith', u'-Wsign-compare', u'-Wtype-limits', u'-Wunreachable-code', u'-Wduplicated-cond', u'-Wno-error=maybe-uninitialized', u'-Wno-error=deprecated-declarations', u'-Wno-error=array-bounds', u'-Wno-error=coverage-mismatch', u'-Wno-error=free-nonheap-object', u'-Wno-error=multistatement-macros', u'-Wno-error=class-memaccess', u'-Wno-error=deprecated-copy', u'-Wformat', u'-Wformat-security', u'-Wformat-overflow=2'], u'BASE_INCLUDES': [u'-I/var/tmp/portage/www-client/firefox-68.0/work/firefox-68.0/security/sandbox/linux', u'-I/var/tmp/portage/www-client/firefox-68.0/work/firefox-68.0/ff/security/sandbox/linux'], u'DSO': [], u'FRAMEPTR': [u'-fomit-frame-pointer', u'-funwind-tables'], u'OS_CFLAGS': [u'-pipe', u'-march=znver1', u'-O2', u'-U_FORTIFY_SOURCE', u'-D_FORTIFY_SOURCE=2', u'-fstack-protector-strong', u'-fno-strict-aliasing', u'-ffunction-sections', u'-fdata-sections', u'-fno-math-errno', u'-pthread', u'-fPIC', u'-pipe'], u'OS_INCLUDES': [u'-I/usr/include/nspr', u'-I/usr/include/nss', u'-I/var/tmp/portage/www-client/firefox-68.0/work/firefox-68.0/ff/dist/include/nss', u'-I/usr/include/pixman-1'], u'OS_CPPFLAGS': [u'-U_FORTIFY_SOURCE', u'-D_FORTIFY_SOURCE=2', u'-fstack-protector-strong'], u'DEBUG': [], u'MOZBUILD_CXXFLAGS': None, u'LOCAL_INCLUDES': None, u'MOZBUILD_CFLAGS': None, u'OS_COMPILE_CFLAGS': [u'-include', u'/var/tmp/portage/www-client/firefox-68.0/work/firefox-68.0/ff/mozilla-config.h', u'-DMOZILLA_CLIENT']}

... I'm switching to USE=-lto.
Comment 9 Jeroen Roovers gentoo-dev 2019-07-11 06:23:24 UTC
*** Bug 689646 has been marked as a duplicate of this bug. ***
Comment 10 tt_1 2019-07-11 08:04:40 UTC
the user from #689646 had this happen with glibc, so I wonder what is going on here. Are you all on binutils-2.32 yet?
Comment 11 Søren Dalby Larsen 2019-07-11 08:09:43 UTC
I'm on binutils-2.32-r1 and glibc
Comment 12 tt_1 2019-07-11 09:15:39 UTC
Created attachment 582496 [details]
compressed build log from firefox-68.0

please share your emerge --info's
Comment 13 tt_1 2019-07-11 09:17:27 UTC
Created attachment 582498 [details]
output from emerge --info (glibc)

I was able to reproduce on my main glibc system, my toolchain is stable unless sandbox itself; trying with a downgrade of it now.
Comment 14 tt_1 2019-07-11 10:12:49 UTC
Downgrade of sandbox didn't help much. 

Is there somewhere a stage3 available, with the old stable binutils-2.31.1? I took a fresh one, and it seems to be impossible to downgrade the binutils.
Comment 15 Maciej S. Szmigiero 2019-07-11 13:49:20 UTC
Created attachment 582504 [details, diff]
firefox build patch

You can try the attached patch, applying it via, for example,
"/etc/portage/patches" mechanism.
Comment 16 Maciej S. Szmigiero 2019-07-11 15:25:54 UTC
(In reply to tt_1 from comment #6)
> I'm happy that you seem to have found the problem, but to reproduce (and
> confirm) your finding I need a patch against the ebuild, or the firefox
> source tree.

You have a patch now, which has worked for me, against the Firefox
source tree in comment 15.
Comment 17 Jeroen Roovers gentoo-dev 2019-07-12 06:47:38 UTC
*** Bug 689712 has been marked as a duplicate of this bug. ***
Comment 18 Klaus Kusche 2019-07-20 09:45:00 UTC
Same problem here.
Could this be added to the ebuild?
Comment 19 Sergei Trofimovich gentoo-dev 2019-07-20 13:32:20 UTC
*** Bug 690286 has been marked as a duplicate of this bug. ***
Comment 20 Attila Tóth 2019-07-23 10:23:50 UTC
Patch works for me, thanks!
Comment 21 Thomas Deutschmann gentoo-dev Security 2019-08-03 15:32:51 UTC
I don't understand why this patch should be necessary. I am also unable to reproduce:

With USE=lto we will set

> --enable-lto=thin

which will do a variant of

> cflags.append("-flto")

in build/moz.configure/toolchain.configure

In other words:

> if f.startswith('-flto') and CONFIG['CC_TYPE'] != 'clang':
>    ^^^^^^^^^^^^^^^^^^^^^

should trigger.
Comment 22 fkater 2019-08-06 10:46:41 UTC
FYI: The above patch did not work for me with firefox-68.0.1.
Comment 23 Attila Tóth 2019-08-06 11:04:43 UTC
(In reply to fkater from comment #22)
> FYI: The above patch did not work for me with firefox-68.0.1.

The attached patch or the modification in Comment #7?
Comment 24 tt_1 2019-08-10 10:12:43 UTC
I'm going to grab a amd64 glibc stage3 and try to reproduce soon. The bug was intentionally opened for musl, but since people got it on glibc as well, I changed that. In any case, musl has a broken ld.gold linker, a possible solution for the problem discussed here isn't enough to make it play nicely with lto enabled. 

@whissi: did you use anything else but stable keywords for the toolchain, in your efforts to reproduce this?
Comment 25 Zdenek Sojka 2019-08-12 08:15:32 UTC
(In reply to Maciej S. Szmigiero from comment #15)
> Created attachment 582504 [details, diff] [details, diff]
> firefox build patch
> 
> You can try the attached patch, applying it via, for example,
> "/etc/portage/patches" mechanism.

I can confirm this patch fixes the ~amd64 GCC 8.3.0-r1 LTO build for me:
[ebuild   R   ~] www-client/firefox-68.0.1::gentoo  USE="dbus gmp-autoupdate lto screenshot startup-notification system-av1 system-harfbuzz system-icu system-jpeg system-libevent system-sqlite system-webp wayland -bindist -clang -custom-cflags -custom-optimization -debug -eme-free -geckodriver -hardened -hwaccel -jack (-neon) -pgo -pulseaudio (-selinux) -system-libvpx -test -wifi" CPU_FLAGS_X86="avx2" L10N="-ach -af -an -ar -ast -az -be -bg -bn -br -bs -ca -cak -cs -cy -da -de -dsb -el -en-CA -en-GB -eo -es-AR -es-CL -es-ES -es-MX -et -eu -fa -ff -fi -fr -fy -ga -gd -gl -gn -gu -he -hi -hr -hsb -hu -hy -ia -id -is -it -ja -ka -kab -kk -km -kn -ko -lij -lt -lv -mk -mr -ms -my -nb -nl -nn -oc -pa -pl -pt-BR -pt-PT -rm -ro -ru -si -sk -sl -son -sq -sr -sv -ta -te -th -tr -uk -ur -uz -vi -xh -zh-CN -zh-TW" 0 KiB
Comment 26 Jory A. Pratt gentoo-dev 2019-09-03 00:09:54 UTC
*** Bug 693312 has been marked as a duplicate of this bug. ***
Comment 27 Andrew Udvare 2019-09-03 00:42:00 UTC
This patches fixes USE="lto" for mail-client/thunderbird-68.0-r1.
Comment 28 Zdenek Sojka 2019-09-03 06:56:18 UTC
(In reply to Maciej S. Szmigiero from comment #15)
> Created attachment 582504 [details, diff] [details, diff]
> firefox build patch
> 
> You can try the attached patch, applying it via, for example,
> "/etc/portage/patches" mechanism.


I can confirm this patch fixes the ~amd64 GCC 8.3.0-r1 LTO build for me:
[ebuild   R   ~] mail-client/thunderbird-68.0-r1::gentoo  USE="dbus gmp-autoupdate lto startup-notification system-av1 system-harfbuzz system-icu system-jpeg system-libevent system-libvpx system-sqlite system-webp wayland -bindist -clang -custom-cflags -custom-optimization -debug -eme-free -geckodriver -hardened -jack -lightning (-neon) (-pgo) -pulseaudio (-selinux) -test -wifi" CPU_FLAGS_X86="avx2" L10N="-ar -ast -be -bg -br -ca -cs -cy -da -de -el -en-GB -es-AR -es-ES -et -eu -fi -fr -fy -ga -gd -gl -he -hr -hsb -hu -hy -id -is -it -ja -ko -lt -nb -nl -nn -pl -pt-BR -pt-PT -rm -ro -ru -si -sk -sl -sq -sr -sv -tr -uk -vi -zh-CN -zh-TW" 0 KiB
Comment 29 Attila Tóth 2019-09-03 10:05:27 UTC
Pat(In reply to Andrew Udvare from comment #27)
> This patches fixes USE="lto" for mail-client/thunderbird-68.0-r1.

Patch works for me, thanks!
Comment 30 Thomas Deutschmann gentoo-dev Security 2019-09-03 20:17:59 UTC
I am now able to reproduce with USE=-pgo but USE=lto.
Comment 31 Jory A. Pratt gentoo-dev 2019-09-03 21:12:26 UTC
Created attachment 589016 [details, diff]
append-cppflags -flto for lto builds

This is the work around I have used in mozilla overlay, if someone wants to test and report your findings would be appreciated.
Comment 32 Larry the Git Cow gentoo-dev 2019-09-03 21:40:58 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=f215ad14657d298836e3fc2755d9c03df68fa4ca

commit f215ad14657d298836e3fc2755d9c03df68fa4ca
Author:     Thomas Deutschmann <whissi@gentoo.org>
AuthorDate: 2019-09-03 21:34:53 +0000
Commit:     Thomas Deutschmann <whissi@gentoo.org>
CommitDate: 2019-09-03 21:34:53 +0000

    www-client/firefox: fix USE=lto when USE=-pgo
    
    Closes: https://bugs.gentoo.org/689358
    Package-Manager: Portage-2.3.75, Repoman-2.3.17
    Signed-off-by: Thomas Deutschmann <whissi@gentoo.org>

 www-client/firefox/firefox-68.1.0.ebuild | 3 +++
 www-client/firefox/firefox-69.0.ebuild   | 3 +++
 2 files changed, 6 insertions(+)

Additionally, it has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=cd069c6534cb766995d0c62ea5c5cd38b0f763ea

commit cd069c6534cb766995d0c62ea5c5cd38b0f763ea
Author:     Thomas Deutschmann <whissi@gentoo.org>
AuthorDate: 2019-09-03 21:36:24 +0000
Commit:     Thomas Deutschmann <whissi@gentoo.org>
CommitDate: 2019-09-03 21:36:24 +0000

    mail-client/thunderbird: fix USE=lto when USE=-pgo
    
    Bug: https://bugs.gentoo.org/689358
    Package-Manager: Portage-2.3.75, Repoman-2.3.17
    Signed-off-by: Thomas Deutschmann <whissi@gentoo.org>

 mail-client/thunderbird/thunderbird-68.0-r1.ebuild | 3 +++
 1 file changed, 3 insertions(+)
Comment 33 Steffen Hau 2019-09-06 09:56:20 UTC
Please reopen, as this is just a workaround.

There must be something wrong with the ebuilds and/or eclass, because my {C,CXX,LD}FLAGS all contain "-flto":
CFLAGS="-march=native -O3 -pipe -flto=5 -fuse-linker-plugin"
CXXFLAGS="-march=native -O3 -pipe -flto=5 -fuse-linker-plugin -fno-delete-null-pointer-checks -flifetime-dse=1"
LDFLAGS="-march=native -O3 -pipe -flto=5 -fuse-linker-plugin -Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu -Wl,--gc-sections -Wl,--icf=safe"

But emerge --info reports:
FF
CFLAGS="-march=native -pipe -fuse-linker-plugin"
CXXFLAGS="-march=native -pipe -fuse-linker-plugin -fno-delete-null-pointer-checks -flifetime-dse=1 -flifetime-dse=1 -flto"
LDFLAGS="-march=native -pipe -fuse-linker-plugin -Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu -Wl,--gc-sections -Wl,--icf=safe -Wl,-rpath=/usr/lib64/firefox,--enable-new-dtags -Wl,--compress-debug-sections=zlib"

TB
CFLAGS="-march=native -pipe -fuse-linker-plugin"
CXXFLAGS="-march=native -pipe -fuse-linker-plugin -fno-delete-null-pointer-checks -flifetime-dse=1 -flifetime-dse=1 -flto"
LDFLAGS="-march=native -pipe -fuse-linker-plugin -Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu -Wl,--gc-sections -Wl,--icf=safe -Wl,-rpath=/usr/lib64/thunderbird,--enable-new-dtags -Wl,--compress-debug-sections=zlib"

Please note that the "-flto" in CXXFLAGS has been added by the recent ebuild change. Both FF and TB have USE="custom-cflags custom-optimization" enabled. Something is removing "-flto=5" and "-O3" from my flags. This explains why the check in moz.build does not recognize lto and therefor does not append '--param lto-partitions=1' to LDFLAGS.

"-O3" is stripped by this commit: https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=cf29bfea148f8a2fe960c495bd1887eab42e6599. But I have no Idea what is causing "-flto=5" to be removed.


Complete emerge --info output for FF and TB:
Portage 2.3.75 (python 3.7.4-final-0, default/linux/amd64/17.1/systemd, gcc-9.2.0, glibc-2.29-r5, 5.2.11-HAUIHAU x86_64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-5.2.11-HAUIHAU-x86_64-Intel-R-_Core-TM-_i5-6200U_CPU_@_2.30GHz-with-gentoo-2.6
KiB Mem:     8099596 total,   4788892 free
KiB Swap:    8388604 total,   6677244 free
Timestamp of repository gentoo: Thu, 05 Sep 2019 20:45:01 +0000
Head commit of repository gentoo: 7d6d2e014e3c20b332aeacf4b62225034fc9cf2e
sh bash 5.0_p11
ld GNU gold (Gentoo 2.32 p2 2.32.0) 1.16
app-shells/bash:          5.0_p11::gentoo
dev-java/java-config:     2.2.0-r4::gentoo
dev-lang/perl:            5.30.0::gentoo
dev-lang/python:          2.7.16::gentoo, 3.7.4-r1::gentoo
dev-util/cmake:           3.15.3::gentoo
sys-apps/baselayout:      2.6-r1::gentoo
sys-apps/sandbox:         2.18::gentoo
sys-devel/autoconf:       2.13-r1::gentoo, 2.69-r4::gentoo
sys-devel/automake:       1.16.1-r1::gentoo
sys-devel/binutils:       2.32-r1::gentoo
sys-devel/gcc:            9.2.0::gentoo
sys-devel/gcc-config:     2.0::gentoo
sys-devel/libtool:        2.4.6-r5::gentoo
sys-devel/make:           4.2.1-r4::gentoo
sys-kernel/linux-headers: 5.2::gentoo (virtual/os-headers)
sys-libs/glibc:           2.29-r5::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://rsync.de.gentoo.org/gentoo-portage
    priority: -1000
    sync-rsync-extra-opts:
    sync-rsync-verify-jobs: 1
    sync-rsync-verify-metamanifest: yes
    sync-rsync-verify-max-age: 24

hauihau
    location: /usr/local/portage/hauihau
    masters: gentoo
    priority: 0

ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O3 -pipe -flto=5 -fuse-linker-plugin"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=native -O3 -pipe -flto=5 -fuse-linker-plugin -fno-delete-null-pointer-checks -flifetime-dse=1"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--autounmask=n --keep-going=y --quiet-build=y --quiet-fail=y --with-bdeps=y --changed-deps-report=n"
ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fakeroot fixlafiles ipc-sandbox merge-sync metadata-transfer multilib-strict network-sandbox news parallel-fetch parallel-install pid-sandbox preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="de_DE.UTF-8"
LC_ALL="de_DE.utf8"
LDFLAGS="-march=native -O3 -pipe -flto=5 -fuse-linker-plugin -Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu -Wl,--gc-sections -Wl,--icf=safe"
LINGUAS="de"
MAKEOPTS="-j5"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/home/gentoo/tmp/"
USE="X a52 aac aalib acl alsa amd64 avx avx2 bash-completion berkdb bluetooth bluray branding bzip2 cairo caps cdda cddb cdparanoia cdr cli cracklib crypt cups curl cxx dbus dga dri dts dv dvd egl encode exif ffmpeg flac fontconfig fortran ftp gd gdbm gif gmp gstreamer iconv icu imagemagick imlib ipv6 jpeg jpeg2k kde kerberos lame libcaca libnotify libsamplerate libtirpc lzma lzo mad matroska mmx mmxext mng mp3 mpeg mtp multilib musepack ncurses networkmanager nls nptl nsplugin ogg openal opengl openmp opus pam pcre pdf png policykit pulseaudio qt5 quicktime readline seccomp sndfile spell split-usr sse sse2 sse3 sse4 sse4_1 sse4_2 ssl ssse3 svg syslog systemd tcpd theora threads tiff truetype udev unicode upower usb v4l vaapi vcd vim-syntax vorbis wavpack wayland webkit x264 xattr xcb xcomposite xinerama xml xmp xorg xosd xpm xv xvid zlib" ABI_X86="64" ADA_TARGET="gnat_2018" ALSA_CARDS="hda-intel" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64" INPUT_DEVICES="libinput" KERNEL="linux" L10N="de" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" NETBEANS_MODULES="apisupport cnd groovy gsf harness ide identity j2ee java mobility nb php profiler soa visualweb webcommon websvccommon xml" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-2" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_7" PYTHON_TARGETS="python3_7" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby26" USERLAND="GNU" VIDEO_CARDS="i965 intel iris" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

=================================================================
                        Package Settings
=================================================================

www-client/firefox-69.0::gentoo was built with the following:
USE="custom-cflags custom-optimization gmp-autoupdate lto pulseaudio screenshot system-av1 system-harfbuzz system-icu system-jpeg system-libevent system-libvpx system-sqlite system-webp wayland -bindist -clang -debug -eme-free -geckodriver -hardened -hwaccel -jack (-neon) -pgo (-selinux) -startup-notification -test -wifi" ABI_X86="(64)" CPU_FLAGS_X86="avx2" L10N="de -ach -af -an -ar -ast -az -be -bg -bn -br -bs -ca -cak -cs -cy -da -dsb -el -en-CA -en-GB -eo -es-AR -es-CL -es-ES -es-MX -et -eu -fa -ff -fi -fr -fy -ga -gd -gl -gn -gu -he -hi -hr -hsb -hu -hy -ia -id -is -it -ja -ka -kab -kk -km -kn -ko -lij -lt -lv -mk -mr -ms -my -nb -nl -nn -oc -pa -pl -pt-BR -pt-PT -rm -ro -ru -si -sk -sl -son -sq -sr -sv -ta -te -th -tr -uk -ur -uz -vi -xh -zh-CN -zh-TW"
CFLAGS="-march=native -pipe -fuse-linker-plugin"
CXXFLAGS="-march=native -pipe -fuse-linker-plugin -fno-delete-null-pointer-checks -flifetime-dse=1 -flifetime-dse=1 -flto"
FEATURES="metadata-transfer ipc-sandbox ebuild-locks sfperms distlocks usersandbox merge-sync config-protect-if-modified unmerge-orphans fixlafiles news sandbox userfetch userpriv network-sandbox parallel-install unmerge-logs xattr strict parallel-fetch pid-sandbox usersync binpkg-logs unknown-features-warn preserve-libs assume-digests fakeroot protect-owned multilib-strict binpkg-docompress binpkg-dostrip"
LDFLAGS="-march=native -pipe -fuse-linker-plugin -Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu -Wl,--gc-sections -Wl,--icf=safe -Wl,-rpath=/usr/lib64/firefox,--enable-new-dtags -Wl,--compress-debug-sections=zlib"


mail-client/thunderbird-68.0-r2::gentoo was built with the following:
USE="custom-cflags custom-optimization dbus gmp-autoupdate lto pulseaudio system-av1 system-harfbuzz system-icu system-jpeg system-libevent system-libvpx system-sqlite system-webp wayland -bindist -clang -debug -eme-free -hardened -jack -lightning (-neon) (-pgo) (-selinux) -startup-notification -test -wifi" ABI_X86="(64)" CPU_FLAGS_X86="avx2" L10N="de -ar -ast -be -bg -br -ca -cs -cy -da -el -en-GB -es-AR -es-ES -et -eu -fi -fr -fy -ga -gd -gl -he -hr -hsb -hu -hy -id -is -it -ja -ko -lt -nb -nl -nn -pl -pt-BR -pt-PT -rm -ro -ru -si -sk -sl -sq -sr -sv -tr -uk -vi -zh-CN -zh-TW"
CFLAGS="-march=native -pipe -fuse-linker-plugin"
CXXFLAGS="-march=native -pipe -fuse-linker-plugin -fno-delete-null-pointer-checks -flifetime-dse=1 -flifetime-dse=1 -flto"
FEATURES="metadata-transfer ipc-sandbox ebuild-locks sfperms distlocks usersandbox merge-sync config-protect-if-modified unmerge-orphans fixlafiles news sandbox userfetch userpriv network-sandbox parallel-install unmerge-logs xattr strict parallel-fetch pid-sandbox usersync binpkg-logs unknown-features-warn preserve-libs assume-digests fakeroot protect-owned multilib-strict binpkg-docompress binpkg-dostrip"
LDFLAGS="-march=native -pipe -fuse-linker-plugin -Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu -Wl,--gc-sections -Wl,--icf=safe -Wl,-rpath=/usr/lib64/thunderbird,--enable-new-dtags -Wl,--compress-debug-sections=zlib"
Comment 34 Attila Tóth 2019-09-06 11:26:51 UTC
Note, that I have "-custom-cflags -custom-optimization" for both FF and TB and the patch works for me. I haven't inspected flags like Steffen so far, but I can take a look if it is relevant...
Comment 35 Thomas Deutschmann gentoo-dev Security 2019-09-06 13:43:16 UTC
@ Steffen: While I agree that this is somehow a workaround (I still think --enable-lto should be enough but we didn't spend much time on this given that with upcoming Mozilla 70.x release LTO handling will change), your conclusion is wrong. See

https://gitweb.gentoo.org/repo/gentoo.git/tree/www-client/firefox/firefox-69.0.ebuild?id=a8af3151b4ecb713101d6dab3f6cb2ea045d259d#n367

We are filtering your flags on purpose.

Feel free to file a new bug for this but we will probably close as WONTFIX: LTO/PGO stuff is very fragile. There are so many options which aren't supported/will cause problems. We don't have the man power to fully support all possible PGO/LTO options. We only try to be binary-compatible with firefox-bin (i.e. support the same PGO/LTO options upstream is using so you can get the same performance with your own firefox build from source like with firefox-bin). Sorry.
Comment 36 Jory A. Pratt gentoo-dev 2019-09-07 01:49:46 UTC
Created attachment 589270 [details, diff]
Properly fix lto gcc builds.

If someone would please comment out the append line we added to fix this bug and test the updated patch would be appreciated. This is the fix I am gonna be sending upstream which will actually fix the problem instead of working around it.
Comment 37 Thomas Deutschmann gentoo-dev Security 2019-09-07 17:45:53 UTC
Patch works.
Comment 38 Steffen Hau 2019-09-09 11:00:18 UTC
(In reply to Thomas Deutschmann from comment #35)
> @ Steffen: While I agree that this is somehow a workaround (I still think
> --enable-lto should be enough but we didn't spend much time on this given
> that with upcoming Mozilla 70.x release LTO handling will change), your
> conclusion is wrong. See
> 
> https://gitweb.gentoo.org/repo/gentoo.git/tree/www-client/firefox/firefox-69.
> 0.ebuild?id=a8af3151b4ecb713101d6dab3f6cb2ea045d259d#n367
> 
> We are filtering your flags on purpose.
> 
> Feel free to file a new bug for this but we will probably close as WONTFIX:
> LTO/PGO stuff is very fragile. There are so many options which aren't
> supported/will cause problems. We don't have the man power to fully support
> all possible PGO/LTO options. We only try to be binary-compatible with
> firefox-bin (i.e. support the same PGO/LTO options upstream is using so you
> can get the same performance with your own firefox build from source like
> with firefox-bin). Sorry.
Thanks for clarifying, I haven't seen the strip-flags in the ebuild, sorry.


(In reply to Jory A. Pratt from comment #36)
> Created attachment 589270 [details, diff] [details, diff]
> Properly fix lto gcc builds.
> 
> If someone would please comment out the append line we added to fix this bug
> and test the updated patch would be appreciated. This is the fix I am gonna
> be sending upstream which will actually fix the problem instead of working
> around it.
I can confirm that the patch works.
Comment 39 Larry the Git Cow gentoo-dev 2019-09-14 20:34:57 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/proj/mozilla.git/commit/?id=4694c68a6b60520d02af98fc1fced6dfc642ee93

commit 4694c68a6b60520d02af98fc1fced6dfc642ee93
Author:     Jory Pratt <anarchy@gentoo.org>
AuthorDate: 2019-09-10 16:06:47 +0000
Commit:     Jory Pratt <anarchy@gentoo.org>
CommitDate: 2019-09-10 16:06:47 +0000

    www-client/firefox: fix lto builds
    
    Closes: https://bugs.gentoo.org/689358
    Closes: https://bugs.gentoo.org/693654
    Package-Manager: Portage-2.3.75, Repoman-2.3.17
    Signed-off-by: Jory Pratt <anarchy@gentoo.org>

 .../firefox/files/firefox-69.0-lto-gcc-fix.patch   | 26 ++++++++++++++++++++++
 ...{firefox-69.0.ebuild => firefox-69.0-r1.ebuild} |  6 ++---
 2 files changed, 28 insertions(+), 4 deletions(-)