Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 502950 - sys-apps/portage: -flto combined with -frecord-gcc-switches prints invalid QA warnings about ignored CFLAGS
Summary: sys-apps/portage: -flto combined with -frecord-gcc-switches prints invalid QA...
Status: RESOLVED INVALID
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All All
: Normal minor (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
: 853979 (view as bug list)
Depends on:
Blocks: lto
  Show dependency tree
 
Reported: 2014-02-27 07:39 UTC by Arfrever Frehtes Taifersar Arahesis
Modified: 2024-03-11 21:08 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build.log.xz (was too large for plain text) (build.log.xz,20.05 KB, application/octet-stream)
2014-02-27 09:41 UTC, Samuli Suominen (RETIRED)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Arfrever Frehtes Taifersar Arahesis 2014-02-27 07:39:45 UTC
sys-fs/udev-210 ignores CFLAGS during compilation of some files.
This problem is absent in sys-fs/udev-208.

 * QA Notice: Files built without respecting CFLAGS have been detected
 *  Please include the following list of files in your report:
 * /usr/lib64/libgudev-1.0.so.0.2.0
 * /lib64/libudev.so.1.4.0
Comment 1 Samuli Suominen (RETIRED) gentoo-dev 2014-02-27 08:06:29 UTC
I also use CFLAGS="-frecord-gcc-switches" to catch pkgs not respecting flags, and I'm not seeing this problem. So, you'll have to provide more information.
Comment 2 Arfrever Frehtes Taifersar Arahesis 2014-02-27 08:07:52 UTC
Please show output of:
portageq envvar -v CFLAGS CXXFLAGS FFLAGS FCFLAGS
Comment 3 Samuli Suominen (RETIRED) gentoo-dev 2014-02-27 08:11:58 UTC
(In reply to Arfrever Frehtes Taifersar Arahesis from comment #2)
> Please show output of:
> portageq envvar -v CFLAGS CXXFLAGS FFLAGS FCFLAGS

$ portageq envvar -v CFLAGS CXXFLAGS FFLAGS FCFLAGS
CFLAGS="-O2 -pipe -march=native -frecord-gcc-switches"
CXXFLAGS="-O2 -pipe -march=native -frecord-gcc-switches"
FFLAGS="-O2 -pipe"
FCFLAGS="-O2 -pipe"
Comment 4 Samuli Suominen (RETIRED) gentoo-dev 2014-02-27 08:20:49 UTC
reproduced by fixing -frecord-gcc-switches to be in every *FLAGS like required
Comment 5 Samuli Suominen (RETIRED) gentoo-dev 2014-02-27 08:33:16 UTC
Post the complete build.log and point out the part where CFLAGS aren't used. I suspect it's the QA check itself which is broken since I can't find anything wrong in my build.log
Comment 6 Samuli Suominen (RETIRED) gentoo-dev 2014-02-27 09:41:11 UTC
Created attachment 371376 [details]
build.log.xz (was too large for plain text)

Here is my build.log, showing these in the end:

 * QA Notice: Files built without respecting CFLAGS have been detected
 *  Please include the following list of files in your report:
 * /lib64/libudev.so.1.4.0
 * /usr/lib64/libgudev-1.0.so.0.2.0
 * /usr/lib32/libudev.so.1.4.0

However I can't see anywhere in the log the flags being ignored. Is the QA check broken? I'm using gcc-4.8.x.
Comment 7 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-02-27 10:16:43 UTC
Maybe it's because -210 appends '-flto' by default... I'll check that.

But do we want it to auto-enable LTO when it's available?
Comment 8 Samuli Suominen (RETIRED) gentoo-dev 2014-02-27 10:17:51 UTC
(In reply to Michał Górny from comment #7)
> But do we want it to auto-enable LTO when it's available?

Is there any reason not to?
Comment 9 Samuli Suominen (RETIRED) gentoo-dev 2014-02-27 10:35:11 UTC
configure.ac of systemd checks if compiler supports -lfto, like it checks for eg. -Wl,--as-needed and only adds it when it's supported

we all know global lto is currently a dead end

so is it really wrong for packages that know they support lto, use it? maybe the portage's check should just be ignored if lto is detected?
Comment 10 Samuli Suominen (RETIRED) gentoo-dev 2014-02-27 10:37:51 UTC
CC toolchain@ for comment
Comment 11 Ryan Hill (RETIRED) gentoo-dev 2014-02-28 03:19:53 UTC
I don't have a problem with packages using compiler features like this.  Just like some packages use potentially unsafe flags because they've been designed to work with them.  I think packages are free to test for and use whatever compiler features they find useful - as long as upstream addresses any bugs that they might uncover of course.  There's a lot of precedent for this.

LTO does wipe out the .GCC.command.line sections in those libraries.  But I don't know of any way to tell if a binary was built with LTO after the fact, so I'm not sure how we would detect it.
Comment 12 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-02-28 08:38:56 UTC
Well, I was more wondering if -flto doesn't go in line with -On options that we expect user to control himself. AFAIU -flto also goes with some memory overhead but I don't think that'd be important in this case.
Comment 13 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-03-02 16:28:26 UTC
Ok, I've got an argument against -flto: I've set distcc to make compilations faster. Now that systemd implicitly adds -flto, all the workload ends up on my local machine anyway...
Comment 14 Samuli Suominen (RETIRED) gentoo-dev 2014-03-02 16:52:41 UTC
(In reply to Michał Górny from comment #13)
> Ok, I've got an argument against -flto: I've set distcc to make compilations
> faster. Now that systemd implicitly adds -flto, all the workload ends up on
> my local machine anyway...

Darn! If I remember correctly it was NOT okay to test $FEATURES from .ebuild, so we can't even do anything like...

has distcc ${FEATURES} && sed -i -e '-flto' configure{,.ac}

...but I see many ebuilds like eg. media-tv/mythtv doing just that:

media-tv/mythtv/mythtv-0.25.3_p20130101.ebuild:	has distcc ${FEATURES} || myconf="${myconf} --disable-distcc"

But the generated binaries without -lfto are different from those of built with -flto, so while removing -flto makes the compilation faster for distcc case, it potentially makes the application (systemd, udev, ...) slower?
Comment 15 Samuli Suominen (RETIRED) gentoo-dev 2014-03-02 16:53:25 UTC
I meant 'has distcc ${FEATURES} && sed -i -e 's:-flto::' configure{,.ac}' of course. Sorry. Typing error.
Comment 16 Samuli Suominen (RETIRED) gentoo-dev 2014-03-02 16:55:33 UTC
I don't understand, and sorry for my ignorance, why doesn't -flto work with distcc? My googling foo failed me.
Comment 17 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-03-02 19:17:19 UTC
LTO moves the optimization step (which is quite CPU-heavy) from compilation to linking step. Since distcc does linking locally, this means that all the heaviest effort will be done only locally. In my case, this doubles the build time for really minor benefit. A low-end computer will be hurt even more.

The solution is to leave '-flto' as user choice for CFLAGS. If I want LTO, I enable LTO. I don't want some random build system to decide that I want LTO much like I don't want random build systems to override my -O because they think it'd be better for me to get more optimization when I want to debug stuff.
Comment 18 Ryan Hill (RETIRED) gentoo-dev 2014-03-04 03:26:46 UTC
All good reasons.  It also completely scrambles debug info.  I'm actually prefer to see it stripped out but i didn't want to be the bad guy this time.
Comment 19 Samuli Suominen (RETIRED) gentoo-dev 2014-03-04 06:35:33 UTC
(In reply to Ryan Hill from comment #18)
> All good reasons.  It also completely scrambles debug info.  I'm actually
> prefer to see it stripped out but i didn't want to be the bad guy this time.

OK. This is now disabled in both sys-apps/systemd (by mgorny) and sys-fs/udev (by me).

But the package manager's QA check is still broken and should be fixed, so leaving bug open.
Comment 20 Eli Schwartz 2024-03-11 20:10:38 UTC
elt-patches currently fixes this:

https://gitweb.gentoo.org/proj/elt-patches.git/tree/patches/color-record

You also really need https://gitweb.gentoo.org/proj/elt-patches.git/tree/patches/werror in order to QA check LTO using the relevant -Werror flags to guarantee that dangerous runtime issues get detected.

The overall issue here is twofold:

- packages whose custom build systems (raw Makefiles?) don't pass CFLAGS
  at link time

- packages using libtool but not a recent enough one, or not using
  elibtoolize


It is overall an actual legitimate issue I think, because the compiler does document:

"""
To use the link-time optimizer, -flto and optimization options should
be specified at compile time and during the final link. It is
recommended that you compile all the files  participating  in
the same link with the same options and also specify those options
at link time.  For example:
"""


automake does pass flags to libtool, and leaves it up to libtool to filter flags it (often wrongly) thinks shouldn't be used at link time -- one of these flags ***used*** to be -frecord-gcc-switches.

We should be good on that count now.

Build systems other than autotools, meson and cmake may or may not pass CFLAGS at link time.
Comment 21 Eli Schwartz 2024-03-11 20:10:59 UTC
I think probably there's nothing for portage to do here?
Comment 22 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-03-11 20:44:59 UTC
yep, per comment20.

There ws a fix to be done, but it was in libtool (which I've got landed upstream) and we've backported it in elt-patches too.
Comment 23 Eli Schwartz 2024-03-11 21:08:14 UTC
*** Bug 853979 has been marked as a duplicate of this bug. ***