Created attachment 576036 [details, diff] Patch to enable building GCC itself with LTO GCC as of version 4.4 supports building itself with Link Time Optimizations enabled. Since the infrastructure was already present in toolchain.eclass, I simply modified it to expose the functionality via the lto USE flag. A check is present to ensure that GCC is new enough to enable this functionality. Ping @gentoo/toolchain Package-Manager: Portage-2.3.66, Repoman-2.3.12
> confgcc+=( --with-build-config=bootstrap-lto ) toolchain.eclass uses `GCC_MAKE_TARGET` to specify build config, namely: # default target if is_crosscompile || tc-is-cross-compiler ; then # 3 stage bootstrapping doesnt quite work when you cant run the # resulting binaries natively ^^; GCC_MAKE_TARGET=${GCC_MAKE_TARGET-all} else if tc_version_is_at_least 3.3 && use_if_iuse pgo; then GCC_MAKE_TARGET=${GCC_MAKE_TARGET-profiledbootstrap} else GCC_MAKE_TARGET=${GCC_MAKE_TARGET-bootstrap-lean} fi fi It is equivalent to --with-build-config=. You might need to expand it to add `bootstrap-lto-lean` and do something about `profiledbootstrap` (USE="lto pgo"). I don't immediately see the equivalent in https://gcc.gnu.org/install/build.html
Are you sure? Using this patch, I get the following output in my compile log: ~~~ /tmp/portage/sys-devel/gcc-9.1.0/work/build/./prev-gcc/xg++ ... -fprofile-generate -flto=jobserver ... ~~~ It seems that the build config and the make targets may be be slightly different from one another. I'm unsure exactly how the machinery works, but the documentation seems to suggest this as well: https://gcc.gnu.org/install/build.html See section: ~~~ ... ‘bootstrap-lto-lean’ This option is similar to bootstrap-lto, but is intended for faster build by only using LTO in the final bootstrap stage. With ‘make profiledbootstrap’ the LTO frontend is trained only on generator files. ... ~~~ Which seems to suggest that `--with-build-config=bootstrap-lto[-lean]` and `make profiledbootstrap` can be used at the same time. Indeed, I'm getting both LTO and `-fprofile-generate` and `-fprofile-use` in my build log. I will attach a partial log here in case it's useful.
Created attachment 576182 [details] build.log using new ebuild (partial log)
Also, this is an interesting mailing list thread: https://www.mail-archive.com/gcc-patches@gcc.gnu.org/msg210066.html Quote: > >>>>>> Please mention in both bootstrap-lto-lean.mk and the documentation > >>>>>> that the intended make target for this config is profiledbootstrap > >>>>>> since for non-profiledbootstrap it ends up not using LTO at all. A > >>>>>> "lean" > >>>>>> mode for non-profiledbootstrap would need to set up things to > >>>>>> use LTO only for stage3 which means not doing a bootstrap comparison > >>>>>> which means we could "skip" stage2 as well here.
Interesting. Yeah, bootstrap/final distinction makes sense. I suggest enabling IUSE=lto only for gcc-9.1 for now and extend it to older versions when gcc-9.1 gets stable. That way we'll avoid gcc rebuild for majority of stable users.
Created attachment 576458 [details, diff] Updated patch to build GCC with LTO I agree -- also, it seems like using bootstrap-lto-lean is the way to go, as there aren't any benefits in using LTO in the earlier bootstrapping stages. I've amended the patch to only apply to GCC 9.1 and use bootstrap-lto-lean as a result.
OK -- there seems to be a minor hiccup with applying LTO only to GCC 9.1 and above, which is that dev-lang/gnat-gpl doesn't use a high enough version of GCC to ever actually have IUSE=lto. This seems to make repoman unhappy. What's the best course of action to avoid breakage here?
Created attachment 576466 [details, diff] Updated patch: bootstrap-lto-lean, >=gcc-9.1, no gnat-gpl Found the problem -- it was that I had incorrectly changed dev-lang/gnat-gpl's metadata.xml. Should be fixed now.
Just a comment on bootstrap-lto vs bootstrap-lto-lean although it shouldn't be a big deal. The distinction between them is more than bootstrap vs bootstrap-lean. In a non-lto build with bootstrap-lean stage2 and stage3 are still compared: what the lean does is to delete stage1 compiler once stage2 is built since it's not needed any more, and delete stage2 once stage3 is built and compared with stage2. Without PGO, in bootstrap-lto, stage2 and stage3 are both built with LTO and compared. With bootstrap-lto-lean since stage2 is not built with LTO, they cannot be compared, so you lose some double-checking. With regular -O2 compiling this should not be a great loss, though you may want to keep the compare if we eventually allow overriding the flags. With PGO, no comparison is done in any case, so that part doesn't matter. However, notice the "With ‘make profiledbootstrap’ the LTO frontend is trained only on generator files." That means you potentially lose some profiling coverage as the full compiler is not built with LTO in stage3 (this is the stage that creates the profiling data), only a set of helper programs.
Ah yes, the config files seem to indicate such as well. It sounds like bootstrap-lto is the safest and possibly most performant option.
Created attachment 576516 [details, diff] bootstrap-lto, >=gcc-9.1, no gnat-gpl Same as previous patch, but uses bootstrap-lto instead. I'm not marking the other patch as obsolete because there may be reasons to use bootstrap-lto-lean that I'm not aware of.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ee307a9f5d9e77ce1dbb3655d4ba7f12fffaf554 commit ee307a9f5d9e77ce1dbb3655d4ba7f12fffaf554 Author: Shane Peelar <lookatyouhacker@gmail.com> AuthorDate: 2019-05-08 18:55:53 +0000 Commit: Sergei Trofimovich <slyfox@gentoo.org> CommitDate: 2019-05-13 22:38:22 +0000 {sys-devel/gcc,dev-lang/gnat-gpl}: add a USE for building GCC with LTO Bug: https://bugs.gentoo.org/685634 Signed-off-by: Shane Peelar <lookatyouhacker@gmail.com> Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org> eclass/toolchain.eclass | 6 ++++++ sys-devel/gcc/metadata.xml | 1 + 2 files changed, 7 insertions(+)
Pushed as-is. Thank you!
