When compiling sys-devel/gcc[lto] with >=sys-devel/make-4.4, the gcc build system fails to understand the new --jobserver-auth flag syntax. Instead of serializing the build however it will spawn N lto processes for each N child process, leading to N^2 LTO processes which will very quickly grind to a halt any machine with >8 cores and/or limited memory, and eventually result in an OOM. This may be fixed in gcc master if I am reading correctly, but not backported to 12: https://savannah.gnu.org/bugs/index.php?57242 https://github.com/gcc-mirror/gcc/commit/1270ccda70ca09f7d4fe76b5156dca8992bd77a6 If this is not backported it should IMO be justification for masking >=sys-devel/make-4.4. Reproducible: Always Portage 3.0.41 (python 3.10.8-final-0, default/linux/amd64/17.1, gcc-12, glibc-2.36-r6, 6.0.10-gentoo-dist x86_64) ================================================================= System uname: Linux-6.0.10-gentoo-dist-x86_64-AMD_Ryzen_9_3950X_16-Core_Processor-with-glibc2.36 KiB Mem: 131830396 total, 38634324 free KiB Swap: 20971516 total, 20971516 free Head commit of repository gentoo: 22a3b4fbee781e0daa6a952ba76e02662894e00b Timestamp of repository haskell: Mon, 14 Nov 2022 01:01:51 +0000 sh bash 5.2_p12 ld GNU ld (Gentoo 2.39 p5) 2.39.0 app-misc/pax-utils: 1.3.5::gentoo app-shells/bash: 5.2_p12::gentoo dev-java/java-config: 2.3.1::gentoo dev-lang/perl: 5.36.0-r1::gentoo dev-lang/python: 3.9.15_p3::gentoo, 3.10.8_p3::gentoo, 3.11.0_p2::gentoo dev-lang/rust: 1.65.0::gentoo dev-util/cmake: 3.25.1::gentoo dev-util/meson: 0.64.1::gentoo sys-apps/baselayout: 2.9::gentoo sys-apps/openrc: 0.45.2-r1::gentoo sys-apps/sandbox: 2.29::gentoo sys-devel/autoconf: 2.13-r7::gentoo, 2.71-r5::gentoo sys-devel/automake: 1.16.5::gentoo sys-devel/binutils: 2.39-r4::gentoo sys-devel/binutils-config: 5.4.1::gentoo sys-devel/clang: 11.1.0::gentoo, 13.0.1::gentoo, 14.0.6-r1::gentoo, 15.0.5::gentoo sys-devel/gcc: 12.2.1_p20221008::gentoo sys-devel/gcc-config: 2.8::gentoo sys-devel/libtool: 2.4.7::gentoo sys-devel/lld: 13.0.1-r1::gentoo, 15.0.5::gentoo sys-devel/llvm: 11.1.0::gentoo, 13.0.1::gentoo, 14.0.6-r2::gentoo, 15.0.5::gentoo sys-devel/make: 4.3::gentoo sys-kernel/linux-headers: 6.0::gentoo (virtual/os-headers) sys-libs/glibc: 2.36-r6::gentoo Repositories: gentoo location: /usr/portage sync-type: git sync-uri: https://github.com/gentoo/gentoo priority: -1000 emu-dev location: /usr/local/portage/emu-dev masters: gentoo guru location: /var/lib/layman/guru sync-type: laymansync sync-uri: https://anongit.gentoo.org/git/repo/proj/guru.git masters: gentoo priority: 50 haskell location: /var/lib/layman/haskell sync-type: laymansync sync-uri: https://github.com/gentoo-haskell/gentoo-haskell.git masters: gentoo priority: 50 lto-overlay location: /var/lib/layman/lto-overlay sync-type: laymansync sync-uri: https://github.com/InBetweenNames/gentooLTO.git masters: gentoo mv priority: 50 menelkir location: /var/lib/layman/menelkir sync-type: laymansync sync-uri: https://gitlab.com/menelkir/gentoo-overlay.git masters: gentoo priority: 50 mv location: /var/lib/layman/mv sync-type: laymansync sync-uri: https://anongit.gentoo.org/git/user/mv.git masters: gentoo priority: 50 pentoo location: /var/lib/layman/pentoo sync-type: laymansync sync-uri: https://github.com/pentoo/pentoo-overlay.git masters: gentoo priority: 50 ricerlay location: /var/lib/layman/ricerlay sync-type: laymansync sync-uri: https://github.com/azahi/ricerlay.git masters: gentoo priority: 50 steam-overlay location: /var/lib/layman/steam-overlay sync-type: laymansync sync-uri: https://github.com/anyc/steam-overlay.git masters: gentoo priority: 50 ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="@FREE ValveSteamLicense" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -mtune=native -O3 -fgraphite-identity -floop-nest-optimize -fdevirtualize-at-ltrans -fipa-pta -fno-semantic-interposition -flto=32 -fuse-linker-plugin -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/gnupg/qualified.txt /usr/share/maven-bin-3.8/conf" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-march=native -mtune=native -O3 -fgraphite-identity -floop-nest-optimize -fdevirtualize-at-ltrans -fipa-pta -fno-semantic-interposition -flto=32 -fuse-linker-plugin -pipe" DISTDIR="/var/cache/distfiles" EMERGE_DEFAULT_OPTS="--usepkg --autounmask=n --backtrack=300 --complete-graph --with-bdeps=y --keep-going" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg buildpkg-live compress-build-logs compressdebug config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-install pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="https://gentoo.osuosl.org/ https://mirror.leaseweb.com/gentoo/ https://mirror.rackspace.com/gentoo/" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LEX="flex" MAKEOPTS="-j30" PKGDIR="/var/cache/binpkgs" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" SHELL="/bin/bash" USE="X acl alsa amd64 bash-completion bluetooth bzip2 caps cli crypt cups dri elogind evdev fortran gdbm graphite iconv ipv6 libglvnd libtirpc lm-sensors lto multilib ncurses nls nptl opengl openmp pam pcre pgo pulseaudio readline seccomp split-usr ssl test-rust threads udev unicode vaapi vdpau verify-sig vim-syntax vulkan x264 xattr xdg zlib" ABI_X86="64" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64 pc" INPUT_DEVICES="evdev joystick libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="luajit" LUA_TARGETS="luajit" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-4 php8-0" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_10" PYTHON_TARGETS="python3_10" RUBY_TARGETS="ruby31" USERLAND="GNU" VIDEO_CARDS="amdgpu radeonsi" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
The commit you linked seems unrelated, however the one straight after it seems quite related (53e3b2bf16a486c15c20991c6095f7be09012b55). [[ or, more likely, the commit you linked was made to facilitate 53e3b2bf16a486c15c20991c6095f7be09012b55 ]] I'll try reproducing now, I hesitated at first since I first need a copy of 12.2.1_p20221008 (I'm on 12.2.1_p20221203 and 13)
Ok, the latest GCC 12 snapshot indeed omits support for fifo: jobservers. I think it's probably better to backport that than to mask make, though.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/gcc-patches.git/commit/?id=f91690729a34d00f90b7303a51f7952105696196 commit f91690729a34d00f90b7303a51f7952105696196 Author: Sam James <sam@gentoo.org> AuthorDate: 2022-12-09 11:04:30 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-09 11:24:01 +0000 12.2: backport make 4.4 jobserver fixes Bug: https://bugs.gentoo.org/884633 Signed-off-by: Sam James <sam@gentoo.org> .../78_all_Factor-out-jobserver_active_p.patch | 301 +++++++++++++++++++++ ...-jobserver-style-fifo-for-recent-GNU-make.patch | 73 +++++ ...spect-jobserver-in-parallel-WPA-streaming.patch | 237 ++++++++++++++++ ...ix-fifo-mode-by-opening-pipe-in-proper-mo.patch | 37 +++ 12.2.0/gentoo/README.history | 7 +- 5 files changed, 654 insertions(+), 1 deletion(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=fb34ab687d83e8b93245944d6a59b46390594148 commit fb34ab687d83e8b93245944d6a59b46390594148 Author: Sam James <sam@gentoo.org> AuthorDate: 2022-12-09 11:24:29 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-09 11:25:09 +0000 sys-devel/gcc: bump 12.2.1_p20221203 to patchset 12.2.0 patchset 4 This is for the make 4.4 jobserver fixes. Note that GCC 10 & 11 still need backports. Bug: https://bugs.gentoo.org/884633 Signed-off-by: Sam James <sam@gentoo.org> sys-devel/gcc/Manifest | 2 +- sys-devel/gcc/gcc-12.2.1_p20221203.ebuild | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
Note: - GCC 10 & 11 are pending backports. - No revbump done for 12 yet (this can affect packages building using -flto=jobserver, not just GCC itself, but this isn't very common) as I plan to keyword the next snapshot which is due in 2 days anyway
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/gcc-patches.git/commit/?id=90d6e02b9980b278cb63f2b8e79b0a29dab5262f commit 90d6e02b9980b278cb63f2b8e79b0a29dab5262f Author: Sam James <sam@gentoo.org> AuthorDate: 2022-12-17 02:49:35 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-17 04:54:05 +0000 Revert "11.4.0: backport make 4.4 jobserver fixes" Not complete but wanted to commit it for posterity in case we want to resume it. This reverts commit 54c43babcc6a7b957ef59725b5b54608517e6fb0. Bug: https://bugs.gentoo.org/884633 Signed-off-by: Sam James <sam@gentoo.org> 11.4.0/gentoo/README.history | 4 ---- 1 file changed, 4 deletions(-) https://gitweb.gentoo.org/proj/gcc-patches.git/commit/?id=6b7210b745d69eee33e5eda1a7ed5fa6ee93c003 commit 6b7210b745d69eee33e5eda1a7ed5fa6ee93c003 Author: Sam James <sam@gentoo.org> AuthorDate: 2022-12-17 02:49:23 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-17 04:54:05 +0000 Revert "10.5.0: backport make 4.4 jobserver fixes" Not complete but wanted to commit it for posterity in case we want to resume it. This reverts commit b0521a3be4d9d67a7190126e58440c4904924744. Bug: https://bugs.gentoo.org/884633 Signed-off-by: Sam James <sam@gentoo.org> .../77_all_Factor-out-jobserver_active_p.patch | 320 --------------------- ...-jobserver-style-fifo-for-recent-GNU-make.patch | 66 ----- 10.5.0/gentoo/README.history | 4 - 3 files changed, 390 deletions(-) https://gitweb.gentoo.org/proj/gcc-patches.git/commit/?id=e8a0be0130af739172f5f694a782c684fd341c09 commit e8a0be0130af739172f5f694a782c684fd341c09 Author: Arsen Arsenović <arsen@aarsen.me> AuthorDate: 2022-12-09 14:41:40 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-17 04:54:04 +0000 11.4.0: backport make 4.4 jobserver fixes Bug: https://bugs.gentoo.org/884633 Signed-off-by: Arsen Arsenović <arsen@aarsen.me> Signed-off-by: Sam James <sam@gentoo.org> 11.4.0/gentoo/README.history | 4 ++++ 1 file changed, 4 insertions(+) https://gitweb.gentoo.org/proj/gcc-patches.git/commit/?id=a535dea3848bd10f950f72281bb7d1563586d1d8 commit a535dea3848bd10f950f72281bb7d1563586d1d8 Author: Arsen Arsenović <arsen@aarsen.me> AuthorDate: 2022-12-09 14:37:14 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-17 04:54:04 +0000 10.5.0: backport make 4.4 jobserver fixes Bug: https://bugs.gentoo.org/884633 Signed-off-by: Arsen Arsenović <arsen@aarsen.me> Signed-off-by: Sam James <sam@gentoo.org> .../77_all_Factor-out-jobserver_active_p.patch | 320 +++++++++++++++++++++ ...-jobserver-style-fifo-for-recent-GNU-make.patch | 66 +++++ 10.5.0/gentoo/README.history | 4 + 3 files changed, 390 insertions(+)
Done upstream in 10 and 11 respectively: https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=51e819e99f3e800d2fb1cb22b4648e6d0558e35b https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=c4de50835f731b301bd8161be5318de701f2864a
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=77c516ccbbb90dd43e12ebd7cfa3642f786762ac commit 77c516ccbbb90dd43e12ebd7cfa3642f786762ac Author: Sam James <sam@gentoo.org> AuthorDate: 2022-12-22 22:46:47 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-22 22:46:47 +0000 sys-devel/gcc: add 10.4.1_p20221222 Bug: https://bugs.gentoo.org/884633 Signed-off-by: Sam James <sam@gentoo.org> sys-devel/gcc/Manifest | 1 + sys-devel/gcc/gcc-10.4.1_p20221222.ebuild | 58 +++++++++++++++++++++++++++++++ 2 files changed, 59 insertions(+)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=316d250ee4d7bf08015da05b04e56264735cb178 commit 316d250ee4d7bf08015da05b04e56264735cb178 Author: Sam James <sam@gentoo.org> AuthorDate: 2022-12-24 01:42:05 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-24 01:44:30 +0000 sys-devel/gcc: add 11.3.1_p20221223 As remarked on IRC, this + the last GCC 10 snapshot + GCC 12 tomorrow are likely the last ones to be keyworded for a little while, as we want the make 4.4 fixes, with the exception of 12 which might get another one kewyorded beyond this weekend if there's a fix for the PPC + LLVM issues. Bug: https://bugs.gentoo.org/884633 Signed-off-by: Sam James <sam@gentoo.org> sys-devel/gcc/Manifest | 1 + sys-devel/gcc/gcc-11.3.1_p20221223.ebuild | 52 +++++++++++++++++++++++++++++++ 2 files changed, 53 insertions(+)
(In reply to Arsen Arsenovic from comment #7) > Done upstream in 10 and 11 respectively: > > https://gcc.gnu.org/git/?p=gcc.git;a=commit; > h=51e819e99f3e800d2fb1cb22b4648e6d0558e35b > https://gcc.gnu.org/git/?p=gcc.git;a=commit; > h=c4de50835f731b301bd8161be5318de701f2864a Thanks for doing these!
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=bdf7258fd2b4c203d62aeaaa38d63227ad36f384 commit bdf7258fd2b4c203d62aeaaa38d63227ad36f384 Author: Sam James <sam@gentoo.org> AuthorDate: 2022-12-25 17:02:10 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-25 17:02:18 +0000 sys-devel/gcc: add 12.2.1_p20221224 See 316d250ee4d7bf08015da05b04e56264735cb178. Closes: https://bugs.gentoo.org/884633 Signed-off-by: Sam James <sam@gentoo.org> sys-devel/gcc/Manifest | 2 ++ sys-devel/gcc/gcc-12.2.1_p20221224.ebuild | 52 +++++++++++++++++++++++++++++++ 2 files changed, 54 insertions(+)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/gcc-patches.git/commit/?id=d182e15bbb3333f34afd4a6396108c1e581e4872 commit d182e15bbb3333f34afd4a6396108c1e581e4872 Author: Sam James <sam@gentoo.org> AuthorDate: 2022-12-25 16:57:30 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-25 16:57:30 +0000 12.2.0: drop upstreamed make 4.4 patches Bug: https://bugs.gentoo.org/884633 Signed-off-by: Sam James <sam@gentoo.org> .../78_all_Factor-out-jobserver_active_p.patch | 284 --------------------- ...-jobserver-style-fifo-for-recent-GNU-make.patch | 64 ----- ...spect-jobserver-in-parallel-WPA-streaming.patch | 225 ---------------- ...ix-fifo-mode-by-opening-pipe-in-proper-mo.patch | 31 --- 12.2.0/gentoo/README.history | 6 + 5 files changed, 6 insertions(+), 604 deletions(-)
