I was updating a Linode VM this afternoon, and compilation of the current gcc failed twice. The host is provisioned with 4 GB RAM, which has always been sufficient in the past, but when I checked dmesg after the second failed compile, it indicated that the out-of-memory killer had killed a cc1plus process that was part of the gcc ebuild. I've enabled 4 GB of swap (the host normally runs without swap) that should enable the build to complete, but perhaps gcc is getting big enough that it now needs a pre-build memory check like some other large packages (Chromium, etc.). Reproducible: Always
Please add 'emerge --info' in a comment.
Created attachment 855836 [details] archived build log
(In reply to Wapuk-cobaka from comment #2) > Created attachment 855836 [details] > archived build log Tried to update a VM with 4gb of RAM. Process fails on: sys-devel/gcc-12.2.1_p20230121-r1 emerge --info '=sys-devel/gcc-12.2.1_p20230121-r1::gentoo' Portage 3.0.44 (python 3.10.10-final-0, default/linux/amd64/17.1/systemd/merged-usr, gcc-11.3.0, glibc-2.36-r5, 5.15.77- gentoo-dist x86_64) ================================================================= System Settings ================================================================= System uname: Linux-5.15.77-gentoo-dist-x86_64-Intel-R-_Xeon-R-_Silver_4210_CPU_@_2.20GHz-with-glibc2.36 KiB Mem: 4022052 total, 3538300 free KiB Swap: 0 total, 0 free Timestamp of repository gentoo: Wed, 01 Mar 2023 19:15:01 +0000 Head commit of repository gentoo: 5484ef531559e140fb261d20f5b3c224c7a4604c sh bash 5.1_p16-r2 ld GNU ld (Gentoo 2.39 p5) 2.39.0 app-misc/pax-utils: 1.3.5::gentoo app-shells/bash: 5.1_p16-r2::gentoo dev-lang/perl: 5.36.0-r1::gentoo dev-lang/python: 3.10.10_p2::gentoo, 3.11.1-r1::gentoo dev-util/cmake: 3.24.3::gentoo dev-util/meson: 1.0.0::gentoo sys-apps/baselayout: 2.9::gentoo sys-apps/sandbox: 2.29::gentoo sys-apps/systemd: 252.4-r1::gentoo sys-devel/autoconf: 2.71-r5::gentoo sys-devel/automake: 1.16.5::gentoo sys-devel/binutils: 2.39-r4::gentoo sys-devel/binutils-config: 5.4.1::gentoo sys-devel/gcc: 11.3.0::gentoo sys-devel/gcc-config: 2.8::gentoo sys-devel/libtool: 2.4.7-r1::gentoo sys-devel/make: 4.3::gentoo sys-kernel/linux-headers: 6.1::gentoo (virtual/os-headers) sys-libs/glibc: 2.36-r5::gentoo Repositories: gentoo location: /var/db/repos/gentoo sync-openpgp-key-refresh: no sync-type: rsync sync-uri: rsync: / / repo.update/gentoo-portage priority: -1000 volatile: True sync-rsync-extra-opts: sync-rsync-verify-jobs: 1 sync-rsync-verify-max-age: 24 sync-rsync-verify-metamanifest: yes Binary Repositories: binhost priority: 9999 sync-uri: http: / / binpkgs.update ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="@FREE" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -march=cascadelake -mtune=cascadelake" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbo x.d" CXXFLAGS="-O2 -pipe -march=cascadelake -mtune=cascadelake" DISTDIR="/var/cache/distfiles" EMERGE_DEFAULT_OPTS=" --getbinpkg --usepkg --usepkg-exclude-live" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME " FCFLAGS="-O2 -pipe -march=cascadelake -mtune=cascadelake" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg buildpkg-live candy config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync metadata-transfer multilib-strict n etwork-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms sig n strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe -march=cascadelake -mtune=cascadelake" GENTOO_MIRRORS="http: / / distfiles.gentoo.org" LANG="en_US.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LEX="flex" LINGUAS="ru en" MAKEOPTS="-j4 -l4" PKGDIR="/var/cache/binpkgs" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file -- delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" SHELL="/bin/bash" USE="acl amd64 bash-completion bzip2 cli crypt dri fortran gdbm iconv ipv6 libglvnd libtirpc lzma multilib ncurses nls n ptl openmp pam pcre readline seccomp ssl systemd test-rust udev unicode vim vim-syntax xattr zlib" ABI_X86="64" ADA_TARG ET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_ano n authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhos t_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU _FLAGS_X86="aes avx avx2 avx512f avx512dq avx512cd avx512bw avx512vl f16c fma3 mmx mmxext pclmul popcnt rdrand sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timi ng tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64 pc" INPUT_DEVICES="libinput" KERNEL="linux" L10N="ru en" LCD_DEVI CES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console pr esenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7 -4 php8-0" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_10" PYTHON_TARGETS="python3_10" RUBY_T ARGETS="ruby31" USERLAND="GNU" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy v4l" XTABLES_ADDONS="q uota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat l ogmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTR A_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PO RTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READ ELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
After increasing VM's RAM from 4gb to 8gb system successfully updated.
All we can suggest for this is lower MAKEOPTS, dropping -pipe from *FLAGS, and adding -Wl,--no-keep-memory to LDFLAGS.
(In reply to Greg Kubaryk from comment #1) > Please add 'emerge --info' in a comment. Portage 3.0.44 (python 3.10.10-final-0, default/linux/amd64/17.1, gcc-12, glibc-2.36-r7, 6.1.19-gentoo-x86_64 x86_64) ================================================================= System uname: Linux-6.1.19-gentoo-x86_64-x86_64-Intel-R-_Xeon-R-_Gold_6148_CPU_@_2.40GHz-with-glibc2.36 KiB Mem: 4027844 total, 180876 free KiB Swap: 524284 total, 340220 free Head commit of repository gentoo: 8d75300ebec012a04f3ff6fe12b604bb7b161b25 sh bash 5.1_p16-r2 ld GNU ld (Gentoo 2.39 p5) 2.39.0 ccache version 4.7.4 [enabled] app-misc/pax-utils: 1.3.5::gentoo app-shells/bash: 5.1_p16-r2::gentoo dev-lang/perl: 5.36.0-r2::gentoo dev-lang/python: 3.10.10_p3::gentoo, 3.11.2_p2::gentoo dev-lang/rust: 1.66.1::gentoo dev-util/ccache: 4.7.4::gentoo dev-util/cmake: 3.25.3::gentoo dev-util/meson: 1.0.1::gentoo sys-apps/baselayout: 2.13-r1::gentoo sys-apps/openrc: 0.46::gentoo sys-apps/sandbox: 2.29::gentoo sys-devel/autoconf: 2.71-r5::gentoo sys-devel/automake: 1.16.5::gentoo sys-devel/binutils: 2.39-r4::gentoo sys-devel/binutils-config: 5.5::gentoo sys-devel/gcc: 12.2.1_p20230121-r1::gentoo sys-devel/gcc-config: 2.10::gentoo sys-devel/libtool: 2.4.7-r1::gentoo sys-devel/make: 4.3::gentoo sys-kernel/linux-headers: 6.1::gentoo (virtual/os-headers) sys-libs/glibc: 2.36-r7::gentoo Repositories: gentoo location: /var/db/repos/gentoo sync-type: git sync-uri: https://github.com/gentoo/gentoo priority: -1000 volatile: True salfter location: /var/lib/layman/salfter sync-type: laymansync sync-uri: https://gitlab.com/salfter/portage.git masters: gentoo priority: 50 volatile: True ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -mtune=native" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/php/apache2-php7.4/ext-active/ /etc/php/apache2-php8.0/ext-active/ /etc/php/apache2-php8.1/ext-active/ /etc/php/cgi-php7.4/ext-active/ /etc/php/cgi-php8.0/ext-active/ /etc/php/cgi-php8.1/ext-active/ /etc/php/cli-php7.4/ext-active/ /etc/php/cli-php8.0/ext-active/ /etc/php/cli-php8.1/ext-active/ /etc/php/fpm-php7.4/ext-active/ /etc/php/fpm-php8.0/ext-active/ /etc/php/fpm-php8.1/ext-active/ /etc/php/phpdbg-php7.4/ext-active/ /etc/php/phpdbg-php8.0/ext-active/ /etc/php/phpdbg-php8.1/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -pipe -mtune=native" DISTDIR="/var/cache/distfiles" EMERGE_DEFAULT_OPTS="--autounmask-write --quiet-build=y" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg-live ccache config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="https://gentoo.osuosl.org/" LANG="en_US.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LEX="flex" MAKEOPTS="-j2" PKGDIR="/var/cache/binpkgs" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" SHELL="/bin/bash" USE="acl amd64 bzip2 cli crypt dri fortran gdbm iconv ipv6 libglvnd libtirpc multilib ncurses nls nptl openmp pam pcre readline seccomp split-usr ssl test-rust unicode vhosts xattr zlib" ABI_X86="64" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 avx512f avx512dq avx512cd avx512bw avx512vl f16c fma3 mmx mmxext pclmul popcnt rdrand sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-4 php8-0" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_10" PYTHON_TARGETS="python3_10" RUBY_TARGETS="ruby30" USERLAND="GNU" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
I'm in the middle of trying to build gcc with -pipe disabled in /etc/portage/make.conf...hopefully it'll work. If it does, is there a way to override CFLAGS on a per-ebuild basis?
(In reply to Scott Alfter from comment #7) > I'm in the middle of trying to build gcc with -pipe disabled in > /etc/portage/make.conf...hopefully it'll work. If it does, is there a way > to override CFLAGS on a per-ebuild basis? Yeah, you can do it via https://wiki.gentoo.org/wiki//etc/portage/package.env.
(In reply to Sam James from comment #8) > (In reply to Scott Alfter from comment #7) > > I'm in the middle of trying to build gcc with -pipe disabled in > > /etc/portage/make.conf...hopefully it'll work. If it does, is there a way > > to override CFLAGS on a per-ebuild basis? > > Yeah, you can do it via > https://wiki.gentoo.org/wiki//etc/portage/package.env. I found https://wiki.gentoo.org/wiki/Handbook:AMD64/Portage/Advanced#Per-package_environment_variables, which looks like it describes the same thing. Testing again...
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/gcc-patches.git/commit/?id=7a6b2d23ec02e75475a6123254ccd44d73827f39 commit 7a6b2d23ec02e75475a6123254ccd44d73827f39 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-05-26 02:26:36 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-05-26 02:26:36 +0000 13.2.0: backport split match.pd changes See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109927#c21. This should improve (reduce) the peak RAM usage when building GCC as it builds gimple-match in smaller chunks (previously was one huge generated file) as well as speeding up parallel build when used. i.e. This has benefits for both people building on constrained hardware (because -j1 is more effective and just means each smaller chunk is built one-at-a-time, not the massive blob) and also for people building with many jobs, as there's one less synchronisation point so the build is less serial. Motivated in particular by the upstream report (PR109927) of this being an issue on m68k with RAM usage for the single GCC process building gimple-match. (IIRC immolo and possibly dilfridge had mentioned similar issues when building some arches in qemu as well.) Note that upstream are planning on splitting insn-* as well so there are further improvements in the pipeline (see PR54179, but discussion has occurred far more recently on IRC about this and tamar is likely to look at it.). Bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54179 (old bug for splitting insn-*, will be revisited) Bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84402 (tracker bug for parallelisation, interesting discussion) Bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109927 Bug: https://bugs.gentoo.org/891909 (not the same issue per-se but related to reduced resource usage) Bug: https://bugs.gentoo.org/894256 Bug: https://bugs.gentoo.org/901317 Signed-off-by: Sam James <sam@gentoo.org> ...l_match.pd-don-t-emit-label-if-not-needed.patch | 114 + ...move-commented-out-line-pragmas-unless-vv.patch | 41 + ...78_all_match.pd-CSE-the-dump-output-check.patch | 70 + ...lit-shared-code-to-gimple-match-exports.c.patch | 2560 ++++++++++++++++++++ ...pd-automatically-partition-match.cc-files.patch | 524 ++++ ...e-splits-in-makefile-and-make-configurabl.patch | 258 ++ 13.2.0/gentoo/README.history | 10 + 7 files changed, 3577 insertions(+)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=aa5725d7e4407bdfa4137cb0518c9d6ffaf5d8ae commit aa5725d7e4407bdfa4137cb0518c9d6ffaf5d8ae Author: Sam James <sam@gentoo.org> AuthorDate: 2023-05-26 02:40:06 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-05-26 02:40:51 +0000 sys-devel/gcc: backport split match.pd changes to 13.1.1_p20230520 See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109927#c21. This should improve (reduce) the peak RAM usage when building GCC as it builds gimple-match in smaller chunks (previously was one huge generated file) as well as speeding up parallel build when used. i.e. This has benefits for both people building on constrained hardware (because -j1 is more effective and just means each smaller chunk is built one-at-a-time, not the massive blob) and also for people building with many jobs, as there's one less synchronisation point so the build is less serial. Motivated in particular by the upstream report (PR109927) of this being an issue on m68k with RAM usage for the single GCC process building gimple-match. (IIRC immolo and possibly dilfridge had mentioned similar issues when building some arches in qemu as well.) Note that upstream are planning on splitting insn-* as well so there are further improvements in the pipeline (see PR54179, but discussion has occurred far more recently on IRC about this and tamar is likely to look at it.). Bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54179 (old bug for splitting insn-*, will be revisited) Bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84402 (tracker bug for parallelisation, interesting discussion) Bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109927 Bug: https://bugs.gentoo.org/891909 (not the same issue per-se but related to reduced resource usage) Bug: https://bugs.gentoo.org/894256 Bug: https://bugs.gentoo.org/901317 Signed-off-by: Sam James <sam@gentoo.org> sys-devel/gcc/Manifest | 2 +- sys-devel/gcc/gcc-13.1.1_p20230520.ebuild | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
