Latest update breaks qtwebengine ebuild if one tries to compile it with a more powerful host or if distcc is enabled. Latest patch to the ebuild looks at the `make -j` options to compute the memory requirements like so: local CHECKREQS_MEMORY=$(($(makeopts_jobs)*multiplier/10))G However this does not take into account that -j can be high due to distcc usage when compiling said package and blocks the ebuild with large memory requirements. Even without distcc this memory requirement seems to be overly aggressive. Having 16-32 cores is relatively accessible these days (e.g. Ryzen 3950X, which is at this point 2 years old), which would compute to a 32-64GB memory requirement, which I feel is probably less common (due to memory prices or motherboard limitations, or lack of need, e.g. 32 core + 32GB host would fail this requirement). Qtwebengine is based on Chromium 87.0.4280, which is older than the actual browser (www-client/chromium-98.0.4758.9) we have in tree and the latter can be built with a 16GB memory requirement. While there might be some differences between qtwebengine and Chromium build process, I doubt it's this big (though I'm not an expert here), given that the previous version (5.15.2_p20211019) of this ebuild built just fine. Reproducible: Always Steps to Reproduce: 1. try to update qtwebengine Actual Results: >>> emerge -1 qtwebengine Calculating dependencies... done! >>> Verifying ebuild manifests >>> Running pre-merge checks for dev-qt/qtwebengine-5.15.2_p20211216 * Checking for at least 192 GiB RAM ... [ !! ] * There is NOT at least 192 GiB RAM * Checking for at least 7 GiB disk space at "/tmp/portage/dev-qt/qtwebengine-5.15.2_p20211216/temp" ... [ ok ] * Checking for at least 150 MiB disk space at "/usr" ... [ ok ] * * Space constraints set in the ebuild were not met! * The build will most probably fail, you should enhance the space * as per failed tests. * * ERROR: dev-qt/qtwebengine-5.15.2_p20211216::gentoo failed (pretend phase): * Build requirements not met! * * Call stack: * ebuild.sh, line 127: Called pkg_pretend * qtwebengine-5.15.2_p20211216.ebuild, line 145: Called qtwebengine_check-reqs * qtwebengine-5.15.2_p20211216.ebuild, line 141: Called check-reqs_pkg_pretend * check-reqs.eclass, line 98: Called check-reqs_pkg_setup * check-reqs.eclass, line 89: Called _check-reqs_output * check-reqs.eclass, line 296: Called die * The specific snippet of code: * [[ ${EBUILD_PHASE} == "pretend" && -z ${CHECKREQS_DONOTHING} ]] && \ * die "Build requirements not met!" * * If you need support, post the output of `emerge --info '=dev-qt/qtwebengine-5.15.2_p20211216::gentoo'`, * the complete build log and the output of `emerge -pqv '=dev-qt/qtwebengine-5.15.2_p20211216::gentoo'`. * The complete build log is located at '/tmp/portage/dev-qt/qtwebengine-5.15.2_p20211216/temp/build.log'. * The ebuild environment file is located at '/tmp/portage/dev-qt/qtwebengine-5.15.2_p20211216/temp/die.env'. * Working directory: '/tmp/portage/dev-qt/qtwebengine-5.15.2_p20211216/empty' * S: '/tmp/portage/dev-qt/qtwebengine-5.15.2_p20211216/work/qtwebengine-5.15.2_p20211216' Expected Results: qtwebengine proceeds and compiles >>> emerge --info qtwebengine Portage 3.0.30 (python 3.9.9-final-0, default/linux/amd64/17.1/desktop/plasma/systemd, gcc-10.3.1, glibc-2.34-r4, 5.15.11-gentoo x86_64) ================================================================= System Settings ================================================================= System uname: Linux-5.15.11-gentoo-x86_64-Intel-R-_Core-TM-_i7-8550U_CPU_@_1.80GHz-with-glibc2.34 KiB Mem: 32773940 total, 24793584 free KiB Swap: 2097128 total, 2097128 free Timestamp of repository gentoo: Wed, 05 Jan 2022 08:30:01 +0000 Head commit of repository gentoo: 51126a5987daca8a9ceb8dd9545300efb32373c7 sh bash 5.1_p12 ld GNU ld (Gentoo 2.37_p1 p1) 2.37 distcc 3.4 x86_64-pc-linux-gnu [enabled] app-misc/pax-utils: 1.3.3::gentoo app-shells/bash: 5.1_p12::gentoo dev-java/java-config: 2.3.1::gentoo dev-lang/perl: 5.34.0-r6::gentoo dev-lang/python: 2.7.18_p13::gentoo, 3.8.12_p1::gentoo, 3.9.9::gentoo, 3.10.1-r1::gentoo dev-lang/rust: 1.57.0::gentoo dev-util/cmake: 3.22.1::gentoo dev-util/meson: 0.60.3::gentoo sys-apps/baselayout: 2.8::gentoo sys-apps/sandbox: 2.29::gentoo sys-apps/systemd: 250-r1::gentoo sys-devel/autoconf: 2.13-r1::gentoo, 2.71-r1::gentoo sys-devel/automake: 1.16.5::gentoo sys-devel/binutils: 2.37_p1-r1::gentoo sys-devel/binutils-config: 5.4::gentoo sys-devel/clang: 13.0.0::gentoo sys-devel/gcc: 10.3.1_p20211126::gentoo sys-devel/gcc-config: 2.5-r1::gentoo sys-devel/libtool: 2.4.6-r6::gentoo sys-devel/llvm: 13.0.0::gentoo sys-devel/make: 4.3::gentoo sys-kernel/linux-headers: 5.15-r1::gentoo (virtual/os-headers) sys-libs/glibc: 2.34-r4::gentoo Repositories: gentoo location: /var/db/repos/gentoo sync-type: rsync sync-uri: rsync://lambda/gentoo-portage priority: -1000 sync-rsync-extra-opts: sync-rsync-verify-metamanifest: yes sync-rsync-verify-max-age: 24 sync-rsync-verify-jobs: 1 sinustrom-test location: /var/db/repos/sinustrom-test masters: gentoo sinustrom location: /var/lib/layman/sinustrom sync-type: laymansync sync-uri: https://github.com/zpuskas/sinustrom-gentoo-overlay.git masters: gentoo priority: 50 Installed sets: @openwrt-prerequisites ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="@FREE" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=skylake -mmmx -msse -msse2 -msse3 -mssse3 -mcx16 -msahf -mmovbe -maes -mpclmul -mpopcnt -mabm -mfma -mbmi -msgx -mbmi2 -mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mrdrnd -mf16c -mfsgsbase -mrdseed -mprfchw -madx -mfxsr -mxsave -mxsaveopt -mclflushopt -mxsavec -mxsaves --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=skylake -O3 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=skylake -mmmx -msse -msse2 -msse3 -mssse3 -mcx16 -msahf -mmovbe -maes -mpclmul -mpopcnt -mabm -mfma -mbmi -msgx -mbmi2 -mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mrdrnd -mf16c -mfsgsbase -mrdseed -mprfchw -madx -mfxsr -mxsave -mxsaveopt -mclflushopt -mxsavec -mxsaves --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=skylake -O3 -pipe" DISTDIR="/var/cache/distfiles" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live cgroup collision-protect config-protect-if-modified distcc distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict strict-keepdir unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="rsync://lambda/gentoo-distfiles http://mirror.csclub.uwaterloo.ca/gentoo-distfiles/" LANG="en_US.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--defsym=__gentoo_check_ldflags__=0" MAKEOPTS="-j96 -l8" PKGDIR="/var/cache/binpkgs" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/tmp" SHELL="/bin/zsh" USE="X a52 aac acl acpi activities aes alsa amd64 avx avx2 bash-completion bluetooth branding bzip2 cairo cdda cdr cli crypt cups dbus declarative dri dts dvd dvdr emboss encode exif f16c flac fma3 fortran gdbm gif gphoto2 gpm gtk gui hardened iconv icu ipv6 jpeg jpeg2k kde kipi kwallet lcms libglvnd libnotify libtirpc lm_sensors lzma mad mmx mmxext mng mp3 mp4 mpeg multilib ncurses networkmanager nls nptl ogg opencl opengl openmp pam pango pclmul pcre pdf plasma png policykit popcnt posix ppds pulseaudio qml qt5 readline sdl seccomp smp spell split-usr sse sse2 sse3 sse4_1 sse4_2 ssl ssse3 startup-notification svg systemd threads tiff truetype udev udisks unicode upower usb uxa vaapi vdpau vim-syntax vorbis widgets wifi wxwidgets x264 xattr xcb xml xpm xv xvid zlib zstd" ABI_X86="64" ADA_TARGET="gnat_2020" ALSA_CARDS="hda-intel usb-audio" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput wacom synaptics joystick" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LLVM_TARGETS="x86 BPF" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-3 php7-4" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_9" PYTHON_TARGETS="python3_9" QEMU_SOFTMMU_TARGETS="x86_64 arm aarch64" QEMU_USER_TARGETS="x86_64 arm aarch64" RUBY_TARGETS="ruby27 ruby30" SANE_BACKENDS="plustek genesys pixma" USERLAND="GNU" VIDEO_CARDS="intel i965 nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LEX, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS ================================================================= Package Settings ================================================================= dev-qt/qtwebengine-5.15.2_p20211216::gentoo was built with the following: USE="alsa jumbo-build pulseaudio system-ffmpeg system-icu widgets -bindist -debug -designer -geolocation -kerberos -test" ABI_X86="(64)" FEATURES="binpkg-dostrip pid-sandbox usersandbox userpriv fixlafiles ipc-sandbox preserve-libs qa-unresolved-soname-deps parallel-fetch sandbox xattr strict sfperms buildpkg-live network-sandbox binpkg-docompress config-protect-if-modified userfetch cgroup merge-sync collision-protect protect-owned strict-keepdir unmerge-logs binpkg-logs fail-clean distlocks multilib-strict unmerge-orphans distcc binpkg-multi-instance news ebuild-locks usersync unknown-features-warn assume-digests"
CHECKREQS_MEMORY_MANGLE_JOBS=no in make.conf should prevent this from happening. it will be fixed for distcc usecase a bit later. details: https://archives.gentoo.org/gentoo-dev/message/2c399e67d1445d73ff467e5e27782e26
please ignore ^ comment. this will not help and is not merged yet.
1. check-reqs can be bypassed anyway 2. It's a valid point that we should skip on distcc 3. I really don't think 2GB per job is aggressive at all for this.. it's not about core count.
typical usecase with distcc is to specify load-avg equal to local cores. so we could detect if distcc enabled and use $(makeopts_loadavg) and fall-back to $(get_nproc) if former is not specified.
Created attachment 761388 [details, diff] 0001-dev-qt-qtwebengine-UNTESTED.-extend-ram-check-for-di.patch sample patch. I think I've covered the gotchas, but it's completely untested.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/qt.git/commit/?id=61c6b4e97824f4547983dffead5ca392d825cbbc commit 61c6b4e97824f4547983dffead5ca392d825cbbc Author: Sam James <sam@gentoo.org> AuthorDate: 2022-01-06 19:53:32 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-01-06 19:57:15 +0000 dev-qt/qtwebengine: disable CHECKREQS_MEMORY for distcc It's harder to calculate properly for this case. Bug: https://bugs.gentoo.org/830661 Signed-off-by: Sam James <sam@gentoo.org> dev-qt/qtwebengine/qtwebengine-5.15.2.9999.ebuild | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=2b2510ae5cffece67dc8b03cd463d23eaaa3bebe commit 2b2510ae5cffece67dc8b03cd463d23eaaa3bebe Author: Sam James <sam@gentoo.org> AuthorDate: 2022-01-06 19:53:32 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-01-06 19:59:43 +0000 dev-qt/qtwebengine: disable CHECKREQS_MEMORY for distcc It's harder to calculate properly for this case. Bug: https://bugs.gentoo.org/830661 Signed-off-by: Sam James <sam@gentoo.org> dev-qt/qtwebengine/qtwebengine-5.15.2_p20211216.ebuild | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
same on my end. Really sucks :). See attached relevant outputs. Machine = h t t p s : // linux-hardware.org/?probe=597fae9cb6
Created attachment 762713 [details] emerge --ask dev-qt/qtwebengine $ emerge --ask dev-qt/qtwebengine
Created attachment 762714 [details] /etc/portage/make.conf cat /etc/portage/make.conf
Created attachment 762715 [details] $ cat /etc/fstab $ cat /etc/fstab
Created attachment 762716 [details] emerge --info '=dev-qt/qtwebengine-5.15.2_p20211216::gentoo'
Created attachment 762717 [details] emerge -pqv '=dev-qt/qtwebengine-5.15.2_p20211216::gentoo' emerge -pqv '=dev-qt/qtwebengine-5.15.2_p20211216::gentoo'
`export I_KNOW_WHAT_I_AM_DOING=1` works
(In reply to zafarella from comment #8) > same on my end. Really sucks :). See attached relevant outputs. Machine = > h t t p s : // linux-hardware.org/?probe=597fae9cb6 I don't actually see the problem though. Please share the log of it failing and explain why you feel it's okay. We already added the exception for distcc and the general RAM requirement for qtwebengine is correct anyway.
