This was working as expected since around Jan 8th or 9th till 9th Feb, but when I upgraded on 9th Feb, it started to produce output as -- >>> Emerging (1 of 6) app-cdr/xfburn-0.7.0::gentoo >>> Emerging (2 of 6) media-sound/audacity-3.3.3::gentoo >>> Emerging (3 of 6) app-emulation/virtualbox-7.0.12-r2::gentoo >>> Emerging (4 of 6) media-video/parole-4.18.1::gentoo >>> Emerging (5 of 6) net-im/telegram-desktop-4.13.1::gentoo >>> Failed to emerge net-im/telegram-desktop-4.13.1, Log file: >>> '/tmp/portage/net-im/telegram-desktop-4.13.1/temp/build.log' >>> Jobs: 0 of 6 complete, 1 running, 1 failed Load avg: 8.89, 7.43, 4.53 Note there is no 'installing' or 'completed' phase as in the previous commit of portage -- it does come but at the wrong time. In the above e.g only 1 ebuild is being compiled (check the jobs:) and it looks like it's compiling 5. Mysteriously even the output of quickpkg has changed and follows similar behavior. It's going to list 5 or 6 packages at the same time making the user believe that it's packaging in parallel, but in reality it's only packaging 1 package at a time. Portage 3.0.61_p43 (python 3.11.7-final-0, default/linux/amd64/17.1/desktop/plasma/systemd/merged-usr, gcc-13, glibc-2.38-r10, 6.1.70-gentoo-gentoo-dist x86_64) ================================================================= System uname: Linux-6.1.70-gentoo-gentoo-dist-x86_64-12th_Gen_Intel-R-_Core-TM-_i3-12100-with-glibc2.38 KiB Mem: 32613760 total, 14350044 free KiB Swap: 5242876 total, 5242876 free Head commit of repository oubliette: d74b8676d696ec4a5304fda055d2c7fe9b528288 Timestamp of repository gentoo: Fri, 09 Feb 2024 00:45:01 +0000 sh bash 5.1_p16-r6 ld GNU ld (Gentoo 2.41 p4) 2.41.0 ccache version 4.9.1 [enabled] app-misc/pax-utils: 1.3.7::gentoo app-shells/bash: 5.1_p16-r6::gentoo dev-build/autoconf: 2.13-r8::gentoo, 2.71-r6::gentoo dev-build/automake: 1.16.5-r2::gentoo dev-build/cmake: 3.27.9::gentoo dev-build/libtool: 2.4.7-r1::gentoo dev-build/make: 4.4.1-r1::gentoo dev-build/meson: 1.3.0-r2::gentoo dev-java/java-config: 2.3.3-r1::gentoo dev-lang/perl: 5.38.2-r1::gentoo dev-lang/python: 3.11.7::gentoo, 3.12.1_p1::gentoo dev-lang/rust: 1.74.1::gentoo dev-util/ccache: 4.9.1::gentoo sys-apps/baselayout: 2.14-r2::gentoo sys-apps/openrc: 0.53::gentoo sys-apps/sandbox: 2.38::gentoo sys-apps/systemd: 255.3::gentoo sys-devel/binutils: 2.41-r3::gentoo sys-devel/binutils-config: 5.5::gentoo sys-devel/clang: 15.0.7-r3::gentoo, 17.0.6::gentoo sys-devel/gcc: 13.2.1_p20240113-r1::gentoo sys-devel/gcc-config: 2.11::gentoo sys-devel/lld: 15.0.7::gentoo, 17.0.6::gentoo sys-devel/llvm: 15.0.7-r3::gentoo, 17.0.6::gentoo sys-kernel/linux-headers: 6.6::gentoo (virtual/os-headers) sys-libs/glibc: 2.38-r10::gentoo sys-libs/libselinux: 3.5-r1::gentoo Repositories: oubliette location: /var/db/repos/oubliette sync-type: git sync-uri: https://github.com/nabbi/oubliette-overlay masters: gentoo priority: 50 volatile: False gentoo location: /usr/portage sync-type: git sync-uri: https://github.com/gentoo-mirror/gentoo priority: 100 volatile: True ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -fomit-frame-pointer -floop-interchange -floop-strip-mine -floop-block -fgraphite-identity -ftree-loop-distribution -O3 -pipe -flto=1 -fuse-linker-plugin -ffat-lto-objects" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt /var/bind" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php8.1/ext-active/ /etc/php/cgi-php8.1/ext-active/ /etc/php/cli-php8.1/ext-active/ /etc/php/fpm-php8.1/ext-active/ /etc/php/phpdbg-php8.1/ext-active/ /etc/revdep-rebuild /etc/sandbox.d" CXXFLAGS="-march=native -fomit-frame-pointer -floop-interchange -floop-strip-mine -floop-block -fgraphite-identity -ftree-loop-distribution -O3 -pipe -flto=1 -fuse-linker-plugin -ffat-lto-objects" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--with-bdeps=n --complete-graph --binpkg-respect-use=y --binpkg-changed-deps=n --changed-deps=n --rebuild-if-new-slot=y" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live candy ccache config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync merge-wait multilib-strict network-sandbox news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://gentoo.gossamerhost.com/" LANG="en_US.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LEX="flex" MAKEOPTS="-j8" PKGDIR="/mnt/systmp/packages" PORTAGE_BINHOST="/mnt/systmp/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/tmp" SHELL="/bin/bash" USE="0libedit X a52 a53 aac acc acpi adns aio amd64 amr asm atm audiofile avif bash-completion bluray branding brotli bzip2 c++0x c++11 cdda cdr cli consolekit cracklib css custom-cflags custom-optimization dav1d dbx declarative default-gold dga dirac dri dri3 dts dv dvd dvdr encode enscript exif faad ffmpeg firefox flac fortran gdu gif gles gles1 gles2 gles2-only gles3 glslang glut gmp gstreamer gui gzip heif http2 http3 iconv idn imlib io-uring ipc ipv6 jemalloc jit jpeg jpeg2k kde lame libevent libnotify libsamplerate libtirpc libwww lto lz4 lzma lzo mad matroska matrox mikmod mime minimal mmap mmx mozilla mp3 mp4 mpeg mplayer mpris mudflap multilib native-extensions networkmanager nntp nocd nptl nptlonly nsplugin offensive ogg openal opencl opengl openmp optimization opus orc osc pam pango pdf pgo phonon pie plasma png pnm policykit posix ppds pppd pulseaudio qt5 quicktime raw readline schroedinger seccomp sharedmem shorten smp sndfile sockets socks5 sound spell sse sse2 sse3 sse4 sse4_1 sse4_2 ssse3 startup-notification strip strong-optimization svg svg2 symlink sysfs systemd taglib tcmalloc test-rust tfo theora threads tiff toolame truetype twolame udev udisks unicode unixd upower usb vaapi vcd verify-sig vhosts vorbis vpx vulkan webp widevine widgets win32codecs wxwidgets x264 x265 xcb xft xine xml xorg xpm xv xvid xxhash zlib" ABI_X86="64" ADA_TARGET="gnat_2021" APACHE2_MODULES="unixd authn_core authz_core authz_host access_compat alias auth_basic authn_file log_config mime log_config proxy proxy_http rewrite proxy_balancer lbmethod_bybusyness lbmethod_byrequests lbmethod_bytraffic lbmethod_heartbeat slotmem_shm env cache dir cgid headers proxy_connect" APACHE2_MPMS="event" CALLIGRA_FEATURES="karbon sheets words" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" GRUB_PLATFORMS="pc" INPUT_DEVICES="evdev" KERNEL="linux" L10N="en" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LIBREOFFICE_EXTENSIONS="nlpsolver pdfimport" LLVM_TARGETS="X86" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" NGINX_MODULES_HTTP="access fastcgi gzip gzip_static limit_req rewrite upstream_check proxy" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-1" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_11" QEMU_SOFTMMU_TARGETS="x86_64" RUBY_TARGETS="ruby31" VIDEO_CARDS="intel i965 i915" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
It's supposed to compile them in parallel if you use emerge --jobs, but delay the install until no builds are running due to the new default FEATURES=merge-wait setting related to bug 663324. If you set FEATURES="-merge-wait" in make.conf then you'll get back the previous behavior. (In reply to dE from comment #0) > This was working as expected since around Jan 8th or 9th till 9th Feb, but > when I upgraded on 9th Feb, it started to produce output as -- > >>> Emerging (1 of 6) app-cdr/xfburn-0.7.0::gentoo > >>> Emerging (2 of 6) media-sound/audacity-3.3.3::gentoo > >>> Emerging (3 of 6) app-emulation/virtualbox-7.0.12-r2::gentoo > >>> Emerging (4 of 6) media-video/parole-4.18.1::gentoo > >>> Emerging (5 of 6) net-im/telegram-desktop-4.13.1::gentoo > >>> Failed to emerge net-im/telegram-desktop-4.13.1, Log file: > >>> '/tmp/portage/net-im/telegram-desktop-4.13.1/temp/build.log' > >>> Jobs: 0 of 6 complete, 1 running, 1 failed Load avg: 8.89, 7.43, 4.53 > > Note there is no 'installing' or 'completed' phase as in the previous commit > of portage -- it does come but at the wrong time. That's the FEATURES=merge-wait change. > In the above e.g only 1 > ebuild is being compiled (check the jobs:) and it looks like it's compiling > 5. That's odd because I'd expect them all to be in the running state rather than just "1 running" as shown there. I tested locally with 3 packages just now and FEATURES=merge-wait, and all 3 packages immediately entered the running state: >>> Jobs: 0 of 3 complete, 3 running Load avg: 0.78, 0.97, 1.39 FEATURES=merge-wait is only supposed to affect what happens when a build completes, since it's implemented where PackageMerge instances are created and handled in a _build_exit callback after a build completes: https://gitweb.gentoo.org/proj/portage.git/commit/?id=825db01b91a37dcd9890ee5bf9f462ea524ac5cc
With FEATURE=merge-wait, if some package has finish compiling while others are With -merge-wait things start working fine.
(In reply to dE from comment #2) > With FEATURE=merge-wait, if some package has finish compiling while others > are With -merge-wait things start working fine. This typically works pretty well, but be warned that there are certainly some rare cases like bug 663324 where you could experience "random" build failures, so that is why it is no longer the default behavior. (In reply to Zac Medico from comment #1) > (In reply to dE from comment #0) > > In the above e.g only 1 > > ebuild is being compiled (check the jobs:) and it looks like it's compiling > > 5. > > That's odd because I'd expect them all to be in the running state rather > than just "1 running" as shown there. I tested locally with 3 packages just > now and FEATURES=merge-wait, and all 3 packages immediately entered the > running state: > > >>> Jobs: 0 of 3 complete, 3 running Load avg: 0.78, 0.97, 1.39 We still have this oddity to reconcile. I assume that you did not observe an issue with the number of running jobs with FEATURES="-merge-wait", but such an issue also should not have been triggered by merge-wait for any known reason.
Sorry, the last message was half baked. No, I did not face the issue with FEATURES="-merge-wait" and it did happen with merge-wait.
