I'm performing mixed source / binary updates with emerge --getbinpkg --keep-going using current portage HEAD (eb855b8cd1248f49649003dcfb9bf009b70e88cb), and after resume due to a build failure it chose different builds of binary packages: [binary R ] app-crypt/gpgme-1.23.2-6:1/11.6.15.2::gentoo USE="cxx -common-lisp -debug -python -qt5* -qt6 -static-libs -test -verify-sig" PYTHON_TARGETS="python3_12* -python3_10 -python3_11*" 630 KiB [binary U ] dev-qt/qtwebengine-5.15.14_p20240510-2:5/5.15::gentoo [5.15.13_p20240510:5/5.15::gentoo] USE="alsa bindist jumbo-build pulseaudio system-icu widgets -debug -designer -geolocation -kerberos -screencast* -test" 46,310 KiB [binary R ] sys-devel/clang-17.0.6-6:17::gentoo USE="extra (pie) static-analyzer -debug -doc (-ieee-long-double) -test -verify-sig -xml*" ABI_X86="(64) -32 (-x32)" LLVM_TARGETS="(AArch64) (AMDGPU) (ARM) (AVR) (BPF) (Hexagon) (Lanai) (LoongArch) (MSP430) (Mips) (NVPTX) (PowerPC) (RISCV) (Sparc) (SystemZ) (VE) (WebAssembly) (X86) (XCore) -ARC -CSKY -DirectX -M68k -SPIRV -Xtensa" PYTHON_SINGLE_TARGET="python3_12* (-python3_10) -python3_11*" 49,020 KiB [binary R ] kde-plasma/kwin-5.27.11-r1-4:5::gentoo USE="handbook lock -accessibility -caps -debug -gles2-only -multimedia -plasma* -screencast* -test" 8,690 KiB These where the previously downloaded packages from the first calculation: [binary R ] app-crypt/gpgme-1.23.2-5:1/11.6.15.2::gentoo USE="cxx qt5 -common-lisp -debug -python -qt6 -static-libs -test -verify-sig" PYTHON_TARGETS="python3_12* -python3_10 -python3_11*" 0 KiB [binary U ] dev-qt/qtwebengine-6.7.1-1:6/6.7.1::gentoo [6.7.0:6/6.7.0::gentoo] USE="alsa bindist jumbo-build opengl pulseaudio qml screencast system-icu vulkan widgets -accessibility -custom-cflags -designer -geolocation -kerberos -pdfium -test -vaapi -webdriver" 0 KiB [binary R ] sys-devel/clang-17.0.6-4:17::gentoo USE="extra (pie) static-analyzer xml -debug -doc (-ieee-long-double) -test -verify-sig" ABI_X86="(64) -32 (-x32)" LLVM_TARGETS="(AArch64) (AMDGPU) (ARM) (AVR) (BPF) (Hexagon) (Lanai) (LoongArch) (MSP430) (Mips) (NVPTX) (PowerPC) (RISCV) (Sparc) (SystemZ) (VE) (WebAssembly) (X86) (XCore) -ARC -CSKY -DirectX -M68k -SPIRV -Xtensa" PYTHON_SINGLE_TARGET="python3_12* (-python3_10) -python3_11*" 0 KiB [binary R ] kde-plasma/kwin-5.27.11-r1-3:5::gentoo USE="handbook lock plasma screencast -accessibility -caps -debug -gles2-only -multimedia -test" 0 KiB The emerge command: emerge -auvgD --changed-use --binpkg-respect-use --binpkg-changed-deps=n --rebuilt-binaries --keep-going emerge --info: Portage 3.0.64-32+geb855b8cd1 (python 3.12.3-final-0, default/linux/amd64/23.0/desktop/systemd, gcc-13, glibc-2.39-r6, 6.6.30-0512-x86_64 x86_64) ================================================================= System uname: Linux-6.6.30-0512-x86_64-x86_64-AMD_Ryzen_7_3700U_with_Radeon_Vega_Mobile_Gfx-with-glibc2.39 KiB Mem: 30695412 total, 4285812 free KiB Swap: 0 total, 0 free Head commit of repository gentoo: ea360b781f73aa1f450deb99aa9fa5fc43b6cbf6 sh bash 5.1_p16-r6 ld GNU ld (Gentoo 2.42 p3) 2.42.0 distcc 3.4 x86_64-pc-linux-gnu [disabled] ccache version 4.9.1 [disabled] app-misc/pax-utils: 1.3.7::gentoo app-shells/bash: 5.1_p16-r6::gentoo dev-build/autoconf: 2.13-r8::gentoo, 2.71-r7::gentoo dev-build/automake: 1.16.5-r2::gentoo dev-build/cmake: 3.28.5::gentoo dev-build/libtool: 2.4.7-r4::gentoo dev-build/make: 4.4.1-r1::gentoo dev-build/meson: 1.4.0-r1::gentoo dev-java/java-config: 2.3.3-r1::gentoo dev-lang/perl: 5.38.2-r3::gentoo dev-lang/python: 3.9.19_p3::gentoo, 3.10.14_p1-r1::gentoo, 3.11.9-r1::gentoo, 3.12.3-r1::gentoo, 3.13.0_beta1_p3::gentoo dev-lang/rust-bin: 1.77.1::gentoo dev-util/ccache: 4.9.1-r1::gentoo sys-apps/baselayout: 2.15::gentoo sys-apps/sandbox: 2.38::gentoo sys-apps/systemd: 255.4::gentoo sys-devel/binutils: 2.42-r1::gentoo sys-devel/binutils-config: 5.5::gentoo sys-devel/clang: 17.0.6::gentoo sys-devel/gcc: 13.2.1_p20240210::gentoo sys-devel/gcc-config: 2.11::gentoo sys-devel/lld: 17.0.6::gentoo sys-devel/llvm: 17.0.6::gentoo sys-kernel/linux-headers: 6.6-r1::gentoo (virtual/os-headers) sys-libs/glibc: 2.39-r6::gentoo Repositories: gentoo location: /var/db/repos/gentoo sync-type: git sync-uri: https://anongit.gentoo.org/git/repo/gentoo.git priority: -1000 volatile: False Binary Repositories: gentoobinhost priority: 9999 sync-uri: https://gentoo.osuosl.org/releases/amd64/binpackages/23.0/x86-64 frozen: true ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="@FREE linux-fw-redistributable" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -frecord-gcc-switches" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/share/config/kdm /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/easy-rsa /usr/share/gnupg/qualified.txt /usr/share/themes/oxygen-gtk/gtk-2.0 /var/bind" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/init.d /etc/laptop-mode /etc/logrotate.d/elog-save-summary /etc/make.conf.example /etc/revdep-rebuild /etc/sandbox.d /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /etc/udev/rules.d /usr/NX/etc /usr/NX/home /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/lib/mozilla/defaults/pref /usr/share/X11/xkb /usr/share/config /var/bind" CXXFLAGS="-O2 -pipe -frecord-gcc-switches" DISTDIR="/var/cache/distfiles" EMERGE_DEFAULT_OPTS="--with-bdeps=y --noconfmem --ask-enter-invalid --autounmask-unrestricted-atoms=y --package-moves=n --verbose-conflicts --dynamic-deps=n --autounmask=y -j4 --load-average=4 -j8 -l8" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" FCFLAGS="-O2 -pipe -frecord-gcc-switches" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-multi-instance buildpkg buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch parallel-install pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms sign split-elog split-log strict unknown-features-warn unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe -frecord-gcc-switches" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_US.utf8" LDFLAGS="-Wl,--as-needed -Wl,--defsym=__gentoo_check_ldflags__=0" LEX="flex" LINGUAS="en en_US" MAKEOPTS="-j8 -l8" PKGDIR="/var/cache/binpkgs" PORTAGE_COMPRESS="" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" SHELL="/bin/bash" USE="3dnow X Xaw3d a52 aac acl acpi activities aim alsa amd64 apache2 arts asm audiofile avi bash-completion bidi bindist bluetooth bonobo branding bzip2 bzlib cairo ccache cdda cdparanoia cdr cet colord cpdflib crypt cups curl curlwrappers dbase dbus dga dri dts dv dvb dvd dvdr dvdread eds egl emboss encode ethereal evo exif fakeroot fam fbcon fftw filepro firefox flac foomaticdb fortran freetype ftp gb gd gdbm ggi gif ginac gles2 glut gnutls gpm graphviz gstreamer gtk gui hal iconv icq icu imap imlib innodb introspection iodbc ipod ipv6 irda javascript joystick jpeg kdeenablefinal keybinder ladcca lcms leim libass libg++ libgda libinput libkms libnotify libtirpc libwww llvm logrotate ltsp lzo mad matroska mcal memlimit mikmod mime ming mmap mmx mng mod mp3 mp4 mpeg msn mssql mule multilib musicbrainz mysqli ncurses netboot netcdf nls nptl nptlonly numa ogg oggvorbis opengl openmp opus oscar pam pam_console pango pcmcia pcre pda pdf pdflib pic png pnp policykit posix postproc ppds prelude pulseaudio qml qt5 quicktime rdesktop readline sdl seccomp sip sipim slang slp sndfile snmp sound sox spell sqlite sse sse2 ssl stack-realign startup-notification svg sysprof systemd test-rust theora threads tidy tiff transcode truetype truetype-fonts trusted type1-fonts udev udisks unicode upower usb v4l2 vcd vhosts vorbis vulkan wavelan wayland webdav win32codecs wxwidgets wxwindows x264 xattr xcb xcomposite xface xft xml xml2 xmlrpc xosd xsl xv xvid yahoo zeo zlib" ABI_X86="64" ADA_TARGET="gcc_12" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2 popcnt" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" INPUT_DEVICES="libinput digitaledge evdev jamstudio keyboard magellan mouse palmax spaceorb summa synaptics tek4957 ur98 vmmouse" KERNEL="linux" L10N="en" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LLVM_TARGETS="AMDGPU" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-2" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_12" PYTHON_TARGETS="python3_12" RUBY_TARGETS="ruby31 ruby32 ruby25" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa apm chips cirrus cyrix dummy epson i128 i740 i810 i965 imstt nsc nv rendition s3 s3virge savage sis tdfx tga trident tseng v4l vga voodoo" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
Created attachment 894969 [details] emerge output showing the original merge list and merge list after --keep-going
The selected BUILD_ID is not saved in the mtimedb resume list, and this code in the depgraph _loadResumeCommand method makes poor package selection choices: pkg = None for pkg in self._iter_match_pkgs(root_config, pkg_type, atom): if not self._pkg_visibility_check( pkg ) or self._frozen_config.excluded_pkgs.findAtomForPackage( pkg, modified_use=self._pkg_use_enabled(pkg) ): continue break
In order to extend the resume mergelist format while keeping this schema backward compatibly, I think we should add an additional key next to the mergelist, and use it to store the build it, build time, file size, and mtime of selected binary packages. Older versions of portage can simply ignore the added key.
That was the first thing on my list to check - I had a feeling we didn't save it at all...
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=3a9f2c09eb75f47cf3ae15fa4ebe671548a66870 commit 3a9f2c09eb75f47cf3ae15fa4ebe671548a66870 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2024-06-03 01:18:44 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2024-06-03 01:55:08 +0000 Add binpkgs info to mtimedb resume data In order to fix emerge --resume and --keep-going to make appropriate binary package selections, store a list of binpkgs in the resume data. By adding the data as a new key which older versions of portage will ignore, the extension is backward compatible. Without this fix, emerge --resume and --keep-going make poor package selection choices which do not account for the --binpkg-respect-use option. Bug: https://bugs.gentoo.org/933442 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/_emerge/Scheduler.py | 13 +++++++++++++ lib/_emerge/actions.py | 7 +++++++ lib/_emerge/depgraph.py | 21 +++++++++++++++++++++ lib/portage/tests/util/test_mtimedb.py | 30 +++++++++++++++++++++++++----- 4 files changed, 66 insertions(+), 5 deletions(-)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=1dd08f05355b14c7594d0e0ccf46cdb2b1486cbe commit 1dd08f05355b14c7594d0e0ccf46cdb2b1486cbe Author: Sam James <sam@gentoo.org> AuthorDate: 2024-06-04 01:31:07 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2024-06-04 01:31:07 +0000 sys-apps/portage: add 3.0.65 Closes: https://bugs.gentoo.org/930802 Closes: https://bugs.gentoo.org/787623 Closes: https://bugs.gentoo.org/922629 Closes: https://bugs.gentoo.org/927241 Closes: https://bugs.gentoo.org/930992 Closes: https://bugs.gentoo.org/932739 Closes: https://bugs.gentoo.org/932804 Closes: https://bugs.gentoo.org/933297 Closes: https://bugs.gentoo.org/933385 Closes: https://bugs.gentoo.org/933442 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.65.ebuild | 242 +++++++++++++++++++++++++++++++++ 2 files changed, 243 insertions(+)
