Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 908717 - sys-apps/portage-9999: can't resolve upgrade to LLVM 17 snapshots
Summary: sys-apps/portage-9999: can't resolve upgrade to LLVM 17 snapshots
Status: CONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard: Reverted for now
Keywords:
Depends on:
Blocks: 155723 463976 622270
  Show dependency tree
 
Reported: 2023-06-17 21:15 UTC by Sam James
Modified: 2023-08-08 04:55 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge -p -uvDU @world (portage-9999, bad) (file_908717.txt,11.69 KB, text/plain)
2023-06-17 21:15 UTC, Sam James
Details
emerge -p -uvDU @world (3.0.48.1-r1, good) (file_908717.txt,5.25 KB, text/plain)
2023-06-17 21:16 UTC, Sam James
Details
emerge -p -uvDU @world --debug (3.0.48.1-r1, good) (emerge-good-debug.log.xz,612.38 KB, application/x-xz)
2023-06-17 21:22 UTC, Sam James
Details
emerge -p -uvDU @world --debug (9999, bad) (emerge-bad-debug.log.xz,612.10 KB, application/x-xz)
2023-06-17 21:27 UTC, Sam James
Details
diff -ruN emerge-good-debug.log emerge-bad-debug.log (emerge-good-bad-debug.diff.xz,112.20 KB, application/x-xz)
2023-06-17 21:29 UTC, Sam James
Details
/etc/portage/package.accept_keywords/llvm (file_908717.txt,790 bytes, text/plain)
2023-06-18 01:25 UTC, Sam James
Details
portage-bug908717-logs.tar.xz (portage-bug908717-logs.tar.xz,970.38 KB, application/x-xz)
2023-06-18 02:16 UTC, Sam James
Details
reproducer (test_todo.py,2.61 KB, text/x-python)
2023-06-18 16:16 UTC, YiFei Zhu
Details
reproducer 2 (test_todo.py,3.09 KB, text/x-python)
2023-06-18 16:52 UTC, YiFei Zhu
Details
POC patch (patch,1.56 KB, patch)
2023-06-18 18:17 UTC, YiFei Zhu
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-06-17 21:15:38 UTC
Created attachment 864097 [details]
emerge -p -uvDU @world (portage-9999, bad)

portage-9999 (at commit c5fe39fa9bae5f77e50b8130fbf66b9865e9d1b0) can't seem to figure this out, while portage-3.0.48.1-r1 can.

---

# emerge --info
Portage 3.0.48.1 (python 3.12.0-beta-2, default/linux/amd64/17.1/hardened, gcc-13, glibc-2.37-r3, 6.1.34-sam_box x86_64)
=================================================================
System uname: Linux-6.1.34-sam_box-x86_64-AMD_Ryzen_9_3950X_16-Core_Processor-with-glibc2.37
KiB Mem:    65757432 total,  36070192 free
KiB Swap:   16662520 total,  16545016 free
Timestamp of repository gentoo: Sat, 17 Jun 2023 20:47:03 +0000
Head commit of repository gentoo: 2052cb55c714abdb6ac55eb121a8755aa44efc86

Timestamp of repository emacs: Fri, 16 Jun 2023 18:17:23 +0000
Head commit of repository emacs: 9debdbee45e86c5890e408eadd8e30f15319aebe

Timestamp of repository kde: Sat, 17 Jun 2023 20:46:55 +0000
Head commit of repository kde: d89f6e877c0e17245c367904e6f51e77b19ac236

Timestamp of repository qt: Sat, 17 Jun 2023 10:17:27 +0000
Head commit of repository qt: e12221943544a937c7a7547c264ea908d1d9d809

Timestamp of repository sam_c: Fri, 16 Jun 2023 18:17:41 +0000
Head commit of repository sam_c: 94cd0a1245e56155457f512a7768d880a38b3ca3

Timestamp of repository steam-overlay: Sat, 17 Jun 2023 10:17:28 +0000
Head commit of repository steam-overlay: 6d66cb6aaaeceafa8b4df65009bc5f851e6e7b01

sh dash 0.5.12
ld GNU ld (Gentoo 2.40 p5) 2.40.0
ccache version 4.8.2 [disabled]
app-misc/pax-utils:        1.3.7::gentoo
app-shells/bash:           5.2_p15-r3::gentoo
dev-java/java-config:      2.3.1-r1::gentoo
dev-lang/perl:             5.38.0_rc1::gentoo
dev-lang/python:           3.10.12::gentoo, 3.11.4::gentoo, 3.12.0_beta2::gentoo
dev-lang/rust:             1.69.0-r1::gentoo
dev-util/ccache:           4.8.2::gentoo
dev-util/cmake:            3.27.0_rc2::gentoo
dev-util/meson:            1.1.1::gentoo
sys-apps/baselayout:       2.13-r1::gentoo
sys-apps/sandbox:          2.31::gentoo
sys-apps/systemd:          253.5::gentoo
sys-devel/autoconf:        2.13-r8::gentoo, 2.69-r9::gentoo, 2.71-r6::gentoo, 9999::gentoo
sys-devel/automake:        1.16.5-r1::gentoo
sys-devel/binutils:        2.38-r2::gentoo, 2.39-r5::gentoo, 2.40-r5::gentoo
sys-devel/binutils-config: 5.5::gentoo
sys-devel/clang:           15.0.7-r3::gentoo, 16.0.6::gentoo, 17.0.0_pre20230609::gentoo
sys-devel/gcc:             10.4.1_p20230614::gentoo, 11.4.1_p20230615::gentoo, 12.3.1_p20230616::gentoo, 13.1.1_p20230610::gentoo, 14.0.0_pre20230611::gentoo
sys-devel/gcc-config:      2.11::gentoo
sys-devel/libtool:         2.4.7-r1::gentoo
sys-devel/lld:             17.0.0_pre20230609::gentoo
sys-devel/llvm:            15.0.7-r3::gentoo, 16.0.6::gentoo, 17.0.0_pre20230609::gentoo
sys-devel/make:            4.4.1-r1::gentoo
sys-kernel/linux-headers:  6.3::gentoo (virtual/os-headers)
sys-libs/glibc:            2.37-r3::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/gentoo.git
    priority: -1000
    eclass-overrides: sam_c
    volatile: False
    sync-git-verify-commit-signature: yes
    sync-git-clone-extra-opts: -b stable -c gc.reflogExpire=0 -c gc.reflogExpireUnreachable=0 -c gc.rerereresolved=0 -c gc.rerereunresolved=0 -c gc.pruneExpire=now

crossdev
    location: /var/db/repos/crossdev
    masters: gentoo
    eclass-overrides: sam_c
    volatile: True

emacs
    location: /var/db/repos/emacs
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/emacs.git
    masters: gentoo
    eclass-overrides: sam_c
    volatile: False

kde
    location: /var/db/repos/kde
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/kde.git
    masters: gentoo
    eclass-overrides: sam_c
    volatile: False

local
    location: /var/db/repos/local
    masters: gentoo
    eclass-overrides: sam_c
    volatile: False

qt
    location: /var/db/repos/qt
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/qt.git
    masters: gentoo
    eclass-overrides: sam_c
    volatile: False

sam_c
    location: /var/db/repos/sam_c
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/sam_c.git
    masters: gentoo
    eclass-overrides: sam_c
    volatile: False

steam-overlay
    location: /var/db/repos/steam-overlay
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/steam-overlay.git
    masters: gentoo
    eclass-overrides: sam_c
    volatile: False

test
    location: /var/db/repos/test
    masters: gentoo
    eclass-overrides: sam_c
    volatile: False

Installed sets: @sam-emacs, @sam-fonts, @sam-fortune
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="@FREE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=native -fdiagnostics-color=always -frecord-gcc-switches -Wreturn-type"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php8.2/ext-active/ /etc/php/cgi-php8.2/ext-active/ /etc/php/cli-php8.2/ext-active/ /etc/php/fpm-php8.2/ext-active/ /etc/php/phpdbg-php8.2/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -pipe -march=native -fdiagnostics-color=always -frecord-gcc-switches -Wreturn-type"
DISTDIR="/var/cache/distfiles"
EMERGE_DEFAULT_OPTS="--keep-going --with-bdeps=y --complete-graph --deep --dynamic-deps=n --changed-deps=n --usepkg-exclude sys-fs/zfs --usepkg-exclude sys-fs/zfs-kmod --usepkg=y --implicit-system-deps=n --jobs=4 --load-average 32"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-O2 -pipe -march=native -fdiagnostics-color=always -frecord-gcc-switches -Wreturn-type"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg cgroup clean-logs config-protect-if-modified distlocks fixlafiles ipc-sandbox mount-sandbox multilib-strict network-sandbox news parallel-fetch parallel-install pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms split-log strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe -march=native -fdiagnostics-color=always -frecord-gcc-switches -Wreturn-type"
GENTOO_MIRRORS="http://www.mirrorservice.org/sites/distfiles.gentoo.org/ http://mirrors.soeasyto.com/distfiles.gentoo.org/"
LANG="en_GB.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--defsym=__gentoo_check_ldflags__=0 -Wl,-z,pack-relative-relocs"
LEX="reflex"
LINGUAS="en en_GB"
MAKEOPTS="-j32 -l32"
PKGDIR="/var/cache/binpkgs"
PORTAGE_BUNZIP2_COMMAND="lbunzip2"
PORTAGE_COMPRESS="xz"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
RUSTFLAGS="-C target-cpu=native -C link-arg=-Wl,-z,pack-relative-relocs"
SHELL="/bin/bash"
USE="PIC X a52 aac acl acpi activities aes alsa amd64 avx avx2 bash-completion bluetooth branding bzip2 cairo caps cdda cdr cet clang cli crypt dbus declarative dist-kernel dri dts dvd dvdr editorconfig emacs encode exif f16c filecaps firewalld flac fma3 fortran freetype gdbm gif gmp gpm graphite gtk gui hardened harfbuzz hunspell iconv icu ipv6 jit jpeg kde kdesu kwallet lcms libnotify libtirpc lto mad mmx mmxext mng mp3 mp4 mpeg multilib ncurses nftables nls nptl ogg opengl openmp pam pango pclmul pcre pdf pgo pie pipewire plasma png policykit popcnt ppds pulseaudio qml qt5 rdrand readline screencast sdl seccomp semantic-desktop sha sound spell sse sse2 sse3 sse4_1 sse4_2 sse4a ssl ssp ssse3 startup-notification svg system-av1 system-binutils system-boost system-bootstrap system-cairo system-clang system-digest system-ffmpeg system-harfbuzz system-heimdal system-icu system-jpeg system-leveldb system-libevent system-libs system-libvpx system-libyaml system-llvm system-lz4 system-mitkrb5 system-sqlite system-ssl system-tbb system-uulib system-webp system-zlib systemd test-rust tiff truetype udev udisks unicode upower usb valgrind verify-sig vorbis vulkan wayland widgets x264 xattr xcb xft xml xtpax xv xvid zlib zsh-completion" ABI_X86="32 64" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en en-GB" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LLVM_TARGETS="AArch64 AMDGPU ARM AVR BPF Hexagon Lanai MSP430 Mips NVPTX PowerPC RISCV Sparc SystemZ WebAssembly X86 XCore" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-4 php8-0" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_11 pypy3 python3_10 python3_12" QEMU_USER_TARGETS="aarch64 aarch64_be alpha armeb hppa m68k ppc ppc64 ppc64le sparc sparc64" RUBY_TARGETS="ruby31 ruby32" VIDEO_CARDS="amdgpu radeonsi radeon" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account"
YACC="byacc"
Unset:  ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, SIZE, STRINGS, STRIP, YFLAGS
Comment 1 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-06-17 21:16:52 UTC
Created attachment 864098 [details]
emerge -p -uvDU @world (3.0.48.1-r1, good)
Comment 2 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-06-17 21:22:22 UTC
Created attachment 864099 [details]
emerge -p -uvDU @world --debug (3.0.48.1-r1, good)
Comment 3 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-06-17 21:27:29 UTC
Created attachment 864100 [details]
emerge -p -uvDU @world --debug (9999, bad)
Comment 4 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-06-17 21:29:03 UTC
Created attachment 864101 [details]
diff -ruN emerge-good-debug.log emerge-bad-debug.log
Comment 5 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-06-17 22:25:34 UTC
44afa8445dc46464200fe46c1e09e0c7475067bf ('depgraph: Don't ignore downgrades as missed_updates ', https://github.com/gentoo/portage/commit/44afa8445dc46464200fe46c1e09e0c7475067bf) is the first bad commit, interestingly (I was worried it'd be the subslot one).
Comment 6 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-06-18 01:25:09 UTC
Created attachment 864108 [details]
/etc/portage/package.accept_keywords/llvm
Comment 7 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-06-18 02:16:26 UTC
Created attachment 864109 [details]
portage-bug908717-logs.tar.xz
Comment 8 YiFei Zhu 2023-06-18 13:10:53 UTC
First backtrack: handles first slot conflicts rebuilds, not surprised.

Second backtrack: prune slot rebuilds. What surprises me here is that this backtrack causes perf to pull in the original installed version of clang. I would think that would recompute the slot conflict rebuilds but no, that doesn't happen until 4th backtrack. This backtrack will fail due to the slot conflict as both versions of clang:17 is pulled in.

What's also interesting is that I installed clang-17.0.0_pre20230609, and perf[clang] on my own machine and couldn't reproduce this. The runtime_pkg_mask stays empty so the second backtrack does not happen.
Comment 9 YiFei Zhu 2023-06-18 15:00:23 UTC
Ok the sequence of events that caused this to happen is clear to me now. To reproduce this you need both lldb and perf[clang] installed.

During _solve_non_slot_operator_slot_conflicts, it tries to evaluate which versions of clang:17 to keep.
- (dev-util/perf-6.3:0/0::gentoo, installed) pulls in sys-devel/clang-17.0.0_pre20230609. At this point it seems unaware that it needs to be rebuilt by the slot ABI upgrade, so this pull should be void. (i.e. it sells like a bug to me)
- (dev-util/lldb-17.0.0_pre20230615:0/17.0.0_pre20230615::gentoo, ebuild scheduled for merge) pulls in sys-devel/clang-17.0.0_pre20230615. This is an upgrade with a strict version requirement, and lldb is non-slotted. This is correct.

Slot conflict resolution then goes to be like:

   Conflict: (/, sys-devel/clang:17)
      keep:   (sys-devel/clang-17.0.0_pre20230615:17/17.0.0_pre20230615::gentoo, ebuild scheduled for merge)
      keep:   (sys-devel/clang-17.0.0_pre20230609-1:17/17.0.0_pre20230609::gentoo, installed)

This in turn pulls in the runtime dependencies of clang:

   Conflict: (/, sys-devel/clang-runtime:17)
      keep:   (sys-devel/clang-runtime-17.0.0_pre20230615:17/17::gentoo, ebuild scheduled for merge)
      keep:   (sys-devel/clang-runtime-17.0.0_pre20230609:17/17::gentoo, installed)

This then causes sys-devel/clang-runtime, among a few other packages, to be inserted into the feedback of the backtrack to have a slot conflict, and the first of the feedback is inserted into runtime_pkg_mask. Each individual "keep" is entered as a backtrack attempt.

The patch "depgraph: Don't ignore downgrades as missed_updates" then sees that there was a runtime_pkg_mask entry for sys-devel/clang-runtime-17.0.0_pre20230609. This entry implies that portage had an attempted merge on sys-devel/clang-runtime-17.0.0_pre20230609. This triggers the logic to perform a backtrack, pruning rebuilds.
Comment 10 YiFei Zhu 2023-06-18 15:23:40 UTC
I should also note, I *think* the backtrack failing in the end has something to do with prune rebuild causing the dependency to fail, and it doesn't enter slot_operator_update_probe to know that perf needs to be rebuilt, so this backtrack fails.

backtrack 2 3 and 4 are each individual attempts to mask each individual of the triple in the backtrack feedback:

> 'slot conflict': [[((<Package ('ebuild', '/', 'sys-devel/clang-runtime-17.0.0_pre20230615', 'merge', 'gentoo')>,
>                      {(<Package ('installed', '/', 'sys-devel/clang-17.0.0_pre20230609', 'nomerge', 'installed')>,
>                        '~sys-devel/clang-runtime-17.0.0_pre20230609')}),),
>                    ((<Package ('ebuild', '/', 'sys-devel/clang-runtime-17.0.0_pre20230609', 'merge', 'gentoo')>,
>                      {(<Package ('ebuild', '/', 'sys-devel/clang-17.0.0_pre20230615', 'merge', 'gentoo')>,
>                        '~sys-devel/clang-runtime-17.0.0_pre20230615')}),
>                     (<Package ('installed', '/', 'sys-devel/clang-runtime-17.0.0_pre20230609', 'nomerge', 'installed')>,
>                      {(<Package ('ebuild', '/', 'sys-devel/clang-17.0.0_pre20230615', 'merge', 'gentoo')>,
>                        '~sys-devel/clang-runtime-17.0.0_pre20230615')}))],

backtrack 2 masks Package ('installed', '/', 'sys-devel/clang-runtime-17.0.0_pre20230609', 'nomerge', 'installed')
backtrack 3 masks Package ('ebuild', '/', 'sys-devel/clang-runtime-17.0.0_pre20230609', 'merge', 'gentoo')
backtrack 4 masks Package ('ebuild', '/', 'sys-devel/clang-runtime-17.0.0_pre20230615', 'merge', 'gentoo')

Both 2 and 3 performs an upgrade, whereas 4 aborts the upgrade. 2 and 3 both fail because perf pulls in the old version (since perf's slot rebuild is pruned)... and 4 is what we get in the end, the upgrade unable to be "figured out".

So there are several parts of this that I think are bugs:
1. slot conflict doesn't know about slot ABI rebuilds and pulls in old versions as slot conflict resolution.
2. slot rebuild pruning is way too trigger happy. It prunes everything, even if that will result in a dependency failure in the next backtrack.
3. after slot rebuild pruning, the package doesn't get selected for rebuild before dependency failure.
Comment 11 YiFei Zhu 2023-06-18 16:05:08 UTC
> backtrack 2 3 and 4 are each individual attempts to mask each individual of
> the triple in the backtrack feedback:
> 
> > 'slot conflict': [[((<Package ('ebuild', '/', 'sys-devel/clang-runtime-17.0.0_pre20230615', 'merge', 'gentoo')>,
> >                      {(<Package ('installed', '/', 'sys-devel/clang-17.0.0_pre20230609', 'nomerge', 'installed')>,
> >                        '~sys-devel/clang-runtime-17.0.0_pre20230609')}),),
> >                    ((<Package ('ebuild', '/', 'sys-devel/clang-runtime-17.0.0_pre20230609', 'merge', 'gentoo')>,
> >                      {(<Package ('ebuild', '/', 'sys-devel/clang-17.0.0_pre20230615', 'merge', 'gentoo')>,
> >                        '~sys-devel/clang-runtime-17.0.0_pre20230615')}),
> >                     (<Package ('installed', '/', 'sys-devel/clang-runtime-17.0.0_pre20230609', 'nomerge', 'installed')>,
> >                      {(<Package ('ebuild', '/', 'sys-devel/clang-17.0.0_pre20230615', 'merge', 'gentoo')>,
> >                        '~sys-devel/clang-runtime-17.0.0_pre20230615')}))],
> 
> backtrack 2 masks Package ('installed', '/',
> 'sys-devel/clang-runtime-17.0.0_pre20230609', 'nomerge', 'installed')
> backtrack 3 masks Package ('ebuild', '/',
> 'sys-devel/clang-runtime-17.0.0_pre20230609', 'merge', 'gentoo')
> backtrack 4 masks Package ('ebuild', '/',
> 'sys-devel/clang-runtime-17.0.0_pre20230615', 'merge', 'gentoo')
> 
> Both 2 and 3 performs an upgrade, whereas 4 aborts the upgrade. 2 and 3 both
> fail because perf pulls in the old version (since perf's slot rebuild is
> pruned)... and 4 is what we get in the end, the upgrade unable to be
> "figured out".

Just checked runtime_pkg_mask, this does not seem to be correct. It always masks sys-devel/clang-runtime-17.0.0_pre20230609.
Comment 12 YiFei Zhu 2023-06-18 16:16:14 UTC
Created attachment 864145 [details]
reproducer
Comment 13 YiFei Zhu 2023-06-18 16:37:48 UTC
> 3. after slot rebuild pruning, the package doesn't get selected for rebuild
> before dependency failure.

It seems at at least in my reproducer, and in Sam's stage3 tarball, this part eventually figures things out, so this is probably not really that bad.
Comment 14 YiFei Zhu 2023-06-18 16:51:43 UTC
(In reply to YiFei Zhu from comment #12)
> Created attachment 864145 [details]
> reproducer

I didn't add asserts for the merge list, and since it resolves after a few backtracks anyways it's not as useful.

I'll attach a test that distinguishes good and bad.
Comment 15 YiFei Zhu 2023-06-18 16:52:03 UTC
Created attachment 864154 [details]
reproducer 2
Comment 16 YiFei Zhu 2023-06-18 18:17:46 UTC
Created attachment 864155 [details, diff]
POC patch
Comment 17 YiFei Zhu 2023-06-18 18:21:55 UTC
> 1. slot conflict doesn't know about slot ABI rebuilds and pulls in old
> versions as slot conflict resolution.

This patch addresses this part but I'm not very confident in the logic that it won't cause regressions. However, it passes all tests, including the one I attached (which this is a trivial solution of, actually). I'd have to construct some non-trivial tests in which a shot rebuild will cause more slot conflicts to truly test how this patch behaves.

And I think I want more opinions on this approach, if it even makes sense.
Comment 18 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-06-18 18:52:48 UTC
(In reply to YiFei Zhu from comment #17)
> > 1. slot conflict doesn't know about slot ABI rebuilds and pulls in old
> > versions as slot conflict resolution.
> 

See also bug 528836.
Comment 19 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2023-06-19 15:44:22 UTC
(In reply to YiFei Zhu from comment #16)
> Created attachment 864155 [details, diff] [details, diff]
> POC patch

The code is definitely above my paygrade but your argumentation seems sound and I don't think there's really any way to know better than to try it.  So +1 to merging it from me.  Worst case, we can revert.
Comment 20 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-06-19 16:58:58 UTC
(In reply to Michał Górny from comment #19)
> (In reply to YiFei Zhu from comment #16)
> > Created attachment 864155 [details, diff] [details, diff] [details, diff]
> > POC patch
> 
> The code is definitely above my paygrade but your argumentation seems sound
> and I don't think there's really any way to know better than to try it.  So
> +1 to merging it from me.  Worst case, we can revert.

+1. I think we need to rip the bandaid off. And work on improving our test coverage for depgraph bits.

Also, what I _might_ do is cherry-pick the Perl fix (https://github.com/gentoo/portage/commit/224207c7d1988a354d004507bb7ecfb90b4ef097) first, release that, and then we can commit a tidied up version of this. Not sure.
Comment 21 YiFei Zhu 2023-06-19 18:33:06 UTC
(In reply to Sam James from comment #20)
> (In reply to Michał Górny from comment #19)
> > (In reply to YiFei Zhu from comment #16)
> > > Created attachment 864155 [details, diff] [details, diff] [details, diff] [details, diff]
> > > POC patch
> > 
> > The code is definitely above my paygrade but your argumentation seems sound
> > and I don't think there's really any way to know better than to try it.  So
> > +1 to merging it from me.  Worst case, we can revert.
> 
> +1. I think we need to rip the bandaid off. And work on improving our test
> coverage for depgraph bits.
> 
> Also, what I _might_ do is cherry-pick the Perl fix
> (https://github.com/gentoo/portage/commit/
> 224207c7d1988a354d004507bb7ecfb90b4ef097) first, release that, and then we
> can commit a tidied up version of this. Not sure.

I think this alternative is better, yeah.

The more I think about this code (the one I uploaded) the more I hate it. I described why just now on IRC:
> 11:02:33 <zhuyifei1999_> uh so, what my patch does it it causes the slot operator rebuilds to be processed first before performing a backtrack on slot conflict masks. This works well if, by processing slot operator rebuilds, slot conflicts are gone
> 11:03:24 <zhuyifei1999_> however, if by processing slot operator rebuilds, there are still slot conflicts, when they are solved it will cause another backtrack to prune slot rebuilds
> 11:03:54 <zhuyifei1999_> and that may prune more than it should and cause it to break exactly like how we see in the bug
> 11:04:06 <zhuyifei1999_> this is my "brain simulation". I didn't test it

I think it's just more bandaid that makes the problem harder to appear but can appear anyways.