With sys-libs/readline-7.0_p1-r1 installed and the corresponding ebuild unavailable, and with app-shells/bash-4.4_p12, endless rebuilds are triggered with --autounmask=n for packages with slot operator dependencies on readline: > # emerge -puvD @world > > These are the packages that would be merged, in order: > > Calculating dependencies... done! > [ebuild rR ] dev-db/sqlite-3.17.0:3::gentoo USE="readline -debug -doc -icu -secure-delete -static-libs -tcl {-test} -tools" 0 KiB > [ebuild rR ] sys-apps/gawk-4.1.3::gentoo USE="readline -mpfr (-nls)" 0 KiB > [ebuild rR ] net-misc/socat-1.7.3.1::gentoo USE="ipv6 readline ssl tcpd" 0 KiB > [ebuild rR ] sys-apps/ipmitool-1.8.17-r1::gentoo USE="(-libressl) -openipmi -static" 0 KiB > [ebuild rR ] app-crypt/gnupg-2.1.21::gentoo USE="bzip2 readline smartcard -doc -gnutls -ldap (-nls) (-selinux) -tofu -tools -usb -wks-server" 0 KiB > [ebuild rR ] dev-lang/python-2.7.12:2.7::gentoo USE="gdbm ipv6 ncurses readline sqlite ssl (threads) (wide-unicode) xml (-berkdb) -build -doc -examples -hardened (-libressl) -tk -wininst" 0 KiB > [ebuild rR ] dev-lang/python-3.4.5:3.4/3.4m::gentoo USE="gdbm ipv6 ncurses readline sqlite ssl (threads) xml -build -examples -hardened (-libressl) -tk -wininst" 0 KiB > [ebuild rR ] dev-libs/libpcre-8.40-r1:3::gentoo USE="bzip2 cxx jit readline recursion-limit (unicode) zlib -libedit -pcre16 -pcre32 -static-libs" 0 KiB > [ebuild rR ] net-dns/bind-tools-9.11.0_p5::gentoo USE="ipv6 readline seccomp ssl -doc -gost -gssapi -idn (-libressl) -urandom -xml" 0 KiB > [ebuild rR ] sys-apps/util-linux-2.28.2::gentoo USE="caps cramfs ncurses pam python readline tty-helpers udev unicode -build -fdformat -kill (-nls) (-selinux) -slang -static-libs -suid -systemd {-test}" PYTHON_SINGLE_TARGET="python3_4 -python2_7 (-python3_5) (-python3_6)" PYTHON_TARGETS="python2_7 python3_4 (-python3_5) (-python3_6)" 0 KiB > [ebuild rR ] dev-libs/libxml2-2.9.4-r1:2::gentoo USE="ipv6 lzma python readline -debug -examples -icu -static-libs {-test}" PYTHON_TARGETS="python2_7 python3_4 (-python3_5) (-python3_6)" 0 KiB > [ebuild rR ] net-misc/ntp-4.2.8_p10-r1::gentoo USE="caps ipv6 readline snmp ssl threads vim-syntax -debug (-libressl) -openntpd -parse-clocks -samba (-selinux) -zeroconf" 0 KiB > [ebuild rR ] sys-fs/lvm2-2.02.145-r2::gentoo USE="device-mapper-only readline udev (-clvm) (-cman) -corosync -lvm1 -lvm2create_initrd -openais (-selinux) -static -static-libs -systemd -thin" 0 KiB > [ebuild rR ] sys-fs/xfsprogs-4.5.0::gentoo USE="readline -libedit (-nls) -static -static-libs" 0 KiB > > Total: 14 packages (14 reinstalls), Size of downloads: 0 KiB The debug log shows that it wants to downgrade to sys-libs/readline-6.3_p8-r3, but it can't because app-shells/bash-4.4_p12 requires a newer version: > slot_operator_update_probe: > existing child package: (sys-libs/readline-7.0_p1-r1-3:0/7::gentoo, installed) > existing parent package: (net-misc/socat-1.7.3.1-18:0/0::gentoo, installed) > new child package: (sys-libs/readline-6.3_p8-r3:0/0::gentoo, ebuild scheduled for merge) > new parent package: (net-misc/socat-1.7.3.1:0/0::gentoo, ebuild scheduled for merge) With --autounmask, emerge gives a reasonable upgrade to go with the rebuilds: > # emerge -puvD --autounmask=y @world > > These are the packages that would be merged, in order: > > Calculating dependencies... done! > [ebuild r U ~] sys-libs/readline-7.0_p3:0/7::gentoo [7.0_p1-r1:0/7::gentoo] USE="-static-libs -utils" 5 KiB > [ebuild rR ] dev-db/sqlite-3.17.0:3::gentoo USE="readline -debug -doc -icu -secure-delete -static-libs -tcl {-test} -tools" 0 KiB > [ebuild rR ] sys-apps/gawk-4.1.3::gentoo USE="readline -mpfr (-nls)" 0 KiB > [ebuild rR ] dev-libs/libpcre-8.40-r1:3::gentoo USE="bzip2 cxx jit readline recursion-limit (unicode) zlib -libedit -pcre16 -pcre32 -static-libs" 0 KiB > [ebuild rR ] net-misc/socat-1.7.3.1::gentoo USE="ipv6 readline ssl tcpd" 0 KiB > [ebuild rR ] sys-apps/ipmitool-1.8.17-r1::gentoo USE="(-libressl) -openipmi -static" 0 KiB > [ebuild rR ] net-dns/bind-tools-9.11.0_p5::gentoo USE="ipv6 readline seccomp ssl -doc -gost -gssapi -idn (-libressl) -urandom -xml" 0 KiB > [ebuild rR ] app-crypt/gnupg-2.1.21::gentoo USE="bzip2 readline smartcard -doc -gnutls -ldap (-nls) (-selinux) -tofu -tools -usb -wks-server" 0 KiB > [ebuild rR ] dev-lang/python-3.4.5:3.4/3.4m::gentoo USE="gdbm ipv6 ncurses readline sqlite ssl (threads) xml -build -examples -hardened (-libressl) -tk -wininst" 0 KiB > [ebuild rR ] dev-lang/python-2.7.12:2.7::gentoo USE="gdbm ipv6 ncurses readline sqlite ssl (threads) (wide-unicode) xml (-berkdb) -build -doc -examples -hardened (-libressl) -tk -wininst" 0 KiB > [ebuild rR ] sys-apps/util-linux-2.28.2::gentoo USE="caps cramfs ncurses pam python readline tty-helpers udev unicode -build -fdformat -kill (-nls) (-selinux) -slang -static-libs -suid -systemd {-test}" PYTHON_SINGLE_TARGET="python3_4 -python2_7 (-python3_5) (-python3_6)" PYTHON_TARGETS="python2_7 python3_4 (-python3_5) (-python3_6)" 0 KiB > [ebuild rR ] dev-libs/libxml2-2.9.4-r1:2::gentoo USE="ipv6 lzma python readline -debug -examples -icu -static-libs {-test}" PYTHON_TARGETS="python2_7 python3_4 (-python3_5) (-python3_6)" 0 KiB > [ebuild rR ] net-misc/ntp-4.2.8_p10-r1::gentoo USE="caps ipv6 readline snmp ssl threads vim-syntax -debug (-libressl) -openntpd -parse-clocks -samba (-selinux) -zeroconf" 0 KiB > [ebuild rR ] sys-fs/lvm2-2.02.145-r2::gentoo USE="device-mapper-only readline udev (-clvm) (-cman) -corosync -lvm1 -lvm2create_initrd -openais (-selinux) -static -static-libs -systemd -thin" 0 KiB > [ebuild rR ] sys-fs/xfsprogs-4.5.0::gentoo USE="readline -libedit (-nls) -static -static-libs" 0 KiB > > Total: 15 packages (1 upgrade, 14 reinstalls), Size of downloads: 5 KiB > > The following keyword changes are necessary to proceed: > (see "package.accept_keywords" in the portage(5) man page for more details) > # required by app-shells/bash-4.4_p12::gentoo > # required by @system > # required by @world (argument) > =sys-libs/readline-7.0_p3 ~amd64 Alternatively, with app-shells/bash-4.4_p12 masked, emerge gives a reasonable downgrade to go with the rebuilds: > # emerge -puvD @world > > These are the packages that would be merged, in order: > > Calculating dependencies... done! > [ebuild r UD ] sys-libs/readline-6.3_p8-r3:0/0::gentoo [7.0_p1-r1:0/7::gentoo] USE="-static-libs -utils" 0 KiB > [ebuild UD ] app-shells/bash-4.3_p48-r1::gentoo [4.4_p12::gentoo] USE="net (readline) -afs -bashlogger -examples -mem-scramble (-nls) -plugins" 0 KiB > [ebuild rR ] dev-db/sqlite-3.17.0:3::gentoo USE="readline -debug -doc -icu -secure-delete -static-libs -tcl {-test} -tools" 0 KiB > [ebuild rR ] sys-apps/gawk-4.1.3::gentoo USE="readline -mpfr (-nls)" 0 KiB > [ebuild rR ] dev-libs/libpcre-8.40-r1:3::gentoo USE="bzip2 cxx jit readline recursion-limit (unicode) zlib -libedit -pcre16 -pcre32 -static-libs" 0 KiB > [ebuild rR ] net-misc/socat-1.7.3.1::gentoo USE="ipv6 readline ssl tcpd" 0 KiB > [ebuild rR ] net-dns/bind-tools-9.11.0_p5::gentoo USE="ipv6 readline seccomp ssl -doc -gost -gssapi -idn (-libressl) -urandom -xml" 0 KiB > [ebuild rR ] sys-apps/ipmitool-1.8.17-r1::gentoo USE="(-libressl) -openipmi -static" 0 KiB > [ebuild rR ] app-crypt/gnupg-2.1.21::gentoo USE="bzip2 readline smartcard -doc -gnutls -ldap (-nls) (-selinux) -tofu -tools -usb -wks-server" 0 KiB > [ebuild rR ] dev-lang/python-3.4.5:3.4/3.4m::gentoo USE="gdbm ipv6 ncurses readline sqlite ssl (threads) xml -build -examples -hardened (-libressl) -tk -wininst" 0 KiB > [ebuild rR ] dev-lang/python-2.7.12:2.7::gentoo USE="gdbm ipv6 ncurses readline sqlite ssl (threads) (wide-unicode) xml (-berkdb) -build -doc -examples -hardened (-libressl) -tk -wininst" 0 KiB > [ebuild rR ] sys-apps/util-linux-2.28.2::gentoo USE="caps cramfs ncurses pam python readline tty-helpers udev unicode -build -fdformat -kill (-nls) (-selinux) -slang -static-libs -suid -systemd {-test}" PYTHON_SINGLE_TARGET="python3_4 -python2_7 (-python3_5) (-python3_6)" PYTHON_TARGETS="python2_7 python3_4 (-python3_5) (-python3_6)" 0 KiB > [ebuild rR ] dev-libs/libxml2-2.9.4-r1:2::gentoo USE="ipv6 lzma python readline -debug -examples -icu -static-libs {-test}" PYTHON_TARGETS="python2_7 python3_4 (-python3_5) (-python3_6)" 0 KiB > [ebuild rR ] net-misc/ntp-4.2.8_p10-r1::gentoo USE="caps ipv6 readline snmp ssl threads vim-syntax -debug (-libressl) -openntpd -parse-clocks -samba (-selinux) -zeroconf" 0 KiB > [ebuild rR ] sys-fs/lvm2-2.02.145-r2::gentoo USE="device-mapper-only readline udev (-clvm) (-cman) -corosync -lvm1 -lvm2create_initrd -openais (-selinux) -static -static-libs -systemd -thin" 0 KiB > [ebuild rR ] sys-fs/xfsprogs-4.5.0::gentoo USE="readline -libedit (-nls) -static -static-libs" 0 KiB > > Total: 16 packages (2 downgrades, 14 reinstalls), Size of downloads: 0 KiB
An implementation of --autounmask-backtrack=n that works with --autounmask=n would be helpful here, as proposed in bug 618228.
I hit this bug a few months ago and a few days ago saw this again on Reddit (https://www.reddit.com/r/Gentoo/comments/143mp3a/portage_keeps_rebuilding_the_same_packages_when/) I did a bit of debugging. Reddit OP triggered this bug by un-accepting ~amd64 and rolling back to amd64 keyword, which would cause openssl to be downgraded from =dev-libs/openssl=3.0.9 to =dev-libs/openssl-1.1.1u, but they also had dev-util/rustup, which does not have a version with amd64 keyword. This then caused a bunch of packages to be rebuilt endlessly. I debugged their emerge --debug and found the issue and suggested to accept ~amd64 for rustup, which fixed the issue for them. So I then wrote a test for portage and tried to do bisection but did not find it to be a regression (didn't find a version that passed the test), but then I found that if I have another package, whose upgrade was prevented because of an unsolved dependency, the bug doesn't trigger. In my case I had net-vpn/networkmanager-openconnect 1.2.8-r1 -> 1.2.10 prevented but it depends on webkit-gtk and I have it masked. This led me through the debug logs and I found the backtrack trigger for bug 439688, and a bit of minor debugging led me to the root cause here: > lib/_emerge/depgraph.py > def _get_missed_updates(self): > [...] > if chosen_pkg > pkg or ( > not chosen_pkg.installed and chosen_pkg.version == pkg.version > ): bug 439688 expects a backtrack to prune these rebuilds, but it doesn't happen, because chosen_pkg (here =dev-libs/openssl=3.0.9) is > pkg (here =dev-libs/openssl-1.1.1u) I will send the patches for text + fix.
Oh, fantastic, thank you!
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=44afa8445dc46464200fe46c1e09e0c7475067bf commit 44afa8445dc46464200fe46c1e09e0c7475067bf Author: YiFei Zhu <zhuyifei1999@gmail.com> AuthorDate: 2023-06-12 02:23:09 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-06-16 03:34:46 +0000 depgraph: Don't ignore downgrades as missed_updates Missed updates can also come in the form of package downgrades, when, for example, there are keyword changes. They can cause rebuilds, and these rebuilds may be not possible due to reasons such as keywords or masks. In this case, prior to this patch, portage would cancel the downgrade, but the rebuilds would be requested endlessly, because bug 439688's backtrack code does not trigger. To reproduce, on an ACCEPT_KEYWORDS=~amd64 machine, emerge =dev-libs/openssl=3.0.9, dev-util/rustup, and something else that depends on openssl. Then un-accept ~amd64 for openssl and rustup. Prior to this patch, a @world upgrade would cause: These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild rR ] dev-libs/libevent-2.1.12-r1:0/2.1-7::gentoo [ebuild rR ] net-misc/rsync-3.2.7-r2::gentoo [...] Total: 71 packages (71 reinstalls), Size of downloads: 0 KiB There are no packages marked "R", only "rR". There are no section labeled "The following packages are causing rebuilds:" either. After this patch, we have: These are the packages that would be merged, in order: Calculating dependencies... done! Total: 0 packages, Size of downloads: 0 KiB WARNING: One or more updates/rebuilds have been skipped due to a dependency conflict: dev-libs/openssl:0 (dev-libs/openssl-1.1.1u:0/1.1::gentoo, ebuild scheduled for merge) dev-libs/openssl:0/3= required by (dev-util/rustup-1.25.2:0/0::gentoo, installed) I also updated the test from the previous patch to account for this change. No other tests seems affected. Bug: https://bugs.gentoo.org/439688 Bug: https://bugs.gentoo.org/622270 Signed-off-by: YiFei Zhu <zhuyifei1999@gmail.com> Closes: https://github.com/gentoo/portage/pull/1053 Signed-off-by: Sam James <sam@gentoo.org> lib/_emerge/depgraph.py | 4 +--- lib/portage/tests/resolver/test_slot_conflict_blocked_prune.py | 2 +- 2 files changed, 2 insertions(+), 4 deletions(-) https://gitweb.gentoo.org/proj/portage.git/commit/?id=2af4bc17d1b4ea1581bb191dfa7f34ec64ab140b commit 2af4bc17d1b4ea1581bb191dfa7f34ec64ab140b Author: YiFei Zhu <zhuyifei1999@gmail.com> AuthorDate: 2023-06-12 02:07:39 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-06-16 03:34:46 +0000 tests: resolver: Test the (bad) behavior of bug #622270 Without fixing the bug. This is so we have a baseline behavior to compare against. Bug: https://bugs.gentoo.org/439688 Bug: https://bugs.gentoo.org/622270 Signed-off-by: YiFei Zhu <zhuyifei1999@gmail.com> Signed-off-by: Sam James <sam@gentoo.org> .../resolver/test_slot_conflict_blocked_prune.py | 78 ++++++++++++++++++++++ 1 file changed, 78 insertions(+)
We reverted the fix for this temporarily for bug 908717.
