Today I got this: # emerge -pvuDN --complete-graph y --with-bdeps y --rebuild-if-new-rev y world [ebuild rR ] sys-apps/util-linux-2.22.2 USE="cramfs crypt ncurses nls {test} udev unicode -ddate -old-linux -perl (-selinux) -slang -static-libs -suid" 0 kB [ebuild U ] sys-fs/udev-init-scripts-25 [23] 0 kB [ebuild U ] sys-apps/hwids-20130329 [20130302] USE="udev" 0 kB [ebuild N ] app-text/build-docbook-catalog-1.19.1 0 kB [ebuild N ] app-text/docbook-xsl-stylesheets-1.78.0 USE="-ruby" 0 kB [ebuild U ] sys-fs/udev-200 [197-r8] USE="firmware-loader%* kmod openrc -acl -doc -gudev -hwdb -introspection -keymap (-selinux) -static-libs" 0 kB [ebuild rR ] x11-base/xorg-server-1.13.1 USE="minimal nptl udev xorg xvfb -dmx -doc -ipv6 -kdrive (-selinux) -static-libs -suid -tslib -xnest" 0 kB I can't seem to find out what triggered the rebuild of util-linux, but whatever it is, it makes no sense to rR-rebuild something before everything else. Maybe it's the dependency on virtual/udev that's somehow tied to sys-fs/udev, but wrongly processed by rebuild-if-new-rev. Reproducible: Always
Created attachment 343816 [details] Debug output of the original emerge command
Created attachment 343818 [details] emerge --info
The --rebuild-if-new-rev code "sees through" virtuals, so that explains why a virtual/udev dependency and sys-fs/udev upgrade would trigger a rebuild of util-linux. The odd build order is probably due to circular dependencies, which is common in system packages. If so, then I don't think there's any good way to fix this, other than to stop using --rebuild-if-new-rev. Ideally, we'd be able to automatically trigger all desirable rebuilds without using options like --rebuild-if-new-rev. For example, with EAPI 5, we have automatic rebuilds triggered by slot-operators and sub-slots: http://wiki.gentoo.org/wiki/Sub-slots_and_Slot-Operators
First off, I looked at the wiki article and I must say that if this strategy is adopted by all/most ebuilds, the tree will be a better place. But as of now, I suppose the circular deps situation can cause trouble with revdep-if-new-rev in case pkgX and pkgY are to be upgraded while depending on each other. But that's not the case here with udev and util-linux. Only udev needs to be upgraded, so why not merge udev first and util-linux afterwards? rebuild-if-unbuilt cannot be satisfied, but rebuild-if-new-* can be, it seems. And even if both udev and util-linux were in need of upgrade while depending on each other, rebuild-if-new-* can function: one of the two packages is upgraded, then the other and finally the first one is re-compiled. I suppose that would require implementing multiple builds of a single package in a single emerge run. Alternatively, one of the two packages is upgraded, then the other and the next call of "emerge --deep --rebuild-if-new-* world" (or something similar) knows to rR-merge the former package. Actually, this feature would be a nice thing to have even outside any circular deps matters. I mean, the feature of the unfinished, but necessary rR builds being remembered and included in the merge set next time emerge is run with rebuild-if-new-*. I wonder, though, what kind of amount of people use rebuild-if-new-* at all.
This also reminds me that the stable sys-fs/udev is now -200, while virtual/udev stays at -197. I have no idea if that's any kind of problem at all; just FYI.
(In reply to comment #4) Yeah, it seems like we could handle this better. Also, it's possible for the same issue to affect rebuilds involving slot-operators and sub-slots.
*** Bug 569938 has been marked as a duplicate of this bug. ***
*** Bug 567552 has been marked as a duplicate of this bug. ***
*** Bug 575960 has been marked as a duplicate of this bug. ***
*** Bug 685280 has been marked as a duplicate of this bug. ***
*** Bug 787806 has been marked as a duplicate of this bug. ***
*** Bug 789966 has been marked as a duplicate of this bug. ***
I'm not sure why but this seems to be hit more often right now on fresh installs. bug 787806, bug 789966
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=224207c7d1988a354d004507bb7ecfb90b4ef097 commit 224207c7d1988a354d004507bb7ecfb90b4ef097 Author: YiFei Zhu <zhuyifei1999@gmail.com> AuthorDate: 2023-06-13 00:47:52 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-06-16 03:34:46 +0000 depgraph: Do not allow slotted deps to be satisfied by wrong slots This may be part of what caused the "perl rebuild bug". The "priority" of perl, when seen by perl modules, may have "satisfied" set to an installed perl of a wrong slot. Compounding this factor with bug #756199 where find_smallest_cycle would select a single node, in this case because the dependency of perl is satisfied and the priority then gets ignored, the "cycle" becomes a perl module alone and gets rebuilt early. I also updated the test from the previous patch to account for this change. No other tests seems affected. For a larger scale test, I reproduced this initially with a stage3 chroot from a Jan 1 2022 stage3 snapshot, and testing in an equivalent dockerfile would work too: FROM gentoo/stage3:amd64-openrc-20220101 RUN emerge-webrsync COPY . /portage Before this patch (USE flags omitted): # cd /portage && bin/emerge -puDN @world 2>&1 | grep -i perl [ebuild U ] app-admin/perl-cleaner-2.30-r1 [2.30] [ebuild rR ] virtual/perl-File-Temp-0.231.100 [ebuild rR ] dev-perl/Locale-gettext-1.70.0-r1 [ebuild rR ] dev-perl/MIME-Charset-1.12.2-r1 [ebuild rR ] dev-perl/Module-Build-0.423.100 [ebuild rR ] dev-perl/Text-CharWidth-0.40.0-r2 [ebuild U ] dev-lang/perl-5.36.0-r2 [5.34.0-r3] [ebuild N ] virtual/perl-CPAN-2.330.0 [ebuild U ] virtual/perl-ExtUtils-MakeMaker-7.640.0 [7.620.0] [ebuild U ] virtual/perl-File-Spec-3.840.0 [3.800.0] [...] After this patch: # cd /portage && bin/emerge -puDN @world 2>&1 | grep -i perl [ebuild U ] app-admin/perl-cleaner-2.30-r1 [2.30] [ebuild U ] dev-lang/perl-5.36.0-r2:0/5.36 [5.34.0-r3:0/5.34] [ebuild N ] virtual/perl-CPAN-2.330.0 [ebuild U ] virtual/perl-ExtUtils-MakeMaker-7.640.0 [7.620.0] [ebuild U ] virtual/perl-Data-Dumper-2.184.0 [2.179.0] [ebuild U ] virtual/perl-File-Spec-3.840.0 [3.800.0] [ebuild U ] virtual/perl-Test-Harness-3.440.0-r1 [3.430.0] [ebuild rR ] dev-perl/Pod-Parser-1.630.0-r1 [ebuild rR ] dev-perl/Text-CharWidth-0.40.0-r2 [ebuild rR ] dev-perl/Text-WrapI18N-0.60.0-r2 [...] Bug: https://bugs.gentoo.org/463976 Bug: https://bugs.gentoo.org/592880 Bug: https://bugs.gentoo.org/596664 Bug: https://bugs.gentoo.org/631490 Bug: https://bugs.gentoo.org/764365 Bug: https://bugs.gentoo.org/793992 Signed-off-by: YiFei Zhu <zhuyifei1999@gmail.com> Closes: https://github.com/gentoo/portage/pull/1055 Signed-off-by: Sam James <sam@gentoo.org> lib/_emerge/depgraph.py | 18 ++++++++++++++++++ lib/portage/tests/resolver/test_perl_rebuild_bug.py | 4 ++-- 2 files changed, 20 insertions(+), 2 deletions(-) https://gitweb.gentoo.org/proj/portage.git/commit/?id=ff457f742b513a65e6609054e5310f8beee488ec commit ff457f742b513a65e6609054e5310f8beee488ec Author: YiFei Zhu <zhuyifei1999@gmail.com> AuthorDate: 2023-06-13 08:08:51 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-06-16 03:34:46 +0000 tests: resolver: Test the broken behavior of Perl rebuild bug Bug: https://bugs.gentoo.org/463976 Bug: https://bugs.gentoo.org/592880 Bug: https://bugs.gentoo.org/596664 Bug: https://bugs.gentoo.org/631490 Bug: https://bugs.gentoo.org/764365 Bug: https://bugs.gentoo.org/793992 Signed-off-by: YiFei Zhu <zhuyifei1999@gmail.com> Signed-off-by: Sam James <sam@gentoo.org> .../tests/resolver/test_perl_rebuild_bug.py | 121 +++++++++++++++++++++ 1 file changed, 121 insertions(+)
I think this was actually commit 44afa8445dc46464200fe46c1e09e0c7475067bf Author: YiFei Zhu <zhuyifei1999@gmail.com> Date: Sun Jun 11 19:23:09 2023 -0700 depgraph: Don't ignore downgrades as missed_updates but we had to revert it commit a2da08b6dfbfd2ef803aac6302f9de1f7c00a143 Author: Sam James <sam@gentoo.org> Date: Wed Jun 21 19:54:24 2023 +0100 Revert "depgraph: Don't ignore downgrades as missed_updates" This reverts commit 44afa8445dc46464200fe46c1e09e0c7475067bf. This is just so we can make a release for the other fixes in master like the Perl bits and keep a linear history. Signed-off-by: Sam James <sam@gentoo.org> for bug 908717.