I just synced my tree and today an upgrade for dev-python/cffi was included, that triggered rebuilds: The following packages are causing rebuilds: (dev-python/cffi-1.14.4:0/1.14.4::gentoo, ebuild scheduled for merge) causes rebuilds for: (dev-python/bcrypt-3.2.0:0/0::gentoo, ebuild scheduled for merge) (dev-python/cryptography-3.2.1:0/0::gentoo, ebuild scheduled for merge) The problem is, that cryptography was rebuild before this cffi update, so that it could not fix to cause for this rebuild. Reproducible: Always Steps to Reproduce: 1. emerge --sync 2. emerge -auDNv world Actual Results: ariolc ~ # emerge -auDNv world These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild U ] dev-cpp/tbb-2020.3::gentoo [2019.8::gentoo] USE="-debug -examples" ABI_MIPS="(-n32) (-n64) (-o32)" ABI_S390="(-32) (-64)" ABI_X86="(64) -32 (-x32)" 0 KiB [ebuild U ] x11-misc/shared-mime-info-2.0-r2::gentoo [1.15::gentoo] 0 KiB [ebuild U ] dev-libs/expat-2.2.10::gentoo [2.2.8::gentoo] USE="(split-usr) unicode -examples -static-libs" ABI_MIPS="(-n32) (-n64) (-o32)" ABI_RISCV="(-lp64%) (-lp64d%)" ABI_S390="(-32) (-64)" ABI_X86="(64) -32 (-x32)" ELIBC="(-FreeBSD)" 0 KiB [ebuild U ] media-libs/openjpeg-2.4.0:2/7::gentoo [2.3.1-r1:2/7::gentoo] USE="doc -static-libs -test" ABI_MIPS="(-n32) (-n64) (-o32)" ABI_S390="(-32) (-64)" ABI_X86="(64) -32 (-x32)" 0 KiB [ebuild U ] dev-libs/libverto-0.3.1-r1::gentoo [0.3.1::gentoo] USE="glib libev threads -libevent -static-libs -tevent" ABI_MIPS="(-n32) (-n64) (-o32)" ABI_S390="(-32) (-64)" ABI_X86="(64) -32 (-x32)" 0 KiB [ebuild N ] dev-python/backports-zoneinfo-0.2.1-r1::gentoo USE="-test" PYTHON_TARGETS="python3_8 (-pypy3) (-python3_6) -python3_7" 0 KiB [ebuild rR ] dev-python/cryptography-3.2.1::gentoo USE="-idna -libressl -test" PYTHON_TARGETS="python3_8 python3_9 (-pypy3) (-python3_6) -python3_7" 0 KiB [ebuild U ] dev-python/sphinx-3.3.1-r1::gentoo [3.2.1::gentoo] USE="doc latex -test" PYTHON_TARGETS="python3_8 python3_9 (-pypy3) -python3_7 (-python3_6%)" 0 KiB [ebuild r U ] dev-python/cffi-1.14.4:0/1.14.4::gentoo [1.14.0-r3:0/1.14.0::gentoo] USE="doc -test" PYTHON_TARGETS="python3_8 python3_9 (-python3_6) -python3_7" 0 KiB [ebuild U ] dev-python/pyopenssl-20.0.0-r1::gentoo [19.1.0-r1::gentoo] USE="doc -test" PYTHON_TARGETS="python3_8 python3_9 (-pypy3) (-python3_6) -python3_7" 0 KiB [ebuild U ] dev-python/pygments-2.7.3::gentoo [2.7.2::gentoo] USE="doc -test" PYTHON_TARGETS="python3_8 python3_9 (-pypy3) (-python3_6) -python3_7" 0 KiB [ebuild U ] dev-python/Babel-2.9.0::gentoo [2.8.1::gentoo] USE="doc -test" PYTHON_TARGETS="python3_8 python3_9 (-pypy3) (-python3_6) -python3_7" 0 KiB [ebuild U ] sys-apps/busybox-1.32.0-r1::gentoo [1.31.1-r3::gentoo] USE="ipv6 math mdev static syslog -debug -livecd -make-symlinks -pam -savedconfig (-selinux) -sep-usr -systemd" 0 KiB [ebuild U ] sys-fs/xfsprogs-5.10.0-r1::gentoo [5.10.0::gentoo] USE="icu nls (split-usr) -libedit" 0 KiB [ebuild rR ] dev-python/bcrypt-3.2.0::gentoo USE="-test" PYTHON_TARGETS="python3_8 python3_9 (-pypy3) (-python3_6) -python3_7" 0 KiB [ebuild U ] net-libs/nodejs-14.15.4:0/14::gentoo [14.15.1:0/14::gentoo] USE="doc icu npm snapshot ssl system-icu system-ssl -debug -inspector -pax_kernel -systemtap -test" CPU_FLAGS_X86="sse2" 32,516 KiB [ebuild U ] dev-python/zope-interface-5.2.0::gentoo [5.1.2::gentoo] USE="-test" PYTHON_TARGETS="python3_8 python3_9 (-pypy3) (-python3_6) -python3_7" 0 KiB [ebuild U ] dev-python/jaraco-packaging-8.1.1-r1::gentoo [8.1.0-r2::gentoo] USE="doc -test" PYTHON_TARGETS="python3_8 python3_9 (-pypy3) (-python3_6) -python3_7" 0 KiB [ebuild U ] dev-python/cheroot-8.5.0::gentoo [8.4.8::gentoo] USE="-test" PYTHON_TARGETS="python3_8 python3_9 (-python3_6) -python3_7" 0 KiB Total: 19 packages (16 upgrades, 1 new, 2 reinstalls), Size of downloads: 32,516 KiB The following packages are causing rebuilds: (dev-python/cffi-1.14.4:0/1.14.4::gentoo, ebuild scheduled for merge) causes rebuilds for: (dev-python/bcrypt-3.2.0:0/0::gentoo, ebuild scheduled for merge) (dev-python/cryptography-3.2.1:0/0::gentoo, ebuild scheduled for merge) Would you like to merge these packages? [Yes/No] yes >>> Verifying ebuild manifests >>> Running pre-merge checks for net-libs/nodejs-14.15.4 >>> Emerging (1 of 19) dev-cpp/tbb-2020.3::gentoo >>> Emerging (2 of 19) x11-misc/shared-mime-info-2.0-r2::gentoo >>> Installing (1 of 19) dev-cpp/tbb-2020.3::gentoo >>> Installing (2 of 19) x11-misc/shared-mime-info-2.0-r2::gentoo >>> Emerging (3 of 19) dev-libs/expat-2.2.10::gentoo >>> Installing (3 of 19) dev-libs/expat-2.2.10::gentoo >>> Emerging (4 of 19) media-libs/openjpeg-2.4.0::gentoo >>> Installing (4 of 19) media-libs/openjpeg-2.4.0::gentoo >>> Emerging (5 of 19) dev-libs/libverto-0.3.1-r1::gentoo >>> Installing (5 of 19) dev-libs/libverto-0.3.1-r1::gentoo >>> Emerging (6 of 19) dev-python/backports-zoneinfo-0.2.1-r1::gentoo >>> Installing (6 of 19) dev-python/backports-zoneinfo-0.2.1-r1::gentoo >>> Emerging (7 of 19) dev-python/cryptography-3.2.1::gentoo >>> Installing (7 of 19) dev-python/cryptography-3.2.1::gentoo >>> Emerging (8 of 19) dev-python/sphinx-3.3.1-r1::gentoo >>> Installing (8 of 19) dev-python/sphinx-3.3.1-r1::gentoo >>> Emerging (9 of 19) dev-python/cffi-1.14.4::gentoo >>> Installing (9 of 19) dev-python/cffi-1.14.4::gentoo >>> Emerging (10 of 19) dev-python/pyopenssl-20.0.0-r1::gentoo >>> Installing (10 of 19) dev-python/pyopenssl-20.0.0-r1::gentoo >>> Emerging (11 of 19) dev-python/pygments-2.7.3::gentoo >>> Installing (11 of 19) dev-python/pygments-2.7.3::gentoo >>> Emerging (12 of 19) dev-python/Babel-2.9.0::gentoo >>> Installing (12 of 19) dev-python/Babel-2.9.0::gentoo >>> Emerging (13 of 19) sys-apps/busybox-1.32.0-r1::gentoo >>> Emerging (14 of 19) sys-fs/xfsprogs-5.10.0-r1::gentoo >>> Emerging (15 of 19) dev-python/bcrypt-3.2.0::gentoo >>> Emerging (16 of 19) net-libs/nodejs-14.15.4::gentoo >>> Installing (15 of 19) dev-python/bcrypt-3.2.0::gentoo >>> Emerging (17 of 19) dev-python/zope-interface-5.2.0::gentoo >>> Installing (17 of 19) dev-python/zope-interface-5.2.0::gentoo >>> Installing (14 of 19) sys-fs/xfsprogs-5.10.0-r1::gentoo >>> Emerging (18 of 19) dev-python/jaraco-packaging-8.1.1-r1::gentoo >>> Installing (18 of 19) dev-python/jaraco-packaging-8.1.1-r1::gentoo >>> Emerging (19 of 19) dev-python/cheroot-8.5.0::gentoo >>> Installing (19 of 19) dev-python/cheroot-8.5.0::gentoo >>> Installing (16 of 19) net-libs/nodejs-14.15.4::gentoo >>> Installing (13 of 19) sys-apps/busybox-1.32.0-r1::gentoo >>> Jobs: 19 of 19 complete Load avg: 2.24, 3.35, 3.41 Expected Results: First upgrading dev-pyton/cffi, only then rebuilding dev-python/cryptography and dev-python/bcrypt. An emerge -a --depclean after this seemingly successful word update showed: Calculating dependencies... done! * Dependencies could not be completely resolved due to * the following required packages not being installed: * * >=dev-python/cffi-1.8:0/1.14.0=[-python_single_target_python3_6(-),-python_single_target_python3_7(-),python_targets_python3_8(-),-python_single_target_python3_8(-),python_targets_python3_9(-),-python_single_target_python3_9(-)] pulled in by: * dev-python/cryptography-3.2.1 * * Have you forgotten to do a complete update prior to depclean? The A second emerge -auDNv showed that cryptography still needed to be rebuild and after that --depclean did work correctly again. The bug seems purely the order the packages where build/merged. Instead of cffi -> cryptography+bcrypt portage choose the wrong order of cryptography -> cffi -> bcrypt. My guess would be some kind of circular dependency between cffi -> sphinx -> cryptography -> cffi, because I have USE=doc active. But moving a rebuild merge before the packages that causes the rebuild can never be the correct solution for this kind of dependency chains. I did see something similar on my ~amd64 system, but do not recall what packages were part of that. But now it is the second time I noticed a wrong order wrt. to rebuilds, so I'm filing this bug.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=224207c7d1988a354d004507bb7ecfb90b4ef097 commit 224207c7d1988a354d004507bb7ecfb90b4ef097 Author: YiFei Zhu <zhuyifei1999@gmail.com> AuthorDate: 2023-06-13 00:47:52 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-06-16 03:34:46 +0000 depgraph: Do not allow slotted deps to be satisfied by wrong slots This may be part of what caused the "perl rebuild bug". The "priority" of perl, when seen by perl modules, may have "satisfied" set to an installed perl of a wrong slot. Compounding this factor with bug #756199 where find_smallest_cycle would select a single node, in this case because the dependency of perl is satisfied and the priority then gets ignored, the "cycle" becomes a perl module alone and gets rebuilt early. I also updated the test from the previous patch to account for this change. No other tests seems affected. For a larger scale test, I reproduced this initially with a stage3 chroot from a Jan 1 2022 stage3 snapshot, and testing in an equivalent dockerfile would work too: FROM gentoo/stage3:amd64-openrc-20220101 RUN emerge-webrsync COPY . /portage Before this patch (USE flags omitted): # cd /portage && bin/emerge -puDN @world 2>&1 | grep -i perl [ebuild U ] app-admin/perl-cleaner-2.30-r1 [2.30] [ebuild rR ] virtual/perl-File-Temp-0.231.100 [ebuild rR ] dev-perl/Locale-gettext-1.70.0-r1 [ebuild rR ] dev-perl/MIME-Charset-1.12.2-r1 [ebuild rR ] dev-perl/Module-Build-0.423.100 [ebuild rR ] dev-perl/Text-CharWidth-0.40.0-r2 [ebuild U ] dev-lang/perl-5.36.0-r2 [5.34.0-r3] [ebuild N ] virtual/perl-CPAN-2.330.0 [ebuild U ] virtual/perl-ExtUtils-MakeMaker-7.640.0 [7.620.0] [ebuild U ] virtual/perl-File-Spec-3.840.0 [3.800.0] [...] After this patch: # cd /portage && bin/emerge -puDN @world 2>&1 | grep -i perl [ebuild U ] app-admin/perl-cleaner-2.30-r1 [2.30] [ebuild U ] dev-lang/perl-5.36.0-r2:0/5.36 [5.34.0-r3:0/5.34] [ebuild N ] virtual/perl-CPAN-2.330.0 [ebuild U ] virtual/perl-ExtUtils-MakeMaker-7.640.0 [7.620.0] [ebuild U ] virtual/perl-Data-Dumper-2.184.0 [2.179.0] [ebuild U ] virtual/perl-File-Spec-3.840.0 [3.800.0] [ebuild U ] virtual/perl-Test-Harness-3.440.0-r1 [3.430.0] [ebuild rR ] dev-perl/Pod-Parser-1.630.0-r1 [ebuild rR ] dev-perl/Text-CharWidth-0.40.0-r2 [ebuild rR ] dev-perl/Text-WrapI18N-0.60.0-r2 [...] Bug: https://bugs.gentoo.org/463976 Bug: https://bugs.gentoo.org/592880 Bug: https://bugs.gentoo.org/596664 Bug: https://bugs.gentoo.org/631490 Bug: https://bugs.gentoo.org/764365 Bug: https://bugs.gentoo.org/793992 Signed-off-by: YiFei Zhu <zhuyifei1999@gmail.com> Closes: https://github.com/gentoo/portage/pull/1055 Signed-off-by: Sam James <sam@gentoo.org> lib/_emerge/depgraph.py | 18 ++++++++++++++++++ lib/portage/tests/resolver/test_perl_rebuild_bug.py | 4 ++-- 2 files changed, 20 insertions(+), 2 deletions(-) https://gitweb.gentoo.org/proj/portage.git/commit/?id=ff457f742b513a65e6609054e5310f8beee488ec commit ff457f742b513a65e6609054e5310f8beee488ec Author: YiFei Zhu <zhuyifei1999@gmail.com> AuthorDate: 2023-06-13 08:08:51 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-06-16 03:34:46 +0000 tests: resolver: Test the broken behavior of Perl rebuild bug Bug: https://bugs.gentoo.org/463976 Bug: https://bugs.gentoo.org/592880 Bug: https://bugs.gentoo.org/596664 Bug: https://bugs.gentoo.org/631490 Bug: https://bugs.gentoo.org/764365 Bug: https://bugs.gentoo.org/793992 Signed-off-by: YiFei Zhu <zhuyifei1999@gmail.com> Signed-off-by: Sam James <sam@gentoo.org> .../tests/resolver/test_perl_rebuild_bug.py | 121 +++++++++++++++++++++ 1 file changed, 121 insertions(+)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=9502761c5bef818dbec90f062909d46dc22289df commit 9502761c5bef818dbec90f062909d46dc22289df Author: Sam James <sam@gentoo.org> AuthorDate: 2023-06-21 19:09:31 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-06-21 19:11:05 +0000 sys-apps/portage: add 3.0.49 Closes: https://bugs.gentoo.org/485100 Cloess: https://bugs.gentoo.org/592880 Closes: https://bugs.gentoo.org/596664 Closes: https://bugs.gentoo.org/631490 Closes: https://bugs.gentoo.org/764365 Closes: https://bugs.gentoo.org/793992 Closes: https://bugs.gentoo.org/890812 Closes: https://bugs.gentoo.org/905660 Closes: https://bugs.gentoo.org/907949 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.49.ebuild | 296 +++++++++++++++++++++++++++++++++ 2 files changed, 297 insertions(+)
