Created attachment 872423 [details] emerge --info After today's regular updates somehow portage wants to reinstall this set of packages again and again. The problem goes away with FEATURES="-getbinpkg": * Updating gnupg key ring for package signatures Local copy of remote index is up-to-date and will be used. * Updating gnupg key ring for package signatures Local copy of remote index is up-to-date and will be used. These are the packages that would be merged, in order: Calculating dependencies... done! Dependency resolution took 483.89 s (backtrack: 17/100000). [binary rR ] media-video/ffmpeg-4.4.4-r4-2 [binary rR ] media-video/pipewire-0.3.81-1 [binary rR ] kde-frameworks/kfilemetadata-5.110.0-r1-1 [binary rR ] kde-plasma/kpipewire-5.27.8-1 [binary rR ] media-plugins/gst-plugins-libav-1.22.3-1 [binary rR ] media-libs/tg_owt-0_pre20230921-1 [binary rR ] sci-libs/vtk-9.2.6-5 [binary rR ] media-plugins/alsa-plugins-1.2.7.1-r1-4 [binary rR ] media-libs/opencv-4.8.0-r1-5 [binary rR ] games-util/chiaki-2.1.1-r2-2 [binary rR ] media-libs/gegl-0.4.46-4 [binary rR ] net-im/telegram-desktop-4.10.3-1 [binary rR ] media-video/obs-studio-29.1.3-r1-3 [binary rR ] media-libs/mlt-7.20.0-1 [binary rR ] kde-apps/k3b-23.08.1-3 After completing these rebuilds, portage wants to do them again (and again) when running the same 'emerge -uDN @world'. Disabling FEATURES="getbinpkg" fixes the problem and then portage returns saying that everything is already up to date (as it should). This happens with the binary host [1] from the binhost project (CCed). Restricting viewing permission of this bug report to Gentoo developers since it is my understanding that binhost wants to keep the binary host under wraps until the big reveal. [1] https://gentoo.osuosl.org/releases/amd64/binpackages/17.1/x86-64
And here's the output of the same command with '--debug' added. The file is huge, and even when compressed it is still to big for bugzilla: https://drive.google.com/file/d/1mjYFZXGz9fsfZVztIePS2El7zXF77Nav/view?usp=sharing
(In reply to Andrew Ammerlaan from comment #0) > Calculating dependencies... done! > Dependency resolution took 483.89 s (backtrack: 17/100000). At least some of the backtracking is probably due to this ffmpeg inconsistency: In https://gentoo.osuosl.org/releases/amd64/binpackages/17.1/x86-64/Packages I can see that there is only one instance of kde-plasma/kpipewire-5.27.8 and it is built against media-video/ffmpeg:0/58.60.60= which matches ffmepg-6* and not your media-video/ffmpeg-4.4.4-r4. Also, media-video/ffmpeg-4.4.4-r4 is not available in the binhost.
Another inconsistency: kde-frameworks/kfilemetadata-5.110.0-r1-1 is not the latest. There's a kde-frameworks/kfilemetadata-5.110.0-r1-2 available in the binhost that requires media-gfx/exiv2:0/0.28= instead of media-gfx/exiv2:0/27.5=.
Another inconsistency: The binhost only has media-libs/opencv-4.7.0-1 with a different subslot that your media-libs/opencv-4.8.0-r1-5.
Wiped all mentioned packages, rebuilt index, added some magic portage options, and started extra rebuild. https://gitweb.gentoo.org/proj/binhost.git/tree/builders/demeter/kde/run-update ^ this is the build command
More inconsistencies: Your media-libs/gegl-0.4.46-4 seems different from the media-libs/gegl-0.4.46-1 and media-libs/gegl-0.4.46-2 available on the binhost. Not sure what triggered backtracking here but maybe related to a slot operator dependency. Your media-libs/mlt-7.20.0-1 has a newer subslot than media-libs/mlt-7.18.0-1 available on the binhost. Your kde-apps/k3b-23.08.1-3 is newer than kde-apps/k3b-23.04.3-1 available on the binhost. Not sure what triggered backtracking here but maybe related to the slot operator dependency like the one on flac.
In seems like this _prune_rebuilds thing from bug 439688 is probably not working as we would hope in the face of all the inconsistencies: https://gitweb.gentoo.org/proj/portage.git/commit/?id=222dfa56e8fb311f4bea54012bdfd5d1a474d56c
We could probably make the code from bug 439688 scan for unnecessary rebuilds and then trigger another prune_rebuilds calculation even if the _prune_rebuilds flag has already been set.
Also seems like bug 354441 may be related, so maybe we can do something different there, possibly check if @__auto_slot_operator_replace_installed__ is involved: https://gitweb.gentoo.org/proj/portage.git/commit/?id=e4bcbdace3e0c28c39fdc9a92da38b21611638bf
(In reply to Zac Medico from comment #7) > In seems like this _prune_rebuilds thing from bug 439688 is probably not > working as we would hope in the face of all the inconsistencies: > > https://gitweb.gentoo.org/proj/portage.git/commit/ > ?id=222dfa56e8fb311f4bea54012bdfd5d1a474d56c Maybe we just need to tweak the logic related to _get_missed_updates so that it triggers the prune_rebuilds backtracking if there are missed updates *or* there are unnecessary reinstalls.
We might also have to remove the installed instances from the backtracking runtime_pkg_mask.
(In reply to Andrew Ammerlaan from comment #1) > And here's the output of the same command with '--debug' added. The file is > huge, and even when compressed it is still to big for bugzilla: > > https://drive.google.com/file/d/1mjYFZXGz9fsfZVztIePS2El7zXF77Nav/ > view?usp=sharing The log shows that none of the installed instances were in the backtracking runtime_pkg_mask, and all of the 15 corresponding binary packages were in @__auto_slot_operator_replace_installed__, so the tweak suggested in comment #10 will probably work.
We should test if https://github.com/gentoo/portage/pull/1126 fixes it.
(In reply to Zac Medico from comment #13) > We should test if https://github.com/gentoo/portage/pull/1126 fixes it. Not quite, portage-9999 plus the 1126 patch still wants to rebuild these packages. Here's the new log: https://drive.google.com/file/d/1ZgFXuRGPZAH5nIAd5wiFed0tNpkznCQg/view?usp=sharing
To add some more potentially relevant info: - Just before I got stuck in this loop, portage inexplicably insisted on downgrading ffmpeg from 4.4.4-r4 to 4.4.4-r3. Manually upgrading again worked. - The available binary packages are a mix from the binhost and locally built binary packages. Note that in my first log portage only pulls in binaries, this is because binaries were just built for this configuration the previous time I ran the upgrade command. Somehow when I try today portage wants to build some of these packages from source but pulls in binaries for others. I suspect this is because of the changes Andreas made to the binhost yesterday.
(In reply to Andrew Ammerlaan from comment #14) > (In reply to Zac Medico from comment #13) > > We should test if https://github.com/gentoo/portage/pull/1126 fixes it. > > Not quite, portage-9999 plus the 1126 patch still wants to rebuild these > packages. Here's the new log: > https://drive.google.com/file/d/1ZgFXuRGPZAH5nIAd5wiFed0tNpkznCQg/ > view?usp=sharing Now it's a mix of ebuilds and binary packages: [ebuild rR ] media-video/ffmpeg-4.4.4-r4 [ebuild rR ] media-video/pipewire-0.3.81 [binary rR ] kde-frameworks/kfilemetadata-5.110.0-r1-1 [ebuild rR ] kde-plasma/kpipewire-5.27.8 [binary rR ] media-plugins/gst-plugins-libav-1.22.3-1 [ebuild rR ] media-libs/tg_owt-0_pre20230921 [binary rR ] sci-libs/vtk-9.2.6-3 [binary rR ] media-plugins/alsa-plugins-1.2.7.1-r1-3 [ebuild rR ] media-libs/opencv-4.8.0-r1 [binary rR ] games-util/chiaki-2.1.1-r2-1 [binary rR ] media-libs/gegl-0.4.46-3 [ebuild rR ] net-im/telegram-desktop-4.10.3 [binary rR ] media-video/obs-studio-29.1.3-r1-2 [ebuild rR ] media-libs/mlt-7.20.0 [binary rR ] kde-apps/k3b-23.08.1-2 I think a good way to handle this might be something like our _solve_non_slot_operator_slot_conflicts method, which instead of solving slot conflicts, replaces ebuild/binary reinstalls with installed instances when appropriate.
> Now it's a mix of ebuilds and binary packages: Yes, but I've let it build what it wants and now its back to only binaries.
(In reply to Andrew Ammerlaan from comment #17) > > Now it's a mix of ebuilds and binary packages: > > Yes, but I've let it build what it wants and now its back to only binaries. Can you check /var/db/pkg/*/*/BUILD_ID to see if the number is the same or different from the binary package that emerge has pulled in (like for kde-apps/k3b-23.08.1-2 the -2 suffix indicates BUILD_ID 2). My patch was for cases where it's identical, but maybe that's not what's happening. I suppose we may have to account for instances that have different BUILD_ID/BUILD_TIME but are practically identical nonetheless.
It seems like we're going to need some new logic to exhaustively compare the installed instance to the chosen replacement instance, and prefer the installed instance when there is no discernible difference (neglecting binary package attributes like BUILD_ID and BUILD_TIME). We've got these tests that trigger prune_rebuilds backtracking, which will be useful for testing the new comparison logic: lib/portage/tests/resolver/test_slot_operator_complete_graph.py::SlotOperatorCompleteGraphTestCase::testSlotOperatorCompleteGraph lib/portage/tests/resolver/test_slot_operator_missed_update.py::BacktrackMissedUpdateTestCase::testBacktrackMissedUpdateTestCase lib/portage/tests/resolver/test_slot_operator_runtime_pkg_mask.py::SlotOperatorRuntimePkgMaskTestCase::testSlotOperatorRuntimePkgMask lib/portage/tests/resolver/soname/test_skip_update.py::SonameSkipUpdateTestCase::testSonameSkipUpdate
Actually this is the only test case that now requires the prune_rebuilds backtracking to succeed: lib/portage/tests/resolver/soname/test_skip_update.py::SonameSkipUpdateTestCase::testSonameSkipUpdate It's good for the purposes of this bug, because if I disable prune_rebuilds backtracking then it comes up with these unwanted reinstalls: [binary rR ] dev-libs/B-1 [binary rR ] app-misc/A-1 WARNING: One or more updates/rebuilds have been skipped due to a dependency conflict: dev-libs/B:0 (dev-libs/B-2:0/0::test_repo, binary scheduled for merge) USE="" conflicts with x86_32: libB.so.1 required by (app-misc/A-1:0/0::test_repo, binary scheduled for merge) USE=""
Created attachment 872556 [details] Checking build IDs (In reply to Zac Medico from comment #18) > (In reply to Andrew Ammerlaan from comment #17) > > > Now it's a mix of ebuilds and binary packages: > > > > Yes, but I've let it build what it wants and now its back to only binaries. > > Can you check /var/db/pkg/*/*/BUILD_ID to see if the number is the same or > different from the binary package that emerge has pulled in (like for > kde-apps/k3b-23.08.1-2 the -2 suffix indicates BUILD_ID 2). My patch was for > cases where it's identical, but maybe that's not what's happening. I suppose > we may have to account for instances that have different BUILD_ID/BUILD_TIME > but are practically identical nonetheless. There seems to be something fishy going on with kde-plasma/kpipewire: - first I had BUILD_ID 3 - portage pulls kpipewire in to compile from source, ignoring the existing binpkg that should also satisfy the dependencies (the BUILD_ID 3 one that is also installed) - portage builds a new binpkg for kpipewire with BUILD_ID 4, this is now the installed version - Running the same upgrade command again results in portage pulling in a kpipewire binpkg with BUILD_ID 1, ignoring the newer versions with BUILD_ID 3 and 4. I noticed that '/var/db/pkg/kde-plasma/kpipewire-5.27.8/BUILD_ID' is the only BUILD_ID file of this set that contains a newline character. Perhaps this is somehow causing problems? If so then we have a different bug because both the BUILD_ID 3 and BUILD_ID 4 version have this new line character, so somehow portage is writing this file differently then the others. All the other BUILD_IDs match before and after running the upgrade command.
(In reply to Andrew Ammerlaan from comment #21) > I noticed that '/var/db/pkg/kde-plasma/kpipewire-5.27.8/BUILD_ID' is the > only BUILD_ID file of this set that contains a newline character. Perhaps > this is somehow causing problems? If so then we have a different bug because > both the BUILD_ID 3 and BUILD_ID 4 version have this new line character, so > somehow portage is writing this file differently then the others. After 'emerge -C kpipewire && emerge -1 kpipewire' this newline character is gone, but the problem persists. Whatever this was it seems to be unrelated.
[Andrew, is it OK if we make this bug public? Thanks for being considerate. I'm not too worried about the binpkg ref in here as it's kind of buried and someone dedicated can find the links anyway, we just don't want to make the big announcement yet. I don't think your --debug output should have anything sensitive.]
(In reply to Sam James from comment #23) > [Andrew, is it OK if we make this bug public? Thanks for being considerate. > I'm not too worried about the binpkg ref in here as it's kind of buried and > someone dedicated can find the links anyway, we just don't want to make the > big announcement yet. I don't think your --debug output should have anything > sensitive.] That's fine by me.
(In reply to Andrew Ammerlaan from comment #21) > Created attachment 872556 [details] > Checking build IDs Thanks for gathering all these details! > There seems to be something fishy going on with kde-plasma/kpipewire: > - first I had BUILD_ID 3 > - portage pulls kpipewire in to compile from source, ignoring the existing > binpkg that should also satisfy the dependencies (the BUILD_ID 3 one that is > also installed) > - portage builds a new binpkg for kpipewire with BUILD_ID 4, this is now the > installed version > - Running the same upgrade command again results in portage pulling in a > kpipewire binpkg with BUILD_ID 1, ignoring the newer versions with BUILD_ID > 3 and 4. This is weird. Let's see how it behaves with my new patch and go from there. Ultimately we might need to open a separate bug for this. > I noticed that '/var/db/pkg/kde-plasma/kpipewire-5.27.8/BUILD_ID' is the > only BUILD_ID file of this set that contains a newline character. Perhaps > this is somehow causing problems? If so then we have a different bug because > both the BUILD_ID 3 and BUILD_ID 4 version have this new line character, so > somehow portage is writing this file differently then the others. > > All the other BUILD_IDs match before and after running the upgrade command. I doubt that the newline is much of an issue, since portage should strip the whitespace internally. It's possible that new gpkg code behaving differently here. Anyway, we'll come back to this if it continues to be an issue after my new patch. The new patch in https://github.com/gentoo/portage/pull/1126 performs an exhaustive metadata comparison (neglecting BUILD_ID and BUILD_TIME), so it should be very aggressive about eliminating the binary package reinstalls (but beware that the installation will always go forward if dependencies are not completely identical). For ebuild rebuilds, it still needs some work (there's a corresponding TODO comment in the new _eliminate_rebuilds method).
(In reply to Zac Medico from comment #25) > The new patch in https://github.com/gentoo/portage/pull/1126 performs an > exhaustive metadata comparison (neglecting BUILD_ID and BUILD_TIME), so it > should be very aggressive about eliminating the binary package reinstalls > (but beware that the installation will always go forward if dependencies are > not completely identical). For ebuild rebuilds, it still needs some work > (there's a corresponding TODO comment in the new _eliminate_rebuilds method). Awesome, this fixes my problem! andrew-gentoo-pc portage # emerge -uDNAa --with-bdeps=y @world --keep-going --backtrack=100000 --verbose-conflicts * Updating gnupg key ring for package signatures Local copy of remote index is up-to-date and will be used. * Updating gnupg key ring for package signatures Local copy of remote index is up-to-date and will be used. These are the packages that would be merged, in order: Calculating dependencies... done! Dependency resolution took 515.91 s (backtrack: 18/100000). WARNING: One or more updates/rebuilds have been skipped due to a dependency conflict: dev-python/cython:0 (dev-python/cython-3.0.2-r1-2:0/0::gentoo, binary scheduled for merge) USE="-debug -doc -test" ABI_X86="(64)" PYTHON_TARGETS="python3_11 -pypy3 -python3_10 -python3_12" conflicts with >=dev-python/cython-3.0.0[python_targets_python3_11(-),python_targets_python3_12(-)] required by (dev-python/msgpack-1.0.7-1:0/0::gentoo, installed) USE="native-extensions -debug -test" ABI_X86="(64)" PYTHON_TARGETS="python3_11 python3_12 -pypy3 -python3_10" >=dev-python/cython-3[python_targets_python3_11(-),python_targets_python3_12(-)] required by (dev-python/zeroconf-0.115.2-1:0/0::gentoo, installed) USE="-debug -test" ABI_X86="(64)" PYTHON_TARGETS="python3_11 python3_12 -python3_10" >=dev-python/cython-3.0.0[python_targets_python3_11(-),python_targets_python3_12(-)] required by (dev-python/numpy-1.26.0-1:0/0::gentoo, installed) USE="lapack -debug -test" ABI_X86="(64)" PYTHON_TARGETS="python3_11 python3_12 -pypy3 -python3_10" media-video/ffmpeg:0 (media-video/ffmpeg-6.0-r6:0/58.60.60::gentoo, ebuild scheduled for merge) USE="X alsa bluray bzip2 dav1d encode fdk fontconfig gnutls gpl iconv lcms libass libdrm libv4l lzma mp3 network openal opencl opengl openssl opus postproc pulseaudio qsv samba sdl speex svg theora threads truetype v4l vaapi verify-sig vorbis vpx vulkanx264 x265 xvid zlib -amf -amr -amrenc (-appkit) -bs2b -cdio -chromaprint -chromium -codec2 -cpudetection -cuda -debug -doc -flite -frei0r -fribidi -gcrypt -gme -gmp -gsm -hardcoded-tables -iec61883 -ieee1394 -jack -jpeg2k -jpegxl -kvazaar -ladspa -libaom -libaribb24 -libcaca -libilbc -libplacebo -librtmp -libsoxr -libtesseract -libxml2 -lv2 (-mipsdspr1) (-mipsdspr2) (-mipsfpu) (-mmal) -modplug -nvenc -openh264 -oss -pic -rav1e -rubberband -snappy -sndio -srt -ssh -static-libs -svt-av1 -test -twolame -vdpau -vidstab -vmaf -webp -zeromq -zimg -zvbi" ABI_X86="32 (64) (-x32)" CPU_FLAGS_X86="aes avx avx2 fma3 mmx mmxext sse sse2 sse3 sse4_1 sse4_2 ssse3 -3dnow -3dnowext -fma4 -xop" FFTOOLS="aviocat cws2fws ffescape ffeval ffhash fourcc2pixfmt graph2dot ismindex pktdumper qt-faststart sidxindex trasher" conflicts with <media-video/ffmpeg-5 required by (media-video/vlc-3.0.18-r3-2:0/5-9::gentoo, installed) USE="X a52 alsa aom bluray dbus dts dvbpsi dvd encode fdk ffmpeg flac fontconfig gcrypt gstreamer gui jpeg kate keyring libass libnotify libsamplerate lua mad matroska mp3 mpeg mtp ncurses ogg opus png projectm pulseaudio samba skins speex ssl svg taglib theora tremor truetype udev v4l vaapi wayland x264 x265 xml zeroconf -archive -aribsub -bidi-cddb -chromaprint -chromecast -dav1d -dc1394 -debug (-directx) -faad -fluidsynth -gme -ieee1394 -jack -libcaca -libtar -libtiger -linsys -lirc -live -macosx-notifications -modplug -musepack -nfs -omxil -optimisememory -rdp -run-as-root -sdl-image -sftp -shout -sid -soxr -srt -test -twolame -upnp -vdpau -vnc -vpx -zvbi" ABI_X86="(64)" CPU_FLAGS_X86="mmx sse" LUA_SINGLE_TARGET="lua5-1" ^ ^ <media-video/ffmpeg-5:0/56.58.58= required by (sci-libs/opencascade-7.6.3-r2-1:0/7.6::gentoo, installed) USE="ffmpeg gles2 json tbb vtk -doc -eigen -examples -freeimage -optimize" ABI_X86="(64)" ^ ^^^^^^^^^^^^ >=media-video/ffmpeg-3.1.3:0/56.58.58=[postproc,vaapi] required by (media-video/vlc-3.0.18-r3-2:0/5-9::gentoo, installed) USE="X a52 alsa aom bluray dbus dts dvbpsi dvd encode fdk ffmpeg flac fontconfig gcrypt gstreamer gui jpeg kate keyring libass libnotify libsamplerate lua mad matroska mp3 mpeg mtp ncurses ogg opus png projectm pulseaudio samba skins speex ssl svg taglib theora tremor truetype udev v4l vaapi wayland x264 x265 xmlzeroconf -archive -aribsub -bidi -cddb -chromaprint -chromecast -dav1d -dc1394 -debug (-directx) -faad -fluidsynth -gme -ieee1394 -jack -libcaca -libtar -libtiger -linsys -lirc -live -macosx-notifications -modplug -musepack -nfs -omxil -optimisememory -rdp -run-as-root -sdl-image -sftp -shout -sid -soxr -srt -test -twolame -upnp -vdpau -vnc -vpx -zvbi" ABI_X86="(64)" CPU_FLAGS_X86="mmx sse" LUA_SINGLE_TARGET="lua5-1" ^^^^^^^^^^^^ <media-video/ffmpeg-5:= required by (sci-libs/opencascade-7.6.3-r2-1:0/7.6::gentoo, installed) USE="ffmpeggles2 json tbb vtk -doc -eigen -examples -freeimage -optimize" ABI_X86="(64)" ^ ^ dev-libs/ell:0 (dev-libs/ell-0.59:0/0::gentoo, ebuild scheduled for merge) USE="-pie -test" ABI_X86="(64)" conflicts with ~dev-libs/ell-0.58 required by (net-wireless/iwd-2.8-r1-2:0/0::gentoo, installed) USE="client monitor systemd wired -ofono -standalone" ABI_X86="(64)" CPU_FLAGS_X86="aes ssse3" ^ ^^^^ Nothing to merge; quitting.
(In reply to Andrew Ammerlaan from comment #26) > (In reply to Zac Medico from comment #25) > > The new patch in https://github.com/gentoo/portage/pull/1126 performs an > > exhaustive metadata comparison (neglecting BUILD_ID and BUILD_TIME), so it > > should be very aggressive about eliminating the binary package reinstalls > > (but beware that the installation will always go forward if dependencies are > > not completely identical). For ebuild rebuilds, it still needs some work > > (there's a corresponding TODO comment in the new _eliminate_rebuilds method). > > Awesome, this fixes my problem! Great! I think we can move this patch from draft to ready if I handle the TODO about ebuild rebuilds, which shouldn't be too difficult. Right now it doesn't really eliminate ebuild rebuilds since it has to resolve all of the slot operator deps in order to do it safely.
We might consider merging the binary package fix without the unbuilt ebuild fix. For the unbuilt ebuild fix, I'll have to adapt our evaluate_slot_operator_equal_deps function a bit, which doesn't seem like too much work but we'll see.
The evaluate_slot_operator_equal_deps function worked nicely, and I think https://github.com/gentoo/portage/pull/1126 is ready to merge now. It can prevent rebuilds from source now, as long as there will be no difference in the dependencies.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=4978b913df39abbb3369739e370856cf8292a203 commit 4978b913df39abbb3369739e370856cf8292a203 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2023-10-12 18:05:29 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2023-10-12 18:37:26 +0000 Detect and handle unnecessary package reinstall Compare package rebuilds/reinstalls to installed packages of the same exact version, and eliminate unecessary rebuilds/reinstalls triggered solely by the @__auto_slot_operator_replace_installed__ set. This is careful to obey the user's wishes if they have explicitly requested for a package to be rebuilt or reinstalled for some reason. The SonameSkipUpdateTestCase::testSonameSkipUpdateNoPruneRebuilds test case shows that the new depgraph._eliminate_rebuilds method eliminates a backtracking run that is needed for the testSonameSkipUpdate test case to succeed via prune_rebuilds backtracking which was added for bug 439688. Bug: https://bugs.gentoo.org/915494 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/_emerge/depgraph.py | 206 ++++++++++++++++++--- lib/portage/dep/_slot_operator.py | 8 +- .../tests/resolver/soname/test_skip_update.py | 19 +- 3 files changed, 199 insertions(+), 34 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=e2bd39cdc9646b84cd32daf372b5b0496c683cbc commit e2bd39cdc9646b84cd32daf372b5b0496c683cbc Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2023-10-12 18:40:42 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2023-10-12 18:41:21 +0000 NEWS: Eliminate unnecessary package reinstalls (bug #915494) Bug: https://bugs.gentoo.org/915494 Signed-off-by: Zac Medico <zmedico@gentoo.org> NEWS | 2 ++ 1 file changed, 2 insertions(+)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=3500483f75789c36e379c1b6546a09df7c11e2b1 commit 3500483f75789c36e379c1b6546a09df7c11e2b1 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-10-20 00:49:33 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-10-20 00:51:00 +0000 sys-apps/portage: add 3.0.53 Closes: https://bugs.gentoo.org/915120 Closes: https://bugs.gentoo.org/821529 Closes: https://bugs.gentoo.org/914441 Closes: https://bugs.gentoo.org/914722 Closes: https://bugs.gentoo.org/914873 Closes: https://bugs.gentoo.org/915099 Closes: https://bugs.gentoo.org/915123 Closes: https://bugs.gentoo.org/915128 Closes: https://bugs.gentoo.org/915136 Closes: https://bugs.gentoo.org/915330 Closes: https://bugs.gentoo.org/915494 Closes: https://bugs.gentoo.org/915834 Closes: https://bugs.gentoo.org/915903 Closes: https://bugs.gentoo.org/900224 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.53.ebuild | 238 +++++++++++++++++++++++++++++++++ 2 files changed, 239 insertions(+)
The depgraph._eliminate_rebuilds method may need to use strip_libc_deps to cope with the injected libc deps from bug 913628.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=70cbb9b693782eaa779cd7f9f5de6f72edc381d1 commit 70cbb9b693782eaa779cd7f9f5de6f72edc381d1 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2023-12-19 05:25:40 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2023-12-19 06:27:26 +0000 depgraph: Use strip_libc_deps in _eliminate_rebuilds The included test case will fail without strip_libc_deps because the "injected" >=sys-libs/glibc-2.37 dependency will be interpreted as a dependency change, triggering unwanted reinstall of app-misc/A-1. Bug: https://bugs.gentoo.org/915494 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/_emerge/depgraph.py | 33 ++++++++++++++-------- .../tests/resolver/soname/test_skip_update.py | 17 +++++++++-- 2 files changed, 36 insertions(+), 14 deletions(-)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0a1f19cdd7a598070b7eb08b3954e677aa4868ad commit 0a1f19cdd7a598070b7eb08b3954e677aa4868ad Author: Sam James <sam@gentoo.org> AuthorDate: 2023-12-27 21:27:55 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-12-27 21:28:01 +0000 sys-apps/portage: add 3.0.59 Closes: https://bugs.gentoo.org/587088 Closes: https://bugs.gentoo.org/822033 Closes: https://bugs.gentoo.org/915494 Closes: https://bugs.gentoo.org/916135 Closes: https://bugs.gentoo.org/917120 Closes: https://bugs.gentoo.org/919862 Closes: https://bugs.gentoo.org/920095 Closes: https://bugs.gentoo.org/920258 Closes: https://bugs.gentoo.org/920537 Closes: https://bugs.gentoo.org/920654 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.59.ebuild | 246 +++++++++++++++++++++++++++++++++ 2 files changed, 247 insertions(+)
This issue has returned for me in 3.0.61-r1. Whether this is due to some change in the portage code, or something new is triggering this I don't know. What is happening now is that portage insists on rebuilding media-plugins/frei0r-plugins-1.8.0 and installing a binpkg of rust-1.72.0-r (even though corresponding rust-bin is already present. Then later emerge -c wants to remove this version of rust again. And a new emerge -uDN triggers the whole cylce again. Will share a --debug log later.
And here's the log, once again via google drive because its way too big for bugzilla: https://drive.google.com/file/d/1c45deFcJnNGjDEFwGyV-0X9jgi3gI6rX/view?usp=sharing
(In reply to Andrew Ammerlaan from comment #37) > And here's the log, once again via google drive because its way too big for > bugzilla: > https://drive.google.com/file/d/1c45deFcJnNGjDEFwGyV-0X9jgi3gI6rX/ > view?usp=sharing Seems like this log got truncated but anyway it's having trouble with media-libs/opencv and virtual/rust updates which both seem like they could be related to built slot operator deps as in bug 598503. Can you try with --pretend --ignore-built-slot-operator-deps=y to see if you can get it to pull in the media-libs/opencv-4.8.1-r1 and virtual/rust-1.74.1 updates? If it doesn't pull them in automatically, it's also worth adding =media-libs/opencv-4.8.1-r1 and =virtual/rust-1.74.1 to the emerge arguments to see if forcing them in will reveal some kind of conflict (do this with --pretend --ignore-built-slot-operator-deps=y to suppress noise from built slot operator deps). You can also try adding =dev-libs/protobuf-23.3-r2 to the emerge arguments since the log show some backtracking involving this package which caused =media-libs/opencv-4.8.1-r1 to get eliminated.
(In reply to Zac Medico from comment #38) > (In reply to Andrew Ammerlaan from comment #37) > > And here's the log, once again via google drive because its way too big for > > bugzilla: > > https://drive.google.com/file/d/1c45deFcJnNGjDEFwGyV-0X9jgi3gI6rX/ > > view?usp=sharing > > Seems like this log got truncated but anyway it's having trouble with > media-libs/opencv and virtual/rust updates which both seem like they could > be related to built slot operator deps as in bug 598503. hmm, not sure what happened here. I'll get a new log in a bit. > Can you try with --pretend --ignore-built-slot-operator-deps=y to see if you > can get it to pull in the media-libs/opencv-4.8.1-r1 and virtual/rust-1.74.1 > updates? This removes the rebuild of media-plugins/frei0r-plugins but it still insists on pulling in rust-1.72.0 (non-bin) > If it doesn't pull them in automatically, it's also worth adding > =media-libs/opencv-4.8.1-r1 and =virtual/rust-1.74.1 to the emerge arguments > to see if forcing them in will reveal some kind of conflict (do this with > --pretend --ignore-built-slot-operator-deps=y to suppress noise from built > slot operator deps). You can also try adding =dev-libs/protobuf-23.3-r2 to > the emerge arguments since the log show some backtracking involving this > package which caused =media-libs/opencv-4.8.1-r1 to get eliminated. andrew-gentoo-pc ~ # emerge -uDNA --with-bdeps=y @world --keep-going --backtrack=100000 --verbose-conflicts --ignore-built-slot-operator-deps=y '=media-libs/opencv-4.8.1-r1' '=virtual/rust-1.74.1' Local copy of remote index is up-to-date and will be used. Local copy of remote index is up-to-date and will be used. These are the packages that would be merged, in order: Calculating dependencies... done! Dependency resolution took 49.38 s (backtrack: 12/100000). [binary UD ] dev-libs/protobuf-21.12-9 [23.3-r2] [ebuild UD ] dev-python/protobuf-python-4.21.12 [4.23.3] [ebuild U ] dev-lang/rust-bin-1.74.1 [1.72.0] [binary U ] virtual/rust-1.74.1-1 [1.72.0-r1] [ebuild U ] media-libs/opencv-4.8.1-r1 [4.8.1] USE="tbb%* -cudnn% -non-free%" PYTHON_TARGETS="python3_12%*" VIDEO_CARDS="intel%*" This helped. The issue with opencv seems to be that I have 4.8.1 installed which did not have any restrictions on the version of protobuf. Now we have 4.8.1-r1 in the tree which depends on version <23 of protobuf. Portage should then pull in protobuf for a downgrade but somehow does not. I'm going to let it do what it wants to do and see if this has allowed me to escape from this loop.
And the issue with rust is caused by: !elibc_glibc? ( || ( dev-lang/rust <dev-lang/rust-bin-1.73 ) ) in firefox. This is a bit weird because I had the non-bin version of rust to begin with, then I got stuck in this loop where portage insisted on pulling in rust-bin and was not happy with the version of rust non-bin I had.
The issue with opencv and media-plugins/frei0r-plugins is now gone. The issue with rust being pulled in, removed, and pulled in again remains. Here's the new --debug log: https://drive.google.com/file/d/14Q4v1KhVy_y3MO1gqx7DFyPJz_8c-kiE/view?usp=sharing
(In reply to Andrew Ammerlaan from comment #41) > The issue with opencv and media-plugins/frei0r-plugins is now gone. The > issue with rust being pulled in, removed, and pulled in again remains. > > Here's the new --debug log: > https://drive.google.com/file/d/14Q4v1KhVy_y3MO1gqx7DFyPJz_8c-kiE/ > view?usp=sharing The debug log shows it oddly select dev-lang/rust instead of dev-lang/rust-bin here: Candidates: virtual/rust-1.72.0-r1: ['~dev-lang/rust-1.72.0[rustfmt,abi_x86_32(-),abi_x86_64(-)]'] Does this command behave normally? Any difference with --usepkg=n? emerge -pv =dev-lang/rust-bin-1.72.0 Also, adding the corresponding virtual to the command could reveal more: emerge -pv =dev-lang/rust-bin-1.72.0 =virtual/rust-1.72.0-r1
(In reply to Zac Medico from comment #42) > (In reply to Andrew Ammerlaan from comment #41) > > The issue with opencv and media-plugins/frei0r-plugins is now gone. The > > issue with rust being pulled in, removed, and pulled in again remains. > > > > Here's the new --debug log: > > https://drive.google.com/file/d/14Q4v1KhVy_y3MO1gqx7DFyPJz_8c-kiE/ > > view?usp=sharing > > The debug log shows it oddly select dev-lang/rust instead of > dev-lang/rust-bin here: If rust-1.72.0-r1 is installed then it does (correctly) pull in rust (non-bin) for an upgrade. But somehow also pulls in rust-bin-1.72.0: andrew-gentoo-pc ~ # emerge -uDNA --with-bdeps=y @world --keep-going --backtrack=100000 --verbose-conflicts Local copy of remote index is up-to-date and will be used. Local copy of remote index is up-to-date and will be used. These are the packages that would be merged, in order: Calculating dependencies... done! Dependency resolution took 79.09 s (backtrack: 5/100000). [binary N ] dev-lang/rust-bin-1.72.0-1 USE="rustfmt verify-sig (-big-endian) -clippy -doc (-prefix) -rust-analyzer -rust-src" ABI_X86="32 (64) (-x32)" CPU_FLAGS_X86="sse2" [binary U ] dev-lang/rust-1.74.1-1 [1.72.0-r1] LLVM_TARGETS="-ARC% -CSKY% -DirectX% -M68k% -SPIRV% -Xtensa%" Disabling getbinpkg/usepkg results in: andrew-gentoo-pc ~ # FEATURES="-getbinpkg" emerge -uDNA --with-bdeps=y @world --keep-going --backtrack=100000 --verbose-conflicts --usepkg=n These are the packages that would be merged, in order: Calculating dependencies... done! Dependency resolution took 48.92 s (backtrack: 3/100000). [ebuild N ] dev-lang/rust-bin-1.74.1 USE="rustfmt verify-sig (-big-endian) -clippy -doc (-prefix) -rust-analyzer -rust-src" ABI_X86="32 (64) (-x32)" CPU_FLAGS_X86="sse2" [ebuild U ] dev-lang/rust-1.74.1 [1.72.0-r1] LLVM_TARGETS="-ARC% -CSKY% -DirectX% -M68k% -SPIRV% -Xtensa%" [ebuild U ] virtual/rust-1.74.1 [1.72.0-r1] [ebuild NS ] sys-devel/lld-17.0.6 [15.0.7, 16.0.6] [ebuild NS ] sys-devel/lld-toolchain-symlinks-17 [15-r2, 16-r2] > > Candidates: virtual/rust-1.72.0-r1: > ['~dev-lang/rust-1.72.0[rustfmt,abi_x86_32(-),abi_x86_64(-)]'] > > Does this command behave normally? Any difference with --usepkg=n? > > emerge -pv =dev-lang/rust-bin-1.72.0 > > Also, adding the corresponding virtual to the command could reveal more: > > emerge -pv =dev-lang/rust-bin-1.72.0 =virtual/rust-1.72.0-r1 These commands do what is expected: andrew-gentoo-pc ~ # emerge -pv '=dev-lang/rust-bin-1.72.0' Local copy of remote index is up-to-date and will be used. Local copy of remote index is up-to-date and will be used. These are the packages that would be merged, in order: Calculating dependencies... done! Dependency resolution took 4.32 s (backtrack: 0/20). [binary N ] dev-lang/rust-bin-1.72.0-1:stable::gentoo USE="rustfmt verify-sig (-big-endian) -clippy -doc(-prefix) -rust-analyzer -rust-src" ABI_X86="32 (64) (-x32)" CPU_FLAGS_X86="sse2" 0 KiB Total: 1 package (1 new, 1 binary), Size of downloads: 0 KiB andrew-gentoo-pc ~ # FEATURES="-getbinpkg" emerge -pv '=dev-lang/rust-bin-1.72.0' --usepkg=n These are the packages that would be merged, in order: Calculating dependencies... done! Dependency resolution took 4.37 s (backtrack: 0/20). [ebuild N ] dev-lang/rust-bin-1.72.0:stable::gentoo USE="rustfmt verify-sig (-big-endian) -clippy -doc (-prefix) -rust-analyzer -rust-src" ABI_X86="32 (64) (-x32)" CPU_FLAGS_X86="sse2" 0 KiB Total: 1 package (1 new), Size of downloads: 0 KiB andrew-gentoo-pc ~ # emerge -pv '=dev-lang/rust-bin-1.72.0' '=virtual/rust-1.72.0-r1' Local copy of remote index is up-to-date and will be used. Local copy of remote index is up-to-date and will be used. These are the packages that would be merged, in order: Calculating dependencies... done! Dependency resolution took 4.39 s (backtrack: 0/20). [binary N ] dev-lang/rust-bin-1.72.0-1:stable::gentoo USE="rustfmt verify-sig (-big-endian) -clippy -doc(-prefix) -rust-analyzer -rust-src" ABI_X86="32 (64) (-x32)" CPU_FLAGS_X86="sse2" 0 KiB [binary R ] virtual/rust-1.72.0-r1-1:0/llvm-16::gentoo USE="rustfmt" ABI_X86="32 (64) (-x32)" 0 KiB Total: 2 packages (1 new, 1 reinstall, 2 binaries), Size of downloads: 0 KiB andrew-gentoo-pc ~ # FEATURES="-getbinpkg" emerge -pv '=dev-lang/rust-bin-1.72.0' '=virtual/rust-1.72.0-r1' --usepkg=n These are the packages that would be merged, in order: Calculating dependencies... done! Dependency resolution took 4.44 s (backtrack: 0/20). [ebuild N ] dev-lang/rust-bin-1.72.0:stable::gentoo USE="rustfmt verify-sig (-big-endian) -clippy -doc (-prefix) -rust-analyzer -rust-src" ABI_X86="32 (64) (-x32)" CPU_FLAGS_X86="sse2" 0 KiB [ebuild R ] virtual/rust-1.72.0-r1:0/llvm-16::gentoo USE="rustfmt" ABI_X86="32 (64) (-x32)" 0 KiB Total: 2 packages (1 new, 1 reinstall), Size of downloads: 0 KiB
(In reply to Andrew Ammerlaan from comment #43) > (In reply to Zac Medico from comment #42) > > (In reply to Andrew Ammerlaan from comment #41) > > > The issue with opencv and media-plugins/frei0r-plugins is now gone. The > > > issue with rust being pulled in, removed, and pulled in again remains. > > > > > > Here's the new --debug log: > > > https://drive.google.com/file/d/14Q4v1KhVy_y3MO1gqx7DFyPJz_8c-kiE/ > > > view?usp=sharing > > > > The debug log shows it oddly select dev-lang/rust instead of > > dev-lang/rust-bin here: > > If rust-1.72.0-r1 is installed then it does (correctly) pull in rust > (non-bin) for an upgrade. But somehow also pulls in rust-bin-1.72.0: > > andrew-gentoo-pc ~ # emerge -uDNA --with-bdeps=y @world --keep-going > --backtrack=100000 --verbose-conflicts > > Local copy of remote index is up-to-date and will be used. > > Local copy of remote index is up-to-date and will be used. > > These are the packages that would be merged, in order: > > Calculating dependencies... done! > Dependency resolution took 79.09 s (backtrack: 5/100000). > > [binary N ] dev-lang/rust-bin-1.72.0-1 USE="rustfmt verify-sig > (-big-endian) -clippy -doc (-prefix) -rust-analyzer -rust-src" ABI_X86="32 > (64) (-x32)" CPU_FLAGS_X86="sse2" > [binary U ] dev-lang/rust-1.74.1-1 [1.72.0-r1] LLVM_TARGETS="-ARC% > -CSKY% -DirectX% -M68k% -SPIRV% -Xtensa%" > > Disabling getbinpkg/usepkg results in: > > andrew-gentoo-pc ~ # FEATURES="-getbinpkg" emerge -uDNA --with-bdeps=y > @world --keep-going --backtrack=100000 --verbose-conflicts --usepkg=n > > These are the packages that would be merged, in order: > > Calculating dependencies... done! > Dependency resolution took 48.92 s (backtrack: 3/100000). > > [ebuild N ] dev-lang/rust-bin-1.74.1 USE="rustfmt verify-sig > (-big-endian) -clippy -doc (-prefix) -rust-analyzer -rust-src" ABI_X86="32 > (64) (-x32)" CPU_FLAGS_X86="sse2" > [ebuild U ] dev-lang/rust-1.74.1 [1.72.0-r1] LLVM_TARGETS="-ARC% -CSKY% > -DirectX% -M68k% -SPIRV% -Xtensa%" > [ebuild U ] virtual/rust-1.74.1 [1.72.0-r1] > [ebuild NS ] sys-devel/lld-17.0.6 [15.0.7, 16.0.6] > [ebuild NS ] sys-devel/lld-toolchain-symlinks-17 [15-r2, 16-r2] > Letting this run resolves the rust mess. It seems the problem is related to the binpkg of rust being built against llvm-17. I think in this case portage should reject the binpkg and rebuild to prevent getting stuck in this loop.
(In reply to Andrew Ammerlaan from comment #44) > (In reply to Andrew Ammerlaan from comment #43) > > Disabling getbinpkg/usepkg results in: > > > > andrew-gentoo-pc ~ # FEATURES="-getbinpkg" emerge -uDNA --with-bdeps=y > > @world --keep-going --backtrack=100000 --verbose-conflicts --usepkg=n > > > > These are the packages that would be merged, in order: > > > > Calculating dependencies... done! > > Dependency resolution took 48.92 s (backtrack: 3/100000). > > > > [ebuild N ] dev-lang/rust-bin-1.74.1 USE="rustfmt verify-sig > > (-big-endian) -clippy -doc (-prefix) -rust-analyzer -rust-src" ABI_X86="32 > > (64) (-x32)" CPU_FLAGS_X86="sse2" > > [ebuild U ] dev-lang/rust-1.74.1 [1.72.0-r1] LLVM_TARGETS="-ARC% -CSKY% > > -DirectX% -M68k% -SPIRV% -Xtensa%" > > [ebuild U ] virtual/rust-1.74.1 [1.72.0-r1] > > [ebuild NS ] sys-devel/lld-17.0.6 [15.0.7, 16.0.6] > > [ebuild NS ] sys-devel/lld-toolchain-symlinks-17 [15-r2, 16-r2] > > > > Letting this run resolves the rust mess. It seems the problem is related to > the binpkg of rust being built against llvm-17. But llvm-17 is the latest stable, so you want binpkgs built against that, no? The previously missed sys-devel/lld-17.x update does suggest that it was clinging to an older llvm slot for the dev-lang/rust-bin-1.72.0 binpkg. Why did this update pull in dev-lang/rust-bin-1.74.1, anyway? This should show any reverse dependencies not satisfied by virtual/rust or dev-lang/rust: emerge -pv --depclean dev-lang/rust-bin > I think in this case portage should reject the binpkg and rebuild to prevent > getting stuck in this loop. That's kind of the idea behind automatic --binpkg-changed-deps added for bug 282927, but --usepkgonly/--getbinpkgonly will disable the automatic behavior since the binpkgs are more valuable in that case: https://gitweb.gentoo.org/proj/portage.git/commit/?id=e99fa094ac73514b23509a0f8305b365f114e9a3 The --binpkg-changed-deps option only compares the dependencies specified in the corresponding ebuild though, it doesn't account for built slot operator deps or soname deps. Typically we rely on built slot operator deps or soname deps being handled by the slot operator rebuild logic.
(In reply to Zac Medico from comment #45) > But llvm-17 is the latest stable, so you want binpkgs built against that, > no? The previously missed sys-devel/lld-17.x update does suggest that it was > clinging to an older llvm slot for the dev-lang/rust-bin-1.72.0 binpkg. Yes the rebuild rebuilt rust from llvm-16 to llvm-17. Sorry I said it the wrong way around in my previous post. > Why did this update pull in dev-lang/rust-bin-1.74.1, anyway? This should > show any reverse dependencies not satisfied by virtual/rust or dev-lang/rust: > > emerge -pv --depclean dev-lang/rust-bin > This outputs nothing, it lets me remove rust-bin, and if I do an emerge -uDN does not pull it back in. So this has resolved the loop I was stuck in. I don't understand why it got pulled in before though.
(In reply to Andrew Ammerlaan from comment #46) > > Why did this update pull in dev-lang/rust-bin-1.74.1, anyway? This should > > show any reverse dependencies not satisfied by virtual/rust or dev-lang/rust: > > > > emerge -pv --depclean dev-lang/rust-bin > > > > This outputs nothing, it lets me remove rust-bin, and if I do an emerge -uDN > does not pull it back in. So this has resolved the loop I was stuck in. I > don't understand why it got pulled in before though. Like bug 649622, this firefox dependency can be problematic because it specifies an opposite preference from virtual/rust, preferring dev-lang/rust over dev-lang/rust-bin: || ( dev-lang/rust <dev-lang/rust-bin-1.73 ) It's also problematic in the sense that it might explain how dev-lang/rust-bin-1.72.0 got pulled into your dependency calculations.
The odd firefox <dev-lang/rust-bin-1.73 dep comes from bug 915651.
I wonder why this matters given Andrew is on glibc. But at the same time, I've seen a bunch of odd reports from users which sound like it might be affecting them.
(In reply to Sam James from comment #49) > I wonder why this matters given Andrew is on glibc. Haha, right, it doesn't. > But at the same time, I've seen a bunch of odd reports from users which > sound like it might be affecting them. I checked the log from comment #41 again just now, and noticed system-bootstrap enabled for the dev-lang/rust-1.72.0-r1-1 binpkg: system-bootstrap? ( || ( =dev-lang/rust-1.71* =dev-lang/rust-bin-1.71* =dev-lang/rust-1.72* =dev-lang/rust-bin-1.72* ) ) This introduces preferences that are inconsistent with virtual/rust, possibly triggering behavior like bug 649622.
This is a handy way to distill the rust package selections from a debug log, here showing candidates selected to satisfy dev-lang/rust-1.72.0-r1, virtual/rust-1.72.0-r1, and virtual/rust-1.74.1 dependencies: > $ grep '^Candidates: .*dev-lang/rust' emerge--debug-out.txt | sort -u > Candidates: ['dev-lang/python:3.12', '>=sys-devel/gcc-4.7', '=dev-lang/rust-1.72*'] > Candidates: ['~dev-lang/rust-1.72.0[rustfmt,abi_x86_32(-),abi_x86_64(-)]'] > Candidates: ['~dev-lang/rust-bin-1.74.1[rustfmt,abi_x86_32(-),abi_x86_64(-)]'] Note how the dev-lang/rust-1.72.0-r1 system-bootstrap dependency seem to influence the behavior of the virtual/rust-1.72.0-r1 dependency, causing it to prefer dev-lang/rust over dev-lang/rust-bin. Meanwhile, the virtual/rust-1.74.1 dependencies prefer dev-lang/rust-bin.
The system-bootstrap USE flag causes a circular dependency for dev-lang/rust, since it's supposed to build itself. The circular dependency backtracking implemented for bug 703440 is designed to pull in rust-bin when necessary to resolve the circular dependency, and allow rust-bin to later be removed by depclean. The "later emerge -c wants to remove this version of rust again" behavior mentioned in comment #36 is therefore expected in cases when rust-bin was only required to satisfy a dev-lang/rust system-bootstrap dependency.
Worth noting that dev-lang/go and dev-lang/openjdk pick the more usual pattern of just depending on || ( $X-bin $X ) (up to order). Rust is the odd one out here.
(In reply to Zac Medico from comment #52) > The "later emerge -c wants to remove this version of > rust again" behavior mentioned in comment #36 is therefore expected in cases > when rust-bin was only required to satisfy a dev-lang/rust system-bootstrap > dependency. I suspect that the default emerge --with-bdeps-auto behavior can cause a problem with the circular dependency handling for binary packages. Since the "bdeps" are optional for binary packages, I think we need to change optional dependency handling for the circular dependency case, in order to prevent rust-bin from being pulled in to solve a circular dependency that doesn't really exist: --- a/lib/_emerge/depgraph.py +++ b/lib/_emerge/depgraph.py @@ -3575,5 +3575,7 @@ class depgraph: # it can skew the merge order calculation in an unwanted # way. - if pkg != dep.parent or (priority.buildtime and not priority.satisfied): + if pkg != dep.parent or ( + priority.buildtime and not (priority.satisfied or priority.optional) + ): self._dynamic_config.digraph.add(pkg, dep.parent, priority=priority) if dep.atom is not None and dep.parent is not None:
(In reply to Zac Medico from comment #54) > I suspect that the default emerge --with-bdeps-auto behavior can cause a > problem with the circular dependency handling for binary packages. I tested it, and --usepkg actually disables the --with-bdeps-auto behavior. With --with-bdeps=y, it adds the direct circular dependency to the graph, and it doesn't trigger any circular dependency backtracking because the dependency is considered optional and therefore is too low priority cause a circular dependency error.
Got stuck in a loop again: Local copy of remote index is up-to-date and will be used. Local copy of remote index is up-to-date and will be used. These are the packages that would be merged, in order: Calculating dependencies... done! Dependency resolution took 174.00 s (backtrack: 56/100000). [binary R ] dev-qt/qtdeclarative-6.6.2-r1-2 [ebuild rR #] kde-frameworks/kwindowsystem-6.0.0-r1 [ebuild rR #] kde-plasma/kwayland-6.0.3 [ebuild rR #] kde-plasma/layer-shell-qt-6.0.3 [ebuild rR #] kde-plasma/plasma-workspace-6.0.3 WARNING: One or more updates/rebuilds have been skipped due to a dependency conflict: dev-qt/qtdeclarative:6 (dev-qt/qtdeclarative-6.7.0:6/6.7.0::gentoo, ebuild scheduled for merge) USE="accessibility network opengl sql ssl svg vulkan widgets -test" ABI_X86="(64)" conflicts with dev-qt/qtdeclarative:6/6.6.2= required by (dev-libs/qcoro-0.10.0-r1-1:0/0::gentoo, installed) USE="dbus network qml -examples -test -websockets" ABI_X86="(64)" ^^^^^^^^^ >=dev-qt/qtdeclarative-6.6.2:6/6.6.2= required by (kde-frameworks/qqc2-desktop-style-6.0.0-1:6/6.0::gentoo, installed) USE="-debug -test" ABI_X86="(64)" ^^^^^^^^^ ~dev-qt/qtdeclarative-6.6.2:6 required by (dev-qt/qtwayland-6.6.2-r1-2:6/6.6.2::gentoo, binary scheduled for merge) USE="accessibility compositor qml vulkan -test" ABI_X86="(64)" ^ ^^^^^ =dev-qt/qtdeclarative-6.6.2*:6[widgets] required by (dev-python/pyside6-6.6.2-r1-1:0/0::gentoo, installed)USE="bluetooth concurrent dbus gui network opengl printsupport qml quick sql svg testlib webchannel webenginewidgets xml -3d -charts -designer -gles2-only -help -location -multimedia -network-auth -nfc -pdfium -positioning -quick3d -scxml -sensors -serialport -spatialaudio -speech -test -websockets" ABI_X86="(64)" LLVM_SLOT="17 -15 -16" PYTHON_TARGETS="python3_11 python3_12 -python3_10" ^ ^^^^^^ ~dev-qt/qtdeclarative-6.6.2:6 required by (dev-qt/qtwebchannel-6.6.2-1:6/6.6.2::gentoo, installed) USE="qml -test" ABI_X86="(64)" ^ ^^^^^ ~dev-qt/qtdeclarative-6.6.2:6 required by (dev-qt/qtwebengine-6.6.2-2:6/6.6.2::gentoo, installed) USE="alsa geolocation jumbo-build opengl pulseaudio qml screencast system-icu vaapi vulkan widgets -bindist -custom-cflags -designer -kerberos -pdfium -test" ABI_X86="(64)" ^ ^^^^^ ~dev-qt/qtdeclarative-6.6.2:6[widgets] required by (dev-qt/qtwebengine-6.6.2-2:6/6.6.2::gentoo, installed)USE="alsa geolocation jumbo-build opengl pulseaudio qml screencast system-icu vaapi vulkan widgets -bindist -custom-cflags -designer -kerberos -pdfium -test" ABI_X86="(64)" ^ ^^^^^ ~dev-qt/qtdeclarative-6.6.2:6 required by (dev-qt/qtpositioning-6.6.2-1:6/6.6.2::gentoo, installed) USE="geoclue qml -nmea -test" ABI_X86="(64)" ^ ^^^^^ !!! The following update(s) have been skipped due to unsatisfied dependencies !!! triggered by backtracking: dev-qt/qttools:6 dev-qt/qtcharts:6 dev-qt/qtmultimedia:6 dev-qt/qtsensors:6 dev-qt/qtspeech:6 dev-qt/qttranslations:6 dev-qt/qtbase:6 dev-qt/qtsvg:6 dev-qt/qtwayland:6 dev-qt/qtshadertools:6 dev-qt/qt5compat:6 dev-qt/qtimageformats:6 dev-qt/qtwebengine:6 dev-qt/qtnetworkauth:6 This one I cannot resolve with --with-bdeps=y, or by force rebuilding dev-qt/qtdeclarative. Somehow portage always insists on pulling in this new binary and rebuilding it's reverse dependencies.
emerge --debug output: https://drive.google.com/file/d/1PaPGObUBtZ4G7PX9U8UO3su7vZ7r-345/view?usp=sharing
The loop is resolved by masking >=dev-qt/*-6.6.3:6. Pyside6 still is stuck on 6.6.2 only and therefore prevents upgrading Qt. Still this requirement should not cause this recursive reinstalling of dev-qt/qtdeclarative.
I am having this problem with net-libs/nodejs-22.7.0:0/22::gentoo, it wants to replace its dev-libs/simdjson dependency from binary dev-libs/simdjson-3.9.1-1:0/22::gentoo to ebuild dev-libs/simdjson-3.10.1::gentoo every time I run emerge --with-bdeps=y -avuDN @world This cycle also breaks emerge --depclean. Dependencies could not be completely resolved due to the following required packages not being installed: >=dev-libs/simdjson-3.9.1:0/22= pulled in by: net-libs/nodejs-22.7.0 Have you forgotten to do a complete update prior to depclean? I am now attempting to use this fix, suggested in #gentoo by Andrew: # emerge -C dev-libs/simdjson net-libs/nodejs # FEATURES=-getbinpkgs emerge --ignore-default-opts -1 net-libs/nodejs It will take ~12h to build nodejs and know if it actually worked. I will post here the result when I have the chance. o/ emanuele6
(In reply to Emanuele Torre from comment #59) > I am now attempting to use this fix, suggested in #gentoo by Andrew: > > # emerge -C dev-libs/simdjson net-libs/nodejs > # FEATURES=-getbinpkgs emerge --ignore-default-opts -1 net-libs/nodejs > > It will take ~12h to build nodejs and know if it actually worked. > I will post here the result when I have the chance. Thank you: this fix worked. > I am having this problem with net-libs/nodejs-22.7.0:0/22::gentoo, it > wants to replace its dev-libs/simdjson dependency > from binary dev-libs/simdjson-3.9.1-1:0/22::gentoo > to ebuild dev-libs/simdjson-3.10.1::gentoo > every time I run emerge --with-bdeps=y -avuDN @world I meant to from, and vice versa. o/ emanuele6
