I use Portage trunk (2.2.10_p39; 9089c2d755b0ecb1b340fc23dda461163f589c43). I was preparing my system for migration from app-emulation/emul-linux-x86-* to USE="abi_x86_x32" on relevant packages and I noticed that Portage dependency resolver makes incorrect choices to satisfy dependencies of virtual/pkgconfig and virtual/*udev. I had the following packages installed: app-emulation/emul-linux-x86-baselibs app-emulation/emul-linux-x86-db app-emulation/emul-linux-x86-gtklibs app-emulation/emul-linux-x86-medialibs app-emulation/emul-linux-x86-opengl app-emulation/emul-linux-x86-sdl app-emulation/emul-linux-x86-soundlibs app-emulation/emul-linux-x86-xlibs These packages were direct or indirect dependencies of: app-emulation/wine sys-devel/gcc:4.7 sys-devel/gcc:4.8 I uninstalled all app-emulation/emul-linux-x86-* packages. I masked all app-emulation/emul-linux-x86-* packages in /etc/portage/package.mask. I ran `ABI_X86="32 64" emerge -ptv --backtrack=1000000 app-emulation/wine sys-devel/gcc:{4.7,4.8}`, which incorrectly suggested installation of dev-util/pkgconf and sys-apps/systemd. virtual/pkgconfig generally depends on: || ( dev-util/pkgconfig dev-util/pkgconf dev-util/pkgconfig-openbsd ) dev-util/pkgconfig is already installed, not masked and able for being reinstalled. Portage should not suggest installation of another pkgconfig implementation. virtual/*udev generally depends on: || ( sys-fs/udev sys-apps/systemd sys-fs/eudev ) sys-fs/udev is already installed, not masked and able for being reinstalled. Portage should not suggest installation of another udev implementation. (I suspended migration, so that I can test potential patches for Portage.)
Created attachment 379740 [details] ABI_X86="32 64" emerge -ptv --backtrack=1000000 app-emulation/wine sys-devel/gcc:{4.7,4.8} Output of: ABI_X86="32 64" emerge -ptv --backtrack=1000000 app-emulation/wine sys-devel/gcc:{4.7,4.8} This output shows problems described in initial comment.
Created attachment 379742 [details] ABI_X86="32 64" emerge -dptv --backtrack=1000000 app-emulation/wine sys-devel/gcc:{4.7,4.8} Output of: ABI_X86="32 64" emerge -dptv --backtrack=1000000 app-emulation/wine sys-devel/gcc:{4.7,4.8}
Created attachment 379744 [details] ABI_X86="32 64" emerge -ptv virtual/pkgconfig Output of: ABI_X86="32 64" emerge -ptv virtual/pkgconfig This output shows that dev-util/pkgconfig is able to be reinstalled.
Created attachment 379746 [details] ABI_X86="32 64" emerge -dptv virtual/pkgconfig Output of: ABI_X86="32 64" emerge -dptv virtual/pkgconfig
Created attachment 379748 [details] ABI_X86="32 64" emerge -ptv virtual/{libgudev,libudev,udev} Output of: ABI_X86="32 64" emerge -ptv virtual/{libgudev,libudev,udev} This output shows that sys-fs/udev is able to be reinstalled.
Created attachment 379750 [details] ABI_X86="32 64" emerge -dptv virtual/{libgudev,libudev,udev} Output of: ABI_X86="32 64" emerge -dptv virtual/{libgudev,libudev,udev}
Created attachment 379752 [details] ABI_X86="32 64" emerge -ptv --backtrack=1000000 app-emulation/wine sys-devel/gcc:{4.7,4.8} (after masking of some packages) Output of: ABI_X86="32 64" emerge -ptv --backtrack=1000000 app-emulation/wine sys-devel/gcc:{4.7,4.8} With the following packages masked in /etc/portage/package.mask: dev-util/pkgconf dev-util/pkgconfig-openbsd sys-apps/systemd sys-fs/eudev Masking of these packages should not be necessary for Portage to correctly calculate dependencies.
Created attachment 379754 [details] ABI_X86="32 64" emerge -dptv --backtrack=1000000 app-emulation/wine sys-devel/gcc:{4.7,4.8} (after masking of some packages) Output of: ABI_X86="32 64" emerge -dptv --backtrack=1000000 app-emulation/wine sys-devel/gcc:{4.7,4.8} With the following packages masked in /etc/portage/package.mask: dev-util/pkgconf dev-util/pkgconfig-openbsd sys-apps/systemd sys-fs/eudev
Just for the record, does adding --deep, --upgrade or --changed-use (if possible, please try each of them separately) help?
I apologize for the bugspam, but maybe sys-apps/portage should be reverted to a version where it still worked correctly? This is causing multiple threads in the forums, such as http://forums.gentoo.org/viewtopic-t-993294.html
My laptop is not 100% idle so the margin for error is larger then I would like. These are what looked like the average run. Installed versions: 2.1.12.2(12:31:50 PM 04/07/14) [blocks b ] >=net-libs/libsoup-2.42 (">=net-libs/libsoup-2.42" is blocking net-libs/libsoup-gnome-2.40.3) [uninstall ] net-libs/libsoup-gnome-2.40.3:2.4 USE="introspection -debug" [blocks b ] sys-power/upower ("sys-power/upower" is blocking sys-power/upower-pm-utils-0.9.23-r2) [uninstall ] sys-power/upower-pm-utils-0.9.23-r2 USE="introspection -ios" [blocks B ] sys-apps/systemd ("sys-apps/systemd" is blocking sys-fs/eudev-1.8) [blocks B ] sys-apps/gentoo-systemd-integration ("sys-apps/gentoo-systemd-integration" is blocking sys-fs/eudev-1.8) Total: 246 packages (192 upgrades, 2 downgrades, 42 new, 2 in new slots, 8 reinstalls, 3 uninstalls), Size of downloads: 377,367 kB Conflict: 6 blocks (2 unsatisfied) real 6m16.539s user 6m6.210s sys 0m9.871s Installed versions: 2.2.1(12:13:43 PM 04/07/14) <same blocks> real 6m34.317s user 6m24.286s sys 0m9.559s Installed versions: 2.2.7(12:03:49 PM 04/07/14) <same blocks> real 6m28.036s user 6m18.138s sys 0m9.295s Installed versions: 2.2.8-r1(11:49:26 AM 04/07/14) < couple extra packages in the blocks but still sane > real 9m33.284s user 9m19.807s sys 0m12.549s And the depgraph for 2.2.10 is insanity, I kill the timed run after 13 minutes because I gave up waiting. RECOMMENDATION: hard mask 2.2.10, put 2.2.8+ back into testing, and then all the stable people get 2.2.7. But I would personally mask everything past 2.2.7, I consider 2.2.7 the last version to work properly.
(In reply to Weedy from comment #11) > RECOMMENDATION: hard mask 2.2.10, put 2.2.8+ back into testing, and then all > the stable people get 2.2.7. > But I would personally mask everything past 2.2.7, I consider 2.2.7 the last > version to work properly. This version also contained resolver bugs that caused lots of duplicated bug reports. The problem is that there are lots of bits in the resolver that interact in a non-trivial way. If you change one bit, the interaction may change and the computed result can be quite different. What in the way of debugging this is, is that those issues are hard to reproduce. Even a debug log doesn't always help. What we need is a minimal test case to reproduce this. A -uDN world log just contains too much noise.
(In reply to Sebastian Luther (few) from comment #12) > What in the way of debugging this is, is that those issues are hard to > reproduce. Even a debug log doesn't always help. We should capture more of the state in order to reproduce it; a debug log doesn't let you reproduce it as it just gives some guesses on what could have happened, a capture state on the other hand allows one to somewhat pinpoint it in the code.
(In reply to Sebastian Luther (few) from comment #12) > This version also contained resolver bugs that caused lots of duplicated bug > reports. Was 2.8 better at resolving? Any way you look at it 2.10 is broken. Like stupidly horrendously broken. And that's ignoring the slowness.
(In reply to Arfrever Frehtes Taifersar Arahesis from comment #2) > Created attachment 379742 [details] > ABI_X86="32 64" emerge -dptv --backtrack=1000000 app-emulation/wine > sys-devel/gcc:{4.7,4.8} > > Output of: ABI_X86="32 64" emerge -dptv --backtrack=1000000 > app-emulation/wine sys-devel/gcc:{4.7,4.8} The problem appears to be a result of the new slot conflict handling code inside the depgraph._solve_non_slot_operator_slot_conflicts method. The debug log shows it handling a conflict for virtual/pkgconfig-0-r1: !!! Slot conflict graph: (virtual/pkgconfig-0-r1:0/0::gentoo, installed) depends on (dev-util/pkgconfig-0.28-r1:0/0::gentoo, installed) (0) Immediately after that point in the log, the depgraph starts pulling in dev-util/pkgconf instead of dev-util/pkgconfig.
Created attachment 383434 [details, diff] package_tracker.match: account for atom.unevaluated_atom Please disregard comment #15. The attached patch fixes the match cache to properly account for atom.unevaluated_atom, which is required since match_from_list output depends on atom.unevaluated_atom. Note that similar issues were also fixed in commits d603f1440c814377fbc1965729fd9b6b008cf76d and 5438bb29c996d777b6343515995176912a7c137f.
Created attachment 383436 [details, diff] package_tracker.match: account for atom.unevaluated_atom This fixed misspelling of unevaluated_atom in the previous patch. Somebody should go through other changes from the past year to see if there are any other match caches that fail to account for atom.unevaluated_atom.
(In reply to Zac Medico from comment #17) > Created attachment 383436 [details, diff] [details, diff] > package_tracker.match: account for atom.unevaluated_atom > > This fixed misspelling of unevaluated_atom in the previous patch. > > Somebody should go through other changes from the past year to see if there > are any other match caches that fail to account for atom.unevaluated_atom. I spoke to Sebastian Luther on IRC, and he doesn't think there are any other match caches with this sort of problem. So, with Brian Dolbec's blessing, I've pushed this patch as commit 5c0f68017e9943f9486ee68005ea3ef7743244bd.
This bug was introduced in Portage 2.2.9. virtual/*udev part of this bug is no longer reproducible due to changes in the tree. virtual/pkgconfig part of this bug is still reproducible at the time of 5c0f68017e9943f9486ee68005ea3ef7743244bd.
Created attachment 383494 [details] relevant part of the debug log I was able to reproduce the problem using an complete copy of Arfrever's configuration. It seems that the problem is due to the the depgraph using _select_atoms_from_graph after just having called _remove_pkg on the dev-util/pkgconfig instance because its only parent was involved in a slot conflict. The relevant _remove_pkg calls show in the log like this: Removing package: (virtual/pkgconfig-0-r1:0/0::gentoo, installed) Removing package: (dev-util/pkgconfig-0.28-r2:0/0::gentoo, installed) Following that, the unbuilt virtual/pkgconfig-0-r1 ebuild gets pulled in, since it's already in the graph (it was involved in the slot conflict with the installed instance that just got removed). Here is the relevant part of the backtrace which corresponds to the state at the end of the attached log: pym/_emerge/depgraph.py(6269)altlist() -> self._resolve_conflicts() pym/_emerge/depgraph.py(6396)_resolve_conflicts() -> self._process_slot_conflicts() pym/_emerge/depgraph.py(1241)_process_slot_conflicts() -> self._solve_non_slot_operator_slot_conflicts() pym/_emerge/depgraph.py(1208)_solve_non_slot_operator_slot_conflicts() -> self._create_graph() pym/_emerge/depgraph.py(2037)_create_graph() -> if not self._pop_disjunction(allow_unsatisfied): pym/_emerge/depgraph.py(3048)_pop_disjunction() -> pkg, dep_root, dep_priority, dep_struct, allow_unsatisfied): pym/_emerge/depgraph.py(2655)_add_pkg_dep_string() -> allow_unsatisfied) > pym/_emerge/depgraph.py(2831)_wrapped_add_pkg_dep_string() And with Pdb we can see the _select_atom state: (Pdb) p self._select_atoms <bound method depgraph._select_atoms_from_graph of <_emerge.depgraph.depgraph object at 0x9bfccec>> So, it seems that we need to update that depgraph._select_atoms reference inside _solve_non_slot_operator_slot_conflicts, before calling _create_graph.
The faulty dep resolution actually begins to occur when _select_atom is still set to _select_atoms_highest_available. The root problem is actually seems to be due to the _dep_check_composite_db visibility logic, as demonstrated by the following Pdb log: (Pdb) p self._select_package('/', Atom('>=dev-util/pkgconfig-0.28-r1[abi_x86_32(-),abi_x86_64(-)]')) (<Package ('ebuild', '/', 'dev-util/pkgconfig-0.28-r2', 'merge', 'gentoo')>, None) (Pdb) p self._dynamic_config._filtered_trees['/']['porttree'].dbapi.match_pkgs(Atom('>=dev-util/pkgconfig-0.28-r1[abi_x86_32(-),abi_x86_64(-)]')) [] As you can see, _select_package appears to return the correct result, while _dep_check_composite_db.match_pkgs returns an empty result. I appears to be due to the unbuilt instance being masked by _dep_check_composite_db._visible, since the installed instance is already in the graph.
Created attachment 383582 [details, diff] _dep_check_composite_db: fix bug #515230 This patches fixes it for me with Arfrever's configuration. Due to the complexity of the state involved, I haven't yet written a standalone test case.
(In reply to Zac Medico from comment #22) This patch fixes this bug, but breaks `emerge -ptv dev-java/icedtea` (unwanted downgrade of dev-java/java-config from >=2.2 to <2.2 and uninstallation of app-admin/eselect-java). Java eclasses add these dependencies to JVM packages: >=dev-java/java-config-2.1.9-r1 =dev-java/java-config-2* || ( app-admin/eselect-java <dev-java/java-config-2.2 )
Created attachment 383824 [details, diff] dep_check: fix bug #515230 This version of the patch includes a new hunk in dep_zapdeps, which fixes the java-config downgrade issue that was reported in comment #23.
(In reply to Zac Medico from comment #24) > Created attachment 383824 [details, diff] [details, diff] > dep_check: fix bug #515230 > > This version of the patch includes a new hunk in dep_zapdeps, which fixes > the java-config downgrade issue that was reported in comment #23. I've pushed this as commit 617a64f77055ddda1dd86a506cf98f5adc545fae.
This is fixed in 2.2.13.
