Summary: | Specify behavior of *TARGET* USE_EXPAND variable settings so that package managers can use them to infer which updates are desirable (related: python-any-r1 pulls in latest Python) | ||
---|---|---|---|
Product: | Gentoo Hosted Projects | Reporter: | alfredthetomato <alfredthetomato> |
Component: | PMS/EAPI | Assignee: | PMS/EAPI <pms> |
Status: | UNCONFIRMED --- | ||
Severity: | enhancement | CC: | arcctgx, charles17, galtgendo, gentoo_eshoes, gotyaoi, jospezial, kingjon3377, matoro_bugzilla_gentoo, me, mgorny, orzel, pacho, python, sam, sveyret, voron1, xavier.miller, xaviermiller |
Priority: | Normal | ||
Version: | unspecified | ||
Hardware: | All | ||
OS: | All | ||
See Also: |
https://bugs.gentoo.org/show_bug.cgi?id=765598 https://bugs.gentoo.org/show_bug.cgi?id=283587 https://bugs.gentoo.org/show_bug.cgi?id=442076 |
||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Attachments: |
portage-bug_702806.patch
portage-bug_702806-r1.patch portage-bug_702806-r2.patch |
Maybe related to the change from bug 701996: https://gitweb.gentoo.org/proj/portage.git/commit/?id=fa7b6ea6ecdc135a01a65e249276e6d75b92791e Reverting that commit restored previous behaviour. (In reply to alfredthetomato from comment #0) > The following USE changes are necessary to proceed: > (see "package.use" in the portage(5) man page for more details) > # required by dev-lang/ruby-2.6.5::gentoo > # required by virtual/rubygems-15::gentoo[ruby_targets_ruby26] > # required by dev-ruby/rdoc-6.2.0::gentoo[-test,ruby_targets_ruby26] > # required by dev-lang/ruby-2.4.9::gentoo[rdoc] > # required by dev-ruby/rubygems-3.0.6::gentoo[ruby_targets_ruby24] > >=dev-ruby/did_you_mean-1.3.1 ruby_targets_ruby26 It looks like you have ruby_targets_ruby26 enabled for dev-ruby/rdoc and virtual/rubygems. Does this command confirm it? emerge -pv --nodeps dev-ruby/rdoc virtual/rubygems (In reply to alfredthetomato from comment #2) > Reverting that commit restored previous behaviour. We're not reverting that commit unless we have a clear test case showing that there's something wrong with it. As noted in comment #3, your output seems to indicate that you had ruby_targets_ruby26 enabled for dev-ruby/rdoc and virtual/rubygems. You have to ensure that you've disabled ruby_targets_ruby26 globally if you don't want it. Also, you can't install dev-lang/ruby-2.6.5 unless ruby_targets_ruby26 is enabled, so you need to ensure that you don't have dev-lang/ruby or something like that in /var/lib/portage/world. It's okay to have specific slots in there, like dev-lang/ruby:2.5 for example. dev-lang/ruby is being pulled in as a dependency only. It is not in @world. The output of emerge -pv --nodeps dev-ruby/rdoc virtual/rubygems shows that I have the default RUBY_TARGETS set, with or without that commit: These are the packages that would be merged, in order: [ebuild N ~] dev-ruby/rdoc-6.2.0::gentoo USE="-doc -test" RUBY_TARGETS="ruby24 ruby25 -ruby26" 653 KiB [ebuild N ] virtual/rubygems-15::gentoo RUBY_TARGETS="ruby24 ruby25 -ruby26" 0 KiB Total: 2 packages (2 new), Size of downloads: 653 KiB A simple test-case for the described issue would be to emerge something which requires dev-lang/ruby as a dependency:
emerge -p dev-libs/ocl-icd
It will happily proceed with the commit reverted, but with it applied:
The following USE changes are necessary to proceed:
(see "package.use" in the portage(5) man page for more details)
# required by dev-lang/ruby-2.6.5::gentoo
# required by virtual/rubygems-15::gentoo[ruby_targets_ruby26]
# required by dev-lang/ruby-2.4.9::gentoo
# required by dev-ruby/bundler-1.17.3::gentoo[ruby_targets_ruby24]
>=dev-ruby/did_you_mean-1.3.1 ruby_targets_ruby26
With the commit applied, it will refuse to proceed until ruby_targets_26 is added to USE. Putting -ruby_targets_26 into USE does not help.
You mentioned that dev-lang/ruby:2.6 cannot be installed without ruby_targets_26 enabled. I also mentioned that dev-lang/python:3.7 had been installed after running emerge -uDN @world. Reverting the commit and running depclean will remove the extra dev-lang/python:3.7 slot.
The underlying issue is more likely due to portage adding the new dev-lang/ruby:2.6 and dev-lang/python:3.7 slots to the dependency graphs despite profile default *_TARGETS being set, which do not call for those newer slots.
(In reply to alfredthetomato from comment #6) > The underlying issue is more likely due to portage adding the new > dev-lang/ruby:2.6 and dev-lang/python:3.7 slots to the dependency graphs > despite profile default *_TARGETS being set, which do not call for those > newer slots. The thing is, portage has no way of knowing that a person's *_TARGETS settings should correlate with the latest version of some particular package such as dev-lang/ruby or dev-lang/python. There's no rule linking these things together. So, it's a person's responsibility to manage latest versions of these packages with mechanisms like ACCEPT_KEYWORDS, package.accept_keywords and package.mask. So, assuming you have ACCEPT_KEYWORDS="amd64 ~amd64", something like this will give you the package versions that you want if we assume that you have *_TARGETS enabled for "stable" versions of dev-lang/python and dev-lang/ruby: $ cat /etc/portage/package.accept_keywords dev-lang/python -~amd64 dev-lang/ruby -~amd64 (In reply to Zac Medico from comment #7) > The thing is, portage has no way of knowing that a person's *_TARGETS > settings should correlate with the latest version of some particular package > such as dev-lang/ruby or dev-lang/python. There's no rule linking these > things together. So, it's a person's responsibility to manage latest > versions of these packages with mechanisms like ACCEPT_KEYWORDS, > package.accept_keywords and package.mask. Portage seemed to manage quite well and did not require user intervention before the commit. Regardless, I came up with a patch to keep the fix for bug 701996 in place whilst restoring the previous behaviour with regards to *_TARGETS dependency resolution. Portage successfully installs with FEATURES=test. Running emerge -pv =virtual/wine-0-r6 dev-libs/ocl-icd with USE=staging just pulls in wine-staging (no wine-vanilla) and will proceed without requiring RUBY_TARGETS changes. Created attachment 599856 [details, diff]
portage-bug_702806.patch
Patch to restore previous *_TARGETS behaviour.
Created attachment 599860 [details, diff]
portage-bug_702806-r1.patch
Previous patch changed depclean behaviour, updated patch fixes that.
(In reply to alfredthetomato from comment #11) > Created attachment 599860 [details, diff] [details, diff] > portage-bug_702806-r1.patch > > Previous patch changed depclean behaviour, updated patch fixes that. This patch is not valid because you're actually dropping dependencies here: > + if disjunctions_appended: > + disjunctions = [] Created attachment 599862 [details, diff]
portage-bug_702806-r2.patch
Updated to hopefully not drop dependencies.
(In reply to alfredthetomato from comment #13) > Created attachment 599862 [details, diff] [details, diff] > portage-bug_702806-r2.patch > > Updated to hopefully not drop dependencies. Do you know why this changes the behavior? I don't want to merge this patch as-is because it goes against the intention of the fix for bug 701996, which is to group all of of the disjunctions (and virtuals) into a single _queue_disjunction call. The behavior that you're observing is entirely expected, and I don't want to merge a patch to change the behavior unless there is a very good reason (and it must not conflict with other concerns such as the fix for bug 701996). (In reply to Zac Medico from comment #14) > Do you know why this changes the behavior? I don't want to merge this patch > as-is because it goes against the intention of the fix for bug 701996, which > is to group all of of the disjunctions (and virtuals) into a single > _queue_disjunction call. > > The behavior that you're observing is entirely expected, and I don't want to > merge a patch to change the behavior unless there is a very good reason (and > it must not conflict with other concerns such as the fix for bug 701996). Calling _queue_disjunction per || dependency but grouping the virtuals will result in the same behaviour before the fix for bug 701996. Conversely, calling _queue_disjunction per virtual dependency, but grouping || dependencies will result in the same behaviour after the commit. Therefore, it is sufficient to group just the || disjunctive dependencies together to fix bug 701996. So my patch results in _queue_disjunction called two times instead of one: firstly with || disjunctions grouped and then for the second time for the virtual dependencies. (In reply to alfredthetomato from comment #15) > Calling _queue_disjunction per || dependency but grouping the virtuals will > result in the same behaviour before the fix for bug 701996. Conversely, > calling _queue_disjunction per virtual dependency, but grouping || > dependencies will result in the same behaviour after the commit. Therefore, > it is sufficient to group just the || disjunctive dependencies together to > fix bug 701996. The intention of the code is to group all of the || dependencies together with the virtual dependencies so that they can all be expanded into DNF together, for the purposes of bug 632026 (there's an _expand_new_virtuals function that will expand a virtual dependency into its component || dependencies). In this context, it's counter-productive to split virtual dependencies into separate _queue_disjunction calls. > So my patch results in _queue_disjunction called two times instead of one: > firstly with || disjunctions grouped and then for the second time for the > virtual dependencies. It can be called any number of times, depending on the composition of the dependency string, and this goes completely against intentions for solving cases like bug 632026, since we want to be able to generate a single DNF that combines all of the disjunctions found in the dependency string. I don't think that your patch guarantees any particular behavior. Even if it appears to produce the "desired" result for the current state of your system at this time, something in the dependencies could easily change that causes it to produce a different behavior. *** Bug 705700 has been marked as a duplicate of this bug. *** I suppose it's possible to add some kind of special logic for *_TARGETS, as long as it doesn't re-introduce issues like bug 693790 or bug 706278. *** Bug 753470 has been marked as a duplicate of this bug. *** *** Bug 754912 has been marked as a duplicate of this bug. *** *** Bug 758878 has been marked as a duplicate of this bug. *** *** Bug 765598 has been marked as a duplicate of this bug. *** I also find this bug (the current behavior) to be an irritant and am glad that I stumbled on this bug report. I do not have dev-lang/python in my world file, but it gets pulled in by other Python programs. They all have the default PYTHON_TARGETS set (python3_8), except python_exec, but that seems to be a special case, as it also has pypy3 set without pulling in dev-lang/pypy3. Nevertheless, Python 3.9 gets pulled in next to Python 3.8, updated, etc. without me ever using it. I see no logic there. I do understand that portage evolved to exhibit this behavior, while fixing other bugs. But still, please consider getting this corner of portage to behave a but more as expected. *** Bug 790431 has been marked as a duplicate of this bug. *** (In reply to Erik Quaeghebeur from comment #24) > I also find this bug (the current behavior) to be an irritant and am glad > that I stumbled on this bug report. > > I do not have dev-lang/python in my world file, but it gets pulled in by > other Python programs. They all have the default PYTHON_TARGETS set > (python3_8), except python_exec, but that seems to be a special case, as it > also has pypy3 set without pulling in dev-lang/pypy3. Nevertheless, Python > 3.9 gets pulled in next to Python 3.8, updated, etc. without me ever using > it. I see no logic there. > > I do understand that portage evolved to exhibit this behavior, while fixing > other bugs. But still, please consider getting this corner of portage to > behave a but more as expected. The root problem is that you've grown accustomed to an inconsistent behavior of old portage versions. The old behavior was truly a bug, and there's no simple way to revert to the old behavior without introducing a risk of inconsistent results and missed updates. That said, there's probably some way to model the behavior that you want, and I'm open to contributions of that nature. Maybe you can overlay a masking layer that models the *TARGET* USE_EXPAND vars correlation. The behavior of *TARGET* USE_EXPAND settings is not covered by PMS, so first we'll have to specify the behavior of *TARGET* USE_EXPAND settings. This is another case like bug 283587, where we'll have extend the package manager specification in order to allow deeper integration with the Gentoo ecosystem. The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=3d081cb997d18d200c71e1ff394e17db2f01f38d commit 3d081cb997d18d200c71e1ff394e17db2f01f38d Author: Sam James <sam@gentoo.org> AuthorDate: 2021-11-11 21:02:52 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2021-11-11 21:02:52 +0000 sys-libs/glibc: drop Python 3.10 from PYTHON_COMPAT to avoid circular deps Needed for now to try help upgrades and avoid circular dependencies with glibc -> python -> libcrypt -> libxcrypt -> glibc -> ... Bug: https://bugs.gentoo.org/699422 Bug: https://bugs.gentoo.org/702806 Bug: https://bugs.gentoo.org/809410 Signed-off-by: Sam James <sam@gentoo.org> sys-libs/glibc/glibc-2.33-r7.ebuild | 5 ++++- sys-libs/glibc/glibc-2.34-r1.ebuild | 5 ++++- sys-libs/glibc/glibc-9999.ebuild | 5 ++++- 3 files changed, 12 insertions(+), 3 deletions(-) |
With previous versions of sys-apps/portage, such as ≤portage-2.3.80, emerge -uDN @world acted as expected and would consider only the highest version of whichever slot satisfied *_TARGETS, namely PYTHON_SINGLE_TARGET and {PYTHON,RUBY}_TARGETS. With sys-apps/portage-2.3.81, emerge -uDN @world wants to update to the latest slot available, despite the profile default *_TARGETS being set. Wrt dev-lang/python, it will happily emerge the new python:3.7 slot, despite PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" being set. However, when it comes to dev-lang/ruby: The following USE changes are necessary to proceed: (see "package.use" in the portage(5) man page for more details) # required by dev-lang/ruby-2.6.5::gentoo # required by virtual/rubygems-15::gentoo[ruby_targets_ruby26] # required by dev-ruby/rdoc-6.2.0::gentoo[-test,ruby_targets_ruby26] # required by dev-lang/ruby-2.4.9::gentoo[rdoc] # required by dev-ruby/rubygems-3.0.6::gentoo[ruby_targets_ruby24] >=dev-ruby/did_you_mean-1.3.1 ruby_targets_ruby26 ... (it continues with everything with a RUBY_TARGETS use_expand flag) Essentially, it will not proceed until the user sets RUBY_TARGETS: ruby26, despite the profile defaults of RUBY_TARGETS="ruby24 ruby25". Adding --backtrack=9999 does not help. I would expect sys-apps/portage to only consider those versions of dev-lang/{python,ruby} which satisfy the slot requirements set with the *_TARGETS use_expand flags. This is the behaviour of previous versions. This is not so much an issue with dev-lang/python, as it will install an otherwise unused slot. With dev-lang/ruby, however, portage will not proceed until the user sets RUBY_TARGETS: ruby26 and deviates from profile defaults. Reproducible: Always