Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 560466 - Behaviour of asterisk with = dependency operator is ill-defined
Summary: Behaviour of asterisk with = dependency operator is ill-defined
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Hosted Projects
Classification: Unclassified
Component: PMS/EAPI (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: PMS/EAPI
URL: https://projects.gentoo.org/pms/5/pms...
Whiteboard:
Keywords:
: 564490 (view as bug list)
Depends on: 560484 560486
Blocks:
  Show dependency tree
 
Reported: 2015-09-14 15:41 UTC by Ulrich Müller
Modified: 2019-03-04 08:09 UTC (History)
9 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Proposed patch for PMS master (0001-The-dependency-operator-can-only-match-complete-vers.patch,2.41 KB, patch)
2015-09-22 18:32 UTC, Ulrich Müller
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ulrich Müller gentoo-dev 2015-09-14 15:41:28 UTC
Section 8.2.6.1 "Operators" (see URL) says about the = operator:

   Exactly equal to the specified version. Special exception: if the
   version specified has an asterisk immediately following it, a string
   prefix comparison is used instead. When an asterisk is used, the
   specification must remain valid if the asterisk were removed.

This is ill-defined:
- cat/foo-1.2* matches cat/foo-1.20, which doesn't follow the principle
  of least surprise (and is not mentioned by the examples given in the
  devmanual).
- According to section 3.3 "Version Comparison", versions 1.020.3 and
  1.02.3 are considered as equal, so they should behave identical in
  dependency specifications. However, =cat/foo-1.020.3* will match the
  former (due to their equal string prefix) but not the latter.
- Neither the Portage nor the Paludis implementation follows the spec,
  see examples below (most of them taken from the discussion in
  gentoo-dev, http://thread.gmane.org/gmane.linux.gentoo.devel/97362).

  dependency spec         matches package          PMS?   Portage?   Paludis?
  ---------------------------------------------------------------------------
  =dev-lang/python-3*     dev-lang/python-3.4.3    yes    yes        yes
  =dev-lang/perl-5.2*     dev-lang/perl-5.22.0     yes    yes        no
  =app-editors/emacs-2*   app-editors/emacs-24.5   yes    yes        no
  =cat/foo-1.020.3*       cat/foo-1.02.3           no     no         yes
  =cat/foo-01.02.3*       cat/foo-1.02.3           no     yes        yes

IMHO we should reword the spec along the lines of:

   [...] if the version specified has an asterisk immediately following it,
   only the specified number of version components is used for comparison,
   i.e. the asterisk acts as a wildcard for any further components. [...]

For the examples given above this would coincide with Paludis's behaviour.
Comment 1 Ciaran McCreesh 2015-09-14 16:31:45 UTC
I'm pretty sure I remember this behaviour changing at some point in the past, to confuse things further...
Comment 2 Ciaran McCreesh 2015-09-14 16:39:56 UTC
Also, Paludis has two behaviours:

    key vo_nice_equal_star  "=* dependency (sensible EAPIs)"
    key vo_stupid_equal_star "=* dependency (Gentooish EAPIs)"

So if you're testing it from the command line, you get one behaviour, but from places that have an EAPI, you get different behaviour.
Comment 3 Ulrich Müller gentoo-dev 2015-09-14 18:21:42 UTC
So, how should we proceed? Fix it in a future EAPI?

Alternatively, we could say that this is a bug and fix it retroactively. In any case, we have to fix either Portage or PMS for existing EAPIs since they don't agree. So we could as well fix both.
Comment 4 Ciaran McCreesh 2015-09-14 18:27:35 UTC
Nuke =* in future EAPIs, because it's silly and horrible.
Comment 5 Zac Medico gentoo-dev 2015-09-14 18:56:35 UTC
(In reply to Ulrich Müller from comment #3)
> Alternatively, we could say that this is a bug and fix it retroactively. In
> any case, we have to fix either Portage or PMS for existing EAPIs since they
> don't agree. So we could as well fix both.

The Paludis behavior shown in comment #0 seems likely to be compatible with the intentions of the atoms as they are used in existing EAPIs, so a retroactive change is likely to be feasible.

(In reply to Ciaran McCreesh from comment #4)
> Nuke =* in future EAPIs, because it's silly and horrible.

That won't address the underlying reasons why people choose =* atoms.
Comment 6 Julian Ospald 2015-09-14 19:00:18 UTC
(In reply to Zac Medico from comment #5)
> (In reply to Ciaran McCreesh from comment #4)
> > Nuke =* in future EAPIs, because it's silly and horrible.
> 
> That won't address the underlying reasons why people choose =* atoms.

It looks like it's mostly a workaround for missing proper SLOTs or just to save a single line of code.
Comment 7 Ulrich Müller gentoo-dev 2015-09-14 19:06:25 UTC
(In reply to Ciaran McCreesh from comment #2)
>     key vo_nice_equal_star  "=* dependency (sensible EAPIs)"

Does that "nice equal star" behaviour agree with this:

(In reply to Ulrich Müller from comment #0)
>    [...] if the version specified has an asterisk immediately following it,
>    only the specified number of version components is used for comparison,
>    i.e. the asterisk acts as a wildcard for any further components. [...]
Comment 8 Ulrich Müller gentoo-dev 2015-09-16 20:36:40 UTC
> dependency spec         matches package          PMS?   Portage?   Paludis?
> ---------------------------------------------------------------------------
> =dev-lang/python-3*     dev-lang/python-3.4.3    yes    yes        yes
> =dev-lang/perl-5.2*     dev-lang/perl-5.22.0     yes    yes        no
> =app-editors/emacs-2*   app-editors/emacs-24.5   yes    yes        no
> =cat/foo-1.020.3*       cat/foo-1.02.3           no     no         yes
> =cat/foo-01.02.3*       cat/foo-1.02.3           no     yes        yes

More examples:

  =cat/foo-00*            cat/foo-001              yes    no         ?
  =cat/foo-1_p*           cat/foo-1_pre2           ?      yes        ?

So there are cases where PMS says that a dependency should match but where it doesn't match in Portage, and vice versa. Also I'm not sure about how to read PMS for the last one, but certainly _p* matching _pre is nonsensical. (Interestingly, [1] says that * deps should only be used with integer versions...)

Since dependencies can also be specified from the command line, I would feel uneasy about introducing EAPI dependent behaviour here. But let's ask the council for some guidance; this could be an agenda item for the October meeting.

[1] https://archives.gentoo.org/gentoo-dev/message/c6ff4820b5bc86d48e0a2b6700df3d3e
Comment 9 Brian Dolbec (RETIRED) gentoo-dev 2015-09-22 01:45:45 UTC
Released in portage-2.2.21 with enforcing * to be at version separator boundaries.
Comment 10 Rick Farina (Zero_Chaos) gentoo-dev 2015-09-22 14:22:18 UTC
How on earth did this go from "Let's ask the council" to "Released"?

Pretty nasty thing to just spring on people...  This significantly breaks my profiles because we like to keep the * as far into the version as possible to avoid unmasking too much.
Comment 11 Brian Dolbec (RETIRED) gentoo-dev 2015-09-22 15:30:43 UTC
The change we released was a minimal amount required to fix the broken behavior that clearly was not intended (a bug) and certainly caused it to incorrectly match in certain cases.  In no way did this specific change require "council" to intervene.

If the spec is to be tightened further restricting the use of *, in EAPI's or future EAPI's, then yes, it would not just be released without warning or approval.

It is also my understanding that there are other ways of getting the same results as you were previously getting exploiting the buggy behaviour.
Comment 12 Rick Farina (Zero_Chaos) gentoo-dev 2015-09-22 17:12:43 UTC
in all cases that I am familiar with, 1* would match both 1.0 and 10.  This is extremely common and expected behavior.

Now portage is magically adding extra information when interpreting a * which leads to significant departure to norms.  1* now literally mean 1.*, portage is forcibly adding a version separator before the *.  If I wanted to add a version separator, I would.

Anyone else fro, QA want to mask this for being broken?
Comment 13 Zac Medico gentoo-dev 2015-09-22 17:17:16 UTC
(In reply to Rick Farina (Zero_Chaos) from comment #12)
> Now portage is magically adding extra information when interpreting a *
> which leads to significant departure to norms.  1* now literally mean 1.*,
> portage is forcibly adding a version separator before the *.  If I wanted to
> add a version separator, I would.

You can't add a version separator, since the spec says that the part before the glob has to be able to stand alone as a valid version.
Comment 14 Alexander Berntsen (RETIRED) gentoo-dev 2015-09-22 17:18:40 UTC
(In reply to Rick Farina (Zero_Chaos) from comment #12)
> Anyone else fro, QA want to mask this for being broken?
If you do that I can assure you that Portage development activity will plunge even further.

It seems that every other time we fix any bug (and let's get this very clear: this *is* a bug fix) we get a bunch of people winging about "but mah scripts!!!", and oftentimes calling us idiots as well. That's one thing. QA starting to actively block progress is at least one step too far.
Comment 15 Ulrich Müller gentoo-dev 2015-09-22 18:32:05 UTC
Created attachment 412594 [details, diff]
Proposed patch for PMS master

Discussion in #gentoo-council on 2015-09-16 (slightly edited):

<ulm> I could need some guidance how to proceed with bug 560466
<ulm> since the council will ultimately decide on this
<ulm> basically, do you think that this is something that could be fixed
      retroactively?
<ulm> or should we go for eapi 6 or 7?
<dilfridge> 6
<ulm> one argument for a retroactive change is that portage doesn't implement
      what pms says
<dilfridge> well, the question boils down to "do we want to keep compatibility  
            with things that are not in the tree anymore"
<dilfridge> yes -> do it in EAPI=6, no -> fix the tree, do it retroactively
<ulm> right
<jlec> I would consider the a state of the tree is bound to the PM at that
       date
<jlec> so we can change things now, even if we break history
<dilfridge> I'm kinda fine with fixing the tree and doing it retroactively,
            but on the other hand when I see people complaining that Perl team
            increases minimum version to 5.12 without big announcement
            first...
* dilfridge bangs head against imagined wall
<ulm> yeah, fixing it in a new eapi is cleaner, but also means more work for
      everyone
<ulm> and then I'm not sure what to do about the disagreement between PMS and
      portage for existing EAPIs
<ulm> it's clearly a boundary case
[... unrelated discussion deleted ...]
<dilfridge> ulm: let's be bold, fix it retroactively and send both portage
            team and paludis team(?) a nice mail asking them to consider
            "fixing" their code
<dilfridge> alternatively, propose some quick council vote on the bug (with
            well-formulated alternatives)
[...]
<ulm> dilfridge: it's not such urgent, but I'll prepare a patch and put it on
      the agenda for the october meeting
Comment 16 Rick Farina (Zero_Chaos) gentoo-dev 2015-09-22 19:11:35 UTC
based on my own check of the current tree profile directory, I am not in favor of a mask at this time.  results from autorepoman may change that opinion at any time.
Comment 17 email200202 2015-09-23 04:02:11 UTC
The new behavior in sys-apps/portage-2.2.21 brook media-video/nvidia-settings-343.22 ebuild. See https://forums.gentoo.org/viewtopic-p-7818522.html#7818522
Comment 18 Zac Medico gentoo-dev 2015-09-23 04:26:18 UTC
(In reply to email200202 from comment #17)
> The new behavior in sys-apps/portage-2.2.21 brook
> media-video/nvidia-settings-343.22 ebuild. See
> https://forums.gentoo.org/viewtopic-p-7818522.html#7818522

That's bug 560486, which is fixed in git:

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=23d792e58e7dc709dc20b68ea4f93883fe483c35
Comment 19 Zac Medico gentoo-dev 2015-09-23 17:46:53 UTC
(In reply to Julian Ospald (hasufell) from comment #6)
> (In reply to Zac Medico from comment #5)
> > (In reply to Ciaran McCreesh from comment #4)
> > > Nuke =* in future EAPIs, because it's silly and horrible.
> > 
> > That won't address the underlying reasons why people choose =* atoms.
> 
> It looks like it's mostly a workaround for missing proper SLOTs or just to
> save a single line of code.

I typically use the =* operator to accept unstable packages of a particular version series in /etc/portage/package.accept_keywords. If I can't continue to do this with a single atom, it seems like it will be a big step backwards. Since users have no control over they subslot settings in ebuilds, the =* operator is an extremely useful way to select a particular version series.
Comment 20 Zac Medico gentoo-dev 2015-09-23 18:35:48 UTC
(In reply to Zac Medico from comment #19)
We also use the =* operator like this in profiles at my workplace (and the atoms need to be valid for the EAPI of the profiles). So, it the =* operator is removed in a future EAPI, then we'll have to come up with some other solution if we want to use the new EAPI in our profiles.
Comment 21 Ulrich Müller gentoo-dev 2015-10-11 19:15:08 UTC
Fixed retroactively in all EAPIs, per today's council decision.

https://gitweb.gentoo.org/proj/pms.git/commit/?id=cb99e4dcb5837626320b1fba0277d0fa7c1c9829
Comment 22 Zac Medico gentoo-dev 2015-10-30 16:02:19 UTC
*** Bug 564490 has been marked as a duplicate of this bug. ***
Comment 23 Ulrich Müller gentoo-dev 2019-03-04 08:09:00 UTC
*** Bug 564490 has been marked as a duplicate of this bug. ***