Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 322049 - use_with 3 arg specification differs in portage for $3=''
Summary: use_with 3 arg specification differs in portage for $3=''
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Hosted Projects
Classification: Unclassified
Component: PMS/EAPI (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: PMS/EAPI
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 273620
  Show dependency tree
 
Reported: 2010-05-30 04:06 UTC by Brian Harring (RETIRED)
Modified: 2011-01-18 11:01 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
quick scan of -x86 looking for potentially affected bits (potentials,56.86 KB, text/plain)
2010-05-30 04:07 UTC, Brian Harring (RETIRED)
Details
Patch based on portage behaviour for EAPI 0-3 (0001-Fix-use_with-3-arg-specification-bug-322049.patch,3.74 KB, patch)
2011-01-02 16:56 UTC, Ulrich Müller
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Brian Harring (RETIRED) gentoo-dev 2010-05-30 04:06:17 UTC
Quoting the spec:

\item[use\_with] Has one-, two-, and three-argument forms. The first argument is a USE flag name, the second a \t{configure} option name (\t{\$\{opt\}}), defaulting to the same as the first argument if not provided, and the third is a string value (\t{\$\{value\}}), defaulting to nothing. If the USE flag is set, outputs \t{-{}-with-\$\{opt\}=\$\{value\}} if the third argument was provided, and \t{-{}-with-\$\{opt\}} otherwise. If the flag is not set, then it outputs \t{-{}-without-\$\{opt\}}.

\item[use\_enable] Works the same as \t{use\_with()}, but outputs \t{-{}-enable-} or \t{-{}-disable-}
instead of \t{-{}-with-} or \t{-{}-without-}. 

then try `use_with flag monkey balls ''`

Specifically PMS's documentation of use_with/use_enable for EAPI0 (which was supposed to match portage behaviour) overlooked this- thus pkgcore/portage both treat $3='' as if there was no third arg.  Paludis written to the spec, I'm told follows the spec.

So, got a conundrum.  3 EAPI's in; my thought is since this can be worked around at the ebuild level, retroactively fix PMS EAPI0-3 to match existing behaviour, mark 4 w/ the desired behaviour.

As of 05/28, arfrever jammed in a fix to portage for this; as indicated above, this isn't particularly sane w/out discussion nor fallout inspection, and should likely be punted till the PMS issue is  resolved in full.
Comment 1 Brian Harring (RETIRED) gentoo-dev 2010-05-30 04:07:08 UTC
Created attachment 233475 [details]
quick scan of -x86 looking for potentially affected bits

Note that this is just -x86; to figure out the full impact we'd need to scan damn near every overlay out there.
Comment 2 Brian Harring (RETIRED) gentoo-dev 2010-05-30 04:08:32 UTC
Sorry, for the example I meant `use_with monkey balls ''`, or `use_enable monkey balls ''`, rather than a non-existant 4 arg form.
Comment 3 Ciaran McCreesh 2010-05-30 07:24:19 UTC
Could just mark it as undefined behaviour. Is there a use for passing an empty third arg?
Comment 4 Brian Harring (RETIRED) gentoo-dev 2010-05-30 08:51:02 UTC
(In reply to comment #3)
> Could just mark it as undefined behaviour. Is there a use for passing an empty
> third arg?
Only usage I'm aware of is dev-libs/xmlsec, and that was introduced by arfrever.  As said, scan the potentials list- that should spot any other -x86 consumers (mozconfig eclass's $@ usage struck me as a potential).

As for undefined... undefined I hate with a passion.  The majority behaviour (pkgcore, but more importantly portage due to time and userbase) is defined, PMS (by extension paludis, a minority the same as pkgcore) just didn't match.

From the standpoint of ebuild devs being able to know what actually occurs, and to avoid people pulling stupid shit like trying to dep on a manager, PMS retroactively for EAPI0-3 should be defined to match the long term behaviour, and EAPI4 fixed to match the PMS specified behaviour.
Comment 5 Brian Harring (RETIRED) gentoo-dev 2010-05-30 08:56:12 UTC
Note also I'm aware that the behaviour will differ with paludis for this prior to any fix, thus could be considered undefined; that said I'm personally of the view that via paludis objectively being a minority share and subjectively that it's users likely keep fairly up to date (versus people trailing current stable for portage by fricking years), undefined isn't the correct solution here.

Further kicker for it, via stating that `use_with monkey balls ''` is equivalent to `use_with monkey balls`, it forces the ebuild dev to do the '' bit themselves- thus always getting the desired result instead of leaving it ambiguous via 'undefined'.
Comment 6 Ulrich Müller gentoo-dev 2010-05-30 10:09:28 UTC
(In reply to comment #1)
> Created an attachment (id=233475) [details]
> quick scan of -x86 looking for potentially affected bits

Looks like your list contains mostly false positives. Could you clean it so that we see where there can be an explicit null string as third argument?
Comment 7 Brian Harring (RETIRED) gentoo-dev 2010-05-30 12:15:37 UTC
(In reply to comment #6)
> (In reply to comment #1)
> > Created an attachment (id=233475) [details] [details]
> > quick scan of -x86 looking for potentially affected bits
> 
> Looks like your list contains mostly false positives. Could you clean it so
> that we see where there can be an explicit null string as third argument?

varying IFS, variable interpolation, and function wrapping mean that this list is about as close as we're going to get without inspecting each/every use_with invocation.  So... yeah, sorry, high false positive in this list but it needs to be checked (or someone needs to invest more time in the potentials identification than I was willing).

Frankly if people really want to push undefined (or just adhoc change portage visible behaviour as arfrever pulled), if it were me I'd be scanning each use_with.  That said I'm pushing the "retroactive PMS tweak" so someone else gets to do that work ;)
Comment 8 Zac Medico gentoo-dev 2010-05-31 06:37:36 UTC
I've reverted the portage behavior for EAPI 0, 1, 2, and 3 here:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=415d33de1692a65c60730da961ad3d787346b375
Comment 9 Ulrich Müller gentoo-dev 2010-05-31 07:27:38 UTC
I see three options to fix it:

  1. Mark it as undefined behaviour in EAPIs 0 to 3, as suggested in comment #3.
     Pro: Portage and Paludis disagree, so ebuilds cannot rely on it.
     Contra: "Undefined" doesn't make a good spec.

  2. Declare previous Portage and Pkgcore behaviour as standard for EAPI 0 to 3.
     Pro: Avoids the "undefined" from 1.
     Contra: May break things for Paludis users. Also behaviour differs
             between Portage versions, e.g. 2.1.8.3 follows the current spec.

  3. Keep the spec as it is and fix Portage retroactively, without EAPI bump.
     Pro: Avoids another table and case distinction in PMS.
     Contra: Ebuilds cannot rely on it for current EAPIs.

I also went though the list of potentially affected ebuilds, and as far as I can see, there is no single ebuild in the tree where it would matter. On first glance, dev-libs/xmlsec seems to be a candidate, but looking into the ebuild shows that it doesn't use the package manager function, but has its private implementation of use_with().

So can't we simply declare it as a Portage bug and leave the spec alone? Previous Portage behaviour doesn't make much sense (I hope we all agree on that), therefore I wouldn't like option 2.
Comment 10 Zac Medico gentoo-dev 2010-12-31 19:36:18 UTC
(In reply to comment #8)
> I've reverted the portage behavior for EAPI 0, 1, 2, and 3 here:
> 
> http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=415d33de1692a65c60730da961ad3d787346b375

This code is active now in portage-2.1.9.27 with EAPI 4.
Comment 11 Brian Harring (RETIRED) gentoo-dev 2011-01-01 03:46:37 UTC
(In reply to comment #10)
> (In reply to comment #8)
> > I've reverted the portage behavior for EAPI 0, 1, 2, and 3 here:
> > 
> > http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=415d33de1692a65c60730da961ad3d787346b375
> 
> This code is active now in portage-2.1.9.27 with EAPI 4.

And that's wrong; it's an incompatibility to the spec again.  With respect, if you want something to change in PMS you change it *prior* to the spec landing- not 4 days after it's been approved.

@council, thoughts on a retag?  Eapi-4 is already in the tree...
Comment 12 Petteri Räty (RETIRED) gentoo-dev 2011-01-01 10:25:51 UTC
(In reply to comment #11)
>
> And that's wrong; it's an incompatibility to the spec again.  With respect, if
> you want something to change in PMS you change it *prior* to the spec landing-
> not 4 days after it's been approved.
> 
> @council, thoughts on a retag?  Eapi-4 is already in the tree...
> 

Why can't Portage be changed to follow the spec? EAPI 4 is not yet approved for ebuild usage.
Comment 13 Ulrich Müller gentoo-dev 2011-01-01 10:33:57 UTC
(In reply to comment #11)
> And that's wrong; it's an incompatibility to the spec again.

Right. The spec says "outputs --with-${opt}=${value} if the third argument was provided, and --with-${opt} otherwise."

So, either
1. the spec should be changed to say "a non-empty third argument", or
2. portage should omit the colon in parameter expansion, i.e. use
   UW_SUFFIX=${3+=$3} .

> @council, thoughts on a retag?

Pick your choice.
Comment 14 Brian Harring (RETIRED) gentoo-dev 2011-01-01 10:42:39 UTC
(In reply to comment #13)
> (In reply to comment #11)
> > And that's wrong; it's an incompatibility to the spec again.
> 
> Right. The spec says "outputs --with-${opt}=${value} if the third argument was
> provided, and --with-${opt} otherwise."

My personal preference is for this to stay, and it be noted that in EAPI4 it can be relied upon- rather than the current undefined that's been in place, and silent change of behaviour that will occur with portage.

> So, either
> 1. the spec should be changed to say "a non-empty third argument", or
> 2. portage should omit the colon in parameter expansion, i.e. use
>    UW_SUFFIX=${3+=$3} .

#2, that's what portage uses for EAPI 0-3, for 4 it's :+= .

My personal views are that if a third arg is given, it should be used- abide the spec namely.  The fact this issue has been left sitting, and portage is swivelling on it complicates that.

So... ought to resolve it before #4 goes widespead.
Comment 15 Ulrich Müller gentoo-dev 2011-01-01 11:09:45 UTC
(In reply to comment #14)
> #2, that's what portage uses for EAPI 0-3, for 4 it's :+= .

Unless I completely misunderstand the code, it's the other way around, i.e. :+= for 0-3 and += for 4:

   if ! has "${EAPI:-0}" 0 1 2 3 ; then
       local UW_SUFFIX=${3+=$3}
   else
       local UW_SUFFIX=${3:+=$3}
   fi
Comment 16 Brian Harring (RETIRED) gentoo-dev 2011-01-01 11:19:37 UTC
(In reply to comment #15)
> (In reply to comment #14)
> > #2, that's what portage uses for EAPI 0-3, for 4 it's :+= .
> 
> Unless I completely misunderstand the code, it's the other way around, i.e. :+=

Pardon, yeah I inverted it in replying :/
Comment 17 Petteri Räty (RETIRED) gentoo-dev 2011-01-01 15:33:59 UTC
(In reply to comment #14)
> 
> My personal views are that if a third arg is given, it should be used- abide
> the spec namely.  The fact this issue has been left sitting, and portage is
> swivelling on it complicates that.
> 

Assigning to Portage people as we don't consider this a bug in the spec.
Comment 18 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2011-01-01 16:05:44 UTC
Portage already respects PMS in EAPI >=4. Should Portage respect PMS also in EAPI <=3?
Comment 19 Brian Harring (RETIRED) gentoo-dev 2011-01-01 16:16:51 UTC
(In reply to comment #18)
> Portage already respects PMS in EAPI >=4. Should Portage respect PMS also in
> EAPI <=3?

Read the comments above; the problem is that portage's long term implementation differs from the spec... meaning the spec wasn't reliable, and effectively this is an unannounced change in EAPI4 for most users (since portage was using it's old behaviour).
Comment 20 Ulrich Müller gentoo-dev 2011-01-01 16:35:03 UTC
As there is no ebuild in the tree where it would make a difference (at least there wasn't in May 2010, see my comment #9), I would say that Portage should be changed to agree with PMS also for EAPIs 0 to 3.

BTW, both the devmanual and ebuild(5) seem to agree with PMS here:

Devmanual:
| If val is not specified, omits the assignment part.

ebuild(5):
| If USE item is in the USE variable and a configure opt is specified,
| then the string --with-[configure name]=[configure opt] will be echoed.
| If configure opt is not specified, then just --with-[configure name]
| will be echoed.
Comment 21 Ulrich Müller gentoo-dev 2011-01-02 16:56:42 UTC
Created attachment 258653 [details, diff]
Patch based on portage behaviour for EAPI 0-3

As far as I've followed the discussion in #-council, at least three council members seem to prefer the "make portage behaviour standard" option. Brian has provided a wording; attached patch is based on this.
Comment 23 Ulrich Müller gentoo-dev 2011-01-11 21:00:38 UTC
(In reply to comment #21)
> Created an attachment (id=258653) [details]
> Patch based on portage behaviour for EAPI 0-3

Pushed to master, following today's council decision.
Comment 24 Ulrich Müller gentoo-dev 2011-01-12 13:54:40 UTC
(In reply to comment #17)
> Assigning to Portage people as we don't consider this a bug in the spec.

Reassigning to pms-bugs as the spec has been changed to follow portage.