Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 373349

Summary: [Future EAPI] Drop the PORTDIR variable
Product: Gentoo Hosted Projects Reporter: Michał Górny <mgorny>
Component: PMS/EAPIAssignee: PMS/EAPI <pms>
Status: RESOLVED FIXED    
Severity: enhancement CC: alexanderyt, esigra, robbat2, tsmksubc
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://bugs.gentoo.org/show_bug.cgi?id=373351
Whiteboard: in-eapi-7
Package list:
Runtime testing required: ---
Bug Depends on: 341653, 373353, 416731, 416733, 416735, 416737, 416739, 416741, 566402, 566412, 566416, 566424, 629764    
Bug Blocks: 174380, 417013    

Description Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2011-06-28 14:39:55 UTC
Right now, PMS defined the PORTDIR variable as:

	The full path to the master repository's base directory.

That doesn't seem actually useful to the ebuilds and is in an unclear relation with overlays.

AFAICS it is used rarely:
1) by eutils.eclass and a few ebuilds to refer to the licenses subdirectory,
2) by games-mods.eclass to grab a fresh Gentoo Foundation header from header.txt,
3) by baselayout ebuild to create /usr/portage,
4) by gentoo-syntax to hardcore PORTDIR path in a site file.

Thus, I'd go with deprecating it and removing in a future EAPI.

The first use could be replaced by a future EAPI providing a clean method of accessing licenses for a given package (alike what ${DISTDIR} and ${FILESDIR} do, overlay-safe).

The second use can be fixed in many ways -- through grabbing that header from whatever other file (the ebuild itself?) or generating it.

The baselayout ebuild shouldn't really rely on PM runtime variables, I think, and just use /usr/portage.

And the gentoo-syntax-provided code should be able to determine the portdir location in runtime.
Comment 1 Jeremy Olexa (darkside) (RETIRED) archtester gentoo-dev Security 2011-06-28 14:43:42 UTC
What do you propose that users do for a custom PORTDIR location? Meaning, I want portage to know that I keep ebuilds in /mnt/nfs-portdir or so instead of /usr/portage/
Comment 2 Ulrich Müller gentoo-dev 2011-06-28 14:50:11 UTC
(In reply to comment #0)
> ... determine the portdir location in runtime.

How?
Comment 3 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2011-06-28 15:04:41 UTC
The only really correct and reliable way is to ask the user, either through his/her PM or directly. Even though the ${PORTDIR} solution works in 99% of cases, it creates a broken package which will start to fail randomly and unexpectedly for users who move their PORTDIR. It results in binpkgs tied to a specific host as well.

The simplest solution is to:
1) let user override that dir,
2) default to portageq when available,
3) fallback to /usr/portage when not.
Comment 4 SpanKY gentoo-dev 2011-06-28 19:06:17 UTC
1) this code is dead and going away, so is irrelevant
2) this is only used at src_* time, has nothing to do with pkg_*, and doesnt break anything if it's not found (it's merely a comment)
3) only baselayout-1 which is already being stabilized away
4) file a bug for the gentoo-syntax people to either do it at runtime, or do it in pkg_preinst

so if those are your only examples, sounds like PMS doesnt need changing
Comment 5 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2011-07-01 15:07:06 UTC
About determining the PORTDIR location at runtime: I have started working on the gentoopm project [1] which will provide an unified Python/shell API to our three PMs. Right now, it's in early development but is already able to iterate over repositories and get names/paths for them using portage & pkgcore. I'd appreciate any comments about it, and especially the 'gentoopmq' syntax as I'm not convinced to the one I'm using now.
Comment 6 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2011-07-01 15:08:40 UTC
[1]:https://github.com/mgorny/gentoopm
Comment 7 SpanKY gentoo-dev 2011-07-05 00:19:49 UTC
sounds a little like overkill and unlikely to get used by projects not already written in python
Comment 8 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2011-07-05 05:27:02 UTC
It has a shell, portageq-like tool.
Comment 9 SpanKY gentoo-dev 2011-07-05 05:32:55 UTC
... which is a frontend to python
Comment 10 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2011-07-05 05:36:32 UTC
(In reply to comment #9)
> ... which is a frontend to python

Like portageq does. And p* are. You've decided to use Paludis?
Comment 11 SpanKY gentoo-dev 2011-07-08 21:07:34 UTC
those arent the only ways to query information

i still dont see an issue that requires changing the EAPI
Comment 12 Brian Harring (RETIRED) gentoo-dev 2012-05-19 21:23:07 UTC
The issue is that portdir needs to go; conceptually it just doesn't fit with multiple repos (masters and such).

I'm *really* not a fan of ebuilds reaching into another repository (since that puts implicit limits on the repository format), but a "give me the pathway to this repo" is good enough via portageq; the ebuild ought to know the masters it has, so explicit PORTDIR shouldn't be needed anymore.

Personally, I'd like this var gone in EAPI5.  counter arguments?
Comment 13 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-05-20 06:59:56 UTC
(In reply to comment #12)
> The issue is that portdir needs to go; conceptually it just doesn't fit with
> multiple repos (masters and such).
> 
> I'm *really* not a fan of ebuilds reaching into another repository (since
> that puts implicit limits on the repository format), but a "give me the
> pathway to this repo" is good enough via portageq; the ebuild ought to know
> the masters it has, so explicit PORTDIR shouldn't be needed anymore.
> 
> Personally, I'd like this var gone in EAPI5.  counter arguments?

Want a current list of packages which would need to be fixed on bump?
Comment 14 Ulrich Müller gentoo-dev 2012-05-20 07:14:32 UTC
I still don't understand why you want to remove this variable. What is wrong with the package manager providing it?
Comment 15 Ciaran McCreesh 2012-05-20 07:21:50 UTC
If the package manager provides it, ebuilds might use it. We don't want that.
Comment 16 Ulrich Müller gentoo-dev 2012-05-20 07:45:55 UTC
And what is wrong with ebuilds using it?
Comment 17 Ciaran McCreesh 2012-05-20 07:49:03 UTC
They shouldn't be, since they've no business sniffing around inside whatever PORTDIR happens to be. There's no legitimate reason for them to have it.
Comment 18 Brian Harring (RETIRED) gentoo-dev 2012-05-20 11:45:01 UTC
(In reply to comment #13)
> (In reply to comment #12)
> > The issue is that portdir needs to go; conceptually it just doesn't fit with
> > multiple repos (masters and such).
> > 
> > I'm *really* not a fan of ebuilds reaching into another repository (since
> > that puts implicit limits on the repository format), but a "give me the
> > pathway to this repo" is good enough via portageq; the ebuild ought to know
> > the masters it has, so explicit PORTDIR shouldn't be needed anymore.
> > 
> > Personally, I'd like this var gone in EAPI5.  counter arguments?
> 
> Want a current list of packages which would need to be fixed on bump?

Not particularly, no; if they need it in EAPI5 they can be explicit and use `portageq get_repo_path / gentoo`


(In reply to comment #16)
> And what is wrong with ebuilds using it?
See my comment above, specifically the 'explicit' part.

Further, it's implicitly overlay/single repo, and dodgy to boot; it basically assumes that PORTDIR is always the 'gentoo' repo.  Invert it, watch shit break, then tell me it's a good design. :)
Comment 19 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-05-20 12:08:34 UTC
I think we should start with deprecating the variable and fix the ebuilds first.
Comment 20 Brian Harring (RETIRED) gentoo-dev 2012-05-20 12:26:59 UTC
(In reply to comment #19)
> I think we should start with deprecating the variable and fix the ebuilds
> first.

If you can explain to me the benefit in 'deprecating' it (IE, letting it sit and cause issues until EAPI6), specifically clarifying why the explicit route I mentioned (which works, and can't misbehave if someone has a portdir as something other than the 'gentoo' repo), sure.

If you can't, then it should go.  Same for ECLASSDIR.

There's no point in letting it rot further; it's still around purely because deprecation == "fucker lives on since people forget to remove it".
Comment 21 Ciaran McCreesh 2012-05-20 12:34:31 UTC
We have extensive procedures in place for phasing out deprecated stuff! Specifically, I grep PMS for 'deprecated' every time we start discussing a new EAPI and suggest anything that matches that I care about.

Having said that, there's no point deprecating things we can kill off immediately.
Comment 22 Brian Harring (RETIRED) gentoo-dev 2012-05-20 12:52:27 UTC
(In reply to comment #21)
> We have extensive procedures in place for phasing out deprecated stuff!
> Specifically, I grep PMS for 'deprecated' every time we start discussing a
> new EAPI and suggest anything that matches that I care about.

Alternative phrasing; this should've been deprecated basically since news/masters showed up (eapi3 for masters, earlier for news), but wasn't marked as such in the source (instead we just had this ticket sitting for the last year), thus sat/rot.

> Having said that, there's no point deprecating things we can kill off
> immediately.

Agreed.
Comment 23 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-05-20 13:26:44 UTC
I'm not saying 'wait with it for another EAPI'. I'm just saying that if we want to actually remove it, we have to consider all uses and provides fixes first. Otherwise, one of the two will happen:

1) people will invent even worse hacks (like messing around with $BASH_SOURCE),
2) there will be ebuilds/eclasses which will never move to a newer EAPI because of missing feature.

And a quick grep suggests that there are more uses of $PORTDIR than when I suggested removing this...
Comment 24 Ciaran McCreesh 2012-05-20 13:31:34 UTC
Good thing there's a QA team to prevent that.
Comment 25 SpanKY gentoo-dev 2012-05-20 15:03:32 UTC
too bad you aren't providing sufficient (or any) replacement functionality.  you can't drop things from the spec that are in active use with "too bad".
Comment 26 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-05-20 15:14:20 UTC
(In reply to comment #25)
> too bad you aren't providing sufficient (or any) replacement functionality. 
> you can't drop things from the spec that are in active use with "too bad".

In your cases, $ECLASSDIR will probably work better. Of course, unless we want to remove that as well...
Comment 27 SpanKY gentoo-dev 2012-05-20 18:49:39 UTC
except ECLASSDIR isn't in PMS.  i should fix libtool.eclass to fallback to PORTDIR when ECLASSDIR doesn't pan out.
Comment 28 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-05-20 18:52:32 UTC
(In reply to comment #27)
> except ECLASSDIR isn't in PMS.  i should fix libtool.eclass to fallback to
> PORTDIR when ECLASSDIR doesn't pan out.

It is. Two rows below PORTDIR...
Comment 29 Brian Harring (RETIRED) gentoo-dev 2012-05-20 21:10:57 UTC
(In reply to comment #25)
> too bad you aren't providing sufficient (or any) replacement functionality. 
> you can't drop things from the spec that are in active use with "too bad".

As I said; portageq get_repo_path / gentoo .  Already required for eselect-news support, and supported in some form by all (nailing it down to "available w/in the ebuild env for all" is perfectly sane).

Note that functionality actually *works* for multi master repo's, configurations that don't have the target repo as PORTDIR, etc.  It's actually a *better* api.
Comment 30 Zac Medico gentoo-dev 2012-05-20 21:15:49 UTC
(In reply to comment #29)
> As I said; portageq get_repo_path / gentoo .

Aren't portageq calls in the ebuild environment frowned upon? If so, then we should add a new function as part of the EAPI.
Comment 31 Arfrever Frehtes Taifersar Arahesis 2012-05-20 21:17:18 UTC
Maybe it would be better to provide MASTER_REPOSITORIES array in ebuild environment, so that `portageq get_repo_path / ${MASTER_REPOSITORIES[0]}` could be used.
Comment 32 Brian Harring (RETIRED) gentoo-dev 2012-05-20 21:36:45 UTC
(In reply to comment #30)
> (In reply to comment #29)
> > As I said; portageq get_repo_path / gentoo .
> 
> Aren't portageq calls in the ebuild environment frowned upon? If so, then we
> should add a new function as part of the EAPI.

Meant wrapping it with a function, but yes.

(In reply to comment #31)
> Maybe it would be better to provide MASTER_REPOSITORIES array in ebuild
> environment, so that `portageq get_repo_path / ${MASTER_REPOSITORIES[0]}`
> could be used.

If we were to do that, should just use an associate array in full rather than adding the function call annoyance (keep in mind any parsing issue we may have the council deemed we stop caring about via the EAPI parsing crap).
Comment 33 Arfrever Frehtes Taifersar Arahesis 2012-05-20 22:42:40 UTC
(In reply to comment #32)
> (In reply to comment #31)
> > Maybe it would be better to provide MASTER_REPOSITORIES array in ebuild
> > environment, so that `portageq get_repo_path / ${MASTER_REPOSITORIES[0]}`
> > could be used.
> 
> If we were to do that, should just use an associate array in full rather
> than adding the function call annoyance

It's a good idea, although associative arrays were introduced in bash 4.0.
Comment 34 Ulrich Müller gentoo-dev 2012-05-21 05:22:23 UTC
(In reply to comment #30)
> > As I said; portageq get_repo_path / gentoo .
> 
> Aren't portageq calls in the ebuild environment frowned upon? If so, then we
> should add a new function as part of the EAPI.

I'd say that we need both get_repos and get_repo_path functions then.

(In reply to comment #26)
> In your cases, $ECLASSDIR will probably work better. Of course, unless we
> want to remove that as well...

Right, keeping ECLASSDIR doesn't make sense if PORTDIR is dropped. So, "get_repo_path -e" for the eclass directory? (And -l for licenses, -p for profiles?)
Comment 35 SpanKY gentoo-dev 2012-05-21 18:07:06 UTC
(In reply to comment #28)

seems that okular fails to do text scanning on the the rotated variables section, so doing a search shows 0 hits

(In reply to comment #29)

so you're proposing replacing a setting that has been in the PMS since the beginning (and has seen very little abuse as documented in earlier comments) with a function that is not ?  and in fact is explicitly mentioned in the "Unspecified Items" section ?

if you're going to kill functionality, you must replace it with something that is specified by PMS.
Comment 36 Brian Harring (RETIRED) gentoo-dev 2012-05-21 18:57:37 UTC
(In reply to comment #35)
> (In reply to comment #28)
> 
> seems that okular fails to do text scanning on the the rotated variables
> section, so doing a search shows 0 hits
> 
> (In reply to comment #29)
> 
> so you're proposing replacing a setting that has been in the PMS since the
> beginning (and has seen very little abuse as documented in earlier comments)
> with a function that is not ?  and in fact is explicitly mentioned in the
> "Unspecified Items" section ?
> 
> if you're going to kill functionality, you must replace it with something
> that is specified by PMS.

Too literal; PORTDIR is fundamentally broken.  I want that var and ECLASSDIR gone (both of which are 'unspecified' in PMS) and replaced w/ a usable "find the root of a given repo" if it's required to be exposed to ebuilds.

I *suspect* if there wasn't stuff like eblits or ebuilds copying licenses out of license dir, this wouldn't be required, but that's a separate mess to cleanse.
Comment 37 Ulrich Müller gentoo-dev 2012-05-21 19:09:50 UTC
(In reply to comment #36)
> (both of which are 'unspecified' in PMS)

PMS specifies both PORTDIR and ECLASSDIR in section 11.1 "Defined Variables".
Comment 38 Ciaran McCreesh 2012-05-21 19:36:51 UTC
(In reply to comment #35)
> if you're going to kill functionality, you must replace it with something
> that is specified by PMS.

Really, the intent is to kill it and have things that rely upon it be redesigned and rewritten to not make all sorts of horrible assumptions about the layout of the repository. If you want access to files, you have FILESDIR and SRC_URI; we shouldn't have to be guaranteeing anything else about where things are.
Comment 39 SpanKY gentoo-dev 2012-05-21 21:54:35 UTC
(In reply to comment #38)

your suggestion is not sufficient for all cases -- there's no way libtool.eclass can utilize FILESDIR or SRC_URI or any package-specific variable/location.  only Brian's would work out, but that's not in PMS.  so if you don't care about that because you implement `portageq`, then i'll implement it.
Comment 40 Ciaran McCreesh 2012-05-22 06:28:34 UTC
Paludis doesn't implement portageq (since a while back someone went to great lengths to remove all portageq calls from ebuilds), so that isn't a solution. You need to rethink how you're distributing files, and SRC_URI is the way to go.
Comment 41 SpanKY gentoo-dev 2012-05-22 14:13:05 UTC
not a solution for libtool.eclass.  sounds like PORTDIR/ECLASSDIR is here to stay until something equivalent is provided.
Comment 42 Ciaran McCreesh 2012-05-22 16:48:59 UTC
SRC_URI *is* the alternative, and if you think there's a specific reason beyond just not wanting to change that you can't use it, you should explain what it is so we can get something in EAPI 5 that will help. We shouldn't be keeping legacy mess that gets in the way of providing things that users want just because you don't feel like redesigning some cruft.
Comment 43 SpanKY gentoo-dev 2012-05-22 17:09:07 UTC
try reading & understanding the actual code before suggesting something that fairly obviously won't scale let alone have any hope of working.  this eclass gets used by thousands of ebuilds (and that number changes frequently), and the patches it applies to fixes things up change, so SRC_URI isn't feasible.

i didn't say i'm against changing the code.  i said give me a *real* solution, not some barely half thought out idea.  especially considering the code in question people want to remove is hardly causing a problem.
Comment 44 Ciaran McCreesh 2012-05-22 17:13:45 UTC
You still haven't explained why SRC_URI "isn't feasible". Simply put the patches in a tarball, and bump the tarball name when you change the patches.
Comment 45 SpanKY gentoo-dev 2012-05-22 17:21:31 UTC
if you can't see how updating SRC_URI of thousands of ebuilds is unfeasible, then i guess there isn't much left to say
Comment 46 Ciaran McCreesh 2012-05-22 17:28:32 UTC
If only someone could invent a mechanism for something to be added metadata variable across all ebuilds using an eclass, without requiring modification to any of those ebuilds.
Comment 47 Brian Harring (RETIRED) gentoo-dev 2012-05-22 19:01:15 UTC
(In reply to comment #43)
> i didn't say i'm against changing the code.  i said give me a *real*
> solution, not some barely half thought out idea.  especially considering the
> code in question people want to remove is hardly causing a problem.

Thing I've not understood in looking at that code is why that was never folded into a pkg itself which libtool eclass then deps on.  Don't see any reason that can't work and avoids storing 500kb of random libtool monkey patches.

Regardless, pulling this discussion back to sanity; any complaints with adding a 'get_repository_path [repo_name]' function to EAPI5 that is fed the repo name to lookup; if fed a repo name that is *not* a master of that repo, it barfs.  If no repo name is given, it returns the root of the current repository.

In the same angle (since eclasses are likely to change at some point), a 'get_eclassdir_path [repo_name]' addition removes the need for ECLASSDIR.   Same logic for that one as get_repository_path.

Doing so means ebuilds can *correctly* find what they're after, rather than implicitly being forced to always have their PORTDIR be forced to gentoo if (for example) libtool.eclass is in use.
Comment 48 Ciaran McCreesh 2012-05-22 19:02:59 UTC
(In reply to comment #47)
> Regardless, pulling this discussion back to sanity; any complaints with
> adding a 'get_repository_path [repo_name]' function to EAPI5 that is fed the
> repo name to lookup; if fed a repo name that is *not* a master of that repo,
> it barfs.  If no repo name is given, it returns the root of the current
> repository.
> 
> In the same angle (since eclasses are likely to change at some point), a
> 'get_eclassdir_path [repo_name]' addition removes the need for ECLASSDIR.  
> Same logic for that one as get_repository_path.
> 
> Doing so means ebuilds can *correctly* find what they're after, rather than
> implicitly being forced to always have their PORTDIR be forced to gentoo if
> (for example) libtool.eclass is in use.

Yeah, I'm against it. They shouldn't be looking in the tree themselves at all, so we shouldn't be providing a way for them to do it.
Comment 49 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-05-22 19:25:22 UTC
Eclasses - acceptable. The whole repo - a bad idea.
Comment 50 Brian Harring (RETIRED) gentoo-dev 2012-05-22 19:34:43 UTC
(In reply to comment #48)
> Yeah, I'm against it. They shouldn't be looking in the tree themselves at
> all, so we shouldn't be providing a way for them to do it.

Right, the stick the head in the sand approach.  Pragmatic and useful.  End result is we just leave PORTDIR in place which is frankly worse, while screaming "You can't do that, it's not allowed by PMS!".  Doesn't get us very far if the option is either to be incompliant and working, or compliant but el busto'd.

Licenses are the primary offender, with one odd header.txt usage, and vapiers elt-patches bits.  If we go your route, we have to add a 'get_license' function in addition...

For vapier's bits, get_eclassdir_path (or similar) addresses it.
Comment 51 Ciaran McCreesh 2012-05-22 19:48:11 UTC
"Pragmatic" is simply another way of saying "wrong". You're talking about replacing one broken thing with another broken thing. The pragmatic approach is to mark SRC_URI as being "merged" across eclasses like DEPEND etc and get rid of PORTDIR in new EAPIs, update PMS to say that ebuilds messing around in the tree is deprecated, and then after a suitable period, retroactively nuke PORTDIR.
Comment 52 Ulrich Müller gentoo-dev 2012-05-22 20:42:59 UTC
What would be the alternative to accessing the tree in the case of 1) licenses (e.g. in corefonts) and 2) eclass-manpages?
Comment 53 Ciaran McCreesh 2012-05-22 20:51:15 UTC
(In reply to comment #52)
> What would be the alternative to accessing the tree in the case of 1)
> licenses (e.g. in corefonts)

SRC_URI

> and 2) eclass-manpages?

SRC_URI
Comment 54 Brian Harring (RETIRED) gentoo-dev 2012-05-22 20:54:42 UTC
(In reply to comment #51)
> "Pragmatic" is simply another way of saying "wrong". You're talking about
> replacing one broken thing with another broken thing. The pragmatic approach
> is to mark SRC_URI as being "merged" across eclasses like DEPEND etc and get
> rid of PORTDIR in new EAPIs, update PMS to say that ebuilds messing around
> in the tree is deprecated, and then after a suitable period, retroactively
> nuke PORTDIR.

You're being academic, mixed w/ your usual healthy dose of cracked out.  A solution involving trying to push every libtool patch into SRC_URI isn't sane from a cache standpoint, sane from a maintenance standpoint, and is just fucking bad design.

Eclass w/ patches alongside or pkg are the options for libtool.  Period (no amount of whinging will change that).

Sorting libtool.eclass still doesn't fix the rest of the usage however (as ulm pointed out).

Reality is, get_repository_path is a bad idea if/when we have wildly variant repository formats in play.  The format is basically the same for the last decade, and to even *do* variant repository formats requires a lot of format marking (and would by definition be outside of PMS, meaning "on the user/developers head" per the norm) thus your concerns there are both academic and moronic.  Specifically, you're stonewalling because you don't like the solution.

Propose a sane one and I'd be game to run with anything you come up with, but stonewalling and leaving the current mess in place isn't viable from where I'm sitting.
Comment 55 Ulrich Müller gentoo-dev 2012-05-22 21:14:09 UTC
(In reply to comment #53)
> (In reply to comment #52)
> > What would be the alternative to accessing the tree in the case of 1)
> > licenses (e.g. in corefonts)
> 
> SRC_URI
> 
> > and 2) eclass-manpages?
> 
> SRC_URI

But this would mean that users would have to download files again that are already present on their system. Doesn't look like a good solution to me.
Comment 56 Ciaran McCreesh 2012-05-22 21:18:20 UTC
(In reply to comment #54)
> You're being academic, mixed w/ your usual healthy dose of cracked out.  A
> solution involving trying to push every libtool patch into SRC_URI isn't
> sane from a cache standpoint, sane from a maintenance standpoint, and is
> just fucking bad design.

Cache is an entirely unrelated issue, and so long as eutils exists, going on about anything else with cache is completely missing the point.

> Eclass w/ patches alongside or pkg are the options for libtool.  Period (no
> amount of whinging will change that).

The SRC_URI approach is perfectly workable. No amount of stonewalling will change that.

> Sorting libtool.eclass still doesn't fix the rest of the usage however (as
> ulm pointed out).

The rest are easy. For that matter, libtool is also easy.

> Reality is, get_repository_path is a bad idea if/when we have wildly variant
> repository formats in play.

You mean like when someone wants to install a binary package using Portage?

> The format is basically the same for the last
> decade, and to even *do* variant repository formats requires a lot of format
> marking (and would by definition be outside of PMS, meaning "on the
> user/developers head" per the norm) thus your concerns there are both
> academic and moronic.

The only reason the layout and ebuild format are so tightly coupled right now is because a few small bits of code make annoying assumptions that were historically valid but that really need to be revisited now that we're looking at a bigger picture. It's exactly the same as ebuilds relying upon VDB.

> Specifically, you're stonewalling because you don't like the solution.
> Propose a sane one and I'd be game to run with anything you come up with,
> but stonewalling and leaving the current mess in place isn't viable from
> where I'm sitting.

You haven't provided a solution. You're just replacing one bad thing with another, equally bad thing.
Comment 57 Ciaran McCreesh 2012-05-22 21:20:03 UTC
(In reply to comment #55)
> But this would mean that users would have to download files again that are
> already present on their system. Doesn't look like a good solution to me.

Uh, it's one tiny file for corefonts (and most packages ship their licence anyway, so users "download files again" zillions of times for every package that includes a COPYING).

As for eclass-manpages, the files shouldn't be being generated on the user's syetem at all. I don't mean download the eclasses, I mean download the built man pages. Otherwise, the "version number" for the ebuild is a complete lie.
Comment 58 Brian Harring (RETIRED) gentoo-dev 2012-05-22 22:27:25 UTC
@Ulm @vapier: your comments as to what is an acceptable solution?

Ciaran's already stated his views, no point in continuing down that pathway (argue it into oblivion via stalling primarily), thus moving on from that part of the discussion.
Comment 59 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-05-23 04:05:34 UTC
(In reply to comment #58)
> Ciaran's already stated his views, no point in continuing down that pathway
> (argue it into oblivion via stalling primarily), thus moving on from that
> part of the discussion.

Your solution just means 'in EAPI 5 we'll change the name of it and everything goes like it was before'.

I think licenses case is not important. It is a single ebuild, with a single tiny file, which I'm even not sure if is really necessary. Maybe someone should try to ping upstream why they're breaking their own license.

While eclassdir is necessary right now. If you really need your fix -- go with it there. I can assume eclass/ has to be untouched. But don't make me require random files like header.txt in partial checkouts.
Comment 60 Ulrich Müller gentoo-dev 2012-05-23 05:00:47 UTC
We shouldn't remove functionality that has existed for a long time and hasn't caused any real problems. Also I don't think that PORTDIR is abused. Ebuilds really aren't "messing around in the tree" (I would interpret that as accessing other packages' directories, e.g.), but use the variable to access a few well-defined locations like eclasses and licenses.

Anyway, I don't see consensus here. So maybe the discussion should be moved to the gentoo-dev ML?
Comment 61 Ciaran McCreesh 2012-05-23 13:05:20 UTC
It only hasn't caused any real problems because we've never tried to fix the tight coupling between the tree format and the package format. The number of offenders here is *really* low.
Comment 62 Arfrever Frehtes Taifersar Arahesis 2012-09-22 01:24:16 UTC
I suggest the following functions:

1. repository_path() function, which prints repository path of specified
   repository.
2. master_repositories() function, which prints space-separated list of
   master repositories for specified repository.
3. eclasses() function, which prints space-separated list of eclasses
   available (not necessarily inherited) for current package. It includes
   eclasses from master repositories.
4. eclass_path() function, which prints path of specified eclass available
   (not necessarily inherited) for current package.
5. license_path() function, which prints path of specified license (not
   necessarily used in LICENSE) for current package.

Examples:
  app-portage/eclass-manpages could use the following code to correctly
  locate eclasses from current repository and master repositories:
    local eclass path
    for eclass in $(eclasses); do
      path=$(eclass_path ${eclass})
      ...
    done

  libtool.eclass could use the following code:
    local path
    path=$(eclass_path libtool)
    path=${path%/*}/ELT-patches
Comment 63 Ciaran McCreesh 2012-09-22 13:05:11 UTC
Why are you so eager to couple ebuilds to where they are?
Comment 64 Brian Harring (RETIRED) gentoo-dev 2012-09-23 02:38:41 UTC
(In reply to comment #62)
> I suggest the following functions:
> 
> 1. repository_path() function, which prints repository path of specified
>    repository.
No, although the reason is subtle; we don't want ebuilds being all that aware of the repo structure;  This allows them to be; if they're sensitive, this severely limits the playing field for future repository formats (since we'd have to do analysis to figure out what all was busted).

> 2. master_repositories() function, which prints space-separated list of
>    master repositories for specified repository.
I suspect this is for repository_path; which case, I say no.  Got a scenario beyond #1 that needs this?

> 3. eclasses() function, which prints space-separated list of eclasses
>    available (not necessarily inherited) for current package. It includes
>    eclasses from master repositories.
I'm not sure I see the point of this; it smells suspiciously like a desire to allow ebuilds to vary the eclass used dependent on whats available- either that or trying to go raiding shit out of another eclasses space.

> 4. eclass_path() function, which prints path of specified eclass available
>    (not necessarily inherited) for current package.
Don't like it, but this solves the eutils scenario.

> 5. license_path() function, which prints path of specified license (not
>    necessarily used in LICENSE) for current package.

No.  get_license() function instead, that returns the text; still don't much like that since it's taking control away from the PM for display control- something like accept_license() and display_license() (if necessary) seems better since the approach allows a hand off to the PM- think about porthole, it would want to pop up an actual UI window for "do you accept this license" rather than trying to scrape/analyze output looking for ebuilds pulling this.

Concept I'm sort of fine with, but the form for #5 limits options on future PM/UI bits, that being the main concern.
Comment 65 Arfrever Frehtes Taifersar Arahesis 2012-10-14 00:08:40 UTC
master_repositories(), repository_path(), available_eclasses(), eclass_path() and license_path() functions have been implemented in EAPI="5-progress":

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commitdiff;h=58c2ef4e37fa635945e19a1650e15206d457830b
Comment 66 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2015-11-21 13:11:51 UTC
How about finally banning it in EAPI 7? We've slept over on this one, and while some of the old uses were removed, a few new appeared.

The current list includes:

1. eutils.eclass referencing ELT-patches (wtf?!).

2. games-mods.eclass referencing header.txt. To be honest, I don't think we should even assume this file will be there...

3. Some random ebuilds referencing licenses... again.

4. A few ebuilds sed-ing PORTDIR variables out of Makefiles to avoid collision with our variable.

5. Portage ebuild doing some ugly hackery but we can fix it.
Comment 67 Ulrich Müller gentoo-dev 2015-11-21 13:59:05 UTC
(In reply to Michał Górny from comment #66)
Does this include dropping the ECLASSDIR variable, too? That one is used by libtool.eclass and app-portage/eclass-manpages.
Comment 68 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2015-11-21 15:06:29 UTC
I'd rather focus on each of them separately. This way, there's at least a slim chance we could remove one of them.
Comment 69 Ulrich Müller gentoo-dev 2015-11-21 15:51:49 UTC
There is at least one instance of ${PORTDIR}/eclass used in the tree.
Conversely, I predict that people will start using ${ECLASSDIR}/.. then.
Comment 70 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2015-11-21 16:35:55 UTC
That doesn't sound like a valid argument. Same way, you could say we ought to expose top ebuild work directory just because people can use ${WORKDIR}/.. instead.

As far as I'm concerned, if we define a particular directory and provide the path to it, people are not expected to go up. If they do, that's a QA violation.

That said, if we have valid reasons to access particular parts of repository structure, we should provide well-defined access methods to them rather than letting people hardcode internals.

So if people need to access licenses, they should have ${LICENSEDIR} rather than ${PORTDIR}/licenses. And probably LICENSEDIR should be populated by the PM with symlinks so that it contains only licenses used in the particular ebuild (much like what we do to DISTDIR), and collected from all relevant repositories (i.e. the repo with ebuild as well as its masters).

If we have ${ECLASSDIR} already, I don't see why people would resort to ${PORTDIR}/eclass. That said, if multiple repositories are involved, it is no longer clear what ECLASSDIR should exactly be. We could go for symlinks again but then both current uses would be broken.

Which pretty much proves that there's no good solution to this since people want to do stuff they're not supposed to do at all.

eclass-manpages makes no sense as it is. If I were to fix it to work reliably, I'd probably make it fetch eclasses from some reference source rather than relying on them being in some specific location relative to ebuild.

libtool.eclass just needs some random common data storage which someone found convenient in putting inside the repo. All those files are completely uncontrolled, lack any kind of signatures and most importantly, probably won't ever be covered by any Manifest files because their existence is not mandated anywhere. If people really want creepy stuff like this, it really belongs installed in the filesystem via an ebuild. Or in some dedicated data directory defined in a future EAPI. But I wouldn't go for something like this for one eclass that is rarely and inconsistently used.
Comment 71 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-09-03 14:24:59 UTC
So what we have left is:

a. gnat-gpl (new use to get patches from sys-devel/gcc, horrible idea),

b. tw_cli (copying one license, probably unnecessary/stupid),

c. games-mods.eclass lazily getting copyright header from random file.

Given there's just one consumer for licenses, I'm not convinced there's a point in debating over that.
Comment 72 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2017-09-26 19:24:27 UTC
(In reply to Michał Górny from comment #71)
> So what we have left is:
> b. tw_cli (copying one license, probably unnecessary/stupid),
The end goal here, to comply with upstream's legal, is that the license must be present if the binary is present. The AdminCD ships the binary, so you can use it to install on new systems, ergo the AdminCD needs the license present. The AdminCD doesn't have a portdir at all, so we can't simply reference it.

The license isn't inside the tarball, so we need to copy it from somewhere. If you want to disallow $PORTDIR/licenses/ access, we can add it to SRC_URI (I'm going to do it creatively, referencing the portage file via gitweb).
Comment 73 Ulrich Müller gentoo-dev 2017-09-26 20:38:23 UTC
(In reply to Robin Johnson from comment #72)
> The end goal here, to comply with upstream's legal, is that the license must
> be present if the binary is present. [...]
> 
> The license isn't inside the tarball, so we need to copy it from somewhere.

That is a very strange constellation, upstream on the one hand insisting that their license must be installed, but on the other hand not including it in their tarball. Let's hope that this remains an exception.

> If you want to disallow $PORTDIR/licenses/ access, we can add it to SRC_URI
> (I'm going to do it creatively, referencing the portage file via gitweb).

I agree that SRC_URI is the way to go here. With 35 kB the file is a bit large for FILESDIR (although the blob is in the repo already).
Comment 74 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2017-09-27 18:27:53 UTC
(In reply to Ulrich Müller from comment #73)
> (In reply to Robin Johnson from comment #72)
> > The end goal here, to comply with upstream's legal, is that the license must
> > be present if the binary is present. [...]
> > 
> > The license isn't inside the tarball, so we need to copy it from somewhere.
> 
> That is a very strange constellation, upstream on the one hand insisting
> that their license must be installed, but on the other hand not including it
> in their tarball. Let's hope that this remains an exception.
The license in the tarball has been superseded by an newer upstream license. If they would just do a bump with a new tarball, it would help a lot yes.

> 
> > If you want to disallow $PORTDIR/licenses/ access, we can add it to SRC_URI
> > (I'm going to do it creatively, referencing the portage file via gitweb).
> 
> I agree that SRC_URI is the way to go here. With 35 kB the file is a bit
> large for FILESDIR (although the blob is in the repo already).
Done.
Comment 75 Larry the Git Cow gentoo-dev 2017-09-27 18:28:00 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=e9dc4206a285c22773c2dea0a11d73855501001b

commit e9dc4206a285c22773c2dea0a11d73855501001b
Author:     Robin H. Johnson <robbat2@gentoo.org>
AuthorDate: 2017-09-27 18:27:10 +0000
Commit:     Robin H. Johnson <robbat2@gentoo.org>
CommitDate: 2017-09-27 18:27:52 +0000

    sys-block/tw_cli: fix PORTDIR/licenses/ access.
    
    Bug: https://bugs.gentoo.org/373349#c72
    Package-Manager: Portage-2.3.8, Repoman-2.3.3

 sys-block/tw_cli/Manifest               |  1 +
 sys-block/tw_cli/tw_cli-10.2.2.1.ebuild | 12 ++++++++----
 2 files changed, 9 insertions(+), 4 deletions(-)}
Comment 76 Larry the Git Cow gentoo-dev 2018-03-11 11:44:16 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=802e7d0bdd967d28c6dbb31fd84e8dfd540d794a

commit 802e7d0bdd967d28c6dbb31fd84e8dfd540d794a
Author:     Michał Górny <mgorny@gentoo.org>
AuthorDate: 2018-02-27 18:53:02 +0000
Commit:     Michał Górny <mgorny@gentoo.org>
CommitDate: 2018-03-11 11:43:41 +0000

    Do not export PORTDIR & ECLASSDIR in EAPI 7
    
    Bug: https://bugs.gentoo.org/373349
    Bug: https://bugs.gentoo.org/373351
    Closes: https://github.com/gentoo/portage/pull/264

 bin/eapi.sh                            |  4 ++++
 bin/phase-functions.sh                 |  6 ++++--
 pym/portage/eapi.py                    | 11 +++++++++++
 pym/portage/package/ebuild/config.py   |  5 +++++
 pym/portage/package/ebuild/doebuild.py |  8 ++++----
 5 files changed, 28 insertions(+), 6 deletions(-)}
Comment 77 Larry the Git Cow gentoo-dev 2018-04-30 22:14:52 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/pms.git/commit/?id=3abcd75afc8a8ed4400ecd66cc4f129a4bf46330

commit 3abcd75afc8a8ed4400ecd66cc4f129a4bf46330
Author:     Michał Górny <mgorny@gentoo.org>
AuthorDate: 2017-09-29 03:17:43 +0000
Commit:     Ulrich Müller <ulm@gentoo.org>
CommitDate: 2018-03-30 16:35:43 +0000

    EAPI 7 removes PORTDIR and ECLASSDIR.
    
    Bug: https://bugs.gentoo.org/373349
    Bug: https://bugs.gentoo.org/373351

 eapi-differences.tex |  8 ++++++++
 ebuild-env-vars.tex  | 17 +++++++++++------
 2 files changed, 19 insertions(+), 6 deletions(-)}