Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 378603 - sys-apps/portage: FHS compliance of default PORTDIR & near friends
Summary: sys-apps/portage: FHS compliance of default PORTDIR & near friends
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core - Configuration (show other bugs)
Hardware: All Linux
: Normal normal with 2 votes (vote)
Assignee: Portage team
URL: https://projects.gentoo.org/council/m...
Whiteboard:
Keywords: InVCS
: 498870 (view as bug list)
Depends on: 479062
Blocks: 662982 683434
  Show dependency tree
 
Reported: 2011-08-10 07:26 UTC by Michał Górny
Modified: 2019-08-13 05:58 UTC (History)
16 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2011-08-10 07:26:48 UTC
As the discussion was raised again on the gentoo-dev ml [1], I'm opening a bug to track any changes related to it.

Saying shortly, the problems are:
- /usr/portage is not really FHS-y location for a PORTDIR,
- storing distfiles & binpackages in PORTDIR subdirs is basically a bad idea (I think we should rather convince users to use separate fs for PORTDIR),
- also the 'suggested' PORTDIR_OVERLAY sounds better being inside PORTDIR than outside it (for the sake of having separate fs for it).

Distfiles should obviously go to /var/cache. Binpackages are more complicated. PORTDIR sounds /var/cache-y too but unsure if portage could sync it without any additional action.

[1]:http://archives.gentoo.org/gentoo-dev/msg_c35325d13ef3c1259449f3a7004da303.xml
Comment 1 Zac Medico gentoo-dev 2011-08-10 08:16:14 UTC
In order to ensure that the locations don't unexpectedly change for existing installs, I'd like to run a migration tool during pkg_preinst that will detect when a user has not overridden the default settings of variables like PORTDIR, and will automatically add the old default settings to make.conf.
Comment 2 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2011-08-10 08:51:37 UTC
Isn't CONFIG_PROTECT all about nicely telling users defaults change?
Comment 3 Zac Medico gentoo-dev 2011-08-10 23:03:42 UTC
(In reply to comment #2)
> Isn't CONFIG_PROTECT all about nicely telling users defaults change?

Yes, and I plan for the migration tool to respect CONFIG_PROTECT when it modifies make.conf. If you're thinking about changes to the defaults in make.globals though, that won't be visible via CONFIG_PROTECT, since make.globals has resided in /usr/share/portage/config/ for years now because it's not intended to be modified by users.
Comment 4 Sebastian Pipping gentoo-dev 2011-08-11 22:21:41 UTC
(In reply to comment #0)
> - storing distfiles & binpackages in PORTDIR subdirs is basically a bad idea (I
> think we should rather convince users to use separate fs for PORTDIR),

Could you provide a linked or inlined rationale?


> Distfiles should obviously go to /var/cache. Binpackages are more complicated.
> PORTDIR sounds /var/cache-y too but unsure if portage could sync it without any
> additional action.

PORTDIR (and PORTDIR_OVERLAY) -- as a remote cache -- is no true cache in the sense of FHS: I see these two points of FSH for /var/cache violated:

  1) "The application must be able to regenerate or restore the data."

  2) "[T]he cached files can be deleted without data loss."

(1) may fail when ebuilds get deleted (please save me a discussion on how version control makes up for that or not), (2) fails on failure of (1).

For that reason, layman's home moved to /var/lib/ last time, instead of /var/cache: it is currently located at /var/lib/layman/.
Comment 5 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2011-08-11 23:25:16 UTC
(In reply to comment #4)
> (In reply to comment #0)
> > - storing distfiles & binpackages in PORTDIR subdirs is basically a bad idea (I
> > think we should rather convince users to use separate fs for PORTDIR),
> 
> Could you provide a linked or inlined rationale?

You pasted the rationale below. PORTDIR is a lot of often-changing, small files which can be regenerated with some effort on user side. Distfiles are random size files which can be removed at a random point in time.

> > Distfiles should obviously go to /var/cache. Binpackages are more complicated.
> > PORTDIR sounds /var/cache-y too but unsure if portage could sync it without any
> > additional action.
> 
> PORTDIR (and PORTDIR_OVERLAY) -- as a remote cache -- is no true cache in the
> sense of FHS: I see these two points of FSH for /var/cache violated:
> 
>   1) "The application must be able to regenerate or restore the data."
> 
>   2) "[T]he cached files can be deleted without data loss."
> 
> (1) may fail when ebuilds get deleted (please save me a discussion on how
> version control makes up for that or not), (2) fails on failure of (1).

(1) is what I mentioned, (2) makes /var/cache a completely bad choice indeed.
Comment 6 William Hubbs gentoo-dev 2011-08-12 01:12:19 UTC
(In reply to comment #3)
> (In reply to comment #2)
> > Isn't CONFIG_PROTECT all about nicely telling users defaults change?
> 
> Yes, and I plan for the migration tool to respect CONFIG_PROTECT when it
> modifies make.conf. If you're thinking about changes to the defaults in
> make.globals though, that won't be visible via CONFIG_PROTECT, since
> make.globals has resided in /usr/share/portage/config/ for years now because
> it's not intended to be modified by users.

There is a symlink in /etc/make.globals, shouldn't we remove that also then?
 
What about /etc/make.profile and /etc/make.conf, should they be moved under /etc/portage?
Comment 7 Zac Medico gentoo-dev 2011-08-12 01:23:12 UTC
(In reply to comment #6)
> There is a symlink in /etc/make.globals, shouldn't we remove that also then?

We can. It's only there for backward compatibility people who source it from scripts.

> What about /etc/make.profile and /etc/make.conf, should they be moved under
> /etc/portage?

It's already been supported for some time now. There's support for /etc/portage/make.profile in eselect-profile too.
Comment 8 William Hubbs gentoo-dev 2011-08-12 01:28:16 UTC
(In reply to comment #4)
> (In reply to comment #0)
> > Distfiles should obviously go to /var/cache. Binpackages are more complicated.
> > PORTDIR sounds /var/cache-y too but unsure if portage could sync it without any
> > additional action.

Yes it can, just point your portdir in your make.conf somewhere else then do emerge --sync and you will create a complete portdir.

> 
> PORTDIR (and PORTDIR_OVERLAY) -- as a remote cache -- is no true cache in the
> sense of FHS: I see these two points of FSH for /var/cache violated:
> 
>   1) "The application must be able to regenerate or restore the data."

How can portage not do this? as I said above, that's what emerge --sync does.

>   2) "[T]he cached files can be deleted without data loss."

They can, just use emerge --sync to get portdir back and or emerge -f to get distfiles back.

> 
> (1) may fail when ebuilds get deleted (please save me a discussion on how
> version control makes up for that or not), (2) fails on failure of (1).

Given my comments above, I do not see how either of these fail.

> For that reason, layman's home moved to /var/lib/ last time, instead of
> /var/cache: it is currently located at /var/lib/layman/.

From the definition of /var/lib in the fhs:

"This hierarchy holds state information pertaining to an application or the system. State information is data that programs modify while they run, and that pertains to one specific host. Users must never need to modify files in /var/lib to configure a package's operation."

I think portdir and distfiles cannot go in /var/lib because anything in /var/lib must not be shared between different computers, and I have heard of folks sharing portdir and distfiles.

I think /var/cache is the best place for this.

[1] http://www.pathname.com/fhs/pub/fhs-2.3.html#VARLIBVARIABLESTATEINFORMATION
Comment 9 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2011-08-12 07:11:43 UTC
(In reply to comment #8)
> > PORTDIR (and PORTDIR_OVERLAY) -- as a remote cache -- is no true cache in the
> > sense of FHS: I see these two points of FSH for /var/cache violated:
> > 
> >   1) "The application must be able to regenerate or restore the data."
> 
> How can portage not do this? as I said above, that's what emerge --sync does.
> 
> >   2) "[T]he cached files can be deleted without data loss."
> 
> They can, just use emerge --sync to get portdir back and or emerge -f to get
> distfiles back.

That's not exactly that. It is rather that a random files inside /var/cache can disappear for no reason, and portage would have to be able to re-get them live, e.g. when performing @world update.

Although this is possible to implement, it would mean an increase in bandwidth (as portage would have to ping rsync over and over again) and real pain for users. Not really worth it.

> > (1) may fail when ebuilds get deleted (please save me a discussion on how
> > version control makes up for that or not), (2) fails on failure of (1).
> 
> Given my comments above, I do not see how either of these fail.

It fails if user doesn't know he/she has to do --sync because a random file has been removed.

> > For that reason, layman's home moved to /var/lib/ last time, instead of
> > /var/cache: it is currently located at /var/lib/layman/.
> 
> From the definition of /var/lib in the fhs:
> 
> "This hierarchy holds state information pertaining to an application or the
> system. State information is data that programs modify while they run, and that
> pertains to one specific host. Users must never need to modify files in
> /var/lib to configure a package's operation."
> 
> I think portdir and distfiles cannot go in /var/lib because anything in
> /var/lib must not be shared between different computers, and I have heard of
> folks sharing portdir and distfiles.

PORTDIR may stay in /usr but would have to go to some nice, clean subdir. Probably /usr/local/share or */lib, sth like that.

> [1] http://www.pathname.com/fhs/pub/fhs-2.3.html#VARLIBVARIABLESTATEINFORMATION
Comment 10 Sebastian Pipping gentoo-dev 2011-08-12 17:03:53 UTC
(In reply to comment #8)
> I think portdir and distfiles cannot go in /var/lib because anything in
> /var/lib must not be shared between different computers, and I have heard of
> folks sharing portdir and distfiles.
> 
> I think /var/cache is the best place for this.

Good point.  Just /var/cache doesn't seem a logical consequence to me.


> > >   2) "[T]he cached files can be deleted without data loss."
> > 
> > They can, just use emerge --sync to get portdir back and or emerge -f to get
> > distfiles back.
> 
> That's not exactly that. It is rather that a random files inside /var/cache can
> disappear for no reason, and portage would have to be able to re-get them live,
> e.g. when performing @world update.

To be very clear: my point is on avoiding loss of data, not just on the trouble and time re-creation could take in a good case.


> PORTDIR may stay in /usr but would have to go to some nice, clean subdir.
> Probably /usr/local/share or */lib, sth like that.

I wonder: what advantage does /usr/local/share give over plain /usr/share ?

According to FHS 2.3 [1] our ebuilds tree would qualify for being architecture independent because both a sparc and amd64 maschine could share that mount.

[1] http://www.pathname.com/fhs/pub/fhs-2.3.html#USRSHAREARCHITECTUREINDEPENDENTDATA

Unfortunately, one problem is that program app-misc/gentoo (a file manager) installs to files /usr/share/gentoo/.  With my puristic hat on I would say that /usr/local/share/gentoo/ inherits too much of that problem.


On another one: would /opt be an alternative?  Inside /opt there seem few restrictions, we could either use /opt/portage (if we aim at minimal changes felt) or something like this for a more thorough switch:
- /opt/gentoo/tree/gentoo  # PORTDIR
- /opt/gentoo/tree/betagarden
- /opt/gentoo/tree/sunrise
- /opt/gentoo/distfiles/

As a side effect, this scheme could would (1) force less portage branding on other package managers and (2) help getting rid of the mis-use of "portage" for "the gentoo main tree" (and leave that term it to the piece of software, only).

But maybe someone comes up with something better yet, I'm curious.
Comment 11 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2011-08-12 19:35:04 UTC
(In reply to comment #10)
> > PORTDIR may stay in /usr but would have to go to some nice, clean subdir.
> > Probably /usr/local/share or */lib, sth like that.
> 
> I wonder: what advantage does /usr/local/share give over plain /usr/share ?

Files not controlled by PM should go into /usr/local. And by 'controlled' I mean recorded in vdb. Of course, most of our eselect thingies break that rule.

> Unfortunately, one problem is that program app-misc/gentoo (a file manager)
> installs to files /usr/share/gentoo/.  With my puristic hat on I would say that
> /usr/local/share/gentoo/ inherits too much of that problem.

It's just about choosing the right name.

> On another one: would /opt be an alternative?  Inside /opt there seem few
> restrictions, we could either use /opt/portage (if we aim at minimal changes
> felt) or something like this for a more thorough switch:
> - /opt/gentoo/tree/gentoo  # PORTDIR
> - /opt/gentoo/tree/betagarden
> - /opt/gentoo/tree/sunrise
> - /opt/gentoo/distfiles/

I'd personally like to get rid of /opt. We're mis-using it completely and it's just another directory outside of /usr which can grow ridiculously large for no good reason.

> As a side effect, this scheme could would (1) force less portage branding on
> other package managers and (2) help getting rid of the mis-use of "portage" for
> "the gentoo main tree" (and leave that term it to the piece of software, only).

Other schemes can do that as well.
Comment 12 William Hubbs gentoo-dev 2011-08-13 15:07:57 UTC
(In reply to comment #9)
> PORTDIR may stay in /usr but would have to go to some nice, clean subdir.

FHS defines /usr as:

"/usr is the second major section of the filesystem. /usr is shareable, read-only data. That means that /usr should be shareable between various FHS-compliant hosts and must not be written to. Any information that is host-specific or varies with time is stored elsewhere."

The entire reason for this discussion is to get these directories out of /usr.

> Probably /usr/local/share or */lib, sth like that.

/usr/local is defined as:

"The /usr/local hierarchy is for use by the system administrator when installing software locally. It needs to be safe from being overwritten when the system software is updated. It may be used for programs and data that are shareable amongst a group of hosts, but not found in /usr."

So we should not attempt to put portdir or any of these directories there.

(In reply to comment #10)
> (In reply to comment #8)
> > > >   2) "[T]he cached files can be deleted without data loss."
> > > 
> > > They can, just use emerge --sync to get portdir back and or emerge -f to get
> > > distfiles back.
> > 
> > That's not exactly that. It is rather that a random files inside /var/cache can
> > disappear for no reason, and portage would have to be able to re-get them live,
> > e.g. when performing @world update.

The worst that would happen if an individual ebuild disappeared is that a downgrade might be triggered. But, again, if the user does an emerge --sync, this would be reversed.

If something in distfiles disappears, it will be downloaded again when the user updates the package.

> To be very clear: my point is on avoiding loss of data, not just on the trouble
> and time re-creation could take in a good case.

I'm still not understanding you. I don't see where the data loss that you are wanting to avoid would occur
Comment 13 Lars Wendler (Polynomial-C) (RETIRED) gentoo-dev 2011-08-13 15:35:25 UTC
Maybe it's about the fact that all these "recovery" steps need the system being online.
I can imagine that there are quite some Gentoo users who don't have permanent access to the internet. So for these users loosing data in ${PORTDIR} or ${DISTDIR} could prevent clean emerges at the worst until the can go online again.
Comment 14 Sebastian Pipping gentoo-dev 2011-08-13 16:54:01 UTC
Right.

In the case of PORTDIR_OVERLAY, ebuils may get lost into nowhere because a repo goes 404 or a forced push wipes away ebuilds that I need to restore a certain defined state of my system.  Maybe I need that very version of that very outdated non-standard tool.  If there was an ebuild before, there is loss after.
Comment 15 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2011-08-13 17:37:08 UTC
(In reply to comment #12)
> (In reply to comment #9)
> > PORTDIR may stay in /usr but would have to go to some nice, clean subdir.
> 
> FHS defines /usr as:
> 
> "/usr is the second major section of the filesystem. /usr is shareable,
> read-only data. That means that /usr should be shareable between various
> FHS-compliant hosts and must not be written to. Any information that is
> host-specific or varies with time is stored elsewhere."
> 
> The entire reason for this discussion is to get these directories out of /usr.

If we could find a clear location for repositories in /usr, these could be shared nicely.

> > Probably /usr/local/share or */lib, sth like that.
> 
> /usr/local is defined as:
> 
> "The /usr/local hierarchy is for use by the system administrator when
> installing software locally. It needs to be safe from being overwritten when
> the system software is updated. It may be used for programs and data that are
> shareable amongst a group of hosts, but not found in /usr."
> 
> So we should not attempt to put portdir or any of these directories there.

I agree.

> (In reply to comment #10)
> > (In reply to comment #8)
> > > > >   2) "[T]he cached files can be deleted without data loss."
> > > > 
> > > > They can, just use emerge --sync to get portdir back and or emerge -f to get
> > > > distfiles back.
> > > 
> > > That's not exactly that. It is rather that a random files inside /var/cache can
> > > disappear for no reason, and portage would have to be able to re-get them live,
> > > e.g. when performing @world update.
> 
> The worst that would happen if an individual ebuild disappeared is that a
> downgrade might be triggered. But, again, if the user does an emerge --sync,
> this would be reversed.

This is much like a HTTP proxy showing 'hey, this page has been removed from cache. Please open the following link to refetch it.'
Comment 16 William Hubbs gentoo-dev 2011-08-13 22:12:32 UTC
(In reply to comment #15)
> (In reply to comment #12)
> > (In reply to comment #9)
> > > PORTDIR may stay in /usr but would have to go to some nice, clean subdir.
> > 
> > FHS defines /usr as:
> > 
> > "/usr is the second major section of the filesystem. /usr is shareable,
> > read-only data. That means that /usr should be shareable between various
> > FHS-compliant hosts and must not be written to. Any information that is
> > host-specific or varies with time is stored elsewhere."
> > 
> > The entire reason for this discussion is to get these directories out of /usr.
> 
> If we could find a clear location for repositories in /usr, these could be
> shared nicely.

Please take a look at the definition of /usr above again. In particular, notice that /usr is sharable, read-only data. Later in the definition, it is echoed again that /usr must not be written to.

/usr/portage is the wrong place for these repositories for that reason; we need to get them out of /usr. That is why we have this bug.
Comment 17 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2011-08-14 06:42:10 UTC
(In reply to comment #16)
> Please take a look at the definition of /usr above again. In particular, notice
> that /usr is sharable, read-only data. Later in the definition, it is echoed
> again that /usr must not be written to.
> 
> /usr/portage is the wrong place for these repositories for that reason; we need
> to get them out of /usr. That is why we have this bug.

With that strict assumption, /usr would never be populated. The idea is that user should not write to /usr directly (except for /usr/local) while the PM can do it. Considering that repo-syncing is mostly done by PM (or layman), I guess it's acceptable even if done with limited user privileges.

Syncs are done near @world updates. If user wants to do a world update, he/she needs to have /usr r/w anyway.
Comment 18 Sebastian Pipping gentoo-dev 2011-08-14 16:03:56 UTC
(In reply to comment #16)
> [N]otice that /usr is sharable, read-only data.

No contradictions to our case - I agree with mgorny.


> /usr/portage is the wrong place for these repositories for that reason; we need
> to get them out of /usr. That is why we have this bug.

I thought we have it because FHS would not allow adding new dirs right into "/usr", e.g. "/usr/portage".
Comment 19 Zac Medico gentoo-dev 2012-12-18 22:54:45 UTC
We could have catalyst write the new defaults in the make.conf that it generates. Having the new defaults explicitly recorded there does not require any changes to portage (though I will bring portage's internal defaults into sync as soon as possible). Also, I think it might help avoid confusion for users if they are able to see the new defaults explicitly recorded in make.conf (as opposed to /usr/share/portage/config/make.globals which is somewhat obscure).
Comment 20 SpanKY gentoo-dev 2014-01-22 05:18:27 UTC
*** Bug 498870 has been marked as a duplicate of this bug. ***
Comment 21 Brian Dolbec (RETIRED) gentoo-dev 2014-01-25 02:53:30 UTC
Just as an update.  Catalyst's old poor code has been the hold up for migrating these directories to new defaults.  I am starting the process now of merging the 3.0 rewrite code back into the master branch.  With that it removes the hard coded paths form the code which prevented the defaults from changing.  I would think it will take 3 - 4 months to get a new release stabilized and producing new stages with the new defaults (wherever they end up). Who knows, perhaps it will be sooner if I can help it.

There is still lots more rewriting to do in catalyst.  But I think we can get a decently stable version out within a few months, then do some additional testing/debugging before continuing the rewrites.
Comment 22 Guilherme Amadio gentoo-dev 2014-03-09 02:41:51 UTC
I've recently become interested in this topic (the location of the portage tree), as I always thought that /usr/portage was an awkward place for it, and I am happy to see that the devs are reaching an agreement that /var makes a lot more sense. So, what are the current locations preferred by the devs of where to put the portage directories at the moment? I noticed that portage started to complain if PORTDIR wasn't explicitly set in make.conf, so I assume this is going somewhere in the near future. I read some of the old discussions on the topic, and I think that the /var/cache approach is the most appropriate. In particular, I decided to use the following in my systems:

PORTDIR=/var/cache/portage/tree
DISTDIR=/var/cache/portage/distfiles
PKGDIR=/var/cache/portage/packages

Also, I don't use any overlays, but I think that /var/cache/portage/overlays or /var/lib/portage/overlays would be a nice location, so one would have their local overlay at /var/lib/portage/overlays/local instead of /usr/local/portage for example (which would also avoid confusion with /usr/portage/local and loss of data). Other overlays would be subdirectories of /var/lib/portage/overlays as well. What do you guys think?
Comment 23 Matt Whitlock 2016-12-30 01:19:13 UTC
I've read the entire thread entitled "Moving our/portage stuff to var" from the gentoo-dev discussion list in 2012, and I've read all the comments on this bug report, and I would now like to summarize the arguments and give my opinions.


1. Shortness of paths (for ease of typing) seems to be a high priority for certain individuals. To them I say, create some symlinks on your system if you like. Let's not eschew a logical containment hierarchy in favor of laziness.


2. Are sync'd ebuild repositories "cache"? There are good arguments on both sides.

* Yes, sync'd ebuild repositories are cache:

- * They need not be backed up.
- * They can be repopulated with a single command invocation.

* No, sync'd ebuild repositories are not cache:

- * They cannot be repopulated without an Internet connection. (FHS requires that cache be "locally generated.")
- * Arbitrarily deleted files are not automatically regenerated by the application. (An explicit sync must be invoked.)

My opinion: The usual intent of a cache dir is to memoize an application's computed results to save time when the same results are needed again in the future. Cache dirs are typically populated in the due course of an application's performing its work, not by explicit request of the user. For this reason, I believe ebuild repositories do *not* belong in /var/cache. There is in fact no FHS-approved location for "system-wide, architecture-independent, shareable data files, fetched/synchronized from the network by a system application only upon user request." We can either invent something or just reuse "/var/db".


3. Are distfiles "cache"?

* Yes, distfiles are cache:

- * They need not be backed up.
- * They are automatically (re)populated as needed.

* No, distfiles are not cache:

- * They cannot be repopulated without an Internet connection.
- * Fetch-restricted distfiles are not automatically repopulated.
- * Very old distfiles may no longer be available for download from the mirrors.

My opinion: Unlike ebuilds, distfiles are automatically populated by Portage in the due course of its activities. Individual distfiles can be deleted, and they will be automatically replaced as needed. This is very much the expected operation of a cache. Continuous Internet connectivity is already nearly ubiquitous and is becoming more so over time, so I'm inclined to disregard FHS's "locally generated" requirement.

N.B.: Fetch-restricted distfiles can be placed in ${PORTAGE_RO_DISTDIR} if the user wishes to treat them as "not cache."


4. Should the master Portage tree be at the same depth as the overlays?

* Yes:

- * The master Portage tree is simply the lowest-priority "overlay."
- * In the new(ish) "repos.conf" configuration, the concept of "overlays" as distinct from ebuild repositories is disappearing. Really, the master Gentoo repository is just another ebuild repository.
- * The root of each repository should be a directory whose name matches the repo name.

* No:

- * A stable system, out of the box, will have only the master Gentoo repository.
- * The main Portage tree has historically been "special" — but this is no longer strictly the case; repositories can specify a master other than "gentoo".
- * Catalyst will have a problem if the main tree is not in a directory named "portage".

My opinion: The master Gentoo repository is simply an ebuild repository, and thus it should be contained in a directory containing only ebuild repositories. The name of the parent directory containing these repositories should be "repos" (not "repositories"), to agree with "/etc/portage/repos.conf".

N.B.: Catalyst needs to be fixed.


Conclusions:

* /var/cache/portage/{distfiles,packages}
* /var/db/portage/repos/{gentoo,local,sunrise,kde,…}

Optional symlinks (for those who like shorter paths or need backward compatibility):
* /usr/portage -> /var/db/portage/repos/gentoo
* /usr/local/portage -> /var/db/portage/repos/local
Comment 24 Zac Medico gentoo-dev 2018-08-04 20:04:17 UTC
From https://projects.gentoo.org/council/meeting-logs/20180729-summary.txt:

> Default locations for repositories, distfiles, and binpkgs
> ==========================================================
> There was a brief discussion about the color of /var/db vs /var/lib,
> and if /var/cache is suitable for binary packages.
> 
> - Vote: Default locations for the Gentoo repository, distfiles, and
>   binary packages will be, respectively:
>      /var/db/repos/gentoo
>      /var/cache/distfiles
>      /var/cache/binpkgs
>   Accepted with 6 yes votes and 1 no vote.
> 
> - Vote: Change the name of the Gentoo repository snapshot (currently
>   portage-YYYYMMDD) to gentoo-YYYYMMDD.
>   Accepted unanimously.
Comment 25 Matt Whitlock 2018-08-05 00:23:39 UTC
(In reply to Zac Medico from comment #24)
> > - Vote: Default locations for the Gentoo repository, distfiles, and
> >   binary packages will be, respectively:
> >      /var/db/repos/gentoo
> >      /var/cache/distfiles
> >      /var/cache/binpkgs
> >   Accepted with 6 yes votes and 1 no vote.

Hallelujah! It's interesting that none of the new paths contain the word "portage". Is Gentoo setting up to distance itself from this venerable but crusty package manager?

FWIW, I've been using these paths since Comment #23:

/var/db/portage/repos/gentoo
/var/cache/portage/distfiles
/var/cache/portage/packages

So the new paths are almost identical except for dropping the "portage" level, which I'm fine to do.

Amazing to see this "bug" finally addressed after 7 years. Congratulations to everyone involved.
Comment 26 Zac Medico gentoo-dev 2018-08-05 02:22:30 UTC
(In reply to Matt Whitlock from comment #25)
> (In reply to Zac Medico from comment #24)
> > > - Vote: Default locations for the Gentoo repository, distfiles, and
> > >   binary packages will be, respectively:
> > >      /var/db/repos/gentoo
> > >      /var/cache/distfiles
> > >      /var/cache/binpkgs
> > >   Accepted with 6 yes votes and 1 no vote.
> 
> Hallelujah! It's interesting that none of the new paths contain the word
> "portage". Is Gentoo setting up to distance itself from this venerable but
> crusty package manager?

Yeah the idea is that portage is just one implementation of PMS:

https://wiki.gentoo.org/wiki/Project:Package_Manager_Specification

> FWIW, I've been using these paths since Comment #23:
> 
> /var/db/portage/repos/gentoo
> /var/cache/portage/distfiles
> /var/cache/portage/packages
> 
> So the new paths are almost identical except for dropping the "portage"
> level, which I'm fine to do.
> 
> Amazing to see this "bug" finally addressed after 7 years. Congratulations
> to everyone involved.

Yeah it's hard to reach a consensus for this sort of thing, so there needed to be a generally agreeable proposal for the council to vote on.
Comment 28 Ulrich Müller gentoo-dev 2019-02-16 15:15:54 UTC
Any progress here?
Comment 29 Larry the Git Cow gentoo-dev 2019-04-19 04:47:37 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0c25fce6a579216092e321b33ca7d529df59eb56

commit 0c25fce6a579216092e321b33ca7d529df59eb56
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2019-04-19 04:43:12 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2019-04-19 04:46:57 +0000

    sys-apps/portage: version bump to 2.3.64
    
    Bug 378603 - New (council approved) default locations for the Gentoo
    repository, distfiles, and binary packages (does not apply to installed
    systems using the old defaults):
    
      /usr/portage -> /var/db/repos/gentoo
      /usr/portage/distfiles -> /var/cache/distfiles
      /usr/portage/packages -> /var/cache/binpkgs
    
    Bug: https://bugs.gentoo.org/378603
    Bug: https://bugs.gentoo.org/683434
    Package-Manager: Portage-2.3.64, Repoman-2.3.12
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-2.3.64.ebuild | 258 +++++++++++++++++++++++++++++++++
 2 files changed, 259 insertions(+)
Comment 30 Larry the Git Cow gentoo-dev 2019-04-19 04:50:46 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=1471339d008bb42ce9b727ad0bf7aa354797d712

commit 1471339d008bb42ce9b727ad0bf7aa354797d712
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2019-04-19 04:49:57 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2019-04-19 04:50:19 +0000

    sys-apps/portage: update 9999 with backward compat for bug 378603
    
    Bug: https://bugs.gentoo.org/378603
    Package-Manager: Portage-2.3.64, Repoman-2.3.12
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 sys-apps/portage/portage-9999.ebuild | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)
Comment 31 Larry the Git Cow gentoo-dev 2019-04-19 04:51:24 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=a859556ffaa4b3f55123215a269228f2ef89ee21

commit a859556ffaa4b3f55123215a269228f2ef89ee21
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-08-06 00:07:35 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2019-04-18 17:52:54 +0000

    Update /usr/portage references (bug 378603)
    
    Update all relevant references in docs, messages, and comments
    to refer to /var/db/repos/gentoo instead of /usr/portage. Also
    update DISTDIR and PKGDIR references to refer to the new
    /var/cache/{distfiles,binpkgs} locations.
    
    Bug: https://bugs.gentoo.org/378603
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 cnf/make.conf.example                            | 16 +++++-------
 lib/portage/__init__.py                          |  6 ++---
 lib/portage/cache/flat_hash.py                   |  4 +--
 lib/portage/tests/news/test_NewsItem.py          |  4 +--
 lib/portage/tests/resolver/ResolverPlayground.py |  4 +--
 lib/portage/xml/metadata.py                      |  6 ++---
 man/ebuild.5                                     |  9 ++++---
 man/emerge.1                                     |  8 +++---
 man/make.conf.5                                  | 26 ++++++++-----------
 man/portage.5                                    | 32 ++++++++++++------------
 man/quickpkg.1                                   |  4 +--
 repoman/lib/repoman/__init__.py                  |  4 +--
 12 files changed, 57 insertions(+), 66 deletions(-)

https://gitweb.gentoo.org/proj/portage.git/commit/?id=ef8469befbd7b61f1bae8f21ffd95e0c0306a4f8

commit ef8469befbd7b61f1bae8f21ffd95e0c0306a4f8
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-08-05 21:30:57 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2019-04-18 17:52:54 +0000

    Change /usr/portage council approved locations (bug 378603)
    
    This includes a _compat_upgrade.default_locations script that the
    ebuild can call in pkg_preinst in order to maintain backward-compatible
    defaults when appropriate. The new defaults are specified in the
    summary of the 20180729 council meeting:
    
    Vote: Default locations for the Gentoo repository, distfiles, and
    binary packages will be, respectively:
       /var/db/repos/gentoo
       /var/cache/distfiles
       /var/cache/binpkgs
    Accepted with 6 yes votes and 1 no vote.
    
    See: https://projects.gentoo.org/council/meeting-logs/20180729-summary.txt
    Bug: https://bugs.gentoo.org/378603
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 cnf/make.globals                                 |  4 +-
 cnf/repos.conf                                   |  2 +-
 lib/portage/_compat_upgrade/__init__.py          |  0
 lib/portage/_compat_upgrade/default_locations.py | 82 ++++++++++++++++++++++++
 4 files changed, 85 insertions(+), 3 deletions(-)
Comment 32 Matt Whitlock 2019-04-19 05:08:14 UTC
In make.conf.example:

 #     will protect the default locations of DISTDIR and PKGDIR, but users are
 #     warned that any other locations inside PORTDIR are not necessarily safe
 #     for data storage.

This passage no longer makes sense, as the default locations of DISTDIR and PKGDIR are not within the default location of PORTDIR.

For that matter, should PORTDIR still be described at all? Hasn't it been replaced by repos.conf?
Comment 33 Larry the Git Cow gentoo-dev 2019-04-19 05:16:36 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=cde445abb6d3bcc357355589c4d6d9365069d247

commit cde445abb6d3bcc357355589c4d6d9365069d247
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2019-04-19 05:14:57 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2019-04-19 05:16:10 +0000

    cnf/make.conf.example: remove comment about DISTDIR and PKGDIR in PORTDIR
    
    Reported-by: Matt Whitlock <gentoo@mattwhitlock.name>
    Bug: https://bugs.gentoo.org/378603#c32
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 cnf/make.conf.example | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)
Comment 34 Zac Medico gentoo-dev 2019-04-19 05:19:18 UTC
(In reply to Matt Whitlock from comment #32)
> For that matter, should PORTDIR still be described at all? Hasn't it been
> replaced by repos.conf?

It's still supported. We'll have to make sure that catalyst (stage builder) works without it before we drop support entirely.
Comment 35 Zac Medico gentoo-dev 2019-05-02 00:50:07 UTC
(In reply to Zac Medico from comment #34)
> (In reply to Matt Whitlock from comment #32)
> > For that matter, should PORTDIR still be described at all? Hasn't it been
> > replaced by repos.conf?
> 
> It's still supported. We'll have to make sure that catalyst (stage builder)
> works without it before we drop support entirely.

I see that catalyst still uses PORTDIR as seen here:

https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=c52962b6bd2e12dec9fdaea6592d39bedce0a752

The code has moved around and it's here now:

https://gitweb.gentoo.org/proj/catalyst.git/tree/catalyst/base/stagebase.py?h=3.0.4#n1171
Comment 36 Zac Medico gentoo-dev 2019-05-02 02:02:58 UTC
Filed bug 684908 for catalyst.