Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 282296 - [Future EAPI] Allow directories for use.* and package.* entries in profiles
Summary: [Future EAPI] Allow directories for use.* and package.* entries in profiles
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Hosted Projects
Classification: Unclassified
Component: PMS/EAPI (show other bugs)
Hardware: All Linux
: High enhancement with 1 vote (vote)
Assignee: PMS/EAPI
URL:
Whiteboard: in-eapi-7
Keywords:
Depends on:
Blocks: future-eapi
  Show dependency tree
 
Reported: 2009-08-22 10:56 UTC by Robert Buchholz (RETIRED)
Modified: 2018-04-30 22:46 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
PMS patch (0001-EAPI-6-Add-support-for-package.-directories.patch,4.11 KB, patch)
2014-05-11 19:21 UTC, Michael Palimaka (kensington)
Details | Diff
[PATCH 1/2] EAPI 7 supports a directory for package.mask. (0001-EAPI-7-supports-a-directory-for-package.mask.patch,2.98 KB, patch)
2015-11-30 15:47 UTC, Ulrich Müller
Details | Diff
[PATCH 2/2] EAPI 7 supports directories for profile files. (0002-EAPI-7-supports-directories-for-profile-files.patch,5.35 KB, patch)
2015-11-30 15:47 UTC, Ulrich Müller
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Buchholz (RETIRED) gentoo-dev 2009-08-22 10:56:46 UTC
Any file in the profiles/ directory of a repository that begins with "package." or "use." can either be a flat file or a directory. If it is a directory, then all the files in that directory will be sorted in ascending alphabetical order by file name and summed together as if it were a single file.

Currently this affects the following type of files:
package.mask
package.provided
package.use
package.use.force
package.use.mask
use.desc
use.force
use.local.desc
use.mask
Comment 1 Christian Faulhammer (RETIRED) gentoo-dev 2009-08-24 19:37:48 UTC
This is a relativly small change that nonetheless should make it into EAPI 4, as it makes behaviour consistent.
Comment 2 Robert Buchholz (RETIRED) gentoo-dev 2009-08-24 22:34:14 UTC
We should probably create a list of files where this behaviour is wanted instead of using a glob. In that we can strike use.desc and use.local.desc from that list. Many of the other files are mostly tiny, but I see no reason not to include them. I'd leave this up to PMS or PM devs to decide in a refinement step.
Comment 3 Arfrever Frehtes Taifersar Arahesis 2012-09-06 07:57:27 UTC
Suggested list:

  package.mask
  use.force
  use.stable.force
  use.mask
  use.stable.mask
  package.use
  package.use.force
  package.use.stable.force
  package.use.mask
  package.use.stable.mask
Comment 4 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-08-30 11:03:00 UTC
What's the rationale/use case?
Comment 5 Tom Wijsman (TomWij) (RETIRED) gentoo-dev 2013-08-30 12:22:15 UTC
(In reply to Michał Górny from comment #4)
> What's the rationale/use case?

Having multiple groups of packages masked in an overlay, currently they all need to be in the same file; by putting them in separate files you can more easily instruct an user to create a symlink to package.mask/gnome4 from /etc/portage/package.unmask/gnome4.

Currently, you would need to keep separate files in a location that's not PMS compatible and manually generate them into a single package.mask with one or another tool to combine them (or well, perhaps just a `cat * > package.mask`) before you commit after having changed the mask.

If this were consistent with how /etc/ works, the extra deviating work and location would be unnecessary; less work and figuring out for the developer, less oddity towards the user, better workflow, etc...

I'm not so sure how much this makes sense for the other files; but at least for package.mask, this makes sense and would be very appreciated!
Comment 6 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-08-30 14:48:56 UTC
(In reply to Tom Wijsman (TomWij) from comment #5)
> (In reply to Michał Górny from comment #4)
> > What's the rationale/use case?
> 
> Having multiple groups of packages masked in an overlay, currently they all
> need to be in the same file; by putting them in separate files you can more
> easily instruct an user to create a symlink to package.mask/gnome4 from
> /etc/portage/package.unmask/gnome4.

  diffmask -a gnome-something

and then:

  diffmask -u

in postsync.d to update the masks.

Honestly, I don't really see us doing this. We will either end up with some project using one way and some another, or with a few dozens of files for each of the files.

Plus, the good idea of keeping newest masks on the top will no longer be possible since masks will be spread over a few files.
Comment 7 Tom Wijsman (TomWij) (RETIRED) gentoo-dev 2013-08-30 16:14:22 UTC
(In reply to Michał Górny from comment #6)
> (In reply to Tom Wijsman (TomWij) from comment #5)
> > (In reply to Michał Górny from comment #4)
> > > What's the rationale/use case?
> > 
> > Having multiple groups of packages masked in an overlay, currently they all
> > need to be in the same file; by putting them in separate files you can more
> > easily instruct an user to create a symlink to package.mask/gnome4 from
> > /etc/portage/package.unmask/gnome4.
> 
>   diffmask -a gnome-something
> 
> and then:
> 
>   diffmask -u
> 
> in postsync.d to update the masks.

This isn't a solution to this problem at all, eg. if I add or remove a file to package.mask/gnome4, then the user won't have this change with your proposal.

> Honestly, I don't really see us doing this.

Saw one of the KDE overlays do this, I think, there might possible be more overlays that do or want to do this; and I plan to use my dev overlay a bit more, so, I want to use a more consistent approach there too if possible.

> We will either end up with some project using one way and some another, or with a few dozens of files for each of the files.

That's what we already have as a result of a lack of consistency; this proposal tends to deal with that, such that we no longer have to use inconsistent locations for these files.

> Plus, the good idea of keeping newest masks on the top will no longer be
> possible since masks will be spread over a few files.

I don't see the need to do keep them that way in my dev overlay; as for a project overlay, I don't see how it is going to be a too big problem there with the quite limited amount of packages that are actually listed in those files. This is not about the package.mask in the Portage tree.
Comment 8 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-08-30 16:27:46 UTC
(In reply to Tom Wijsman (TomWij) from comment #7)
> (In reply to Michał Górny from comment #6)
> > (In reply to Tom Wijsman (TomWij) from comment #5)
> > > (In reply to Michał Górny from comment #4)
> > > > What's the rationale/use case?
> > > 
> > > Having multiple groups of packages masked in an overlay, currently they all
> > > need to be in the same file; by putting them in separate files you can more
> > > easily instruct an user to create a symlink to package.mask/gnome4 from
> > > /etc/portage/package.unmask/gnome4.
> > 
> >   diffmask -a gnome-something
> > 
> > and then:
> > 
> >   diffmask -u
> > 
> > in postsync.d to update the masks.
> 
> This isn't a solution to this problem at all, eg. if I add or remove a file
> to package.mask/gnome4, then the user won't have this change with your
> proposal.

If you add it to the same block, 'diffmask -u' will add it (it syncs the block).
Comment 9 Tom Wijsman (TomWij) (RETIRED) gentoo-dev 2013-08-30 17:13:06 UTC
(In reply to Michał Górny from comment #8)
> If you add it to the same block, 'diffmask -u' will add it (it syncs the
> block).

From IRC I understood it is similar to `awk -vRS= /.../ ... > ...` on postsync.d.

Still, it would be more neat to just have files that the user can simply symlink; less work for them, less work for us, no matching accidents or breakage involved.
Comment 10 Andreas K. Hüttel gentoo-dev 2013-08-30 17:36:33 UTC
Yes please.
Comment 11 Michael Palimaka (kensington) gentoo-dev 2014-05-11 19:21:36 UTC
Created attachment 376730 [details, diff]
PMS patch

For simplicity, I have opted to support all simple line-based files as directories. This requires one small change to portage, which I will follow up with now.
Comment 12 Michael Palimaka (kensington) gentoo-dev 2014-05-12 10:15:24 UTC
(In reply to Michael Palimaka (kensington) from comment #11)
> This requires one small change to portage, which I will follow
> up with now.

This change is now in portage: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=b4d8e300c04b768be7cd5c64116d6cc0453219b4
Comment 13 Brian Dolbec gentoo-dev 2014-05-12 21:27:25 UTC
cc'ing catalyst@ since I will likely need to modify catalyst for packages.build as a directory.  I need to check, but for stage1 it does use portage's api to get the list.

the relevant code is here:
http://git.overlays.gentoo.org/gitweb/?p=proj/catalyst.git;a=blob;f=targets/stage1/build.py;h=bf20bcf3bd48a1a7f1ea7dc92e02f6855e53e2cb;hb=HEAD
Comment 14 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2014-05-13 01:55:09 UTC
Just to clarify as it was mentioned on IRC, I see no advantage to have packages.build as a dir. This is only used for catalyst to build stage1, the order of the packages is important and it will always be a limited list of packages.
Comment 15 Michael Palimaka (kensington) gentoo-dev 2014-05-13 10:07:37 UTC
Modifying catalyst is not necessary because packages.build is outside the scope of PMS.
Comment 16 Alexander Berntsen (RETIRED) gentoo-dev 2014-06-14 22:01:41 UTC
(In reply to Michael Palimaka (kensington) from comment #12)
> This change is now in portage:
> http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;
> h=b4d8e300c04b768be7cd5c64116d6cc0453219b4

This is now reverted by e3ecbad92c28d793a073019e7df81bb5ec84cc81.

The following does no longer traceback:
mkdir -p /etc/portage/make.profile/packages
emerge foo


In 55b2e1c0a51d64a84e5649c135cdbc2c650ce6ca, this is handled more gracefully.
Comment 18 Ulrich Müller gentoo-dev 2015-11-07 17:41:08 UTC
(In reply to Robert Buchholz (RETIRED) from comment #2)
> We should probably create a list of files where this behaviour is wanted
> instead of using a glob.
Comment 19 Ulrich Müller gentoo-dev 2015-11-07 18:31:27 UTC
(In reply to Robert Buchholz (RETIRED) from comment #2)
> We should probably create a list of files where this behaviour is wanted
> instead of using a glob.

I think we should restart this, beginning at comment #2 or even comment #0. AFAICS the intent is to officially sanction what is used in some overlays.

So, let's first try to get the facts straight.

On the PMS side, we strictly distinguish:
- The profiles/ directory, described in section 4.4:
  https://projects.gentoo.org/pms/5/pms.html#x1-350004.4
- Files that make up a profile, described in section 5.2:
  https://projects.gentoo.org/pms/5/pms.html#x1-460005.2

On the Portage side:
- IIUC, files that can be directories (or that could be, at the time this
  bug report was filed) are listed in the _PORTAGE1_DIRECTORIES variable,
  which is defined in LocationsManager.py. These files are:
  * in the profiles/ directory: package.mask
  * in a profile: package.mask, package.provided, package.use,
    package.use.mask, package.use.force, use.mask, use.force
    (Note that this doesn't include any EAPI 5 stable mask files.)
- What is documented in the portage(5) man page is somewhat different,
  namely that all package.* and use.* files can be directories.
  In addition to the above mentioned files this would include:
  * in the profiles/ dir: use.desc, use.local.desc
  * in a profile: package.use.stable.mask, package.use.stable.force,
    use.stable.mask, use.stable.force

Next, I would suggest that all interested parties should state which of the following files they currently use as a directory, or which they intend to use (and some rationale for it should be provided):

- profiles/package.mask
- profiles/use.desc, profiles/use.local.desc
- any other line-based file in the profiles/ dir

- package.mask (in a profile)
- package.provided
- package.use, package.use.mask, package.use.force,
  package.use.stable.mask, package.use.stable.force,
  use.mask, use.force, use.stable.mask, use.stable.force
- packages
- make.defaults
Comment 20 Ulrich Müller gentoo-dev 2015-11-09 23:56:05 UTC
If we start from the list of files (previously) supported by Portage:

>   * in the profiles/ directory: package.mask
>   * in a profile: package.mask, package.provided, package.use,
>     package.use.mask, package.use.force, use.mask, use.force

Then for consistency, we should also include the new files added in EAPI 5, namely: package.use.stable.mask, package.use.stable.force, use.stable.mask, use.stable.force.

package.provided is "strongly deprecated" (says PMS) so presumably we shouldn't add new features to it.

If we keep in mind that package.mask can occur both in the profiles/ dir and in a profile, we end up with exactly the same list as suggested in comment #3:

>   package.mask [profiles/ and profile]
>   use.force
>   use.stable.force
>   use.mask
>   use.stable.mask
>   package.use
>   package.use.force
>   package.use.stable.force
>   package.use.mask
>   package.use.stable.mask

So I propose that we go with this list of files.

Changes from the EAPI 6 patch in https://archives.gentoo.org/gentoo-pms/message/dfdb88571222ec0b52e8151d46b85f8e are:
- profiles/package.mask added
- packages removed
- packages.build removed (which Catalyst doesn't support as a dir anyway)
- package.provided removed
Comment 21 Ulrich Müller gentoo-dev 2015-11-30 15:47:37 UTC
Created attachment 418208 [details, diff]
[PATCH 1/2] EAPI 7 supports a directory for package.mask.

Patch for section 4.4 "The Profiles Directory", i.e. adding directory support for the profiles/package.mask file.
Comment 22 Ulrich Müller gentoo-dev 2015-11-30 15:47:43 UTC
Created attachment 418210 [details, diff]
[PATCH 2/2] EAPI 7 supports directories for profile files.

Patch for PMS section 5.2 "Files that make up a profile", i.e. adding directory support for the following files in a profile:

   package.mask
   package.use
   use.*
   package.use.*
Comment 23 Larry the Git Cow gentoo-dev 2018-03-11 11:43:58 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=7a5cf68420896ba51a8fda541b973f95278e17b3

commit 7a5cf68420896ba51a8fda541b973f95278e17b3
Author:     Michał Górny <mgorny@gentoo.org>
AuthorDate: 2018-02-21 07:50:04 +0000
Commit:     Michał Górny <mgorny@gentoo.org>
CommitDate: 2018-03-11 11:43:40 +0000

    Allow package.*, use.* directories in EAPI 7
    
    Bug: https://bugs.gentoo.org/282296

 pym/portage/eapi.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)}
Comment 24 Larry the Git Cow gentoo-dev 2018-04-30 22:14:05 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/pms.git/commit/?id=2a6a99b5dd4cb3860b69d40af3d1fa8748423e86

commit 2a6a99b5dd4cb3860b69d40af3d1fa8748423e86
Author:     Ulrich Müller <ulm@gentoo.org>
AuthorDate: 2015-11-29 23:01:54 +0000
Commit:     Ulrich Müller <ulm@gentoo.org>
CommitDate: 2018-03-30 16:35:43 +0000

    EAPI 7 supports directories for profile files.
    
    Bug: https://bugs.gentoo.org/282296

 eapi-differences.tex |  5 +++++
 profiles.tex         | 39 ++++++++++++++++++++++++++++++++-------
 2 files changed, 37 insertions(+), 7 deletions(-)

https://gitweb.gentoo.org/proj/pms.git/commit/?id=8b14dd115c4063b902d566afaf21ca8f961d01e3

commit 8b14dd115c4063b902d566afaf21ca8f961d01e3
Author:     Ulrich Müller <ulm@gentoo.org>
AuthorDate: 2015-11-29 23:00:32 +0000
Commit:     Ulrich Müller <ulm@gentoo.org>
CommitDate: 2018-03-30 16:35:43 +0000

    EAPI 7 supports a directory for package.mask.
    
    Bug: https://bugs.gentoo.org/282296

 eapi-differences.tex |  5 ++++-
 tree-layout.tex      | 20 ++++++++++++++++++++
 2 files changed, 24 insertions(+), 1 deletion(-)}