Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 5622 - enhanced, modular, flexible, locally customizable masking functionality
Summary: enhanced, modular, flexible, locally customizable masking functionality
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Unclassified (show other bugs)
Hardware: x86 Linux
: High normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
: 5648 5754 7814 8037 10431 11772 17544 22522 24554 38402 (view as bug list)
Depends on:
Blocks: 6189
  Show dependency tree
 
Reported: 2002-07-26 18:07 UTC by Seemant Kulleen (RETIRED)
Modified: 2022-07-24 03:58 UTC (History)
22 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Seemant Kulleen (RETIRED) gentoo-dev 2002-07-26 18:07:48 UTC
Hi Daniel,

Just a couple of thoughts about portage allowing the user to have a set of
customisation layers.  Following on the idea of /etc/make.conf to allow local
settings to over-ride those specified in the make.globals file, here are two ideas.

First, this is a followup on the /usr/local/portage idea that was brought up on
-core a few months ago, and that you and I very briefly talked about in -dev
earlier this week (with respect to emerge rsync doing a --clean by default). 
Essentially the idea I'm thinking is to have a replica of the categories in
there so that users can have their own custom ebuilds sitting in portage,
without fear of losing them.  This is especially important, since portage no
longer merges and ebuild from outside the portage tree.  For my own purposes, I
used to appreciate portage installing my custom ebuilds from my ~/ebuildstuff
directory.  A /usr/local/portage would be better, I think, though.


The second idea is to have a file which over-rides package.mask.  The idea
behind this is that some users like to test things.  Inadvertantly it happens
that one would do `emerge rsync; emerge -u world` and install the older version
accidentally.

I think this file would be useful from the developers' point of view as well,
since most of the devs are into testing things in the package.mask file as well.
 An accidental commit of that file (and we're _all_ guilty of having done that
at least once) affects everyone else, whereas overriding it just locally leaves
the fragile package.mask file untouched except for when it really needs to be
touched.  The more dangerous option in packages.mask is the unmasking of
broken/testing things, rather than the masking of things.
Comment 1 Seemant Kulleen (RETIRED) gentoo-dev 2002-07-27 18:41:17 UTC
*** Bug 5648 has been marked as a duplicate of this bug. ***
Comment 2 Karl Trygve Kalleberg (RETIRED) gentoo-dev 2002-07-28 15:56:55 UTC
Related to the package.mask problems:

Below is a list of scenarios I've noticed (not necessarily experienced
myself) related to package.mask:

Scenario #1:
I tell one of my faithful testers to test the latest dev-foo/bar, he
comments out its mask in package.mask, emerge the package, reports back
to me, I issue a fix, he rsyncs, emerges again, whoops, the package is
suddenly masked again, bummer!

Scenario #2:
Spider comes along with GNOME2, it's a big beast, and he wants it
tested. I want to help him since he's such a nice guy, so I go through
package.mask, try to find all GNOME2 related masks (no mean feat,
unless Spider is given temporary leave not to add each and every new
mask at the top of our package.mask file), emerge the thing, report
back bugs, etc, etc, emerge rsync, masked again, bummer!

Scenario #3:
I'm in a hurry, I do cvs up profiles/package.mask, add my package on the
top, commit and leave the house. What I failed to notice was that the
package.mask contained conflicts. In less than 15 minutes, many of our
users will notice that something is not right, they'll be confused, angry
and annoyed, they'll yell at their spouses and there will only be cold
dinner for them for the rest of the week.

Scenario #4:
I'm a ppc dev, I've just fixed a dev-foo/bar-1.0 so that it compiles and
apparently works on the ppc. I add the ppc keyword, and put it in
package.mask. Down the road, riots break out among a group of x86 fanatics,
as dev-foo/bar-1.0, which has been stable for two months is suddenly being
masked for no good reason.


I think all of these scenarios, none of which are far-fetched, tells us that
we need to reconsider the way we use package.mask.

I'd suggest we try to collect a new set of scenarios for how we want this to
work. Forget about the implementation details for a bit, let's concentrate on
_what_ we want to achieve before we decide _how_ to achieve it.

My initial scenarios suggstion:

Scenario #1:
I work on a package that doesn't compile, is written by a bunch of air-headed
academics which do not respond to emails about their sordid build system, so
I want to stash it into CVS while I work on something else. I mark the package
as "broken - Does not compile" and get on with my work.

Scenario #2:
I have package that I want to have tested; I know it compiles on x86, but not
on any other platforms. I mark it "ready for testing on x86", and "broken -
does not compile" on the rest.

Scenario #3:
I have a package that works nicely on x86, I know it compiles on ppc, it
is marked as "broken - does not compile" on sparc. I need to have it
tested. I mark it as "ready for testing on ppc", and I expect that doing
so doesn't have any impact on the x86 users, nor that its status as
"broken" on sparc changes.

Scenario #4:
I am working on KDE 3.1. I know that many of the packages work, but need
testing. They have a lot of interdependencies, so I have to bundle them
somehow, and mark the bundle as "ready for testing on <arch>". However, I
don't want to mark all the packages as "ready for testing", as some of them
don't even compile. I want those to remain "broken - do not compile".

Scenario #5:
I'm a KDE-fan. I know that Dan is working KDE 3.1, and I want to be help
him test it. I say "All KDE 3.1 stuff marked testing is okay to merge",
and do emerge -up to see what goodies lie on store for me today.

This is definitely a big problem that I don't expect will be solved in one go.
Various previous suggestions I've seen on #gentoo-dev could be combined into
solving most of these problems, but it'll take careful, step-wise deployment
if we're going to pull it off smoothly.

Simple, highly effective means to fix some of these problems would be:
1) Have a package.mask verifier that checked for good and proper syntax and
   semantics before package.mask was checked in.
2) Split package.mask into multiple files in a separate directory, say
   /usr/portage/profiles/mask/, then have an allow-file in that dir which
   listed mask-files and individual packages that the user explicitly wants
   to allow; say we have mask/kde-3.1-testing which lists all kde-3.1 stuff
   that is deemed ready for testing. By adding kde-3.1-testing to his
   allow file, the user would then allow all packages listed in
   kde-3.1-testing.
Comment 3 Daniel Robbins (RETIRED) gentoo-dev 2002-07-29 23:56:31 UTC
OK, one step at a time.  beta PORTDIR_OVERLAY (local Portage tree) support is
now in Portage 2.0.21.  As for the other stuff, yes, we need to fix all of this.
 I've been wanting to fix it for a long while.  A rework of how the masks are
organized will probably be added to Portage at the same time we restructure the
files in /etc:

/etc/portage/
/etc/portage/globals
/etc/portage/config
/etc/portage/profile -> (link to profile)
/etc/portage/mask.local (local masking information)

We will likely add some modular masking functionality:

/usr/portage/profiles/mask/
/usr/portage/profiles/profilename/mask/

directories will include names like "gnome", "kde-4", etc.

Then users will be able to selectively enable/disable these masks using a
cumulative variable in their config file:

ACCEPT_MASKS="-kde-4" (to unmask kde-4 for testing, etc.)

I have several portage features I need to add first before doing all this,
however. (extended USE variables needed by our alternate architecture guys, etc.)
Comment 4 Daniel Robbins (RETIRED) gentoo-dev 2002-07-29 23:58:57 UTC
*** Bug 5754 has been marked as a duplicate of this bug. ***
Comment 5 Seemant Kulleen (RETIRED) gentoo-dev 2002-09-11 15:16:44 UTC
*** Bug 7814 has been marked as a duplicate of this bug. ***
Comment 6 Tom von Schwerdtner 2002-09-17 16:11:58 UTC
*** Bug 8037 has been marked as a duplicate of this bug. ***
Comment 7 SpanKY gentoo-dev 2002-11-08 22:51:48 UTC
*** Bug 10431 has been marked as a duplicate of this bug. ***
Comment 8 SpanKY gentoo-dev 2003-01-08 14:59:01 UTC
*** Bug 11772 has been marked as a duplicate of this bug. ***
Comment 9 Richard Stevens 2003-04-17 04:27:48 UTC
It's great news to here there is something like that coming. Karl's great description of scenarios 
probably describes all cases. From a user's point of view (my view :-) overriding masks is probably 
the most needed one. Currently the packages with marked as ~arch are the most annoying. I have 
about 6 Packages I want to have current. Right now I have to edit various ebuilds and remove the 
'~' prior to be able to -u world especially since Gentoo is growing at this incredible speed. The ability 
to accept '~' for certain packages but not globally would be really great.  
 
Please note that I don't want to push anything. I'm glad a thing like that is already planned! 
Comment 10 Thomas R. (TRauMa) 2003-06-21 14:18:44 UTC
As a sidenote to this, please consider to not mask things that actually do compile/work in /usr/portage/profiles/package.mask.

To quote some examples:

> # <pauldv@gentoo.org> (19 Jun 2003)
> # It is a beta version
> =app-office/openoffice-bin-1.1_beta2

Fine, why not put KEYWORDS="-arch" into the ebuild, if "~arch" doesn't seem to be enough (I run OOo-1.1_beta2 fine btw)

>  <gnome@gentoo.org> (some time ago)
> # There are development versions. Dev versions are not right for portage.
> # Do not remove this mask without consulting the gnome team. 
> >=media-gfx/gimp-1.3
> >=app-office/gnumeric-1.1.3

Uhm, sorry, dev versions aren't right for portage? Why do we have the ~arch flag after all?

To rephrase: I don't think that the package.maks-file is a bad thing, as it makes developers life easier. But I do think that it is used far to often. I rarely see the "-arch" flag, only saw it in OpenOffice.org-source-1.1-ebuilds, though I understood right away that I wasn't meant to use the ebuild (and instead downloaded a binary from upstream and installed to /usr/local). Compared to that, the package.mask is counterintuitive and clumsy. It really should only be used in case of emergency (that is, if something is released and has to be revoked ASAP), not as a general catch all file. Most users I know, even experienced ones, don't even know of it's existence; they go insane searching for the spot where the software they want is masked (they usually search at /etc/make.profile/ and inside the ebuild itself).
By masking in the ebuild using "-arch", you still can stop unstable-freaks from auto-installing bad things AND give the user the ability to permanentely override this by cp'ing over to /usr/local/portage and unmasking there WITHOUT making him use some obscure scripts that actually sabotage the whole package.mask idea.
Comment 11 SpanKY gentoo-dev 2003-07-09 11:57:46 UTC
*** Bug 17544 has been marked as a duplicate of this bug. ***
Comment 12 SpanKY gentoo-dev 2003-07-09 12:30:11 UTC
*** Bug 22522 has been marked as a duplicate of this bug. ***
Comment 13 SpanKY gentoo-dev 2003-07-15 17:17:54 UTC
*** Bug 24554 has been marked as a duplicate of this bug. ***
Comment 14 Seemant Kulleen (RETIRED) gentoo-dev 2003-08-23 13:32:37 UTC
the new /etc/portage/package.{mask,unmask} stuff is definitely a step towards this already.
Comment 15 Marius Mauch (RETIRED) gentoo-dev 2003-09-25 23:28:57 UTC
Status update:
- PORTDIR_OVERLAY is there for a long time
- /etc/portage/package.{mask,unmask} overriding package.mask is there since
2.0.48
- /etc/portage/categories for creating custom categories is there since 2.0.49
- /etc/portage/package.keywords will likely come in 2.0.50 or 51
- still missing is some way to override profile settings, e.g. there is no
way to
  mask/remove/unmask things in /etc/make.profile/packages without creating
your
  own profile.
anything I missed that is needed (other that the current system is very complex
and hard to understand for non-devs) ?
Comment 16 Marius Mauch (RETIRED) gentoo-dev 2004-01-16 07:37:32 UTC
*** Bug 38402 has been marked as a duplicate of this bug. ***
Comment 17 Marius Mauch (RETIRED) gentoo-dev 2004-04-01 01:35:40 UTC
Ok, every mask can be overridden now -> closing.