Hi Daniel, Just a couple of thoughts about portage allowing the user to have a set of customisation layers. Following on the idea of /etc/make.conf to allow local settings to over-ride those specified in the make.globals file, here are two ideas. First, this is a followup on the /usr/local/portage idea that was brought up on -core a few months ago, and that you and I very briefly talked about in -dev earlier this week (with respect to emerge rsync doing a --clean by default). Essentially the idea I'm thinking is to have a replica of the categories in there so that users can have their own custom ebuilds sitting in portage, without fear of losing them. This is especially important, since portage no longer merges and ebuild from outside the portage tree. For my own purposes, I used to appreciate portage installing my custom ebuilds from my ~/ebuildstuff directory. A /usr/local/portage would be better, I think, though. The second idea is to have a file which over-rides package.mask. The idea behind this is that some users like to test things. Inadvertantly it happens that one would do `emerge rsync; emerge -u world` and install the older version accidentally. I think this file would be useful from the developers' point of view as well, since most of the devs are into testing things in the package.mask file as well. An accidental commit of that file (and we're _all_ guilty of having done that at least once) affects everyone else, whereas overriding it just locally leaves the fragile package.mask file untouched except for when it really needs to be touched. The more dangerous option in packages.mask is the unmasking of broken/testing things, rather than the masking of things.
*** Bug 5648 has been marked as a duplicate of this bug. ***
Related to the package.mask problems: Below is a list of scenarios I've noticed (not necessarily experienced myself) related to package.mask: Scenario #1: I tell one of my faithful testers to test the latest dev-foo/bar, he comments out its mask in package.mask, emerge the package, reports back to me, I issue a fix, he rsyncs, emerges again, whoops, the package is suddenly masked again, bummer! Scenario #2: Spider comes along with GNOME2, it's a big beast, and he wants it tested. I want to help him since he's such a nice guy, so I go through package.mask, try to find all GNOME2 related masks (no mean feat, unless Spider is given temporary leave not to add each and every new mask at the top of our package.mask file), emerge the thing, report back bugs, etc, etc, emerge rsync, masked again, bummer! Scenario #3: I'm in a hurry, I do cvs up profiles/package.mask, add my package on the top, commit and leave the house. What I failed to notice was that the package.mask contained conflicts. In less than 15 minutes, many of our users will notice that something is not right, they'll be confused, angry and annoyed, they'll yell at their spouses and there will only be cold dinner for them for the rest of the week. Scenario #4: I'm a ppc dev, I've just fixed a dev-foo/bar-1.0 so that it compiles and apparently works on the ppc. I add the ppc keyword, and put it in package.mask. Down the road, riots break out among a group of x86 fanatics, as dev-foo/bar-1.0, which has been stable for two months is suddenly being masked for no good reason. I think all of these scenarios, none of which are far-fetched, tells us that we need to reconsider the way we use package.mask. I'd suggest we try to collect a new set of scenarios for how we want this to work. Forget about the implementation details for a bit, let's concentrate on _what_ we want to achieve before we decide _how_ to achieve it. My initial scenarios suggstion: Scenario #1: I work on a package that doesn't compile, is written by a bunch of air-headed academics which do not respond to emails about their sordid build system, so I want to stash it into CVS while I work on something else. I mark the package as "broken - Does not compile" and get on with my work. Scenario #2: I have package that I want to have tested; I know it compiles on x86, but not on any other platforms. I mark it "ready for testing on x86", and "broken - does not compile" on the rest. Scenario #3: I have a package that works nicely on x86, I know it compiles on ppc, it is marked as "broken - does not compile" on sparc. I need to have it tested. I mark it as "ready for testing on ppc", and I expect that doing so doesn't have any impact on the x86 users, nor that its status as "broken" on sparc changes. Scenario #4: I am working on KDE 3.1. I know that many of the packages work, but need testing. They have a lot of interdependencies, so I have to bundle them somehow, and mark the bundle as "ready for testing on <arch>". However, I don't want to mark all the packages as "ready for testing", as some of them don't even compile. I want those to remain "broken - do not compile". Scenario #5: I'm a KDE-fan. I know that Dan is working KDE 3.1, and I want to be help him test it. I say "All KDE 3.1 stuff marked testing is okay to merge", and do emerge -up to see what goodies lie on store for me today. This is definitely a big problem that I don't expect will be solved in one go. Various previous suggestions I've seen on #gentoo-dev could be combined into solving most of these problems, but it'll take careful, step-wise deployment if we're going to pull it off smoothly. Simple, highly effective means to fix some of these problems would be: 1) Have a package.mask verifier that checked for good and proper syntax and semantics before package.mask was checked in. 2) Split package.mask into multiple files in a separate directory, say /usr/portage/profiles/mask/, then have an allow-file in that dir which listed mask-files and individual packages that the user explicitly wants to allow; say we have mask/kde-3.1-testing which lists all kde-3.1 stuff that is deemed ready for testing. By adding kde-3.1-testing to his allow file, the user would then allow all packages listed in kde-3.1-testing.
OK, one step at a time. beta PORTDIR_OVERLAY (local Portage tree) support is now in Portage 2.0.21. As for the other stuff, yes, we need to fix all of this. I've been wanting to fix it for a long while. A rework of how the masks are organized will probably be added to Portage at the same time we restructure the files in /etc: /etc/portage/ /etc/portage/globals /etc/portage/config /etc/portage/profile -> (link to profile) /etc/portage/mask.local (local masking information) We will likely add some modular masking functionality: /usr/portage/profiles/mask/ /usr/portage/profiles/profilename/mask/ directories will include names like "gnome", "kde-4", etc. Then users will be able to selectively enable/disable these masks using a cumulative variable in their config file: ACCEPT_MASKS="-kde-4" (to unmask kde-4 for testing, etc.) I have several portage features I need to add first before doing all this, however. (extended USE variables needed by our alternate architecture guys, etc.)
*** Bug 5754 has been marked as a duplicate of this bug. ***
*** Bug 7814 has been marked as a duplicate of this bug. ***
*** Bug 8037 has been marked as a duplicate of this bug. ***
*** Bug 10431 has been marked as a duplicate of this bug. ***
*** Bug 11772 has been marked as a duplicate of this bug. ***
It's great news to here there is something like that coming. Karl's great description of scenarios probably describes all cases. From a user's point of view (my view :-) overriding masks is probably the most needed one. Currently the packages with marked as ~arch are the most annoying. I have about 6 Packages I want to have current. Right now I have to edit various ebuilds and remove the '~' prior to be able to -u world especially since Gentoo is growing at this incredible speed. The ability to accept '~' for certain packages but not globally would be really great. Please note that I don't want to push anything. I'm glad a thing like that is already planned!
As a sidenote to this, please consider to not mask things that actually do compile/work in /usr/portage/profiles/package.mask. To quote some examples: > # <pauldv@gentoo.org> (19 Jun 2003) > # It is a beta version > =app-office/openoffice-bin-1.1_beta2 Fine, why not put KEYWORDS="-arch" into the ebuild, if "~arch" doesn't seem to be enough (I run OOo-1.1_beta2 fine btw) > <gnome@gentoo.org> (some time ago) > # There are development versions. Dev versions are not right for portage. > # Do not remove this mask without consulting the gnome team. > >=media-gfx/gimp-1.3 > >=app-office/gnumeric-1.1.3 Uhm, sorry, dev versions aren't right for portage? Why do we have the ~arch flag after all? To rephrase: I don't think that the package.maks-file is a bad thing, as it makes developers life easier. But I do think that it is used far to often. I rarely see the "-arch" flag, only saw it in OpenOffice.org-source-1.1-ebuilds, though I understood right away that I wasn't meant to use the ebuild (and instead downloaded a binary from upstream and installed to /usr/local). Compared to that, the package.mask is counterintuitive and clumsy. It really should only be used in case of emergency (that is, if something is released and has to be revoked ASAP), not as a general catch all file. Most users I know, even experienced ones, don't even know of it's existence; they go insane searching for the spot where the software they want is masked (they usually search at /etc/make.profile/ and inside the ebuild itself). By masking in the ebuild using "-arch", you still can stop unstable-freaks from auto-installing bad things AND give the user the ability to permanentely override this by cp'ing over to /usr/local/portage and unmasking there WITHOUT making him use some obscure scripts that actually sabotage the whole package.mask idea.
*** Bug 17544 has been marked as a duplicate of this bug. ***
*** Bug 22522 has been marked as a duplicate of this bug. ***
*** Bug 24554 has been marked as a duplicate of this bug. ***
the new /etc/portage/package.{mask,unmask} stuff is definitely a step towards this already.
Status update: - PORTDIR_OVERLAY is there for a long time - /etc/portage/package.{mask,unmask} overriding package.mask is there since 2.0.48 - /etc/portage/categories for creating custom categories is there since 2.0.49 - /etc/portage/package.keywords will likely come in 2.0.50 or 51 - still missing is some way to override profile settings, e.g. there is no way to mask/remove/unmask things in /etc/make.profile/packages without creating your own profile. anything I missed that is needed (other that the current system is very complex and hard to understand for non-devs) ?
*** Bug 38402 has been marked as a duplicate of this bug. ***
Ok, every mask can be overridden now -> closing.