as attached, applies to current git at the moment of creation of this bug.
Created attachment 212862 [details, diff] PMS offset-prefix patch (r1)
Created attachment 212873 [details, diff] PMS offset-prefix patch (r2) after feedback from ulm: - fill in feature set for EAPI3 (offset-prefix support) - add wording to make explicit that offset-prefix support is also available in EAPIs after 3 - fixed EPREFIX, ED and EROOT variable table to also include EAPI4
I think we need to get that todo sorted out. Your "+files under the" is a very long line. We've never used the wording "EAPI x and later" before, since EAPIs don't have an order. We've always instead pointed to the appropriate table. You might find it easier to do this if there's a table saying "EAPIs supporting offset prefix" or somesuch. I think we need something in here mentioning whether ebuilds are required to handle offset prefix, or whether it's allowable for ebuilds to behave correctly only if EPREFIX is empty. The way it's worded now more or less implies that an ebuild that barfs if EPREFIX is set to non-empty (for example, if it uses $D where it should use $ED to do some tinkering) is broken.
(In reply to comment #3) > I think we need something in here mentioning whether ebuilds are required to > handle offset prefix, or whether it's allowable for ebuilds to behave correctly > only if EPREFIX is empty. The way it's worded now more or less implies that an > ebuild that barfs if EPREFIX is set to non-empty (for example, if it uses $D > where it should use $ED to do some tinkering) is broken. My understanding from various mails is that the Prefix team expects it to be treated like any other arch, ie if the ebuild isn't keyworded for Prefix, there's no expectation that it'll work under Prefix. grobian: is that correct? Has it been formally stated anywhere, and if not, should it be?
Another thing: can we s/in the environment/in the environment by user configuration/, to make it clear that it's illegal for ebuilds to mess with those three vars?
(In reply to comment #4) > (In reply to comment #3) > > I think we need something in here mentioning whether ebuilds are required to > > handle offset prefix, or whether it's allowable for ebuilds to behave correctly > > only if EPREFIX is empty. The way it's worded now more or less implies that an > > ebuild that barfs if EPREFIX is set to non-empty (for example, if it uses $D > > where it should use $ED to do some tinkering) is broken. > > My understanding from various mails is that the Prefix team expects it to be > treated like any other arch, ie if the ebuild isn't keyworded for Prefix, > there's no expectation that it'll work under Prefix. > > grobian: is that correct? Has it been formally stated anywhere, and if not, > should it be? Correct. I'm not sure it needs to be stated anywhere really. Is it *that* unclear? (We have separate profiles, etc) (In reply to comment #5) > Another thing: can we s/in the environment/in the environment by user > configuration/, to make it clear that it's illegal for ebuilds to mess with > those three vars? To my understanding, the vars should be read-only. I also want to make one thing clear, 'the compiled package' (executable, docs, etc) should never rely on EPREFIX being set in the environment. An exception to this is when you are bootstrapping, but that is not relevant here.
(In reply to comment #6) > Correct. I'm not sure it needs to be stated anywhere really. Is it *that* > unclear? (We have separate profiles, etc) Yeah, it needs to be stated. PMS is allegedly independent of profiles etc.
Technically, if an ebuild has EAPI=3, then it should be Prefix aware, that's the result of doing Prefix as a new EAPI. I personally think this is the cleanest way given the current situation. The current state of gx86 however is that if it carries Prefix keywords, the ebuild works for Prefix people, because we use a Portage that always supports an offset-prefix installation, regardless of EAPI.
That's something we need to know: is it your position that any ebuild using EAPI 3 or 4 must work with arbitrary prefix values? Is it your intent to prevent people from using other new features in EAPIs 3 and 4 if they don't have time to prefixify their package?
I have no strong preference to either options, since I think both suck to a certain degree. I do feel that it is cleanest to require offset-compatability when an offset-enabled EAPI is being used. I think it's worth to put the pros and cons next to each other before deciding which option we will officially put in place.
I think this should probably go to -dev@. Would you like to send an email asking for opinions?
I plan to first address your comments posted in here, so we're all on the same level, and then ask for a discussion on -dev.
(In reply to comment #3) > I think we need to get that todo sorted out. > > Your "+files under the" is a very long line. easily rewrapped, I tried not to introduce any diffs just for cosmetic reasons > We've never used the wording "EAPI x and later" before, since EAPIs don't have > an order. We've always instead pointed to the appropriate table. You might find > it easier to do this if there's a table saying "EAPIs supporting offset prefix" > or somesuch. How about Table 12.3? ulm asked me to add the x and later everywhere. So I guess he's the best to defend that wording. > I think we need something in here mentioning whether ebuilds are required to > handle offset prefix, or whether it's allowable for ebuilds to behave correctly > only if EPREFIX is empty. The way it's worded now more or less implies that an > ebuild that barfs if EPREFIX is set to non-empty (for example, if it uses $D > where it should use $ED to do some tinkering) is broken. It would be interesting to see if this is possible at all. Due to the brokenness of some build-systems removing D (and ROOT) never seemed to be a good idea to me, you need them too often such that ${ED%${EPREFIX}} would get tiresome. I will bring the discussion regarding whether it is ok to have EAPI={3,4} without offset-awareness to -dev ML, later.
(In reply to comment #5) > Another thing: can we s/in the environment/in the environment by user > configuration/, to make it clear that it's illegal for ebuilds to mess with > those three vars? how about just s/environment/calling environment/ there?
(In reply to comment #13) > How about Table 12.3? ulm asked me to add the x and later everywhere. > So I guess he's the best to defend that wording. I think the "and later" is fine, as long as all our EAPIs are integers (which can be ordered). But I have no objections against a table. Probably this would be more in line with the rest of the document.
Well, we've got "The package manager must not attempt to perform any kind of comparison test other than equality upon EAPIs." in there... I'd rather we didn't restrict EAPIs to a hierarchy without very good reason.
As I said, I have no objections against a table.
Created attachment 213134 [details, diff] PMS offset-prefix patch (r3) This revision adds an explicit table specifying which EAPIs are offset-prefix aware, and drops the before and later wording again.
(In reply to comment #12) > I plan to first address your comments posted in here, so we're all on the same > level, and then ask for a discussion on -dev. > For the record, the thread is here: http://archives.gentoo.org/gentoo-dev/msg_bbadeffac202df43a276272dd39297c1.xml
Some review: + value than the built in value is set in the calling environment, a + cross-prefix build is performed where using the existing utilities, + a package is build for the given \t{EPREFIX}, akin to \t{ROOT}. See * built-in? And "built" on the last line? * Do we need three columns and all variables separately in tab:offset-env-vars-table? +The latter do have the variables \t{ED} and \t{EROOT} properly set, +though * Isn't this bad writing style? * For the sake of consistence you should always write ${EPREFIX} instead of $EPREFIX in some cases. * econf should be marked-up with \t always. * The todos need to be sorted out. Any proposals how to handle them?
(In reply to comment #19) > (In reply to comment #12) > > I plan to first address your comments posted in here, so we're all on the same > > level, and then ask for a discussion on -dev. > > > > For the record, the thread is here: > http://archives.gentoo.org/gentoo-dev/msg_bbadeffac202df43a276272dd39297c1.xml Outcome seems to be that offset-prefix support should be optional.
The first hunk in eapi-differences.tex doesn't look right, since it adds the variables to the line containing the default phase functions: - \t{src\_compile}, \t{src\_install}, \t{src\_test}} \\ + \t{src\_compile}, \t{src\_install}, \t{src\_test}, + \t{EPREFIX}, \t{ED}, \t{EROOT}} \\ I suggest to add the following line instead (directly after REPLACED_BY_VERSION seems to be the best place): +\t{EPREFIX}, \t{ED}, \t{EROOT} & \compactfeatureref{offset-prefix-vars} & + No & No & No & Yes & Yes \\
Go ahead. You know what the idea is. All of you ask to either revert, or do it again in another different way, so please just fix it to whatever you're happy with.
(In reply to comment #20) > * The todos need to be sorted out. Any proposals how to handle them? % todo: Portage does not behave like this, and Prefix relies on that % root:root -> 0:0 (for systems where root name/group are different) % PORTAGE_INST_UID:PORTAGE_INST_GID <- Prefix sets that to user ids options: 1) drop the clause from PMS 2) update the clause to say the PM will set ownership to the super user or its equivalent for the system/installation at hand 3) change the clause for offset-prefix aware EAPIs to say that the PM will set ownership to the configured user (PORTAGE_ROOT_USER) % todo: Portage's implementation is different and required for Prefix % these files are unpacked with deb2targz if available, else it % falls back to "ar", which cannot be required to be GNU binutils 1) ignore it, PMS' definition stems from a GNU/Linux dominated environment like with the previous issue is the case, Portage just implements for more than just GNU/Linux 2) relax the clause to just mention that a deb-archive is being unpacked, just defining what's the outcome on disk, instead of restricting it to using GNU binutils to unpack 3) change the clause to read that for offset-aware EAPIs deb-file unpacking is implemented using deb2targz with a fallback to ar (if it's GNU).
(In reply to comment #24) > % todo: Portage does not behave like this, and Prefix relies on that > % root:root -> 0:0 (for systems where root name/group are different) > % PORTAGE_INST_UID:PORTAGE_INST_GID <- Prefix sets that to user ids IMHO, 2) is best here: > 2) update the clause to say the PM will set ownership to the super user or its > equivalent for the system/installation at hand > % todo: Portage's implementation is different and required for Prefix > % these files are unpacked with deb2targz if available, else it > % falls back to "ar", which cannot be required to be GNU binutils Not sure about this one. Maybe split the item, as follows? \item ar archives (\t{*.a}). Ebuilds must ensure that the ar program is installed. \item deb archives (\t{*.deb}). Ebuilds must ensure that the deb2targz program or the ar program is installed.
The whole unpack thing's a frickin' mess. Non-prefix EAPIs will have to carry on using the existing wording, so the question's how to word it for prefix things.
(In reply to comment #24) > (In reply to comment #20) > > * The todos need to be sorted out. Any proposals how to handle them? > > % todo: Portage does not behave like this, and Prefix relies on that > % root:root -> 0:0 (for systems where root name/group are different) > % PORTAGE_INST_UID:PORTAGE_INST_GID <- Prefix sets that to user ids > > options: > 2) update the clause to say the PM will set ownership to the super user or its > equivalent for the system/installation at hand Totally my vote. > % todo: Portage's implementation is different and required for Prefix > % these files are unpacked with deb2targz if available, else it > % falls back to "ar", which cannot be required to be GNU binutils > > 1) ignore it, PMS' definition stems from a GNU/Linux dominated environment like > with the previous issue is the case, Portage just implements for more than just > GNU/Linux > 2) relax the clause to just mention that a deb-archive is being unpacked, just > defining what's the outcome on disk, instead of restricting it to using GNU > binutils to unpack > 3) change the clause to read that for offset-aware EAPIs deb-file unpacking is > implemented using deb2targz with a fallback to ar (if it's GNU). 2 should be fine, maybe saying that GNU binutils is preferred if available.
Created attachment 215910 [details, diff] prefix-r4.patch This is the patch ulm gave me, but applied to the current head.
(In reply to comment #26) > The whole unpack thing's a frickin' mess. Non-prefix EAPIs will have to carry > on using the existing wording, so the question's how to word it for prefix > things. Do you forsee any problems with changing the description (for all existing EAPIs) to allow deb2targz next to GNU binutils ar to be used to unpack a .deb file? If so, which? I ask because Portage is already implemented as such.
Created attachment 215915 [details, diff] PMS offset-prefix patch (r5) This revision of the Prefix patch addresses both TODOs based on the input in this bug.
(In reply to comment #29) > Do you forsee any problems with changing the description (for all existing > EAPIs) to allow deb2targz next to GNU binutils ar to be used to unpack a .deb > file? If so, which? I ask because Portage is already implemented as such. Depends upon how you word it. Non-prefix EAPI ebuilds mustn't just DEPEND="deb2targz", for example. You probably also need to word it such that ebuilds *must* select ar where possible, except there's not really a nice way for ebuilds to do that.
(In reply to comment #31) > (In reply to comment #29) > > Do you forsee any problems with changing the description (for all existing > > EAPIs) to allow deb2targz next to GNU binutils ar to be used to unpack a .deb > > file? If so, which? I ask because Portage is already implemented as such. > > Depends upon how you word it. Non-prefix EAPI ebuilds mustn't just > DEPEND="deb2targz", for example. No, so it must perhaps be said even more clearly that the ebuild must guarantee in some what that the deb2targz program is available, if ... > You probably also need to word it such that > ebuilds *must* select ar where possible, except there's not really a nice way > for ebuilds to do that. Why is that a requirement? It's not nice to force a deb2targz dependency on someone who doesn't need it, but what's the danger in accidentially doing so/having it? Of course its awkward to just unconditionally depend on it while it is known that only a couple of arches need it, but that seems to be a detail that the ebuild writer/arch team member needs to deal with.
Ideally you'd keep it legal for package managers that only support basic unprefixed things to continue using exactly the same unpack code they've always used.
A package manager that chooses not to fully implement offset installation support can do so without problems. It will never face any platform that needs deb2targz to unpack a .deb file.
Depends upon how you word it... If you simply say that either binutils ar or dep2targz must be available, it strongly suggests that it's legal for an ebuild to DEPEND="dep2targz". What with the state of binary packages and all, it's pretty unlikely that anyone's going to be able to get away with not happening to have binutils installed by coincidence anyway, but still, I'd rather not set a precedent of picking wording that makes that kind of situation legal since there's a remote possibility that dependencies might start being accurate and complete some day.
Maybe we're not talking about the same thing. This is taken from the patch: \item deb packages (\t{*.deb}). Ebuilds must ensure that the deb2targz program is installed on those platforms where the GNU binutils ar program is not available and the installed ar program is incompatible with GNU archives. Otherwise, ebuilds must ensure that GNU binutils is installed.
Yeah, but how does an ebuild specify that? What's the syntax for it?
something like: ppc-aix? ( app-arch/deb2targz ) ?
That's going to get really really messy, isn't it? Won't you need dozens of those all over the place? *shrug* ah well. Guess we'll have to push hard for magic/unpack-blah in EAPI 5...
Not really, it's just a single package, or maybe 2. It's just because the package manager doesn't add the correct dependency based on the distfile that this awkwardness needs to take place. Similar to xz-utils issue.
No, that's the point: the package manager *can't* add dependencies automatically, because not everything in A gets unpacked. That's why magic/unpack-blah deps are needed, and why they have to be specified explicitly.
Great, that all sounds plausible, but shows that this issue needs to be resolved correctly at some point. You can drop this entry from the current patch, as I just noted that Portage implements it differently, and that that allows a few Prefix arches to install one tool that we need. It would be more reasonable to allow it per PMS though, IMO.
Created attachment 216131 [details, diff] PMS offset-prefix patch (r5) rebased against current PMS head Find attached a new version of your patch that applies cleanly against the current head (where yesterday's removal of kdebuild conditionals introduced relatively large changes). Except for resolution of conflicts with the kdebuild-removal patch, nothing should have changed.
For the records: The patch is fine for me.
Created attachment 216699 [details, diff] PMS offset-prefix patch (r6) I did another round of review, and here is yet another version. Only tiny changes, as compared to r5: - In subsection "Installation commands": Added word "These" at the beginning of the third sentence (that had been mistakenly removed in r5). - In subsubsection "Misc commands", item "unpack": Add word "the" in phrase: "... where _the_ GNU binutils ar program is not available ..." - Some whitespace polishing.
It has been pushed to the repository.
EAPI 3 approved by council.
