Bug 296716 - PMS patch for EAPI3/offset-prefix (EPREFIX, ED and EROOT)
PMS patch for EAPI3/offset-prefix (EPREFIX, ED and EROOT)
 Status: RESOLVED FIXED None Gentoo Hosted Projects Unclassified PMS/EAPI (show other bugs) All All High normal (vote) PMS/EAPI in-eapi-3 future-eapi Show dependency tree

 Reported: 2009-12-13 11:57 UTC by Fabian Groffen 2010-01-18 22:17 UTC (History) 1 user (show) prefix ---

Attachments
PMS offset-prefix patch (r1) (prefix-r1.patch,14.49 KB, patch)
2009-12-13 11:58 UTC, Fabian Groffen
Details | Diff
PMS offset-prefix patch (r2) (prefix-r2.patch,14.48 KB, patch)
2009-12-13 12:30 UTC, Fabian Groffen
Details | Diff
PMS offset-prefix patch (r3) (prefix-r3,15.85 KB, patch)
2009-12-15 20:39 UTC, Fabian Groffen
Details | Diff
prefix-r4.patch (prefix-r4.patch,16.83 KB, patch)
2010-01-10 10:34 UTC, Fabian Groffen
Details | Diff
PMS offset-prefix patch (r5) (prefix-r5.patch,17.27 KB, patch)
2010-01-10 10:55 UTC, Fabian Groffen
Details | Diff
PMS offset-prefix patch (r5) rebased against current PMS head (0001-Offset-prefix-EPREFIX-ED-and-EROOT.patch,16.67 KB, patch)
2010-01-11 22:11 UTC, Ulrich Müller
Details | Diff
PMS offset-prefix patch (r6) (0001-Offset-prefix-EPREFIX-ED-and-EROOT.patch,17.91 KB, patch)
2010-01-16 19:59 UTC, Ulrich Müller
Details | Diff

 Note You need to log in before you can comment on or make changes to this bug.
 Fabian Groffen 2009-12-13 11:57:31 UTC as attached, applies to current git at the moment of creation of this bug. Fabian Groffen 2009-12-13 11:58:28 UTC Created PMS offset-prefix patch (r1) Fabian Groffen 2009-12-13 12:30:45 UTC Created PMS offset-prefix patch (r2) after feedback from ulm: - fill in feature set for EAPI3 (offset-prefix support) - add wording to make explicit that offset-prefix support is also available in EAPIs after 3 - fixed EPREFIX, ED and EROOT variable table to also include EAPI4 Ciaran McCreesh 2009-12-13 14:42:05 UTC I think we need to get that todo sorted out. Your "+files under the" is a very long line. We've never used the wording "EAPI x and later" before, since EAPIs don't have an order. We've always instead pointed to the appropriate table. You might find it easier to do this if there's a table saying "EAPIs supporting offset prefix" or somesuch. I think we need something in here mentioning whether ebuilds are required to handle offset prefix, or whether it's allowable for ebuilds to behave correctly only if EPREFIX is empty. The way it's worded now more or less implies that an ebuild that barfs if EPREFIX is set to non-empty (for example, if it uses $D where it should use$ED to do some tinkering) is broken. David Leverton 2009-12-13 16:28:14 UTC (In reply to comment #3) > I think we need something in here mentioning whether ebuilds are required to > handle offset prefix, or whether it's allowable for ebuilds to behave correctly > only if EPREFIX is empty. The way it's worded now more or less implies that an > ebuild that barfs if EPREFIX is set to non-empty (for example, if it uses $D > where it should use$ED to do some tinkering) is broken. My understanding from various mails is that the Prefix team expects it to be treated like any other arch, ie if the ebuild isn't keyworded for Prefix, there's no expectation that it'll work under Prefix. grobian: is that correct? Has it been formally stated anywhere, and if not, should it be? Ciaran McCreesh 2009-12-14 18:53:01 UTC Another thing: can we s/in the environment/in the environment by user configuration/, to make it clear that it's illegal for ebuilds to mess with those three vars? Jeremy Olexa (darkside) (RETIRED) 2009-12-15 15:33:20 UTC (In reply to comment #4) > (In reply to comment #3) > > I think we need something in here mentioning whether ebuilds are required to > > handle offset prefix, or whether it's allowable for ebuilds to behave correctly > > only if EPREFIX is empty. The way it's worded now more or less implies that an > > ebuild that barfs if EPREFIX is set to non-empty (for example, if it uses $D > > where it should use$ED to do some tinkering) is broken. > > My understanding from various mails is that the Prefix team expects it to be > treated like any other arch, ie if the ebuild isn't keyworded for Prefix, > there's no expectation that it'll work under Prefix. > > grobian: is that correct? Has it been formally stated anywhere, and if not, > should it be? Correct. I'm not sure it needs to be stated anywhere really. Is it *that* unclear? (We have separate profiles, etc) (In reply to comment #5) > Another thing: can we s/in the environment/in the environment by user > configuration/, to make it clear that it's illegal for ebuilds to mess with > those three vars? To my understanding, the vars should be read-only. I also want to make one thing clear, 'the compiled package' (executable, docs, etc) should never rely on EPREFIX being set in the environment. An exception to this is when you are bootstrapping, but that is not relevant here. Ciaran McCreesh 2009-12-15 15:35:30 UTC (In reply to comment #6) > Correct. I'm not sure it needs to be stated anywhere really. Is it *that* > unclear? (We have separate profiles, etc) Yeah, it needs to be stated. PMS is allegedly independent of profiles etc. Fabian Groffen 2009-12-15 16:21:54 UTC Technically, if an ebuild has EAPI=3, then it should be Prefix aware, that's the result of doing Prefix as a new EAPI. I personally think this is the cleanest way given the current situation. The current state of gx86 however is that if it carries Prefix keywords, the ebuild works for Prefix people, because we use a Portage that always supports an offset-prefix installation, regardless of EAPI. Ciaran McCreesh 2009-12-15 16:24:13 UTC That's something we need to know: is it your position that any ebuild using EAPI 3 or 4 must work with arbitrary prefix values? Is it your intent to prevent people from using other new features in EAPIs 3 and 4 if they don't have time to prefixify their package? Fabian Groffen 2009-12-15 16:33:25 UTC I have no strong preference to either options, since I think both suck to a certain degree. I do feel that it is cleanest to require offset-compatability when an offset-enabled EAPI is being used. I think it's worth to put the pros and cons next to each other before deciding which option we will officially put in place. Ciaran McCreesh 2009-12-15 16:35:30 UTC I think this should probably go to -dev@. Would you like to send an email asking for opinions? Fabian Groffen 2009-12-15 16:38:38 UTC I plan to first address your comments posted in here, so we're all on the same level, and then ask for a discussion on -dev. Fabian Groffen 2009-12-15 18:24:43 UTC (In reply to comment #3) > I think we need to get that todo sorted out. > > Your "+files under the" is a very long line. easily rewrapped, I tried not to introduce any diffs just for cosmetic reasons > We've never used the wording "EAPI x and later" before, since EAPIs don't have > an order. We've always instead pointed to the appropriate table. You might find > it easier to do this if there's a table saying "EAPIs supporting offset prefix" > or somesuch. How about Table 12.3? ulm asked me to add the x and later everywhere. So I guess he's the best to defend that wording. > I think we need something in here mentioning whether ebuilds are required to > handle offset prefix, or whether it's allowable for ebuilds to behave correctly > only if EPREFIX is empty. The way it's worded now more or less implies that an > ebuild that barfs if EPREFIX is set to non-empty (for example, if it uses $D > where it should use$ED to do some tinkering) is broken. It would be interesting to see if this is possible at all. Due to the brokenness of some build-systems removing D (and ROOT) never seemed to be a good idea to me, you need them too often such that ${ED%${EPREFIX}} would get tiresome. I will bring the discussion regarding whether it is ok to have EAPI={3,4} without offset-awareness to -dev ML, later. Fabian Groffen 2009-12-15 18:27:45 UTC (In reply to comment #5) > Another thing: can we s/in the environment/in the environment by user > configuration/, to make it clear that it's illegal for ebuilds to mess with > those three vars? how about just s/environment/calling environment/ there? Ulrich Müller 2009-12-15 19:05:58 UTC (In reply to comment #13) > How about Table 12.3? ulm asked me to add the x and later everywhere. > So I guess he's the best to defend that wording. I think the "and later" is fine, as long as all our EAPIs are integers (which can be ordered). But I have no objections against a table. Probably this would be more in line with the rest of the document. Ciaran McCreesh 2009-12-15 19:12:11 UTC Well, we've got "The package manager must not attempt to perform any kind of comparison test other than equality upon EAPIs." in there... I'd rather we didn't restrict EAPIs to a hierarchy without very good reason. Ulrich Müller 2009-12-15 19:15:45 UTC As I said, I have no objections against a table. Fabian Groffen 2009-12-15 20:39:32 UTC Created PMS offset-prefix patch (r3) This revision adds an explicit table specifying which EAPIs are offset-prefix aware, and drops the before and later wording again. Jeremy Olexa (darkside) (RETIRED) 2009-12-16 22:16:15 UTC (In reply to comment #12) > I plan to first address your comments posted in here, so we're all on the same > level, and then ask for a discussion on -dev. > For the record, the thread is here: http://archives.gentoo.org/gentoo-dev/msg_bbadeffac202df43a276272dd39297c1.xml Christian Faulhammer (RETIRED) 2009-12-16 23:07:00 UTC Some review: + value than the built in value is set in the calling environment, a + cross-prefix build is performed where using the existing utilities, + a package is build for the given \t{EPREFIX}, akin to \t{ROOT}. See * built-in? And "built" on the last line? * Do we need three columns and all variables separately in tab:offset-env-vars-table? +The latter do have the variables \t{ED} and \t{EROOT} properly set, +though * Isn't this bad writing style? * For the sake of consistence you should always write ${EPREFIX} instead of$EPREFIX in some cases. * econf should be marked-up with \t always. * The todos need to be sorted out. Any proposals how to handle them? Fabian Groffen 2009-12-18 18:36:03 UTC (In reply to comment #19) > (In reply to comment #12) > > I plan to first address your comments posted in here, so we're all on the same > > level, and then ask for a discussion on -dev. > > > > For the record, the thread is here: > http://archives.gentoo.org/gentoo-dev/msg_bbadeffac202df43a276272dd39297c1.xml Outcome seems to be that offset-prefix support should be optional. Ulrich Müller 2009-12-18 21:43:19 UTC The first hunk in eapi-differences.tex doesn't look right, since it adds the variables to the line containing the default phase functions: - \t{src\_compile}, \t{src\_install}, \t{src\_test}} \\ + \t{src\_compile}, \t{src\_install}, \t{src\_test}, + \t{EPREFIX}, \t{ED}, \t{EROOT}} \\ I suggest to add the following line instead (directly after REPLACED_BY_VERSION seems to be the best place): +\t{EPREFIX}, \t{ED}, \t{EROOT} & \compactfeatureref{offset-prefix-vars} & + No & No & No & Yes & Yes \\ Fabian Groffen 2009-12-18 21:50:40 UTC Go ahead. You know what the idea is. All of you ask to either revert, or do it again in another different way, so please just fix it to whatever you're happy with. Fabian Groffen 2009-12-19 20:17:15 UTC (In reply to comment #20) > * The todos need to be sorted out. Any proposals how to handle them? % todo: Portage does not behave like this, and Prefix relies on that % root:root -> 0:0 (for systems where root name/group are different) % PORTAGE_INST_UID:PORTAGE_INST_GID <- Prefix sets that to user ids options: 1) drop the clause from PMS 2) update the clause to say the PM will set ownership to the super user or its equivalent for the system/installation at hand 3) change the clause for offset-prefix aware EAPIs to say that the PM will set ownership to the configured user (PORTAGE_ROOT_USER) % todo: Portage's implementation is different and required for Prefix % these files are unpacked with deb2targz if available, else it % falls back to "ar", which cannot be required to be GNU binutils 1) ignore it, PMS' definition stems from a GNU/Linux dominated environment like with the previous issue is the case, Portage just implements for more than just GNU/Linux 2) relax the clause to just mention that a deb-archive is being unpacked, just defining what's the outcome on disk, instead of restricting it to using GNU binutils to unpack 3) change the clause to read that for offset-aware EAPIs deb-file unpacking is implemented using deb2targz with a fallback to ar (if it's GNU). Ulrich Müller 2009-12-20 17:40:33 UTC (In reply to comment #24) > % todo: Portage does not behave like this, and Prefix relies on that > % root:root -> 0:0 (for systems where root name/group are different) > % PORTAGE_INST_UID:PORTAGE_INST_GID <- Prefix sets that to user ids IMHO, 2) is best here: > 2) update the clause to say the PM will set ownership to the super user or its > equivalent for the system/installation at hand > % todo: Portage's implementation is different and required for Prefix > % these files are unpacked with deb2targz if available, else it > % falls back to "ar", which cannot be required to be GNU binutils Not sure about this one. Maybe split the item, as follows? \item ar archives (\t{*.a}). Ebuilds must ensure that the ar program is installed. \item deb archives (\t{*.deb}). Ebuilds must ensure that the deb2targz program or the ar program is installed. Ciaran McCreesh 2009-12-20 17:49:11 UTC The whole unpack thing's a frickin' mess. Non-prefix EAPIs will have to carry on using the existing wording, so the question's how to word it for prefix things. Christian Faulhammer (RETIRED) 2009-12-29 15:49:18 UTC (In reply to comment #24) > (In reply to comment #20) > > * The todos need to be sorted out. Any proposals how to handle them? > > % todo: Portage does not behave like this, and Prefix relies on that > % root:root -> 0:0 (for systems where root name/group are different) > % PORTAGE_INST_UID:PORTAGE_INST_GID <- Prefix sets that to user ids > > options: > 2) update the clause to say the PM will set ownership to the super user or its > equivalent for the system/installation at hand Totally my vote. > % todo: Portage's implementation is different and required for Prefix > % these files are unpacked with deb2targz if available, else it > % falls back to "ar", which cannot be required to be GNU binutils > > 1) ignore it, PMS' definition stems from a GNU/Linux dominated environment like > with the previous issue is the case, Portage just implements for more than just > GNU/Linux > 2) relax the clause to just mention that a deb-archive is being unpacked, just > defining what's the outcome on disk, instead of restricting it to using GNU > binutils to unpack > 3) change the clause to read that for offset-aware EAPIs deb-file unpacking is > implemented using deb2targz with a fallback to ar (if it's GNU). 2 should be fine, maybe saying that GNU binutils is preferred if available. Fabian Groffen 2010-01-10 10:34:45 UTC Created prefix-r4.patch This is the patch ulm gave me, but applied to the current head. Fabian Groffen 2010-01-10 10:47:30 UTC (In reply to comment #26) > The whole unpack thing's a frickin' mess. Non-prefix EAPIs will have to carry > on using the existing wording, so the question's how to word it for prefix > things. Do you forsee any problems with changing the description (for all existing EAPIs) to allow deb2targz next to GNU binutils ar to be used to unpack a .deb file? If so, which? I ask because Portage is already implemented as such. Fabian Groffen 2010-01-10 10:55:52 UTC Created PMS offset-prefix patch (r5) This revision of the Prefix patch addresses both TODOs based on the input in this bug. Ciaran McCreesh 2010-01-10 15:56:07 UTC (In reply to comment #29) > Do you forsee any problems with changing the description (for all existing > EAPIs) to allow deb2targz next to GNU binutils ar to be used to unpack a .deb > file? If so, which? I ask because Portage is already implemented as such. Depends upon how you word it. Non-prefix EAPI ebuilds mustn't just DEPEND="deb2targz", for example. You probably also need to word it such that ebuilds *must* select ar where possible, except there's not really a nice way for ebuilds to do that. Fabian Groffen 2010-01-10 16:17:19 UTC (In reply to comment #31) > (In reply to comment #29) > > Do you forsee any problems with changing the description (for all existing > > EAPIs) to allow deb2targz next to GNU binutils ar to be used to unpack a .deb > > file? If so, which? I ask because Portage is already implemented as such. > > Depends upon how you word it. Non-prefix EAPI ebuilds mustn't just > DEPEND="deb2targz", for example. No, so it must perhaps be said even more clearly that the ebuild must guarantee in some what that the deb2targz program is available, if ... > You probably also need to word it such that > ebuilds *must* select ar where possible, except there's not really a nice way > for ebuilds to do that. Why is that a requirement? It's not nice to force a deb2targz dependency on someone who doesn't need it, but what's the danger in accidentially doing so/having it? Of course its awkward to just unconditionally depend on it while it is known that only a couple of arches need it, but that seems to be a detail that the ebuild writer/arch team member needs to deal with. Ciaran McCreesh 2010-01-10 16:23:14 UTC Ideally you'd keep it legal for package managers that only support basic unprefixed things to continue using exactly the same unpack code they've always used. Fabian Groffen 2010-01-10 16:28:18 UTC A package manager that chooses not to fully implement offset installation support can do so without problems. It will never face any platform that needs deb2targz to unpack a .deb file. Ciaran McCreesh 2010-01-10 16:32:47 UTC Depends upon how you word it... If you simply say that either binutils ar or dep2targz must be available, it strongly suggests that it's legal for an ebuild to DEPEND="dep2targz". What with the state of binary packages and all, it's pretty unlikely that anyone's going to be able to get away with not happening to have binutils installed by coincidence anyway, but still, I'd rather not set a precedent of picking wording that makes that kind of situation legal since there's a remote possibility that dependencies might start being accurate and complete some day. Fabian Groffen 2010-01-10 16:48:22 UTC Maybe we're not talking about the same thing. This is taken from the patch: \item deb packages (\t{*.deb}). Ebuilds must ensure that the deb2targz program is installed on those platforms where the GNU binutils ar program is not available and the installed ar program is incompatible with GNU archives. Otherwise, ebuilds must ensure that GNU binutils is installed. Ciaran McCreesh 2010-01-10 16:54:36 UTC Yeah, but how does an ebuild specify that? What's the syntax for it? Fabian Groffen 2010-01-10 16:58:58 UTC something like: ppc-aix? ( app-arch/deb2targz ) ? Ciaran McCreesh 2010-01-10 17:02:43 UTC That's going to get really really messy, isn't it? Won't you need dozens of those all over the place? *shrug* ah well. Guess we'll have to push hard for magic/unpack-blah in EAPI 5... Fabian Groffen 2010-01-10 17:05:12 UTC Not really, it's just a single package, or maybe 2. It's just because the package manager doesn't add the correct dependency based on the distfile that this awkwardness needs to take place. Similar to xz-utils issue. Ciaran McCreesh 2010-01-10 17:11:04 UTC No, that's the point: the package manager *can't* add dependencies automatically, because not everything in A gets unpacked. That's why magic/unpack-blah deps are needed, and why they have to be specified explicitly. Fabian Groffen 2010-01-10 17:18:48 UTC Great, that all sounds plausible, but shows that this issue needs to be resolved correctly at some point. You can drop this entry from the current patch, as I just noted that Portage implements it differently, and that that allows a few Prefix arches to install one tool that we need. It would be more reasonable to allow it per PMS though, IMO. Ulrich Müller 2010-01-11 22:11:53 UTC Created PMS offset-prefix patch (r5) rebased against current PMS head Find attached a new version of your patch that applies cleanly against the current head (where yesterday's removal of kdebuild conditionals introduced relatively large changes). Except for resolution of conflicts with the kdebuild-removal patch, nothing should have changed. Christian Faulhammer (RETIRED) 2010-01-13 21:09:16 UTC For the records: The patch is fine for me. Ulrich Müller 2010-01-16 19:59:57 UTC Created PMS offset-prefix patch (r6) I did another round of review, and here is yet another version. Only tiny changes, as compared to r5: - In subsection "Installation commands": Added word "These" at the beginning of the third sentence (that had been mistakenly removed in r5). - In subsubsection "Misc commands", item "unpack": Add word "the" in phrase: "... where _the_ GNU binutils ar program is not available ..." - Some whitespace polishing. Christian Faulhammer (RETIRED) 2010-01-17 12:06:03 UTC It has been pushed to the repository. Ulrich Müller 2010-01-18 22:13:51 UTC EAPI 3 approved by council.