an attempt to cross emerge on prefix leads me here. when ROOT is set to ${EPREFIX}/usr/${CHOST}, prefix-portage will install ebuilds into ${EPREFIX}/usr/${CHOST}/${EPREFIX}/. The desired location should be ${EPREFIX}/usr/${CHOST} without another ${EPREFIX}. A straightforward idea would be "ROOT=${EPREFIX}/usr/ EPREFIX="" <other cross compile env vars with crossdev> emerge ...." However prefix-portage fails to find "PORTDIR" "PORTAGE_CONFIGROOT", etc. many crucial locations. Apparently, _prefix-portage mixes the EPREFIX for its own and EPREFIX for installing packages_. I have back traced the warnings, and find: PORTDIR is set in make.gloabals as "PORTDIR=${EPREFIX}/usr/portage", which, as also grobian points out, should be hardcoded instead of relative to EPREFIX. Or, (re)introduce BPREFIX (build prefix, like the idea of CBUILD and CHOST in toolchain). PORTAGE_CONFIGROOT should be set in ${EPREFIX}/usr/${CHOST}, but even I set it in the environment, it is ignored first, gives a warning, and finally comes back. There must be something wrong here. at pym/portage/__init__.py:515 if settings["ROOT"] == "/": trees._running_eroot = trees._target_eroot else: # When ROOT != "/" we only want overrides from the calling # environment to apply to the config that's associated # with ROOT != "/", so pass a nearly empty dict for the env parameter. <....> settings = config(config_root=None, target_root="/", env=clean_env, eprefix=eprefix) <....> config_root is intentionally reset to None (later will default to "/", which is not what we want by specifying PORTAGE_CONFIGROOT in the environment) if ROOT!="/". later despite of the warning portage picks up PORTAGE_CONFIGROOT and can be examined with "EPREFIX='' PORTAGE_CONFIGROOT='blah' emerge --info". BTW, it also seems that some tricks for crossdev is treaked towards new features of portage master branch. (call vapier) What's the block of merging prefix branch entirely into master? I may find some time to push this. regards Reproducible: Always
adding prefix team to Cc
(In reply to comment #0) > PORTDIR is set in make.gloabals as "PORTDIR=${EPREFIX}/usr/portage", which, as > also grobian points out, should be hardcoded instead of relative to EPREFIX. Okay, that's fixed in git now: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=e1c6bba8c26525cca498894237c6421f86c98dfb > <....> > settings = config(config_root=None, target_root="/", > env=clean_env, eprefix=eprefix) > <....> > > config_root is intentionally reset to None (later will default to "/", which is > not what we want by specifying PORTAGE_CONFIGROOT in the environment) if > ROOT!="/". I guess this is a bug in the prefix branch. In the master branch, it defaults to the same value as EPREFIX in this code from pym/portage/package/ebuild/_config/LocationsManager.py: if self.eprefix is None: self.eprefix = portage.const.EPREFIX if self.config_root is None: self.config_root = self.eprefix + os.sep > What's the block of merging prefix branch entirely into master? I may find > some time to push this. It's happening gradually. Since v2.2.0_alpha81, it's possible to install and use the master branch of portage in a prefix installation. However, it currently lacks inter-revision support, so you have to convert inter-revision ebuilds to normal versions before it will work. I'm not sure if it's necessary to add inter-revision support to the master branch, since there are very few ebuilds using it.
(In reply to comment #2) > > <....> > > settings = config(config_root=None, target_root="/", > > env=clean_env, eprefix=eprefix) > > <....> > > > > config_root is intentionally reset to None (later will default to "/", which is > > not what we want by specifying PORTAGE_CONFIGROOT in the environment) if > > ROOT!="/". > > I guess this is a bug in the prefix branch. In the master branch, it defaults > to the same value as EPREFIX in this code from > pym/portage/package/ebuild/_config/LocationsManager.py: > > if self.eprefix is None: > self.eprefix = portage.const.EPREFIX > > if self.config_root is None: > self.config_root = self.eprefix + os.sep I forgot to mention that the config_root=None part is intended regardless of the PORTAGE_CONFIGROOT environment variable, since the config instance that's being constructed there is intended represent the "host" config, which is used to satisfy pure build-time dependencies (DEPEND).
(In reply to comment #0) > The desired location should be ${EPREFIX}/usr/${CHOST} without another > ${EPREFIX}. > > A straightforward idea would be "ROOT=${EPREFIX}/usr/ EPREFIX="" <other cross > compile env vars with crossdev> emerge ...." This seems reasonable. I think a problem with the prefix branch is the following code in pym/portage/const.py: # pick up EPREFIX from the environment if set if "EPREFIX" in os.environ: if os.environ["EPREFIX"] != "": EPREFIX = os.path.normpath(os.environ["EPREFIX"]) else: EPREFIX = os.environ["EPREFIX"] The problem is that portage.const.EPREFIX is being overridden, which corrupts the default config_root setting in LocationsManager. Since you only want the EPREFIX variable to override EPREFIX for your cross-installation, it doesn't make sense for the EPREFIX variable to override portage.const.EPREFIX, since that will corrupt the config instance that's intended to satisfy pure build-time dependencies (DEPEND).
(In reply to comment #2) > currently lacks inter-revision support, so you have to convert inter-revision > ebuilds to normal versions before it will work. I'm not sure if it's necessary > to add inter-revision support to the master branch, since there are very few > ebuilds using it. Ya, don't ever think about supporting "inter-revisions".
1. A trivial question: What is 'inter-revisions' actually? could you point out an example ebuild with inter-revision in prefix tree? 2. from the series of zmedico's comments (thanks :), it seems that the only problem resides in const.py. (ROOT='/' check is intended, reading from portage.const.EPREFIX is also intended, and have strong reasons behind those) I suspect getting EPREFIX from env in const.py is responsible for prefix bootstrapping.
(In reply to comment #6) > 1. A trivial question: What is 'inter-revisions' actually? could you point out > an example ebuild with inter-revision in prefix tree? The docs cover inter-revisions here: http://www.gentoo.org/proj/en/gentoo-alt/prefix/techdocs.xml#doc_chap2_sect5 These are the ones that I copied to a local overlay in order to avoid inter-revisions: sys-devel/libperl sys-devel/gcc-config sys-devel/binutils-config app-arch/rpm2targz > 2. from the series of zmedico's comments (thanks :), it seems that the only > problem resides in const.py. (ROOT='/' check is intended, reading from > portage.const.EPREFIX is also intended, and have strong reasons > behind those) We may need to tweak the create_trees() function and/or callers, to make your EPREFIX override work as desired (so it only applies to the cross-install). > I suspect getting EPREFIX from env in const.py is responsible for prefix > bootstrapping. In the master branch, I'm currently using a PORTAGE_OVERRIDE_EPREFIX variable to override portage.const.EPREFIX when necessary. It only needs to be overridden like this in special situations, so it needs to use a different variable that the one that's used for cross-install applications.
(In reply to comment #7) > The docs cover inter-revisions here: > > http://www.gentoo.org/proj/en/gentoo-alt/prefix/techdocs.xml#doc_chap2_sect5 > > These are the ones that I copied to a local overlay in order to avoid > inter-revisions: > > sys-devel/libperl > sys-devel/gcc-config > sys-devel/binutils-config > app-arch/rpm2targz Great, thanks. I suppose it's time to drop inter-revision. Provided prefix is accepted by gentoo community (not a proof of concept for now), We should apply preifx patches to x86 tree. > > 2. from the series of zmedico's comments (thanks :), it seems that the only > > problem resides in const.py. (ROOT='/' check is intended, reading from > > portage.const.EPREFIX is also intended, and have strong reasons > > behind those) > > We may need to tweak the create_trees() function and/or callers, to make your > EPREFIX override work as desired (so it only applies to the cross-install). to my understanding, portage.create_trees is used for making the dependence tree for all cases. if so, we should not touch it. it ain't broken for cross-install. > In the master branch, I'm currently using a PORTAGE_OVERRIDE_EPREFIX variable > to override portage.const.EPREFIX when necessary. It only needs to be > overridden like this in special situations, so it needs to use a different > variable that the one that's used for cross-install applications. I saw that var in portage.const, too. I tried to delete the two lines (it just sets EPREFIX to '', which is not intended, do not know the logic behind it. Anyway we can always do EPREFIX='/' or PORTAGE_OVERRIDE_EPREFIX='' for such perpose.) # pick up EPREFIX from the environment if set if "EPREFIX" in os.environ: if os.environ["EPREFIX"] != "": EPREFIX = os.path.normpath(os.environ["EPREFIX"]) - else: - EPREFIX = os.environ["EPREFIX"] I am poking through pym/_emerge/Scheduler.py to find out how task.settings is set up. I think we can respect os.environ["EPREFIX"] there, or move the part of code to portage.const for cleanness.
I am doing "emerge busybox" a quick question: how is self._task_queues in Scheduler.py constucted for e.g. compile and install? I looked around and could not find what code actually did the injection into the queue.
forward ported to prefix branch: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=9dee9d50c541da4f5282ce6624d82be64bb88a37
Just a couple of steps back before lots of code is being changed. At some time in the past, Prefix Portage had the ability to do "cross-Prefix" installs. haubi and mduft can correct me if I'm wrong here, they are the ones who "did" it, next to prefix-chaining, which should not be confused with this topic, although slightly related. A cross-Prefix emerge is similar to a ROOT install, but then using a different EPREFIX, that is, the resulting installed files are actually usable on the very system being worked on without any tricks/help whatsoever. An example of intended cross-Prefix emerge: # "bootstrap" new Prefix install from /var/tmp/gentoo-prefix based Prefix # into $HOME/Gentoo % /var/tmp/gentoo-prefix/startprefix Entering Gentoo Prefix /var/tmp/gentoo-prefix (Bootstrap:~) % env EPREFIX="$HOME/Gentoo" emerge system <snip 168 packages> (Bootstrap:~) % /var/tmp/gentoo-prefix/usr/portage/scripts/bootstrap-prefix.sh $HOME/Gentoo startscript <snip> (Bootstrap:~) % logout Leaving Gentoo Prefix with exit status 0 % $HOME/Gentoo/startprefix Entering Gentoo Prefix /export/home/neo/Gentoo (Gentoo:~) % emerge --sync <snip retrieving a tree> (Gentoo:~) % emerge -av happily_live_hereafter <...> To facilitate this, a variable BPREFIX was introduced, which holds the EPREFIX as the running Portage was configured (installed) with. BPREFIX is used to e.g. locate bash, mv, sandbox and fakeroot. The override of EPREFIX was meant for Portage to completely "move" into the given target EPREFIX, and just grab the bare minimals (python, bash, ...) from its own Prefix, to be able to pull itself out of the mud and/or repair a possibly broken Prefix.
BPREFIX first gets introduced here: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=2faf56cef94ad74a529e146ba41769b4a32df58a is there another vcs for prefix-portage that tracks the story from grobian "At some time in the past, Prefix Portage had the ability to do "cross-Prefix" installs."? One puzzle to me is that: $ git checkout 2faf56cef94ad74a529e146ba41769b4a32df58a $ grep BPREFIX -r . ./bin/isolated-functions.sh: unset BPREFIX DEFAULT_PATH EPREFIX EXTRA_PATH PORTAGE_GROUP PORTAGE_USER ./pym/_emerge/main.py:from portage.const import EPREFIX, BPREFIX, EPREFIX_LSTRIP ./pym/portage/__init__.py: EPREFIX, EPREFIX_LSTRIP, BPREFIX, rootuid ./pym/portage/package/ebuild/config.py: USER_VIRTUALS_FILE, EPREFIX, EPREFIX_LSTRIP, BPREFIX ./pym/portage/package/ebuild/_config/special_env_vars.py: "BPREFIX", "DEFAULT_PATH", "EXTRA_PATH", ./pym/portage/const.py:BPREFIX = EPREFIX ./pym/portage/const.py:SANDBOX_BINARY = BPREFIX + "/usr/bin/sandbox" ./pym/portage/const.py:FAKEROOT_BINARY = BPREFIX + "/usr/bin/fakeroot" see? BPREFIX is not used actually. I wonder how cross-Prefix emerge is implemented. Or, I am looking for a wrong clue of BPREFIX.
(In reply to comment #11) > The override of EPREFIX was meant for Portage to completely "move" into the > given target EPREFIX, and just grab the bare minimals (python, bash, ...) from > its own Prefix, to be able to pull itself out of the mud and/or repair a > possibly broken Prefix. after a couple hours of sleep, I can quite reasonate with this definition. Let's do it like this: 1. keep the current status of portage.const: EPREFIX can be overrided from env, although PORTAGE_OVERRIDE_EPREFIX serves the same perpose(? see 3). And keep BPREFIX=EPREFIX 2. replace portage.const.EPREFIX usage in portage with portage.const.BPREFIX when necessary to do the trick. 3. merge BPREFIX things to portage master branch, and convert PORTAGE_OVERRIDE_EPREFIX to PORTAGE_OVERRIDE_BPREFIX. What do you guys think? Let me kick out a patch now.
(In reply to comment #11) > Just a couple of steps back before lots of code is being changed. > > At some time in the past, Prefix Portage had the ability to do "cross-Prefix" > installs. haubi and mduft can correct me if I'm wrong here, they are the ones > who "did" it, next to prefix-chaining, which should not be confused with this > topic, although slightly related. Hmm, prefix chaining is here: http://overlays.gentoo.org/proj/alt/browser/trunk/prefix-overlay/sys-apps/portage/files/portage-2.2.00.15801-prefix-chaining.patch?order=name This is a great feature. when merged to portage master branch with the cross ROOT magic, we will be able to bootstrap gentoo from a working gentoo (prefix or not) *officially*, without catalyst stage3.
a further step would be distributing binary ebootstrap-xxxx.{deb,rpm,etc.} in other distro. emerge gentoo out of other distros ;) just like dev-util/debootstrap (hmm, that's binary packages and easier to bootstrap) At least, bootstrap a prefix before standalone gentoo.
what's the purpose of LocationsManager? It is only used once, no code reuse is achieved. $ grep -n LocationsManager -r . ./pym/portage/package/ebuild/config.py:54:from portage.package.ebuild._config.LocationsManager import LocationsManager ./pym/portage/package/ebuild/config.py:277: locations_manager = LocationsManager(config_root=config_root, ./pym/portage/package/ebuild/_config/LocationsManager.py:5: 'LocationsManager', ./pym/portage/package/ebuild/_config/LocationsManager.py:31:class LocationsManager(object):
(In reply to comment #13) > 2. replace portage.const.EPREFIX usage in portage with portage.const.BPREFIX > when necessary to do the trick. I don't think that portage.const.BPREFIX is really necessary. The way that the master branch currently works is that portage.const.EPREFIX serves as the default prefix. It's useful for python apps that use the portage API to find out the prefix that they're running inside (an portage itself uses it for the same purpose). Any other prefix, aside from portage.const.EPREFIX, can be encapsulated with a config instance using the "eprefix" argument to the config constructor. (In reply to comment #16) > what's the purpose of LocationsManager? It is only used once, no code reuse is > achieved. It's for encapsulation.
(In reply to comment #17) > Any other prefix, aside from portage.const.EPREFIX, can be > encapsulated with a config instance using the "eprefix" argument to the config > constructor. What variable can hold other prefix? we have two alternatives: 1. do not let EPREFIX from env override portage.const.EPREFIX; and pull the var from os.envrion['EPREFIX'] 2. use PORTAGE_OVERRIDE_EPREFIX. either feels not good.
(In reply to comment #18) > (In reply to comment #17) > > > Any other prefix, aside from portage.const.EPREFIX, can be > > encapsulated with a config instance using the "eprefix" argument to the config > > constructor. > > What variable can hold other prefix? we have two alternatives: > > 1. do not let EPREFIX from env override portage.const.EPREFIX; and pull the var > from os.envrion['EPREFIX'] > > 2. use PORTAGE_OVERRIDE_EPREFIX. > > either feels not good. The way that it already works with cross-installs is to create a config instance corresponding to the ROOT variable for the cross-install. So what I'm suggesting is to use the same approach, and have that config instance have an EPREFIX variable which differs from portage.const.EPREFIX. For example, with the master branch, try the following command: ROOT=/tmp/test_root python -c 'import portage ; print portage.settings["ROOT"]' The ROOT override is encapsulated in the portage.settings config instance. I'm suggesting that EPREFIX overrides for cross-install be encapsulated in portage.settings in exactly the same way, with no need to have it affect any constants from the portage.const module.
Created attachment 296717 [details, diff] EPREFIX-handling.patch before building, load EPREFIX from environment this is only a dirty fix. not sure if it breaks prefix bootstrap there is a conflict for the handling of EPREFIX between master and prefix branches (and between zmedico and grobian). we should sit down and settle the scheme.
I'm lenient here, since I no longer understand what Portage is supposed to do with vars. I prefer to leave it to someone who does. Perhaps what we're looking for is not cross-EPREFIX (because it seems to suck by design, and is hard to grasp), but prefix-chaining instead (like heroxbd already pointed out), with the addition of being able to turn a chained-instance into a standalone instance somehow (emerge -e world + some magic switch -- USE="-prefix-chaining" -- maybe)
Jee... The EPREFIX and EROOT in current prefix portage is really a mass. Quite a lot of inconsistencies :(
Created attachment 296767 [details, diff] 0001-before-building-load-EPREFIX-from-environment.patch a well tuned updated patch, seems working.
I've spent my whole day yesterday (~14 hours) tracing portage variables and settings handling with pdb. (way too much abstraction than I expected) Now that I have gained some direct feeling of present portage variable handling, I found some instances of legacy in prefix branch accessing portage.const.EPREFIX when actually settings['EPREFIX'] should be used. I agree cross compiling and emerging is not so exciting for people when don't care about embedded systems. Nevertheless, I regard *EPREFIX='' ROOT='/tmp/test' emerge busybox* to be natural to think with portage. If it breaks, it means something is inconsistant, if not wrong, with present treatment of EPREFIX in portage. We need to agree upon a var policy (oh, unfortunately) before prefix branch diverges too much from master. Please review my newest patch (respecting settings encapsulation) in case I am misusing something or a better way exists. If it is acceptable it can be applied to prefix tree. (need to confirm from prefix members if # pick up EPREFIX from the environment if set if "EPREFIX" in os.environ: if os.environ["EPREFIX"] != "": EPREFIX = os.path.normpath(os.environ["EPREFIX"]) else: EPREFIX = os.environ["EPREFIX"] in const.py is used (only?) for bootstrap)
(In reply to comment #24) > I've spent my whole day yesterday (~14 hours) tracing portage variables and > settings handling with pdb. (way too much abstraction than I expected) Now that > I have gained some direct feeling of present portage variable handling, I found > some instances of legacy in prefix branch accessing portage.const.EPREFIX when > actually settings['EPREFIX'] should be used. Ok, that's possible. History here, is that the code was using portage.const.EPREFIX and portage.const.EPREFIX_LSTRIP throughout. When EPREFIX became part of settings, it could be used instead. I only took over the changes from zmedicom which usually were replacements of the const EPREFIX. > I agree cross compiling and emerging is not so exciting for people when don't > care about embedded systems. Nevertheless, I regard *EPREFIX='' > ROOT='/tmp/test' emerge busybox* to be natural to think with portage. If it > breaks, it means something is inconsistant, if not wrong, with present > treatment of EPREFIX in portage. We need to agree upon a var policy (oh, > unfortunately) before prefix branch diverges too much from master. Prefix branch has never been so close to master as it is right now. I feel we cannot diverge any more now, because it is of much more benefit when the two branches can be merged and maintained as single piece. > Please review my newest patch (respecting settings encapsulation) in case I am > misusing something or a better way exists. If it is acceptable it can be > applied to prefix tree. > > (need to confirm from prefix members if > > # pick up EPREFIX from the environment if set > if "EPREFIX" in os.environ: > if os.environ["EPREFIX"] != "": > EPREFIX = os.path.normpath(os.environ["EPREFIX"]) > else: > EPREFIX = os.environ["EPREFIX"] > > in const.py is used (only?) for bootstrap) This is there since the day that cross-prefix was introduced. That is long before settings had a EPREFIX member. This made Portage able to change based on the environment. I guess the best thing to do now, is to simply remove that bit, since it seems to no longer do anything useful but confuse and look wrong.
I dropped the entire BPREFIX and EPREFIX override code. That should just cleanup some.
Hi Zac, This is a reply to your email at http://article.gmane.org/gmane.linux.gentoo.devel/83548 >> self.eroot = self.target_root.rstrip(os.sep) + self.eprefix + os.sep >> >> What is the logic behind this construction? > ROOT corresponds to a valid chroot point. EPREFIX is an offset inside a > valid chroot point. an EPREFIX offset inside a valid chroot point is legal, meaning we are foreign/cross compiling a Gentoo Prefix, but practically useless. 1. Gentoo Prefix often doesn't have root privilege. It cannot perform a chroot. 2. Things inside ROOT are already fully controled (if the user could chroot into to), there is of little interest run another Gentoo Prefix inside a brand new chroot. At the same time, foreign/cross compiling a Gentoo Prefix from Gentoo Prefix would be an interesting story. Then we will have: ${EPREFIX}/${ROOT}${NEW_EPREFIX} And still, even in this case, ${NEW_EPREFIX} instead of ${EPREFIX} should be used under ${ROOT}. ${EPREFIX} is "at the same level" as ${ROOT} and it is natural to prepend ROOT. Benda
(In reply to comment #27) > an EPREFIX offset inside a valid chroot point is legal, meaning we are > foreign/cross compiling a Gentoo Prefix, but practically useless. > > 1. Gentoo Prefix often doesn't have root privilege. It cannot perform a > chroot. > 2. Things inside ROOT are already fully controled (if the user could chroot > into to), there is of little interest run another Gentoo Prefix inside a > brand new chroot. Agreed. > At the same time, foreign/cross compiling a Gentoo Prefix from Gentoo Prefix > would be an interesting story. Then we will have: > > ${EPREFIX}/${ROOT}${NEW_EPREFIX} > > And still, even in this case, ${NEW_EPREFIX} instead of ${EPREFIX} should be > used under ${ROOT}. ${EPREFIX} is "at the same level" as ${ROOT} and it is > natural to prepend ROOT. ROOT has traditionally been an absolute path, and I don't see how it's useful to change it into a path that's relative to ${EPREFIX}. They're independent of each other, so why should one be relative to another?
(In reply to comment #28) > ROOT has traditionally been an absolute path, and I don't see how it's > useful to change it into a path that's relative to ${EPREFIX}. > They're independent of each other, so why should one be relative to another? Yeah, ROOT and EPREFIX are orthogonal, and ROOT is an absolute path (usually "/" or /usr/${CTARGET}). What concerned us is EROOT: We want to define it as something useful. Up until now all the EROOT usage case is when ROOT=/, where there is no difference. However, EROOT=EPREFIX/ROOT is more useful than EROOT=ROOT EPREFIX/ when we are proceeding to support cross development under Gentoo Prefix.
(In reply to comment #29) > Yeah, ROOT and EPREFIX are orthogonal, and ROOT is an absolute path (usually > "/" or /usr/${CTARGET}). What concerned us is EROOT: We want to define it as > something useful. If it has a new meaning, then maybe use a different variable name, to avoid confusion? EROOT already has a very specific meaning for EAPIs 3, 4, and 5. Giving it a new meaning now just seems a little crazy. > Up until now all the EROOT usage case is when ROOT=/, where there is no > difference. However, EROOT=EPREFIX/ROOT is more useful than EROOT=ROOT > EPREFIX/ when we are proceeding to support cross development under Gentoo > Prefix. Give it an new name please. How about EPREFIXROOT?
(In reply to comment #30) > If it has a new meaning, then maybe use a different variable name, to avoid > confusion? EROOT already has a very specific meaning for EAPIs 3, 4, and 5. > Giving it a new meaning now just seems a little crazy. Provided no one has stepped into the craziness of mixing Gentoo Prefix and cross development seriously yet, we could secretly change the definition without an impact. While it is conceptually crazy(redefining a well accepted variable), cleanness also counts. After all, we are not redefining it in the sense ROOT=/ in all use cases. > > Up until now all the EROOT usage case is when ROOT=/, where there is no > > difference. However, EROOT=EPREFIX/ROOT is more useful than EROOT=ROOT > > EPREFIX/ when we are proceeding to support cross development under Gentoo > > Prefix. > > Give it an new name please. How about EPREFIXROOT? If we have to invent a new variable, how about PROOT, EPROOT, EPREROOT? EPREFIXROOT is long.
(In reply to comment #29) > Up until now all the EROOT usage case is when ROOT=/, where there is no > difference. However, EROOT=EPREFIX/ROOT is more useful than EROOT=ROOT > EPREFIX/ when we are proceeding to support cross development under Gentoo > Prefix. I'm not really used to cross-compiling with ROOT - so maybe simply this is why I fail to see the use-case where EROOT=EPREFIX+ROOT is more useful than EROOT=ROOT+EPREFIX ? For the "prefix-chaining" mentioned earlier: Yes, we (mduft&me) still /do/ prefix-chaining - based on some very old snapshot of both prefix-portage and the prefix-tree, because of being _stable_ on all our platforms (ppc-aix, hppa-hpux, ia64-hpux, x86-linux, sparc-solaris, x86-solaris, x86-interix, x86-winnt). However, we hope to be able to schedule a general update in a not-so-far future, eventually replacing Interix by Cygwin or MinGW (while keeping x86-winnt). For how we (currently) use prefix-chaining: Assume /first is where prefix-portage is installed into and maintained by default due to portage-builtin vars. Assume /second is the chained prefix, configured this way: * EPREFIX=/second environment variable, also set by /second/startprefix * /second/etc/make.profile points to some different profile - eventually setting different CHOST and ACCEPT_KEYWORDS in case of x86-winnt, where /first is x86-interix. Actually, creating native Windows binaries is the hard reason we need prefix-chaining for. And as we need it anyway, we do it on any platform so we have to maintain one uniform layout only. * /second/etc/make.conf contains: READONLY_EPREFIX="/first:DEPEND" This means that portage does resolve + runtime-deps (RDEPEND) from /second as usual, but + buildtime-deps (DEPEND) from both /second _and_ /first. Agreed, this may be a misuse of DEPEND/RDEPEND, and we eventually would need something like BDEPEND - but it does work quite well here. And yes, DEPENDs not found in /first currently are merged into /second. In fact, we still have need for prefix-chaining in one way (EPREFIX=/second emerge ...) or another (with an upgrade path then).
(In reply to comment #31) > If we have to invent a new variable, how about PROOT, EPROOT, EPREROOT? > EPREFIXROOT is long. Yes, any name that's not already taken is fine. Still, I would advise against making ROOT a non-absolute path, because there you go again giving new meanings to variables that already have entrenched meanings.
haubi, thanks a lot for the explanating of Prefix chaining. It's time for us to rethink and redesign the precise meaning of the variables. Let me summaries all the uses cases and get a GLEP out for reviewing. 0. bootstrap vanilla Gentoo from vanilla (vanilla -> vaniila) env ROOT=/chroot emerge @system does it work? 0a. cross bootstrap vanilla Gentoo from vanilla (vanilla ~> vaniila) with crossdev: powerpc-unknown-linux-gnu-emerge @system in which ROOT=/usr/${CTARGET} 1. move a Prefix installation into another place. (Prefix -> Prefix) env PORTAGE_OVERWRITE_EPREFIX=/new_prefix emerge @system 1a. Cross build out a new Prefix, e.g. amd64 to arm, glibc to uclibc (Prefix ~> Prefix) ?????? how the command line should look like? 2. bootstrap a vanilla Gentoo from Prefix (Prefix -> vanilla) env ROOT=/chroot EPREFIX='' emerge @system 2a. cross bootstrap a vanilla Gentoo from Prefix (Preifx ~> vanilla) ??? how the command line should look like? 3. Prefix chaining: build out a second running Prefix based on the first. (Prefix ~ Prefix) READONLY_EPREFIX="/first:DEPEND" ???
(In reply to comment #34) > 0. bootstrap vanilla Gentoo from vanilla (vanilla -> vaniila) > > env ROOT=/chroot emerge @system > > does it work? Haven't tried myself, but this is how I would expect it to be done. > 0a. cross bootstrap vanilla Gentoo from vanilla (vanilla ~> vaniila) > with crossdev: > > powerpc-unknown-linux-gnu-emerge @system > > in which ROOT=/usr/${CTARGET} Haven't used crossdev myself, but as far as I understand, /usr/${CTARGET} _only_ contains the CTARGET toolchain (binutils, gcc and other machine-dependent code-generators), running on the build machine, necessary to build another /chroot, which becomes the root filesystem for the target machine having CTARGET architecture. But looking at toolchain*.eclass, the cross toolchain eventually might be located in /usr/${CHOST}/${CTARGET} instead. So /usr/${CTARGET} actually feels identical to /chroot, except that the cross-compiler has it configured --with-sysroot already. And then I always get confused by canadian cross... > 1. move a Prefix installation into another place. (Prefix -> Prefix) > > env PORTAGE_OVERWRITE_EPREFIX=/new_prefix emerge @system Yeah, this used to be: $ env EPREFIX=/new_prefix emerge @system > 1a. Cross build out a new Prefix, e.g. amd64 to arm, glibc to uclibc (Prefix > ~> Prefix) > > ?????? how the command line should look like? Set up /new_prefix/etc/make.profile (&co) for the target profile, then $ env EPREFIX=/new_prefix emerge @system > 2. bootstrap a vanilla Gentoo from Prefix (Prefix -> vanilla) > > env ROOT=/chroot EPREFIX='' emerge @system How's that different to 1. ? > 2a. cross bootstrap a vanilla Gentoo from Prefix (Preifx ~> vanilla) > > ??? how the command line should look like? Same as 0a., with extra EPREFIX=''. > 3. Prefix chaining: build out a second running Prefix based on the first. > (Prefix ~ Prefix) > > READONLY_EPREFIX="/first:DEPEND" ??? This, and $ env EPREFIX=/second emerge @system There isn't so much difference to 1., except that /second is configured to resolve DEPENDs from /first. On a sidenote, it should be possible to configure RDEPENDs from /first if necessary.
Created attachment 351172 [details, diff] Adds support for cross-prefix building by distinguishing portage's prefix from the package prefix This patch to mainline portage (I haven't tested whether it applies cleanly against prefix-portage) makes portage distinguish between the portage EPREFIX and the to-be-installed package EPREFIX. It does not introduce a BPREFIX variable; instead, it uses the builtin portage.const.EPREFIX for the portage EPREFIX, and the overridable portage.settings["EPREFIX"] for the package prefix. The package prefix defaults to the portage prefix, of course, but can be set with the EPREFIX environment variable; the portage prefix can still be overridden with the old PORTAGE_OVERRIDE_EPREFIX variable, though that should basically never be necessary. Portage uses the package EPREFIX for basically everything. The only big place where the portage EPREFIX is used, is in constructing the ebuild PATH: the package EPREFIX isn't used in the path, as that would royally break cross compiling. With this patch, it should be possible to construct a prefix installation at a different prefix by just doing `EPREFIX=/foo emerge @system`, though a few more unrelated problems need to be fixed before that will actually work.
(In reply to Ruud Koolen from comment #36) > Created attachment 351172 [details, diff] [details, diff] > Adds support for cross-prefix building by distinguishing portage's prefix > from the package prefix Thanks, I've committed your patch with few trivial modifications: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=c9f6aa9f0151adb3c86706eaef1914cdbdcf2b6d Things I changed: * Fixed ResolverPlayground to override portage.const.EPREFIX, which is necessary due to the other changes. * Re-arranged the GLOBAL_CONFIG_PATH and DEPCACHE_PATH adjustments so that the constants are unprefixed, allowing ResolverPlayground override of portage.const.EPREFIX to work properly. * Used PORTAGE_OVERRIDE_EPREFIX to implement the --host-root option for ebuild best_version and has_version functions. * Added EPREFIX and --prefix to the emerge man page.
This is fixed in 2.1.12.6 and 2.2.0_alpha181.
Here are some important fixes in git: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=40400439b0c48fbc822b622c1758be3587f4a1f1
(In reply to Zac Medico from comment #39) Released in 2.1.12.7 and 2.2.0_alpha182.
(In reply to Zac Medico from comment #39) > Here are some important fixes in git: > > http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit; > h=40400439b0c48fbc822b622c1758be3587f4a1f1 Just curious, does this fix count for ROOT!=/ case?
(In reply to Benda Xu from comment #41) > Just curious, does this fix count for ROOT!=/ case? That case should have been handled before, and now both should work.
