Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 395633 - prefix-portage-2.2.01.19981 fails with EPREFIX='' emerge
Summary: prefix-portage-2.2.01.19981 fails with EPREFIX='' emerge
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Unclassified (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on:
Blocks: 384167 395505 454602 472632
  Show dependency tree
 
Reported: 2011-12-22 03:46 UTC by Benda Xu
Modified: 2013-06-20 03:31 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
EPREFIX-handling.patch (0001-do-not-let-EPREFIX-from-environment-to-override-port.patch,1.74 KB, patch)
2011-12-23 06:41 UTC, Benda Xu
Details | Diff
0001-before-building-load-EPREFIX-from-environment.patch (0001-before-building-load-EPREFIX-from-environment.patch,4.60 KB, patch)
2011-12-23 18:17 UTC, Benda Xu
Details | Diff
Adds support for cross-prefix building by distinguishing portage's prefix from the package prefix (distinguish-portage-prefix-from-package-prefix.patch,12.70 KB, patch)
2013-06-17 07:39 UTC, Ruud Koolen (RETIRED)
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Benda Xu gentoo-dev 2011-12-22 03:46:28 UTC
an attempt to cross emerge on prefix leads me here. when ROOT is set to ${EPREFIX}/usr/${CHOST}, prefix-portage will install ebuilds into ${EPREFIX}/usr/${CHOST}/${EPREFIX}/.

The desired location should be ${EPREFIX}/usr/${CHOST} without another ${EPREFIX}.

A straightforward idea would be "ROOT=${EPREFIX}/usr/ EPREFIX="" <other cross compile env vars with crossdev> emerge ...." However prefix-portage fails to find "PORTDIR" "PORTAGE_CONFIGROOT", etc. many crucial locations.

Apparently, _prefix-portage mixes the EPREFIX for its own and EPREFIX for installing packages_.
 
I have back traced the warnings, and find:

PORTDIR is set in make.gloabals as "PORTDIR=${EPREFIX}/usr/portage", which, as also grobian points out, should be hardcoded instead of relative to EPREFIX. Or, (re)introduce BPREFIX (build prefix, like the idea of CBUILD and CHOST in toolchain).

PORTAGE_CONFIGROOT should be set in ${EPREFIX}/usr/${CHOST}, but even I set it in the environment, it is ignored first, gives a warning, and finally comes back. There must be something wrong here. 

at pym/portage/__init__.py:515 

        if settings["ROOT"] == "/":
                trees._running_eroot = trees._target_eroot
        else:

                # When ROOT != "/" we only want overrides from the calling
                # environment to apply to the config that's associated
                # with ROOT != "/", so pass a nearly empty dict for the env parameter.
                <....>
                settings = config(config_root=None, target_root="/",
                        env=clean_env, eprefix=eprefix)
                <....>

config_root is intentionally reset to None (later will default to "/", which is not what we want by specifying PORTAGE_CONFIGROOT in the environment) if ROOT!="/". 

later despite of the warning portage picks up PORTAGE_CONFIGROOT and can be examined with "EPREFIX='' PORTAGE_CONFIGROOT='blah' emerge --info".

BTW, it also seems that some tricks for crossdev is treaked towards new features of portage master branch. (call vapier) What's the block of merging prefix branch entirely into master? I may find some time to push this.

regards


Reproducible: Always
Comment 1 Benda Xu gentoo-dev 2011-12-22 03:53:22 UTC
adding prefix team to Cc
Comment 2 Zac Medico gentoo-dev 2011-12-22 05:06:08 UTC
(In reply to comment #0)
> PORTDIR is set in make.gloabals as "PORTDIR=${EPREFIX}/usr/portage", which, as
> also grobian points out, should be hardcoded instead of relative to EPREFIX.

Okay, that's fixed in git now:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=e1c6bba8c26525cca498894237c6421f86c98dfb

>                 <....>
>                 settings = config(config_root=None, target_root="/",
>                         env=clean_env, eprefix=eprefix)
>                 <....>
> 
> config_root is intentionally reset to None (later will default to "/", which is
> not what we want by specifying PORTAGE_CONFIGROOT in the environment) if
> ROOT!="/". 

I guess this is a bug in the prefix branch. In the master branch, it defaults to the same value as EPREFIX in this code from pym/portage/package/ebuild/_config/LocationsManager.py:

		if self.eprefix is None:
			self.eprefix = portage.const.EPREFIX

		if self.config_root is None:
			self.config_root = self.eprefix + os.sep

> What's the block of merging prefix branch entirely into master? I may find
> some time to push this.

It's happening gradually. Since v2.2.0_alpha81, it's possible to install and use the master branch of portage in a prefix installation. However, it currently lacks inter-revision support, so you have to convert inter-revision ebuilds to normal versions before it will work. I'm not sure if it's necessary to add inter-revision support to the master branch, since there are very few ebuilds using it.
Comment 3 Zac Medico gentoo-dev 2011-12-22 05:09:30 UTC
(In reply to comment #2)
> >                 <....>
> >                 settings = config(config_root=None, target_root="/",
> >                         env=clean_env, eprefix=eprefix)
> >                 <....>
> > 
> > config_root is intentionally reset to None (later will default to "/", which is
> > not what we want by specifying PORTAGE_CONFIGROOT in the environment) if
> > ROOT!="/". 
> 
> I guess this is a bug in the prefix branch. In the master branch, it defaults
> to the same value as EPREFIX in this code from
> pym/portage/package/ebuild/_config/LocationsManager.py:
> 
>         if self.eprefix is None:
>             self.eprefix = portage.const.EPREFIX
> 
>         if self.config_root is None:
>             self.config_root = self.eprefix + os.sep

I forgot to mention that the config_root=None part is intended regardless of the PORTAGE_CONFIGROOT environment variable, since the config instance that's being constructed there is intended represent the "host" config, which is used to satisfy pure build-time dependencies (DEPEND).
Comment 4 Zac Medico gentoo-dev 2011-12-22 05:21:32 UTC
(In reply to comment #0)
> The desired location should be ${EPREFIX}/usr/${CHOST} without another
> ${EPREFIX}.
> 
> A straightforward idea would be "ROOT=${EPREFIX}/usr/ EPREFIX="" <other cross
> compile env vars with crossdev> emerge ...."

This seems reasonable. I think a problem with the prefix branch is the following code in pym/portage/const.py:

# pick up EPREFIX from the environment if set
if "EPREFIX" in os.environ:
	if os.environ["EPREFIX"] != "":
		EPREFIX = os.path.normpath(os.environ["EPREFIX"])
	else:
		EPREFIX = os.environ["EPREFIX"]

The problem is that portage.const.EPREFIX is being overridden, which corrupts the default config_root setting in LocationsManager. Since you only want the EPREFIX variable to override EPREFIX for your cross-installation, it doesn't make sense for the EPREFIX variable to override portage.const.EPREFIX, since that will corrupt the config instance that's intended to satisfy pure build-time dependencies (DEPEND).
Comment 5 Jeremy Olexa (darkside) (RETIRED) archtester Gentoo Infrastructure gentoo-dev Security 2011-12-22 05:35:48 UTC
(In reply to comment #2)
> currently lacks inter-revision support, so you have to convert inter-revision
> ebuilds to normal versions before it will work. I'm not sure if it's necessary
> to add inter-revision support to the master branch, since there are very few
> ebuilds using it.

Ya, don't ever think about supporting "inter-revisions".
Comment 6 Benda Xu gentoo-dev 2011-12-22 06:46:33 UTC
1. A trivial question: What is 'inter-revisions' actually? could you point out an example ebuild with inter-revision in prefix tree?

2. from the series of zmedico's comments (thanks :), it seems that the only problem resides in const.py. (ROOT='/' check is intended, reading from portage.const.EPREFIX is also intended, and have strong reasons 
behind those)

I suspect getting EPREFIX from env in const.py is responsible for prefix bootstrapping.
Comment 7 Zac Medico gentoo-dev 2011-12-22 07:06:20 UTC
(In reply to comment #6)
> 1. A trivial question: What is 'inter-revisions' actually? could you point out
> an example ebuild with inter-revision in prefix tree?

The docs cover inter-revisions here:

http://www.gentoo.org/proj/en/gentoo-alt/prefix/techdocs.xml#doc_chap2_sect5

These are the ones that I copied to a local overlay in order to avoid inter-revisions:

sys-devel/libperl
sys-devel/gcc-config
sys-devel/binutils-config
app-arch/rpm2targz

> 2. from the series of zmedico's comments (thanks :), it seems that the only
> problem resides in const.py. (ROOT='/' check is intended, reading from
> portage.const.EPREFIX is also intended, and have strong reasons 
> behind those)

We may need to tweak the create_trees() function and/or callers, to make your EPREFIX override work as desired (so it only applies to the cross-install).

> I suspect getting EPREFIX from env in const.py is responsible for prefix
> bootstrapping.

In the master branch, I'm currently using a PORTAGE_OVERRIDE_EPREFIX variable to override portage.const.EPREFIX when necessary. It only needs to be overridden like this in special situations, so it needs to use a different variable that the one that's used for cross-install applications.
Comment 8 Benda Xu gentoo-dev 2011-12-22 08:11:43 UTC
(In reply to comment #7)

> The docs cover inter-revisions here:
> 
> http://www.gentoo.org/proj/en/gentoo-alt/prefix/techdocs.xml#doc_chap2_sect5
> 
> These are the ones that I copied to a local overlay in order to avoid
> inter-revisions:
> 
> sys-devel/libperl
> sys-devel/gcc-config
> sys-devel/binutils-config
> app-arch/rpm2targz

Great, thanks. I suppose it's time to drop inter-revision. Provided prefix is accepted by gentoo community (not a proof of concept for now), We should apply preifx patches to x86 tree.

> > 2. from the series of zmedico's comments (thanks :), it seems that the only
> > problem resides in const.py. (ROOT='/' check is intended, reading from
> > portage.const.EPREFIX is also intended, and have strong reasons 
> > behind those)
> 
> We may need to tweak the create_trees() function and/or callers, to make your
> EPREFIX override work as desired (so it only applies to the cross-install).

to my understanding, portage.create_trees is used for making the dependence tree for all cases. if so, we should not touch it. it ain't broken for cross-install.
 
> In the master branch, I'm currently using a PORTAGE_OVERRIDE_EPREFIX variable
> to override portage.const.EPREFIX when necessary. It only needs to be
> overridden like this in special situations, so it needs to use a different
> variable that the one that's used for cross-install applications.

I saw that var in portage.const, too. 

I tried to delete the two lines (it just sets EPREFIX to '', which is not intended, do not know the logic behind it. Anyway we can always do EPREFIX='/' or PORTAGE_OVERRIDE_EPREFIX='' for such perpose.)

# pick up EPREFIX from the environment if set
if "EPREFIX" in os.environ:
    if os.environ["EPREFIX"] != "":
        EPREFIX = os.path.normpath(os.environ["EPREFIX"])
-    else:
-        EPREFIX = os.environ["EPREFIX"]


I am poking through pym/_emerge/Scheduler.py to find out how task.settings is set up. I think we can respect os.environ["EPREFIX"] there, or move the part of code to portage.const for cleanness.
Comment 9 Benda Xu gentoo-dev 2011-12-22 08:34:14 UTC
I am doing "emerge busybox"

a quick question: how is self._task_queues in Scheduler.py constucted for e.g. compile and install? I looked around and could not find what code actually did the injection into the queue.
Comment 11 Fabian Groffen gentoo-dev 2011-12-22 10:15:36 UTC
Just a couple of steps back before lots of code is being changed.

At some time in the past, Prefix Portage had the ability to do "cross-Prefix" installs.  haubi and mduft can correct me if I'm wrong here, they are the ones who "did" it, next to prefix-chaining, which should not be confused with this topic, although slightly related.

A cross-Prefix emerge is similar to a ROOT install, but then using a different EPREFIX, that is, the resulting installed files are actually usable on the very system being worked on without any tricks/help whatsoever.

An example of intended cross-Prefix emerge:

# "bootstrap" new Prefix install from /var/tmp/gentoo-prefix based Prefix
# into $HOME/Gentoo
% /var/tmp/gentoo-prefix/startprefix
Entering Gentoo Prefix /var/tmp/gentoo-prefix
(Bootstrap:~) % env EPREFIX="$HOME/Gentoo" emerge system
<snip 168 packages>
(Bootstrap:~) % /var/tmp/gentoo-prefix/usr/portage/scripts/bootstrap-prefix.sh $HOME/Gentoo startscript
<snip>
(Bootstrap:~) % logout
Leaving Gentoo Prefix with exit status 0
% $HOME/Gentoo/startprefix
Entering Gentoo Prefix /export/home/neo/Gentoo
(Gentoo:~) % emerge --sync
<snip retrieving a tree>
(Gentoo:~) % emerge -av happily_live_hereafter
<...>

To facilitate this, a variable BPREFIX was introduced, which holds the EPREFIX as the running Portage was configured (installed) with.  BPREFIX is used to e.g. locate bash, mv, sandbox and fakeroot.

The override of EPREFIX was meant for Portage to completely "move" into the given target EPREFIX, and just grab the bare minimals (python, bash, ...) from its own Prefix, to be able to pull itself out of the mud and/or repair a possibly broken Prefix.
Comment 12 Benda Xu gentoo-dev 2011-12-22 20:56:11 UTC
BPREFIX first gets introduced here:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=2faf56cef94ad74a529e146ba41769b4a32df58a

is there another vcs for prefix-portage that tracks the story from grobian "At some time in the past, Prefix Portage had the ability to do "cross-Prefix"
installs."?

One puzzle to me is that:

$ git checkout 2faf56cef94ad74a529e146ba41769b4a32df58a
$ grep BPREFIX -r .
./bin/isolated-functions.sh:            unset BPREFIX DEFAULT_PATH EPREFIX EXTRA_PATH PORTAGE_GROUP PORTAGE_USER
./pym/_emerge/main.py:from portage.const import EPREFIX, BPREFIX, EPREFIX_LSTRIP
./pym/portage/__init__.py:              EPREFIX, EPREFIX_LSTRIP, BPREFIX, rootuid
./pym/portage/package/ebuild/config.py: USER_VIRTUALS_FILE, EPREFIX, EPREFIX_LSTRIP, BPREFIX
./pym/portage/package/ebuild/_config/special_env_vars.py:       "BPREFIX", "DEFAULT_PATH", "EXTRA_PATH",
./pym/portage/const.py:BPREFIX = EPREFIX
./pym/portage/const.py:SANDBOX_BINARY           = BPREFIX + "/usr/bin/sandbox"
./pym/portage/const.py:FAKEROOT_BINARY          = BPREFIX + "/usr/bin/fakeroot"

see? BPREFIX is not used actually. I wonder how cross-Prefix emerge is implemented.

Or, I am looking for a wrong clue of BPREFIX.
Comment 13 Benda Xu gentoo-dev 2011-12-22 21:09:57 UTC
(In reply to comment #11)

> The override of EPREFIX was meant for Portage to completely "move" into the
> given target EPREFIX, and just grab the bare minimals (python, bash, ...) from
> its own Prefix, to be able to pull itself out of the mud and/or repair a
> possibly broken Prefix.

after a couple hours of sleep, I can quite reasonate with this definition. Let's do it like this:

1. keep the current status of portage.const: EPREFIX can be overrided from env, although PORTAGE_OVERRIDE_EPREFIX serves the same perpose(? see 3). And keep BPREFIX=EPREFIX

2. replace portage.const.EPREFIX usage in portage with portage.const.BPREFIX when necessary to do the trick.

3. merge BPREFIX things to portage master branch, and convert PORTAGE_OVERRIDE_EPREFIX to PORTAGE_OVERRIDE_BPREFIX.

What do you guys think?

Let me kick out a patch now.
Comment 14 Benda Xu gentoo-dev 2011-12-22 21:36:04 UTC
(In reply to comment #11)
> Just a couple of steps back before lots of code is being changed.
> 
> At some time in the past, Prefix Portage had the ability to do "cross-Prefix"
> installs.  haubi and mduft can correct me if I'm wrong here, they are the ones
> who "did" it, next to prefix-chaining, which should not be confused with this
> topic, although slightly related.

Hmm, prefix chaining is here:

http://overlays.gentoo.org/proj/alt/browser/trunk/prefix-overlay/sys-apps/portage/files/portage-2.2.00.15801-prefix-chaining.patch?order=name

This is a great feature. when merged to portage master branch with the cross ROOT magic, we will be able to bootstrap gentoo from a working gentoo (prefix or not) *officially*, without catalyst stage3.
Comment 15 Benda Xu gentoo-dev 2011-12-22 21:42:26 UTC
a further step would be distributing binary ebootstrap-xxxx.{deb,rpm,etc.} in other distro. emerge gentoo out of other distros ;) just like dev-util/debootstrap (hmm, that's binary packages and easier to bootstrap)

At least, bootstrap a prefix before standalone gentoo.
Comment 16 Benda Xu gentoo-dev 2011-12-23 00:22:41 UTC
what's the purpose of LocationsManager? It is only used once, no code reuse is achieved.

$ grep -n LocationsManager -r .
./pym/portage/package/ebuild/config.py:54:from portage.package.ebuild._config.LocationsManager import LocationsManager
./pym/portage/package/ebuild/config.py:277: locations_manager = LocationsManager(config_root=config_root,
./pym/portage/package/ebuild/_config/LocationsManager.py:5: 'LocationsManager',
./pym/portage/package/ebuild/_config/LocationsManager.py:31:class LocationsManager(object):
Comment 17 Zac Medico gentoo-dev 2011-12-23 04:19:12 UTC
(In reply to comment #13)
> 2. replace portage.const.EPREFIX usage in portage with portage.const.BPREFIX
> when necessary to do the trick.

I don't think that portage.const.BPREFIX is really necessary. The way that the master branch currently works is that portage.const.EPREFIX serves as the default prefix. It's useful for python apps that use the portage API to find out the prefix that they're running inside (an portage itself uses it for the same purpose). Any other prefix, aside from portage.const.EPREFIX, can be encapsulated with a config instance using the "eprefix" argument to the config constructor.

(In reply to comment #16)
> what's the purpose of LocationsManager? It is only used once, no code reuse is
> achieved.

It's for encapsulation.
Comment 18 Benda Xu gentoo-dev 2011-12-23 05:18:13 UTC
(In reply to comment #17)

> Any other prefix, aside from portage.const.EPREFIX, can be
> encapsulated with a config instance using the "eprefix" argument to the config
> constructor.

What variable can hold other prefix? we have two alternatives:

1. do not let EPREFIX from env override portage.const.EPREFIX; and pull the var from os.envrion['EPREFIX']

2. use PORTAGE_OVERRIDE_EPREFIX.

either feels not good.
Comment 19 Zac Medico gentoo-dev 2011-12-23 05:35:34 UTC
(In reply to comment #18)
> (In reply to comment #17)
> 
> > Any other prefix, aside from portage.const.EPREFIX, can be
> > encapsulated with a config instance using the "eprefix" argument to the config
> > constructor.
> 
> What variable can hold other prefix? we have two alternatives:
> 
> 1. do not let EPREFIX from env override portage.const.EPREFIX; and pull the var
> from os.envrion['EPREFIX']
> 
> 2. use PORTAGE_OVERRIDE_EPREFIX.
> 
> either feels not good.

The way that it already works with cross-installs is to create a config instance corresponding to the ROOT variable for the cross-install. So what I'm suggesting is to use the same approach, and have that config instance have an EPREFIX variable which differs from portage.const.EPREFIX.

For example, with the master branch, try the following command:

ROOT=/tmp/test_root python -c 'import portage ; print portage.settings["ROOT"]'

The ROOT override is encapsulated in the portage.settings config instance. I'm suggesting that EPREFIX overrides for cross-install be encapsulated in portage.settings in exactly the same way, with no need to have it affect any constants from the portage.const module.
Comment 20 Benda Xu gentoo-dev 2011-12-23 06:41:27 UTC
Created attachment 296717 [details, diff]
EPREFIX-handling.patch

before building, load EPREFIX from environment

this is only a dirty fix. not sure if it breaks prefix bootstrap

there is a conflict for the handling of EPREFIX between master and
prefix branches (and between zmedico and grobian).

we should sit down and settle the scheme.
Comment 21 Fabian Groffen gentoo-dev 2011-12-23 08:03:44 UTC
I'm lenient here, since I no longer understand what Portage is supposed to do with vars.  I prefer to leave it to someone who does.

Perhaps what we're looking for is not cross-EPREFIX (because it seems to suck by design, and is hard to grasp), but prefix-chaining instead (like heroxbd already pointed out), with the addition of being able to turn a chained-instance into a standalone instance somehow (emerge -e world + some magic switch -- USE="-prefix-chaining" -- maybe)
Comment 22 Benda Xu gentoo-dev 2011-12-23 12:48:32 UTC
Jee... The EPREFIX and EROOT in current prefix portage is really a mass. Quite a lot of inconsistencies :(
Comment 23 Benda Xu gentoo-dev 2011-12-23 18:17:57 UTC
Created attachment 296767 [details, diff]
0001-before-building-load-EPREFIX-from-environment.patch

a well tuned updated patch, seems working.
Comment 24 Benda Xu gentoo-dev 2011-12-24 06:04:10 UTC
I've spent my whole day yesterday (~14 hours) tracing portage variables and settings handling with pdb. (way too much abstraction than I expected) Now that I have gained some direct feeling of present portage variable handling, I found some instances of legacy in prefix branch accessing portage.const.EPREFIX when actually settings['EPREFIX'] should be used.

I agree cross compiling and emerging is not so exciting for people when don't care about embedded systems. Nevertheless, I regard *EPREFIX='' ROOT='/tmp/test' emerge busybox* to be natural to think with portage. If it breaks, it means something is inconsistant, if not wrong, with present treatment of EPREFIX in portage. We need to agree upon a var policy (oh, unfortunately) before prefix branch diverges too much from master. 

Please review my newest patch (respecting settings encapsulation) in case I am misusing something or a better way exists. If it is acceptable it can be applied to prefix tree.

(need to confirm from prefix members if 

# pick up EPREFIX from the environment if set
if "EPREFIX" in os.environ:
    if os.environ["EPREFIX"] != "":
        EPREFIX = os.path.normpath(os.environ["EPREFIX"])
    else:
        EPREFIX = os.environ["EPREFIX"]

in const.py is used (only?) for bootstrap)
Comment 25 Fabian Groffen gentoo-dev 2011-12-24 09:06:36 UTC
(In reply to comment #24)
> I've spent my whole day yesterday (~14 hours) tracing portage variables and
> settings handling with pdb. (way too much abstraction than I expected) Now that
> I have gained some direct feeling of present portage variable handling, I found
> some instances of legacy in prefix branch accessing portage.const.EPREFIX when
> actually settings['EPREFIX'] should be used.

Ok, that's possible.  History here, is that the code was using portage.const.EPREFIX and portage.const.EPREFIX_LSTRIP throughout.  When EPREFIX became part of settings, it could be used instead.  I only took over the changes from zmedicom which usually were replacements of the const EPREFIX.

> I agree cross compiling and emerging is not so exciting for people when don't
> care about embedded systems. Nevertheless, I regard *EPREFIX=''
> ROOT='/tmp/test' emerge busybox* to be natural to think with portage. If it
> breaks, it means something is inconsistant, if not wrong, with present
> treatment of EPREFIX in portage. We need to agree upon a var policy (oh,
> unfortunately) before prefix branch diverges too much from master. 

Prefix branch has never been so close to master as it is right now.  I feel we cannot diverge any more now, because it is of much more benefit when the two branches can be merged and maintained as single piece.

> Please review my newest patch (respecting settings encapsulation) in case I am
> misusing something or a better way exists. If it is acceptable it can be
> applied to prefix tree.
> 
> (need to confirm from prefix members if 
> 
> # pick up EPREFIX from the environment if set
> if "EPREFIX" in os.environ:
>     if os.environ["EPREFIX"] != "":
>         EPREFIX = os.path.normpath(os.environ["EPREFIX"])
>     else:
>         EPREFIX = os.environ["EPREFIX"]
> 
> in const.py is used (only?) for bootstrap)

This is there since the day that cross-prefix was introduced.  That is long before settings had a EPREFIX member.  This made Portage able to change based on the environment.  I guess the best thing to do now, is to simply remove that bit, since it seems to no longer do anything useful but confuse and look wrong.
Comment 26 Fabian Groffen gentoo-dev 2012-01-10 17:45:56 UTC
I dropped the entire BPREFIX and EPREFIX override code.  That should just cleanup some.
Comment 27 Benda Xu gentoo-dev 2013-04-08 03:55:15 UTC
Hi Zac,

This is a reply to your email at

    http://article.gmane.org/gmane.linux.gentoo.devel/83548

>>     self.eroot = self.target_root.rstrip(os.sep) + self.eprefix + os.sep
>> 
>> What is the logic behind this construction?

> ROOT corresponds to a valid chroot point. EPREFIX is an offset inside a
> valid chroot point.

an EPREFIX offset inside a valid chroot point is legal, meaning we are foreign/cross compiling a Gentoo Prefix, but practically useless.

1. Gentoo Prefix often doesn't have root privilege. It cannot perform a chroot.
2. Things inside ROOT are already fully controled (if the user could chroot into to), there is of little interest run another Gentoo Prefix inside a brand new chroot.

At the same time, foreign/cross compiling a Gentoo Prefix from Gentoo Prefix would be an interesting story. Then we will have:

${EPREFIX}/${ROOT}${NEW_EPREFIX}

And still, even in this case, ${NEW_EPREFIX} instead of ${EPREFIX} should be used under ${ROOT}. ${EPREFIX} is "at the same level" as ${ROOT} and it is natural to prepend ROOT.

Benda
Comment 28 Zac Medico gentoo-dev 2013-04-08 04:47:18 UTC
(In reply to comment #27)
> an EPREFIX offset inside a valid chroot point is legal, meaning we are
> foreign/cross compiling a Gentoo Prefix, but practically useless.
> 
> 1. Gentoo Prefix often doesn't have root privilege. It cannot perform a
> chroot.
> 2. Things inside ROOT are already fully controled (if the user could chroot
> into to), there is of little interest run another Gentoo Prefix inside a
> brand new chroot.

Agreed.

> At the same time, foreign/cross compiling a Gentoo Prefix from Gentoo Prefix
> would be an interesting story. Then we will have:
> 
> ${EPREFIX}/${ROOT}${NEW_EPREFIX}
> 
> And still, even in this case, ${NEW_EPREFIX} instead of ${EPREFIX} should be
> used under ${ROOT}. ${EPREFIX} is "at the same level" as ${ROOT} and it is
> natural to prepend ROOT.

ROOT has traditionally been an absolute path, and I don't see how it's useful to change it into a path that's relative to ${EPREFIX}. They're independent of each other, so why should one be relative to another?
Comment 29 Benda Xu gentoo-dev 2013-04-08 05:08:00 UTC
(In reply to comment #28)

> ROOT has traditionally been an absolute path, and I don't see how it's
> useful to change it into a path that's relative to ${EPREFIX}. 

> They're independent of each other, so why should one be relative to another?

Yeah, ROOT and EPREFIX are orthogonal, and ROOT is an absolute path (usually "/" or /usr/${CTARGET}). What concerned us is EROOT: We want to define it as something useful.

Up until now all the EROOT usage case is when ROOT=/, where there is no difference. However, EROOT=EPREFIX/ROOT is more useful than EROOT=ROOT EPREFIX/ when we are proceeding to support cross development under Gentoo Prefix.
Comment 30 Zac Medico gentoo-dev 2013-04-08 05:14:21 UTC
(In reply to comment #29)
> Yeah, ROOT and EPREFIX are orthogonal, and ROOT is an absolute path (usually
> "/" or /usr/${CTARGET}). What concerned us is EROOT: We want to define it as
> something useful.

If it has a new meaning, then maybe use a different variable name, to avoid confusion? EROOT already has a very specific meaning for EAPIs 3, 4, and 5. Giving it a new meaning now just seems a little crazy.

> Up until now all the EROOT usage case is when ROOT=/, where there is no
> difference. However, EROOT=EPREFIX/ROOT is more useful than EROOT=ROOT
> EPREFIX/ when we are proceeding to support cross development under Gentoo
> Prefix.

Give it an new name please. How about EPREFIXROOT?
Comment 31 Benda Xu gentoo-dev 2013-04-08 07:52:00 UTC
(In reply to comment #30)

> If it has a new meaning, then maybe use a different variable name, to avoid
> confusion? EROOT already has a very specific meaning for EAPIs 3, 4, and 5.
> Giving it a new meaning now just seems a little crazy.

Provided no one has stepped into the craziness of mixing Gentoo Prefix and cross development seriously yet, we could secretly change the definition without an impact.

While it is conceptually crazy(redefining a well accepted variable), cleanness also counts. After all, we are not redefining it in the sense ROOT=/ in all use cases.

> > Up until now all the EROOT usage case is when ROOT=/, where there is no
> > difference. However, EROOT=EPREFIX/ROOT is more useful than EROOT=ROOT
> > EPREFIX/ when we are proceeding to support cross development under Gentoo
> > Prefix.
> 
> Give it an new name please. How about EPREFIXROOT?

If we have to invent a new variable, how about PROOT, EPROOT, EPREROOT? EPREFIXROOT is long.
Comment 32 Michael Haubenwallner gentoo-dev 2013-04-08 10:57:13 UTC
(In reply to comment #29)
> Up until now all the EROOT usage case is when ROOT=/, where there is no
> difference. However, EROOT=EPREFIX/ROOT is more useful than EROOT=ROOT
> EPREFIX/ when we are proceeding to support cross development under Gentoo
> Prefix.

I'm not really used to cross-compiling with ROOT - so maybe simply this is why I fail to see the use-case where EROOT=EPREFIX+ROOT is more useful than EROOT=ROOT+EPREFIX ?

For the "prefix-chaining" mentioned earlier:
Yes, we (mduft&me) still /do/ prefix-chaining - based on some very old snapshot of both prefix-portage and the prefix-tree, because of being _stable_ on all our platforms (ppc-aix, hppa-hpux, ia64-hpux, x86-linux, sparc-solaris, x86-solaris, x86-interix, x86-winnt).

However, we hope to be able to schedule a general update in a not-so-far future, eventually replacing Interix by Cygwin or MinGW (while keeping x86-winnt).

For how we (currently) use prefix-chaining:

Assume /first is where prefix-portage is installed into and maintained by default due to portage-builtin vars.

Assume /second is the chained prefix, configured this way:
* EPREFIX=/second environment variable, also set by /second/startprefix

* /second/etc/make.profile points to some different profile - eventually setting different CHOST and ACCEPT_KEYWORDS in case of x86-winnt, where /first is x86-interix. Actually, creating native Windows binaries is the hard reason we need prefix-chaining for. And as we need it anyway, we do it on any platform so we have to maintain one uniform layout only.

* /second/etc/make.conf contains: READONLY_EPREFIX="/first:DEPEND"
This means that portage does resolve
+ runtime-deps (RDEPEND) from /second as usual, but
+ buildtime-deps (DEPEND) from both /second _and_ /first.
Agreed, this may be a misuse of DEPEND/RDEPEND, and we eventually would need something like BDEPEND - but it does work quite well here.
And yes, DEPENDs not found in /first currently are merged into /second.

In fact, we still have need for prefix-chaining in one way (EPREFIX=/second emerge ...) or another (with an upgrade path then).
Comment 33 Zac Medico gentoo-dev 2013-04-08 16:26:00 UTC
(In reply to comment #31)
> If we have to invent a new variable, how about PROOT, EPROOT, EPREROOT?
> EPREFIXROOT is long.

Yes, any name that's not already taken is fine. Still, I would advise against making ROOT a non-absolute path, because there you go again giving new meanings to variables that already have entrenched meanings.
Comment 34 Benda Xu gentoo-dev 2013-04-09 07:21:22 UTC
haubi, thanks a lot for the explanating of Prefix chaining.

It's time for us to rethink and redesign the precise meaning of the variables.

Let me summaries all the uses cases and get a GLEP out for reviewing.

0. bootstrap vanilla Gentoo from vanilla (vanilla -> vaniila)

env ROOT=/chroot emerge @system

does it work?

0a. cross bootstrap vanilla Gentoo from vanilla (vanilla ~> vaniila)
    with crossdev:

    powerpc-unknown-linux-gnu-emerge @system

   in which ROOT=/usr/${CTARGET}

1. move a Prefix installation into another place. (Prefix -> Prefix)

env PORTAGE_OVERWRITE_EPREFIX=/new_prefix emerge @system

1a. Cross build out a new Prefix, e.g. amd64 to arm, glibc to uclibc (Prefix ~> Prefix) 

   ?????? how the command line should look like?

2. bootstrap a vanilla Gentoo from Prefix (Prefix -> vanilla)

env ROOT=/chroot EPREFIX='' emerge @system

2a. cross bootstrap a vanilla Gentoo from Prefix (Preifx ~> vanilla)

    ??? how the command line should look like?

3. Prefix chaining: build out a second running Prefix based on the first. (Prefix ~ Prefix)

READONLY_EPREFIX="/first:DEPEND" ???
Comment 35 Michael Haubenwallner gentoo-dev 2013-04-09 09:25:42 UTC
(In reply to comment #34)
> 0. bootstrap vanilla Gentoo from vanilla (vanilla -> vaniila)
> 
> env ROOT=/chroot emerge @system
> 
> does it work?

Haven't tried myself, but this is how I would expect it to be done.

> 0a. cross bootstrap vanilla Gentoo from vanilla (vanilla ~> vaniila)
>     with crossdev:
> 
>     powerpc-unknown-linux-gnu-emerge @system
> 
>    in which ROOT=/usr/${CTARGET}

Haven't used crossdev myself, but as far as I understand, /usr/${CTARGET} _only_ contains the CTARGET toolchain (binutils, gcc and other machine-dependent code-generators), running on the build machine, necessary to build another /chroot, which becomes the root filesystem for the target machine having CTARGET architecture.

But looking at toolchain*.eclass, the cross toolchain eventually might be located in /usr/${CHOST}/${CTARGET} instead.

So /usr/${CTARGET} actually feels identical to /chroot, except that the cross-compiler has it configured --with-sysroot already.

And then I always get confused by canadian cross...

> 1. move a Prefix installation into another place. (Prefix -> Prefix)
> 
> env PORTAGE_OVERWRITE_EPREFIX=/new_prefix emerge @system

Yeah, this used to be:
$ env EPREFIX=/new_prefix emerge @system

> 1a. Cross build out a new Prefix, e.g. amd64 to arm, glibc to uclibc (Prefix
> ~> Prefix) 
> 
>    ?????? how the command line should look like?

Set up /new_prefix/etc/make.profile (&co) for the target profile, then
$ env EPREFIX=/new_prefix emerge @system

> 2. bootstrap a vanilla Gentoo from Prefix (Prefix -> vanilla)
> 
> env ROOT=/chroot EPREFIX='' emerge @system

How's that different to 1. ?

> 2a. cross bootstrap a vanilla Gentoo from Prefix (Preifx ~> vanilla)
> 
>     ??? how the command line should look like?

Same as 0a., with extra EPREFIX=''.

> 3. Prefix chaining: build out a second running Prefix based on the first.
> (Prefix ~ Prefix)
> 
> READONLY_EPREFIX="/first:DEPEND" ???

This, and
$ env EPREFIX=/second emerge @system

There isn't so much difference to 1., except that /second is configured to resolve DEPENDs from /first.
On a sidenote, it should be possible to configure RDEPENDs from /first if necessary.
Comment 36 Ruud Koolen (RETIRED) archtester Gentoo Infrastructure gentoo-dev Security 2013-06-17 07:39:11 UTC
Created attachment 351172 [details, diff]
Adds support for cross-prefix building by distinguishing portage's prefix from the package prefix

This patch to mainline portage (I haven't tested whether it applies cleanly against prefix-portage) makes portage distinguish between the portage EPREFIX and the to-be-installed package EPREFIX. It does not introduce a BPREFIX variable; instead, it uses the builtin portage.const.EPREFIX for the portage EPREFIX, and the overridable portage.settings["EPREFIX"] for the package prefix.

The package prefix defaults to the portage prefix, of course, but can be set with the EPREFIX environment variable; the portage prefix can still be overridden with the old PORTAGE_OVERRIDE_EPREFIX variable, though that should basically never be necessary.

Portage uses the package EPREFIX for basically everything. The only big place where the portage EPREFIX is used, is in constructing the ebuild PATH: the package EPREFIX isn't used in the path, as that would royally break cross compiling.

With this patch, it should be possible to construct a prefix installation at a different prefix by just doing `EPREFIX=/foo emerge @system`, though a few more unrelated problems need to be fixed before that will actually work.
Comment 37 Zac Medico gentoo-dev 2013-06-18 21:03:44 UTC
(In reply to Ruud Koolen from comment #36)
> Created attachment 351172 [details, diff] [details, diff]
> Adds support for cross-prefix building by distinguishing portage's prefix
> from the package prefix

Thanks, I've committed your patch with few trivial modifications:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=c9f6aa9f0151adb3c86706eaef1914cdbdcf2b6d

Things I changed:

* Fixed ResolverPlayground to override portage.const.EPREFIX, which is necessary due to the other changes.
* Re-arranged the GLOBAL_CONFIG_PATH and DEPCACHE_PATH adjustments so that the constants are unprefixed, allowing ResolverPlayground override of portage.const.EPREFIX to work properly.
* Used PORTAGE_OVERRIDE_EPREFIX to implement the --host-root option for ebuild best_version and has_version functions.
* Added EPREFIX and --prefix to the emerge man page.
Comment 38 Zac Medico gentoo-dev 2013-06-18 21:50:31 UTC
This is fixed in 2.1.12.6 and 2.2.0_alpha181.
Comment 40 Zac Medico gentoo-dev 2013-06-20 00:48:52 UTC
(In reply to Zac Medico from comment #39)
Released in 2.1.12.7 and 2.2.0_alpha182.
Comment 41 Benda Xu gentoo-dev 2013-06-20 03:28:48 UTC
(In reply to Zac Medico from comment #39)
> Here are some important fixes in git:
> 
> http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;
> h=40400439b0c48fbc822b622c1758be3587f4a1f1

Just curious, does this fix count for ROOT!=/ case?
Comment 42 Zac Medico gentoo-dev 2013-06-20 03:31:36 UTC
(In reply to Benda Xu from comment #41)
> Just curious, does this fix count for ROOT!=/ case?

That case should have been handled before, and now both should work.