Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 615594

Summary: sys-apps/portage: wrongly prepends EPREFIX to 1st parameter of dosym
Product: Portage Development Reporter: Michał Górny <mgorny>
Component: Core - Ebuild SupportAssignee: Portage team <dev-portage>
Status: RESOLVED FIXED    
Severity: major CC: oli.huber, pms, prefix
Priority: Normal Keywords: InVCS
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://github.com/gentoo/gentoo/pull/4419
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 706142    
Attachments: List of packages doing dosym with an absolute path (and supporting prefix)

Description Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-04-15 07:37:00 UTC
dosym has:

  # when absolute, prefix with offset for Gentoo Prefix
  target="${1}"
  [[ ${target:0:1} == "/" ]] && target="${EPREFIX}${target}"

which means that all absolute paths in 1st argument are converted to EPREFIX. This is totally wrong:

1. It is against the PMS. The only bit referring to Prefix in the section is:

| [...] all filenames created or modified are relative to the staging directory including the offset-prefix ED in offset-prefix aware EAPIs [...]

which applies only to destination files. Both input file and dosym target should be treated literally.

2. It is against the common sense. As noted in the linked pull request, developers tend to actually add ${EPREFIX} to the target themselves which results in double prefixing.

3. It makes it impossible to intentionally link to system (out-of-prefix) directories, e.g. /dev, /proc, /run, /sys. We do not want those paths implicitly prefixed.
Comment 1 Ulrich Müller gentoo-dev 2017-04-15 07:49:27 UTC
Looks like this needs to be reverted:

commit 3ff6c0d01a074169b8afeddc668191541f5e4330
Author: Jeremy Olexa <darkside@gentoo.org>
Date:   Thu Mar 29 15:02:56 2012 -0700

    dosym: add $EPREFIX to absolute target
Comment 2 Fabian Groffen gentoo-dev 2017-04-15 08:16:46 UTC
1. observation: it makes dosym as helper the only one(?) that doesn't respect EPREFIX
2. dunno what common sense is/was, at some point in time, the aim was to make the necessary changes to ebuilds for Prefix support minimal, thus making helpers dealing with EPREFIX transparantly in the back.  If we want to change this, no problem, just saying where it stems from.
3. seems like an unlikely example, since the helper should deal with D and/or ROOT, shouldn't it?  Therefore the implicit EPREFIX handling.  For those packages that really aim to link stuff from the host system (like native-cctools) it's not odd to manually use ln -s over dosym, IMO.
Comment 3 Ulrich Müller gentoo-dev 2017-04-15 08:37:09 UTC
(In reply to Fabian Groffen from comment #2)
> 2. dunno what common sense is/was, at some point in time, the aim was to
> make the necessary changes to ebuilds for Prefix support minimal, thus
> making helpers dealing with EPREFIX transparantly in the back.

Why isn't this in the spec then? EAPI 3 was approved in 2010, and dosym was changed in 2012. I've searched the mailing list archives for discussion or review of that patch, but couldn't find any.

Also not sure if it makes ebuilds simpler. A quick search surfaces many examples like the following (from dev-python/pygobject and dev-lang/php, respectively):

    dosym "${prefixed_sitedir#${EPREFIX}}/gtk-2.0/codegen/codegen.py" \
        "/usr/lib/python-exec/${EPYTHON}/pygobject-codegen-2.0"

    dosym "${PHP_EXT_INI_DIR#${EPREFIX}}/opcache.ini" \
        "${PHP_EXT_INI_DIR_ACTIVE#${EPREFIX}}/opcache.ini"

> If we want to change this, no problem, just saying where it stems from.

Not even sure how we would proceed there. If we want to change the spec, we could do this for EAPI 7. That wouldn't prevent us from reverting portage for existing EAPIs.
Comment 4 Fabian Groffen gentoo-dev 2017-04-15 09:03:15 UTC
As you mention, since 2012 this behaviour is in there, so one could argue given by the number of years it is in effect a de-facto standard.

Consider the following (I'm repeating darkside's reasoning here)

  insinto /etc/udev
  newins "${FILESDIR}/udev.conf.post_050" udev.conf
  doins extras/cdsymlinks.conf

here insinto gets absolute argument, but $EPREFIX pre-fixed.  Following the devmanual example:

  dosym ../../../sbin/udevsend /etc/hotplug.d/default/10-udev.hotplug

/etc/hotplug.d is taken within ${ED} here (so $EPREFIX is implicit), because this is relative, it is ok that it takes EPREFIX too.  Now the situation this bug is about:

  dosym /etc/rmt /usr/sbin/rmt

So, likewise, /usr/sbin is put in $ED, but the suggestion in this bug is that /etc/rmt is put in verbatim (thus taken from the host).  The ebuild developer surely meant to use /etc/rmt from ${EPREFIX}.  Like I said before, for the really rare cases where one want to link host-based stuff inside the Prefix, ln can be used.  After the many years of using Prefix systems, there never has been the need for such symlink except for cases like native-cctools (system rescue only).  OIW the chance a developer actually wants to link from the host system is nihil.

I hope this sheds a little more light on the issue why darkside (and the prefix team in general) thought absolute arguments from dosym should be pre-fixed with $EPREFIX.
Comment 5 Ulrich Müller gentoo-dev 2017-04-15 09:36:54 UTC
(In reply to Fabian Groffen from comment #4)
> As you mention, since 2012 this behaviour is in there, so one could argue
> given by the number of years it is in effect a de-facto standard.

I thought we were long past that line of argument. :(
Comment 6 Fabian Groffen gentoo-dev 2017-04-15 09:39:38 UTC
(In reply to Ulrich Müller from comment #5)
> (In reply to Fabian Groffen from comment #4)
> > As you mention, since 2012 this behaviour is in there, so one could argue
> > given by the number of years it is in effect a de-facto standard.
> 
> I thought we were long past that line of argument. :(

You can fix it, but how are you going to deal with the reality?  Perhaps this helps?

(ptah:/export/gentoo/gentoo-x86) % git grep -F "dosym /" | wc -l
1364
Comment 7 Fabian Groffen gentoo-dev 2017-04-15 09:44:28 UTC
A bit more realistic classification of the problem:

(ptah:/export/gentoo/gentoo-x86) % git grep -F "dosym /" | cut -d: -f1 | cut -d/ -f1,2 | uniq | wc -l
311

Those either would have to be bumped to EAPI=7, or fixed.  Not impossible, just someone's got to do it without breaking it, so EAPI bump is the only solution, which leaves the current practice in effect, so full-circle unless you want to require $EPREFIX explicitly added to dosym in EAPI=7, the best you can do now, is document it retroactively, or put it in EAPI=7 and bump packages there as soon as feasible.
Comment 8 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-04-15 09:45:07 UTC
(In reply to Fabian Groffen from comment #2)
> 1. observation: it makes dosym as helper the only one(?) that doesn't
> respect EPREFIX

I don't know which helpers you mean. Almost all other helpers use regular paths for input files (relative to cwd or absolute to build env), and obviously don't prepend EPREFIX there. The only helper where this could be relevant is dohard which is banned anyway for all EAPIs supporting Prefix.

For dohard PMS doesn't even clearly define whether for dohard $1 is supposed to contain ${D} or not. Nevertheless, it is not used by any ebuilds in ::gentoo, so we can ignore it.

The destination where symlink is supposed to be created respects EPREFIX which is consistent with other helpers.
Comment 9 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-04-15 09:49:16 UTC
(In reply to Fabian Groffen from comment #7)
> A bit more realistic classification of the problem:
> 
> (ptah:/export/gentoo/gentoo-x86) % git grep -F "dosym /" | cut -d: -f1 | cut
> -d/ -f1,2 | uniq | wc -l
> 311

It would be useful if you reduced that list to ebuilds that actually support Prefix, and not just accidentally get partial support via an ugly hack in Portage.
Comment 10 Fabian Groffen gentoo-dev 2017-04-15 09:52:52 UTC
I get the feeling you just don't appreciate I tried to help you here.  I assume you'll take it from here.  Have a good day.
Comment 11 Ulrich Müller gentoo-dev 2017-04-15 10:03:00 UTC
(In reply to Michał Górny from comment #9)

$ git grep -F "dosym /" | cut -d: -f1 |
xargs grep -E -l 'KEYWORDS.*-(linux|macos|solaris|aix|winnt|cygwin|mint)' |
cut -d/ -f1,2 | uniq | wc -l
69
Comment 12 Ulrich Müller gentoo-dev 2017-04-15 13:25:38 UTC
Created attachment 470094 [details]
List of packages doing dosym with an absolute path (and supporting prefix)

There are a few additional packages that either have multiple blanks after the dosym command, or quote their first argument, or use a path in a variable. This rises the count to 90. See attached list. Most can be changed to use relative symlinks.
Comment 13 Ulrich Müller gentoo-dev 2017-04-15 13:55:40 UTC
(In reply to Fabian Groffen from comment #7)
> Those either would have to be bumped to EAPI=7, or fixed.  Not impossible,
> just someone's got to do it without breaking it,

The breakage was done in 2012 with the above mentioned portage commit 3ff6c0d01. What we try to do is to fix the mess that is resulting from it.

> so EAPI bump is the only solution, which leaves the current practice in
> effect, so full-circle unless you want to require $EPREFIX explicitly added
> to dosym in EAPI=7, the best you can do now, is document it retroactively,
> or put it in EAPI=7 and bump packages there as soon as feasible.

I don't see why we should change the spec retroactively for EAPIs 3 to 6. By council decision in the 2016-03-13 meeting, "all non-PMS-conformant behaviour should be considered a bug". There are some qualifications to this, but only for historical package manager behaviour (i.e., predating PMS), which doesn't apply here.
https://projects.gentoo.org/council/meeting-logs/20160313-summary.txt

Because EAPI 7 won't be released anytime soon, I see fixing Portage to comply with the spec as the only option that is feasible here.


(In reply to Fabian Groffen from comment #10)
> I assume you'll take it from here.  Have a good day.

This isn't helpful.
Comment 14 Zac Medico gentoo-dev 2017-04-15 16:43:35 UTC
A possible compromise would be to retroactively change the spec to say that the first argument of dosym must be relative, and behavior with absolute paths is undefined. Then fix all the ebuilds to use relative paths, since those are better anyway (they resolve correctly for a root filesystem unpacked into a directory or mounted somewhere).
Comment 15 Ulrich Müller gentoo-dev 2017-04-15 18:12:16 UTC
(In reply to Zac Medico from comment #14)
> A possible compromise would be to retroactively change the spec to say that
> the first argument of dosym must be relative, and behavior with absolute
> paths is undefined. Then fix all the ebuilds to use relative paths,

That's not really feasible, because there are commands like the following around (example from dev-tex/feynmf):

    dosym /usr/share/doc/${PF}/manual.ps \
        ${TEXMF}/doc/latex/${PN}/${PN}-manual.ps

Not sure how I would convert this to a relative path (short of implementing a loop that would count path components).

> since those are better anyway (they resolve correctly for a root filesystem
> unpacked into a directory or mounted somewhere).

Generally I agree with this, but see the example above.
Comment 16 Fabian Groffen gentoo-dev 2017-04-16 08:10:11 UTC
(In reply to Ulrich Müller from comment #13)
> (In reply to Fabian Groffen from comment #7)
> > Those either would have to be bumped to EAPI=7, or fixed.  Not impossible,
> > just someone's got to do it without breaking it,
> 
> The breakage was done in 2012 with the above mentioned portage commit
> 3ff6c0d01. What we try to do is to fix the mess that is resulting from it.

For you it is a mess, for me it is what should've been defined in PMS.  I don't know how it happened to be defined like this, but it looks like an oversight to me.  It's all irrelevant though, since you intend to "fix" this, even though there is close to zero valid cases where the "fixed" situation is required/useful -- in particular to non-Prefix environments.

> > so EAPI bump is the only solution, which leaves the current practice in
> > effect, so full-circle unless you want to require $EPREFIX explicitly added
> > to dosym in EAPI=7, the best you can do now, is document it retroactively,
> > or put it in EAPI=7 and bump packages there as soon as feasible.
> 
> I don't see why we should change the spec retroactively for EAPIs 3 to 6. By
> council decision in the 2016-03-13 meeting, "all non-PMS-conformant
> behaviour should be considered a bug". There are some qualifications to
> this, but only for historical package manager behaviour (i.e., predating
> PMS), which doesn't apply here.
> https://projects.gentoo.org/council/meeting-logs/20160313-summary.txt
> 
> Because EAPI 7 won't be released anytime soon, I see fixing Portage to
> comply with the spec as the only option that is feasible here.

Reality is blocking you here.  For your example where you need an absolute path, if you fix it (according to your logic), you'll add $EPREFIX in front of it, however that will make Portage prefix it with $EPREFIX again, so double Prefix.
Another way around would be to make a transition Portage which prefixes with $EPREFIX unless it already is prefixed with $EPREFIX.  Apart from this being messy, it is impossible to tell whether or not someone upgraded Portage for this, to solve that EAPIs where invented.

So, my only concern here is that you don't break Prefix.  How do you plan on addressing this situation?  (And how do you plan on explaining developers they have to do something different/non-intuitive after to support Prefix in their ebuilds?  add another warning for dosym that they likely forgot to use $EPREFIX with dosym?)
Comment 17 Ulrich Müller gentoo-dev 2017-04-16 11:21:31 UTC
(In reply to Fabian Groffen from comment #16)
> > The breakage was done in 2012 with the above mentioned portage commit
> > 3ff6c0d01. What we try to do is to fix the mess that is resulting from it.
> 
> For you it is a mess, for me it is what should've been defined in PMS.
> I don't know how it happened to be defined like this,

See bug 296716 comment 30 (attachment 215915 [details, diff]). While you have explicitly addressed other helpers like dobin or dosed there, the patch doesn't say anything special about dosym.

> but it looks like an oversight to me. It's all irrelevant though, since you
> intend to "fix" this, even though there is close to zero valid cases where
> the "fixed" situation is required/useful -- in particular to non-Prefix
> environments.

It's not so clear that it is an oversight. Both ways of defining it are valid, and both have drawbacks. Prefixing absolute paths with EPREFIX would make dosym behave the same way as dohard. Maybe that's not a strong argument because dohard is banned since EAPI 4. OTOH, adding EPREFIX would make it impossible to link to paths if they are outside of prefix (see 3. in comment 0).

> So, my only concern here is that you don't break Prefix.  How do you plan on
> addressing this situation?

Not sure. Portage behaviour in EAPIs 3 and 4 is effectively undefined, unless we pretend that versions 2.1.10.33 to .45 and 2.2.0_alpha73 to _alpha96 never existed.
Comment 18 Fabian Groffen gentoo-dev 2017-04-16 14:16:33 UTC
(In reply to Ulrich Müller from comment #17)
> (In reply to Fabian Groffen from comment #16)
> > > The breakage was done in 2012 with the above mentioned portage commit
> > > 3ff6c0d01. What we try to do is to fix the mess that is resulting from it.
> > 
> > For you it is a mess, for me it is what should've been defined in PMS.
> > I don't know how it happened to be defined like this,
> 
> See bug 296716 comment 30 (attachment 215915 [details, diff] [details, diff]). While you
> have explicitly addressed other helpers like dobin or dosed there, the patch
> doesn't say anything special about dosym.

Oversight from my side, my bad.

> > but it looks like an oversight to me. It's all irrelevant though, since you
> > intend to "fix" this, even though there is close to zero valid cases where
> > the "fixed" situation is required/useful -- in particular to non-Prefix
> > environments.
> 
> It's not so clear that it is an oversight. Both ways of defining it are
> valid, and both have drawbacks. Prefixing absolute paths with EPREFIX would
> make dosym behave the same way as dohard. Maybe that's not a strong argument
> because dohard is banned since EAPI 4. OTOH, adding EPREFIX would make it
> impossible to link to paths if they are outside of prefix (see 3. in comment
> 0).

(the same as dohard[banned] or insinto or fowners, ...)

The only drawback raised sofar (impossibility to link to paths outside of the prefix) is completely artificial, and by design NOT wanted.  Could you list any more drawbacks of the current approach if there are any?  It would certainly help me to understand why this is made such a problem.

> > So, my only concern here is that you don't break Prefix.  How do you plan on
> > addressing this situation?
> 
> Not sure. Portage behaviour in EAPIs 3 and 4 is effectively undefined,
> unless we pretend that versions 2.1.10.33 to .45 and 2.2.0_alpha73 to
> _alpha96 never existed.

I suggest you think about how to fix this.  It seems little to no use in discussing this if we don't have a solid plan on how to fix this properly.
Comment 19 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-04-16 20:44:13 UTC
(In reply to Fabian Groffen from comment #16)
> (And how do you plan on explaining developers
> they have to do something different/non-intuitive after to support Prefix in
> their ebuilds?

There is a fair number of developers who believe that having a magical forced EPREFIX for 1st argument of dosym if it's an absolute path is counter-intuitive. The best proof of this is that people actually write ebuilds that prepend ${EPREFIX} there. Not to mention all the cases where path is obtained externally, and therefore contains EPREFIX.

Furthermore, I should point out that of all three package managers, only Portage applies this magic, and only after patch coming from Prefix team. I would therefore dare say that people working on package managers (and PMS) also considered taking target literally as the intuitive behavior.


(In reply to Fabian Groffen from comment #18)
> (the same as dohard[banned] or insinto or fowners, ...)

insinto operates on output path only. So does fowner (*modified file* as listed in the PMS). While in dosym, you're altering arbitrary target parameter which is not an output path. Furthermore, unlike dohard it is not even logically limited to install tree.

dosym can take relative path, or absolute path. All other listed helpers work with absolute path only. Most importantly:

  dodir foo

is equivalent to:

  dodir /foo

and in both cases, ED is applied. No such equivalence applies to the first parameter of dosym, since the path is meant to be taken *literally*. Altering it conditionally makes it non-literal, and therefore counter-intuitive.


> The only drawback raised sofar (impossibility to link to paths outside of
> the prefix) is completely artificial, and by design NOT wanted.  Could you
> list any more drawbacks of the current approach if there are any?  It would
> certainly help me to understand why this is made such a problem.

It would certainly be more meaningful if you just straight away didn't discard valid use cases as 'completely artificial, and by design NOT wanted'. Because as I see it, this is *the only* meaningful use case for absolute symlink targets. Using absolute targets for paths inside install tree is just ignorant, and relative targets should be used instead as they provide better portability (and work e.g. when Gentoo is mounted in a mount point, e.g. for recovery).
Comment 20 Fabian Groffen gentoo-dev 2017-04-17 07:58:24 UTC
(In reply to Michał Górny from comment #19)
> (In reply to Fabian Groffen from comment #16)
> > The only drawback raised sofar (impossibility to link to paths outside of
> > the prefix) is completely artificial, and by design NOT wanted.  Could you
> > list any more drawbacks of the current approach if there are any?  It would
> > certainly help me to understand why this is made such a problem.
> 
> It would certainly be more meaningful if you just straight away didn't
> discard valid use cases as 'completely artificial, and by design NOT
> wanted'. Because as I see it, this is *the only* meaningful use case for
> absolute symlink targets. Using absolute targets for paths inside install
> tree is just ignorant, and relative targets should be used instead as they
> provide better portability (and work e.g. when Gentoo is mounted in a mount
> point, e.g. for recovery).

Can you give some examples for those meaningful use cases for absolute symlink targets pointing outside of your EPREFIX install?
Comment 21 Ulrich Müller gentoo-dev 2017-04-17 09:43:22 UTC
(In reply to Fabian Groffen from comment #18)

By the same argument, econf should prefix all path parameters with ${EPREFIX} then. It does so only for the default parameters, but not for the ones that are explicitly passed to it.

Helper commands should behave in a systematic and predictable way, but dosym would be the only helper that is (sometimes) prepending its input data with ${EPREFIX}.

Also, symlink(2) on Linux says: "symlink() creates a symbolic link named linkpath which contains the string target" (or on FreeBSD: "name1 is the string used in creating the symbolic link"). It's really just an arbitrary string, and both symlink(2) and ln(1) treat it as opaque data. Why should dosym interpret it and modify it depending on its contents?
Comment 22 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-04-19 12:48:51 UTC
I'd go for the following plan:

1. Make repoman complain about known-wrong dosym calls, suggesting relative paths instead,

2. Make Portage handle double EPREFIX for the interim period,

3. Fix ebuilds to use relative targets or EPREFIX explicitly, if relative is not feasible,

4. When all is done, remove the EPREFIX hack from Portage.
Comment 23 Fabian Groffen gentoo-dev 2017-04-19 13:05:11 UTC
How do you ensure at point 4 a fixed Portage is in use?
Comment 24 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-04-19 13:32:13 UTC
(In reply to Fabian Groffen from comment #23)
> How do you ensure at point 4 a fixed Portage is in use?

Why would I need to? You dismissed the uses for fixed Portage anyway, so I guess it wouldn't do any harm if we didn't hurry with ensuring that all users use PMS-adhering Portage.
Comment 25 Fabian Groffen gentoo-dev 2017-04-19 13:39:04 UTC
Ok, so you deliberatly want to break Prefix users because you just don't care.  Then why do you need @prefix on Cc?
Comment 26 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-04-19 13:43:13 UTC
(In reply to Fabian Groffen from comment #25)
> Ok, so you deliberatly want to break Prefix users because you just don't
> care.  Then why do you need @prefix on Cc?

I really don't understand what you're talking about. Maybe you should step back and read everything again. Until people upgrade to Portage from step 4, everything will continue to work as it does now.

The only possible breaking point is for users of Portage before patch from step 2. For those, ebuilds explicitly using EPREFIX will continue being broken -- as they are now already. I seriously doubt we'll have many more ebuilds using that since we will be strongly preferring relative targets anyway.
Comment 27 Fabian Groffen gentoo-dev 2017-04-19 13:49:06 UTC
You cannot ensure a version of Portage, that's the reason EAPI was brought into life.

You are changing a long-standing behaviour of Portage, that's in use by some, which may not (be in the position to) upgrade.
Comment 28 Ulrich Müller gentoo-dev 2017-04-19 16:48:37 UTC
(In reply to Fabian Groffen from comment #27)
> You cannot ensure a version of Portage, that's the reason EAPI was brought
> into life.

Already at present you cannot rely on a specific behaviour, because it has changed between Portage versions. I don't see how the upgrade path suggested in comment 22 could make that any worse, especially if the plan is to convert as many ebuilds as possible to relative paths.

The only thing that is really constant here is the specification of dosym in PMS.
Comment 29 Fabian Groffen gentoo-dev 2017-04-19 19:10:49 UTC
When did this change?  Are you referring to 5 years ago?
Comment 30 Ulrich Müller gentoo-dev 2017-04-20 15:11:25 UTC
(In reply to Fabian Groffen from comment #29)
> When did this change?  Are you referring to 5 years ago?

There are certainly installations that are such old. It is indeed unlikely that anybody still uses such an old Portage version. However, there may be packages installed by such a version and not updated since then (cf. the number of EAPI 3 and 4 ebuilds still in the tree).
Comment 31 Fabian Groffen gentoo-dev 2017-04-21 06:49:57 UTC
I fail to see the relevance of your statement.

The "breakage" that was introduced (a Portage change) has certainly become reality for all Prefix users (the only ones affected).  If there was any pain (which I doubt, since at that time Prefix was completely on its own tree and Portage version), then it has been suffered by now.

Therefore, stating that simply changing it back is just as worse is ignorant to the reality.  Changing it back, will cause problems and pain (double prefix errors) for Prefix users, which means it makes it *worse*.

Since you've started anyway, what is your anticipated timeline here?  How long are you going to let the individual steps last?
Comment 32 Ulrich Müller gentoo-dev 2017-04-21 09:48:36 UTC
(In reply to Fabian Groffen from comment #31)
> Therefore, stating that simply changing it back is just as worse is ignorant
> to the reality.

Nobody is suggesting to simply change it back. Our aim should be to avoid breakage on users' systems.

> Changing it back, will cause problems and pain (double prefix errors) for
> Prefix users, which means it makes it *worse*.

Have you looked at the commit that went into Portage? There shouldn't be any double prefixing (but please review again):
https://gitweb.gentoo.org/proj/portage.git/commit/?id=18712aa6924862bbab5814c6199e7d7ab416d005

The only drawback of that patch is that targets outside of EPREFIX cannot be linked to. That's not possible with current Portage either, though.

> Since you've started anyway, what is your anticipated timeline here?  How
> long are you going to let the individual steps last?

Assuming that you refer to the steps outlined in comment 22, I suggest:

> 1. Make repoman complain about known-wrong dosym calls, suggesting relative
> paths instead,

Immediately.

> 2. Make Portage handle double EPREFIX for the interim period,

Immediately.

> 3. Fix ebuilds to use relative targets ...

Immediately. (I hope that everybody agrees that this is a good change.)

> ... or EPREFIX explicitly, if relative is not feasible,

Obviously we must wait for deployment (and stabilisation?) of the Portage version containing the fix from step 2.

> 4. When all is done, remove the EPREFIX hack from Portage.

After completion of step 3, I'd wait with that for another year at least. Also, any new EAPIs won't contain that hack any more.
Comment 33 Fabian Groffen gentoo-dev 2017-04-21 10:03:22 UTC
(In reply to Ulrich Müller from comment #32)
> (In reply to Fabian Groffen from comment #31)
> > Therefore, stating that simply changing it back is just as worse is ignorant
> > to the reality.
> 
> Nobody is suggesting to simply change it back. Our aim should be to avoid
> breakage on users' systems.
> 
> > Changing it back, will cause problems and pain (double prefix errors) for
> > Prefix users, which means it makes it *worse*.
> 
> Have you looked at the commit that went into Portage? There shouldn't be any
> double prefixing (but please review again):
> https://gitweb.gentoo.org/proj/portage.git/commit/
> ?id=18712aa6924862bbab5814c6199e7d7ab416d005

Of course I'm referring to step 3 where actual breakage is introduced when relative approach doesn't work.

> The only drawback of that patch is that targets outside of EPREFIX cannot be
> linked to. That's not possible with current Portage either, though.
> 
> > Since you've started anyway, what is your anticipated timeline here?  How
> > long are you going to let the individual steps last?
> 
> Assuming that you refer to the steps outlined in comment 22, I suggest:
> 
> > 1. Make repoman complain about known-wrong dosym calls, suggesting relative
> > paths instead,
> 
> Immediately.
> 
> > 2. Make Portage handle double EPREFIX for the interim period,
> 
> Immediately.
> 
> > 3. Fix ebuilds to use relative targets ...
> 
> Immediately. (I hope that everybody agrees that this is a good change.)

(Let's agree to not having strong feelings against relative vs absolute links.)

> > ... or EPREFIX explicitly, if relative is not feasible,
> 
> Obviously we must wait for deployment (and stabilisation?) of the Portage
> version containing the fix from step 2.

Right, so here you introduce breakage, because you can't control the usage of a certain version of Portage (hello EAPI=6.1).  Can we build build a list of packages that actually have a strong need for an absolute link target to review if this is actually going to be an issue or not.  In case we actually are lucky, there is no such package, and we need not to bother about this.  I'm looking at things like nano/vim, and would like to see what package maintainer there think should be done.

> 
> > 4. When all is done, remove the EPREFIX hack from Portage.
> 
> After completion of step 3, I'd wait with that for another year at least.
> Also, any new EAPIs won't contain that hack any more.

This sounds managable to me if step 3 is separated into 3relative and 3absolute, and the wait is done after 3relative (unless 3absolute is an empty set as far as Prefix is concerned).
Comment 34 Zac Medico gentoo-dev 2017-05-20 18:38:10 UTC
The double prefix protection is in portage-2.3.6:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=18712aa6924862bbab5814c6199e7d7ab416d005
Comment 35 Ulrich Müller gentoo-dev 2020-01-19 14:44:09 UTC
(In reply to Zac Medico from comment #34)
> The double prefix protection is in portage-2.3.6:
> 
> https://gitweb.gentoo.org/proj/portage.git/commit/
> ?id=18712aa6924862bbab5814c6199e7d7ab416d005

Three years have passed but the "deprecated hack" is still in place. Time to remove it, i.e. revert portage to the correct behaviour it had in 2011?
Comment 36 Zac Medico gentoo-dev 2020-01-19 18:52:43 UTC
(In reply to Ulrich Müller from comment #35)
> Three years have passed but the "deprecated hack" is still in place. Time to
> remove it, i.e. revert portage to the correct behaviour it had in 2011?

Yes, here's a patch:

https://archives.gentoo.org/gentoo-portage-dev/message/2ddab32df7df038ca5f764fd02fc3688
https://github.com/gentoo/portage/pull/499
Comment 37 Larry the Git Cow gentoo-dev 2020-01-23 07:04:44 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=926daa64ce2932685e50c5b87ebe6e1f2578feb2

commit 926daa64ce2932685e50c5b87ebe6e1f2578feb2
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2020-01-23 06:54:10 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2020-01-23 07:04:32 +0000

    sys-apps/portage: Bump to version 2.3.85
    
     #615594 dosym: revert deprecated prefix compat
     #704256 emerge-webrsync: chmod 755 temp dir
     #704320 Add QA check for unresolved soname dependencies
     #704848 doebuild: export SANDBOX_LOG=${T}/sandbox.log
     #705986 solve pypy / pypy-exe dependency cycle
    
    Bug: https://bugs.gentoo.org/706142
    Bug: https://bugs.gentoo.org/615594
    Bug: https://bugs.gentoo.org/704256
    Bug: https://bugs.gentoo.org/704320
    Bug: https://bugs.gentoo.org/704848
    Bug: https://bugs.gentoo.org/705986
    Package-Manager: Portage-2.3.85, Repoman-2.3.20
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-2.3.85.ebuild | 276 +++++++++++++++++++++++++++++++++
 2 files changed, 277 insertions(+)