Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bugzilla DB migration completed. Please report issues to Infra team via email via infra@gentoo.org or IRC
Bug 174407 - [Future EAPI] add support for dynamic SLOTS
Summary: [Future EAPI] add support for dynamic SLOTS
Status: RESOLVED WONTFIX
Alias: None
Product: Gentoo Hosted Projects
Classification: Unclassified
Component: PMS/EAPI (show other bugs)
Hardware: All All
: High enhancement (vote)
Assignee: PMS/EAPI
URL:
Whiteboard:
Keywords:
: 311267 317215 344675 434246 486566 507810 507814 (view as bug list)
Depends on:
Blocks: future-eapi 175358 393203
  Show dependency tree
 
Reported: 2007-04-13 09:51 UTC by SpanKY
Modified: 2018-02-04 13:01 UTC (History)
15 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description SpanKY gentoo-dev 2007-04-13 09:51:57 UTC
add support for things like:
SLOT="moo? ( moo ) !moo? ( 0 )"

i think this will also require SLOT to have whitespace collapsed in the process
Comment 1 Ciaran McCreesh 2007-04-13 14:40:47 UTC
I suspect this isn't appropriate for EAPI 1, since it requires significant VDB changes (slot names will have to be included, e.g. as cat/pkg-ver:slot) and probably can't be implemented quickly in Portage.
Comment 2 Zac Medico gentoo-dev 2007-05-08 09:02:25 UTC
(In reply to comment #0)
> add support for things like:
> SLOT="moo? ( moo ) !moo? ( 0 )"

Maybe whatever you're trying to accomplish with this dynamic slotting can or should be accomplished other ways.  What kind of use cases are there?
Comment 3 SpanKY gentoo-dev 2007-05-08 17:13:54 UTC
above i just mention multislot:
SLOT="multislot? ( $PV ) !multislot? ( 0 )"
so users can choose to only have 1 version of say binutils or all of them

but ultimately we'd need a way to handle ABI (CHOST) and cross-compilers (CTARGET), but portage could probably construct vdb entries based on CHOST/CTARGET combos in a diff way than SLOT
Comment 4 Zac Medico gentoo-dev 2007-07-07 15:44:15 UTC
(In reply to comment #3)
> above i just mention multislot:
> SLOT="multislot? ( $PV ) !multislot? ( 0 )"
> so users can choose to only have 1 version of say binutils or all of them

It doesn't seem appropriate for a USE flag to toggle the this type of behavior on a package. USE flags are supposed to control how the package is built and installed, not the package replacement semantics of the package manager.  Couldn't you have SLOT="${PV}" unconditionally and then users who don't want the lower slots could simple remove them using something like emerge --prune or --depclean?

It seems like it would be simpler to avoid an interaction between USE and SLOT if there are reasonable alternatives.

(In reply to comment #3)
> but ultimately we'd need a way to handle ABI (CHOST) and cross-compilers
> (CTARGET), but portage could probably construct vdb entries based on
> CHOST/CTARGET combos in a diff way than SLOT

Right, and maybe USE-dynamic SLOTs aren't the best solution for the multislot use case either.
Comment 5 Peter Volkov (RETIRED) gentoo-dev 2008-09-08 08:39:55 UTC
Another use case for this feature is currently discussed in gentoo-dev mailing
list possibility to install KDE either slotted (into /usr/kde/<version>) or
not (into /usr). And as how to remove previous kde installation became FAQ on
forums/mailing lists seems that it's useful to have singleslot installation
for general consumption.

Syntax mentioned in comment 0 can be better like: SLOT="moo:moo?0" or any
other symbol instead of '?' ( | ).

I also think that USE flags should not affect SLOTs; it's good idea to have a
separate flag but currently we have no such flag.

BTW, is it really required something more that just having either
singleslot/multislot installation. Do we need more complicated SLOT schemes,
e.g. either install in SLOT=0 or in SLOT="$(get_version_component_range 1-2)"
or in SLOT="${PV}"?
Comment 6 Zac Medico gentoo-dev 2008-09-08 16:13:16 UTC
If you want to have a given ebuild provide more than one SLOT then you also have to provide a protocol for the package manager to select a specific one of those SLOT values. Such a protocol is necessary in order for the package manager to be able to satisfy a SLOT dependency such as foo/bar:blah.
Comment 7 SpanKY gentoo-dev 2008-09-21 06:26:27 UTC
i dont know what kind of "protocol" you refer to.  the SLOT has a value.  if you want to select that value, then you use it.  there is no magic here.
Comment 8 Zac Medico gentoo-dev 2008-10-02 16:27:15 UTC
How do you know which SLOT it is though? By evaluating USE conditionals? If we do it that way, then the specific USE flag settings have to be known before the specific SLOT is known. This causes a circular definition if we want to support package sets in package.use, because package sets support SLOT atoms. Note that we are currently being asked to add support for package sets in package.use:

http://archives.gentoo.org/gentoo-dev/msg_410b417297b7dffad4c54039d3f62d27.xml

For example, suppose the user wants to be able to do something like this in package.use in order to enable the "moo" flag on the entire @kde-meta-4.1 package set:

@kde-meta-4.1 moo

If that package set contains SLOT atoms, then we need to know the SLOT values of the packages before we know which packages belong to the package set. Once we know which packages belong to the set, we can apply the USE flags specified in package.use. If SLOT depends on USE then we get a circular definition of the package set since we don't know which packages belong to the set until we know their SLOT values, we don't know the SLOT values until we know the USE flags, we don't know the USE flags applied to the packages until we know whether or not they belong to the package set, and we don't know whether or not the packages belong to the package set until we know the SLOT. It's circular.
Comment 9 Doktor Notor 2010-03-25 18:29:56 UTC
*** Bug 311267 has been marked as a duplicate of this bug. ***
Comment 10 Zac Medico gentoo-dev 2010-04-26 18:40:58 UTC
*** Bug 317215 has been marked as a duplicate of this bug. ***
Comment 11 SpanKY gentoo-dev 2010-11-09 22:20:26 UTC
*** Bug 344675 has been marked as a duplicate of this bug. ***
Comment 12 SpanKY gentoo-dev 2011-02-10 02:47:08 UTC
*** Bug 354069 has been marked as a duplicate of this bug. ***
Comment 13 Zac Medico gentoo-dev 2011-09-16 16:51:59 UTC
We may want to make the SLOT value remain constant for a given ebuild, and introduce a second variable (perhaps named CONDITIONAL_SLOT) with support for conditionals. This approach will allow SLOT atoms to continue to work as they do in existing EAPIs, by continuing to match the constant SLOT metadata. If necessary, we can add an atom syntax extension for matching the CONDITIONAL_SLOT metadata.
Comment 14 SpanKY gentoo-dev 2012-09-07 20:13:14 UTC
*** Bug 434246 has been marked as a duplicate of this bug. ***
Comment 15 Ben Longbons 2012-09-10 16:34:49 UTC
Having filed bug 434536, I propose a different solution:

For packages where multislot may be useful, have e.g. sys-devel/gcc with :4.6 slots, and sys-devel/gcc-multislot with :4.6.2 slots.
Comment 16 SpanKY gentoo-dev 2012-09-11 06:47:27 UTC
(In reply to comment #15)

yeah, not going to happen
Comment 17 SpanKY gentoo-dev 2013-10-15 19:51:56 UTC
*** Bug 486566 has been marked as a duplicate of this bug. ***
Comment 18 SpanKY gentoo-dev 2013-10-15 19:56:59 UTC
*** Bug 486566 has been marked as a duplicate of this bug. ***
Comment 19 SpanKY gentoo-dev 2013-10-15 20:00:58 UTC
*** Bug 486566 has been marked as a duplicate of this bug. ***
Comment 20 SpanKY gentoo-dev 2013-10-15 20:07:04 UTC
*** Bug 486566 has been marked as a duplicate of this bug. ***
Comment 21 Ulrich Müller gentoo-dev 2013-10-20 19:01:15 UTC
*** Bug 486566 has been marked as a duplicate of this bug. ***
Comment 22 Ciaran McCreesh 2013-10-20 19:06:32 UTC
It's a growing problem, so something needs to be done about people doing this. QA should a) enforce a ban on any new violations, and then b) look to get existing violations removed.
Comment 23 Ulrich Müller gentoo-dev 2013-10-20 19:43:21 UTC
(In reply to Ciaran McCreesh from comment #22)
> It's a growing problem, so something needs to be done about people doing
> this. QA should a) enforce a ban on any new violations,

+1

> and then b) look to get existing violations removed.

Right, but this bug is open since 2007 and nobody has suggested a good solution for the problem. Things that may or may not work:
- Move multislot ebuilds to an overlay.
- Duplicate all affected ebuilds, i.e. one normal slot and one multislot
  revision (like -r100), with the latter being package.masked.
- Duplicate packages, as suggested in comment 15.
- Have SLOT=${PV} for all ebuilds, plus USE-conditional blockers.
Nothing of this looks elegant at all.
Comment 24 Ciaran McCreesh 2013-10-20 19:46:58 UTC
(In reply to Ulrich Müller from comment #23)
> > and then b) look to get existing violations removed.
> 
> Right, but this bug is open since 2007 and nobody has suggested a good
> solution for the problem.

That could be because there isn't a problem worth solving here. What's the real impact of just getting rid of all of the offenders?
Comment 25 Ben Longbons 2013-10-20 20:20:21 UTC
(In reply to Ciaran McCreesh from comment #24)
> That could be because there isn't a problem worth solving here. What's the
> real impact of just getting rid of all of the offenders?

It is *very* useful as developer to install gcc 4.6.{0,1,2,3,4} at once. Other distros, and Gentoo in its default configuration, will only ever install the latest point release (which is what most users want), but when debugging I need to check whether I'm hitting bugs in the older ones.

And gcc is not the kind of package you want to install by hand.


I don't see any solution better than my recommendation of parallel -multislot packages, though an overlay could work from the user's perspective.

But for a maintainer, I expect that it's sensible to have an identical ebuild behave differently based on the package name, and not sensible to have an identical ebuild behave differently based on which overlay it comes from.
Comment 26 Ciaran McCreesh 2013-10-20 20:28:21 UTC
(In reply to Ben Longbons from comment #25)
> (In reply to Ciaran McCreesh from comment #24)
> > That could be because there isn't a problem worth solving here. What's the
> > real impact of just getting rid of all of the offenders?
> 
> It is *very* useful as developer to install gcc 4.6.{0,1,2,3,4} at once.

But you're not getting gcc 4.6.2 (say). You're getting gcc 4.6.2, plus some patches. So it's not useful for a developer who wants to find out whether a package compiles with a particular gcc version, or whether they're affected by a bug in a particular gcc version, or anything like that. So what is the use for developers?
Comment 27 Ben Longbons 2013-10-20 20:52:06 UTC
(In reply to Ciaran McCreesh from comment #26)
> But you're not getting gcc 4.6.2 (say). You're getting gcc 4.6.2, plus some
> patches.

For the kind of compiler bug I usually hit, it's close enough that I've never had a problem. Gentoo doesn't go crazy with gcc patches, other distros typically apply a similar set.

OTOH, Debian does all their GCC packages from SVN snapshots.
Comment 28 SpanKY gentoo-dev 2013-12-23 19:49:16 UTC
(In reply to Ciaran McCreesh from comment #26)

you do if you leverage the USE=vanilla flag.  but Gentoo stopped being a pit of rice wrt compilers years ago (like the 3.4.x timeframe).
Comment 29 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-01-12 08:38:43 UTC
Is this request worth keeping open? It looks like a big pile of potential and known issues. Implementing it would require a lot of changes to the logic and I guess some people here know better how many issues that could cause.

I think PMS team + portage team should finally say 'no', and focus on fixing the toolchain rather than trying to legalize their breakage in a new formal syntax. Multislot is useful but doesn't deserve throwing all PM internals upside down. IMHO using blockers is actually more elegant than having dynamic slots.
Comment 30 Ulrich Müller gentoo-dev 2014-01-12 10:28:26 UTC
Also, before further changes affecting dependency resolution are done, it would be good to have an assessment what impact they have on package managers' performance. Portage is already slower than it used to be (caused by EAPI 5 features? I've not yet seen a decent analysis).
Comment 31 Ciaran McCreesh 2014-01-12 12:39:35 UTC
(In reply to Ulrich Müller from comment #30)
> Also, before further changes affecting dependency resolution are done, it
> would be good to have an assessment what impact they have on package
> managers' performance. Portage is already slower than it used to be (caused
> by EAPI 5 features? I've not yet seen a decent analysis).

If you want to deal with performance, the way to start is by improving the quality and content of the input data. Everything else is minor compared to the hackery needed to work around badly specified and incorrect data. Let's stick to dismissing this misfeature on its own demerits.
Comment 32 Ben Longbons 2014-01-13 02:59:07 UTC
As someone who needs absolutely needs multislot, I have absolutely no objection to rejecting it as a *portage* feature, as long as it is provided as a package feature (either a gcc-multislot package or a multislot overlay).
Comment 33 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-01-13 06:49:23 UTC
Is there any reason we can't have only-multislot gcc? Is there any more differences than when gcc gets switched and when old version gets depcleaned?
Comment 34 Ben Longbons 2014-01-13 16:36:11 UTC
(In reply to Michał Górny from comment #33)
> Is there any reason we can't have only-multislot gcc? Is there any more
> differences than when gcc gets switched and when old version gets depcleaned?

I think it would make all the gcc-config changes very annoying. Additionally, if you upgrade from gcc 4.8.0 to 4.8.2 during a world update, it will still rebuild the rest of the packages with 4.8.0

Also look into why dev-libs/boost stopped offering multislot (/me weeps for his test setup, even though he didn't have that much stuff using boost yet)
Comment 35 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-01-13 17:02:42 UTC
(In reply to Ben Longbons from comment #34)
> (In reply to Michał Górny from comment #33)
> > Is there any reason we can't have only-multislot gcc? Is there any more
> > differences than when gcc gets switched and when old version gets depcleaned?
> 
> I think it would make all the gcc-config changes very annoying.
> Additionally, if you upgrade from gcc 4.8.0 to 4.8.2 during a world update,
> it will still rebuild the rest of the packages with 4.8.0

We could default the ebuild to automatically switch the slot, with some kind of make.conf switch to let user override that.

> Also look into why dev-libs/boost stopped offering multislot (/me weeps for
> his test setup, even though he didn't have that much stuff using boost yet)

I know the story better than you do. I was actually quite close to fixing support for slotted boost in Gentoo when the slotting was removed.
Comment 36 Steve L 2014-02-01 13:36:41 UTC
Can someone give me a technical reason *not* to use a gcc-multislot package, derived from exactly the same ebuild, as outlined in comment #25?

That way it can be hard-masked by name in default/, it's trivial to make it block the same non-multislot ebuild, since they're the same file, and further it's easy to maintain the multislot one separately, perhaps with patches in an overlay. Based on [[ $PN = *-multislot ]] one can inherit a different eclass, so it should work across toolchain packages.

Then it can stay in the tree, which makes collaboration with externals more efficient, and move more quickly in the overlay as well, as you decide. Without leading to QA issues elsewhere, since you have the same effective setup without cache problems.

I cannot see any downsides to the solution proposed, even if it takes a month or so to work out the eclass changes. After all, given that the current flag is not supposed to be set unless you exactly know what you're doing (if you're not experimenting with the toolchain, YDIW) unmasking an ebuild is not too much to ask. It also means you've taken an extraordinary step to get the potential breakage on your machine, which saves hassle for all sides later on.
Comment 37 Ryan Hill (RETIRED) gentoo-dev 2014-02-02 07:56:52 UTC
Why not take it one step further and have separate packages for every USE flag?

Seriously though, it's trivial to create a binpkg of previous micro versions and takes like a minute to switch between them.
Comment 38 Ben Longbons 2014-02-02 17:34:49 UTC
(In reply to Ryan Hill from comment #37)
> Why not take it one step further and have separate packages for every USE
> flag?
That would be silly. USE flags *work* for what they're mostly used for. They *don't* work for this case, and more importantly *can't* work.

I don't care which of the two approaches is used to solve this particularly, but making absurd arguments will not help anything.

> Seriously though, it's trivial to create a binpkg of previous micro versions
> and takes like a minute to switch between them.
Not when running without root, which all sensible auto-build scripts do. And even then, that's a minute too long.

(for CXX in g++-4.6.{0..4} g++-4.7.{0..3} g++-4.8.{0..2}; do mcd build-$CXX; ../configure CXX=$CXX; ... )
Comment 39 Steve L 2014-02-03 05:20:13 UTC
This isn't a conceptual discussion about USE flags in general: effectively the flag leads to a breaking of the whole SLOT mechanism, on purpose. There is a use-case for that, or we wouldn't have a toolchain; however it is not at all a general user, nor even standard developer, setup.

So we need to provide the exact same capability, *both* without breaking the cache resolution mechanism, and without losing the ability for toolchain-development to happens as easily as now. AFAICT the proposed solution does that, and it's much cleaner: it means we can collaborate around a multislot.eclass without extra dubious logic in the package-manglers.

Hopefully that'll lead to more involvement from other devs, since it won't be a frowned-upon area that is a source of contention any more. Hard-masking by name makes it easier to maintain, imo. Still, if you have a _technical_ reason why it won't work, please speak up.
Comment 40 Tom Wijsman (TomWij) (RETIRED) gentoo-dev 2014-02-06 08:29:49 UTC
(In reply to Ben Longbons from comment #25)
> (In reply to Ciaran McCreesh from comment #24)
> > That could be because there isn't a problem worth solving here. What's the
> > real impact of just getting rid of all of the offenders?
> 
> It is *very* useful as developer to install gcc 4.6.{0,1,2,3,4} at once.

When I read this, I'm trying to understand why you can't just do this with what an user has available; why is it needed to make the ebuild more complex for this?

As I see this, would `emerge =sys-devel/gcc-4.6.{0,1,2,3,4}*` work for an user?
Comment 41 Ben Longbons 2014-02-06 15:21:42 UTC
(In reply to Tom Wijsman (TomWij) from comment #40)
> As I see this, would `emerge =sys-devel/gcc-4.6.{0,1,2,3,4}*` work for an
> user?

The whole point of this bug is that even though the ebuild currently lets you do that, it doesn't actually work with all parts of portage. In particular, it is possible to run 'revdep-rebuild' followed by 'emerge --depclean' and end up without a working compiler.

This bug started as a request to fix portage, but that (quite rightly IMO) has been denied and now we are exploring alternate solutions.

Unfortunately, the trend seems to be "just pretend it doesn't exist, and blame the user if he ends up with a broken system".
Comment 42 Tom Wijsman (TomWij) (RETIRED) gentoo-dev 2014-02-06 15:59:15 UTC
(In reply to Tom Wijsman (TomWij) from comment #40)
> As I see this, would `emerge =sys-devel/gcc-4.6.{0,1,2,3,4}*` work for an
> user?

Sorry, was not fully awake when I wrote that; they're in the same slot so that won't work, I see. I'm wondering whether it would fit better if sys-devel/gcc is slotted the way sys-kernel/gentoo-sources is (X.Y.Z as opposed to X.Y). Is there a particular reason the slot is X.Y instead of X.Y.Z?

We could introduce syntax like :X.Y\Z whereas then \ could allow the user to select whether he wants to slot up to that character or up to the end. This would benefit sys-kernel/gentoo-sources as well; because we've had users request to be able to follow a branch, which is currently impossible due to the X.Y.Z slot.
Comment 43 Ryan Hill (RETIRED) gentoo-dev 2014-02-07 06:38:52 UTC
(In reply to Ben Longbons from comment #38)
> > Seriously though, it's trivial to create a binpkg of previous micro versions
> > and takes like a minute to switch between them.
> Not when running without root, which all sensible auto-build scripts do. And
> even then, that's a minute too long.
> 
> (for CXX in g++-4.6.{0..4} g++-4.7.{0..3} g++-4.8.{0..2}; do mcd build-$CXX;
> ../configure CXX=$CXX; ... )

It should also be trivial to unpack them into some local subdir and set the appropriate variables or options to make your build scripts find them.

(In reply to Tom Wijsman (TomWij) from comment #42)
> Sorry, was not fully awake when I wrote that; they're in the same slot so
> that won't work, I see. I'm wondering whether it would fit better if
> sys-devel/gcc is slotted the way sys-kernel/gentoo-sources is (X.Y.Z as
> opposed to X.Y). Is there a particular reason the slot is X.Y instead of
> X.Y.Z?

Portage will always update to the latest version available of an installed SLOT.  It's also "greedy" when it comes to new SLOTs, meaning if a newer SLOT of an installed package becomes available it gets pulled in whether it's required or not.

So picture this situation:  A user has gcc-4.7.3 installed.  4.8.2 becomes available and portage installs it, but the user continues to use 4.7.3.  4.7.4 becomes available with a fix for a wrong-code bug that is breaking people's systems.  With a SLOT of X.Y, portage will update to 4.7.4.  With X.Y.Z it will not because there is no update in an installed SLOT and a later SLOT is already installed.  Basically any further version bumps of any but the newest branch will be ignored by the package manager.  This is not an ideal situation.

This happens to me all the time with gentoo-sources where I usually run an older kernel until I can find the time to update my .config to a new major version.  But, while it annoys me, it makes sense there.  I also wouldn't want portage swapping out the sources in a directory I've already built in.
Comment 44 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-04-16 11:36:24 UTC
(In reply to Ryan Hill from comment #43)
> (In reply to Tom Wijsman (TomWij) from comment #42)
> > Sorry, was not fully awake when I wrote that; they're in the same slot so
> > that won't work, I see. I'm wondering whether it would fit better if
> > sys-devel/gcc is slotted the way sys-kernel/gentoo-sources is (X.Y.Z as
> > opposed to X.Y). Is there a particular reason the slot is X.Y instead of
> > X.Y.Z?
> 
> Portage will always update to the latest version available of an installed
> SLOT.

Is there any special option that needs to be set in order to reproduce this behavior? With plain:

  emerge -1vDtuN @world

portage doesn't want to update gcc:4.7 (4.7.1) to a newer version (stable 4.7.3-r1) when gcc:4.8 is installed.
Comment 45 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-04-16 17:39:31 UTC
(In reply to Ryan Hill from comment #43)
> (In reply to Tom Wijsman (TomWij) from comment #42)
> > Sorry, was not fully awake when I wrote that; they're in the same slot so
> > that won't work, I see. I'm wondering whether it would fit better if
> > sys-devel/gcc is slotted the way sys-kernel/gentoo-sources is (X.Y.Z as
> > opposed to X.Y). Is there a particular reason the slot is X.Y instead of
> > X.Y.Z?
> 
> Portage will always update to the latest version available of an installed
> SLOT.  It's also "greedy" when it comes to new SLOTs, meaning if a newer
> SLOT of an installed package becomes available it gets pulled in whether
> it's required or not.

I've tested this more thoroughly and discussed this with ulm, and it seems that are you not correct.

In general, gcc is pulled in by an atom alike 'sys-devel/gcc:*'. This means that portage pulls into the dependency tree, depending on the context, either the newest available (unmasked) or newest installed sys-devel/gcc version from any slot. Since gcc is slotted by major versions, this always means the latest slot.

This means that portage will always try to upgrade to newest gcc. Once it's installed, it will no longer try to upgrade older slots since packages specify 'sys-devel/gcc:*' which implies that they will supposedly prefer using the newest version available. This is pretty much where PMS and eselect don't like each other.

I'd say this is the expected behavior. Since gcc:4.8 is installed now, portage sees no reason to keep old gcc:4.7. The old version will be pulled in for removal on next --depclean. Why would portage try to update a package (slot) that is supposedly no longer used and practically queued for removal?

So, in order to make your statement true, the user would need to explicitly place 'sys-devel/gcc:4.7' in his world file. But in fact, he can use more general '=sys-devel/gcc-4.7*' atom there that would work as well with USE=multislot enabled. Then, the automatic switching of versions on upgrade is pretty much the case for pkg_postinst().

So, to sum up, I think that:

1. what you described doesn't happen automatically and isn't supposed to happen,

2. in order to achieve that behavior, explicit dependency needs to be placed in world file,

3. the same behavior can be achieved with =gcc-4.7* dependency that will be more generic,

and so I don't think this is an important argument to prefer the current slotting over USE=multislot slotting.
Comment 46 Ryan Hill (RETIRED) gentoo-dev 2014-04-17 02:40:39 UTC
Okay, I have every gcc slot in my world file, so that explains why I see that behaviour.  Portage uninstalling old versions of gcc just because you have the newest version installed is news to me then, and I'm surprised because that's amazingly dangerous.  I can't believe that's the standard quo.  Just because you've built a new version of gcc doesn't mean you're using it, or have rebuilt @world with it, and until you have you shouldn't uninstall the old version (and in general no sane Gentoo user should _ever_ have only one version of gcc installed).

Wrt multislot, I can't imagine any argument that will make me change the SLOT scheme on gcc for a feature that I don't support and don't think should exist.  If we can't support this cleanly with an EAPI feature then please just allow us to mask the flag, document it eats children in the description, require VERY_BRAVE_OR_VERY_STUPID be set in the env, and get on with it.
Comment 47 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-04-17 07:46:37 UTC
(In reply to Ryan Hill from comment #46)
> Okay, I have every gcc slot in my world file, so that explains why I see
> that behaviour.  Portage uninstalling old versions of gcc just because you
> have the newest version installed is news to me then, and I'm surprised
> because that's amazingly dangerous.  I can't believe that's the standard
> quo.

How would you otherwise implement it? Make portage keep old slots indefinitely and require user to manually clean up old versions of libraries? This is nowhere close to friendly.

Please remember that to the package manager, sys-devel/gcc is just another package and it will have no special behavior for it. If you want to enforce a specific behavior, you need to take into consideration how it will affect other packages.

> Just because you've built a new version of gcc doesn't mean you're
> using it, or have rebuilt @world with it, and until you have you shouldn't
> uninstall the old version (and in general no sane Gentoo user should _ever_
> have only one version of gcc installed).

This sounds like a case for sys-devel/gcc:= dep. However, I don't think we can implicitly add it via profiles. And in fact, it won't make any sense until portage starts always using the newest version because otherwise it will record the wrong deps, simple as that.

Any kind of random version switching in middle of dependency graph is broken and must not be relied upon. It simply can't work properly, and I doubt there's anything anyone can do to solve it.

> Wrt multislot, I can't imagine any argument that will make me change the
> SLOT scheme on gcc for a feature that I don't support and don't think should
> exist.

And why are you opposed to it? As far as I'm concerned, the only thing really changing is paths. Other aspects pretty much boil down to ebuild implementation details and can be solved in reasonable amount of code.

> If we can't support this cleanly with an EAPI feature then please
> just allow us to mask the flag, document it eats children in the
> description, require VERY_BRAVE_OR_VERY_STUPID be set in the env, and get on
> with it.

Masking it will not make the issues disappear. The ebuild will still be calling 'use' in global scope and that's invalid. It will only make users a bit more aware of the issue.

A possible solution would to be to drop the flag completely and use a hidden variable to control it. However, that would be no less ugly than the current solution, pretty much it will just work-around the QA/sanity checks.

And users who need that will still hit most of the breakage involved. If you really don't want to support it, then kill the support completely and handle the fallout. Provide alternatives. Sweeping it under another carpet won't improve things long-term.
Comment 48 Ben Longbons 2014-04-17 16:54:08 UTC
I think, regardless of the resolution of *this* bug, portage should absolutely never uninstall old versions of important packages such as the kernel and gcc.

Earlier, this bug very nearly left me without any working compiler - luckily I still had a *very* old version installed (4.2?) that I could use to reinstall all the other versions. It is quite likely that other such bugs exist.
Comment 49 Julian Ospald 2014-04-17 17:21:18 UTC
(In reply to Ben Longbons from comment #48)
> I think, regardless of the resolution of *this* bug, portage should
> absolutely never uninstall old versions of important packages such as the
> kernel and gcc.
> 

Totally disagree. You should read the output of "emerge --depclean".
Comment 50 Ben Longbons 2014-04-17 17:40:09 UTC
(In reply to Julian Ospald (hasufell) from comment #49)
> Totally disagree. You should read the output of "emerge --depclean".

I do. My current work flow is:

1. emerge --depclean --pretend

2. Manually copy out all the packages that are not kernels or gcc.

3. emerge --depclean <list of packages>

This is painful.
Comment 51 Tom Wijsman (TomWij) (RETIRED) gentoo-dev 2014-04-17 19:44:05 UTC
(In reply to Ben Longbons from comment #50)
> (In reply to Julian Ospald (hasufell) from comment #49)
> > Totally disagree. You should read the output of "emerge --depclean".
> 
> I do. My current work flow is [...] painful.

Put those kernels and gcc in /var/lib/portage/world, `emerge -c`, ???, Profit! :)
Comment 52 Ryan Hill (RETIRED) gentoo-dev 2014-04-18 00:17:29 UTC
(In reply to Michał Górny from comment #47)
> And why are you opposed to it? As far as I'm concerned, the only thing
> really changing is paths. Other aspects pretty much boil down to ebuild
> implementation details and can be solved in reasonable amount of code.

Because the problem I outlined in comment #43 is still valid for anyone with sys-devel/gcc:SLOT in their world file, which in my opinion should be everybody.  Because I don't want people to have to remember to switch their active compiler on every minor version bump.  Because I don't want to keep closing the same bug reports over and over when they forget.  :)

And because I think it's ugly.  This might sound dumb, but it doesn't fit my mental model of how gcc is maintained.  Right now each branch gets a SLOT. 

> Masking it will not make the issues disappear. The ebuild will still be
> calling 'use' in global scope and that's invalid.

Does the `use` call have side effects?  I thought the problem was that it's being used to set SLOT, violating metadata invariance.  If the problem is `use` in global scope then I agree masking won't help that.

> And users who need that will still hit most of the breakage involved. If you
> really don't want to support it, then kill the support completely and handle
> the fallout. Provide alternatives. Sweeping it under another carpet won't
> improve things long-term.

Well, this bug was opened 7 years ago to find an alternative.  We've pretty much nailed down a USE flag is the wrong way to go about it.  What else can we look at?
Comment 53 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-04-18 07:29:16 UTC
(In reply to Ryan Hill from comment #52)
> (In reply to Michał Górny from comment #47)
> > And why are you opposed to it? As far as I'm concerned, the only thing
> > really changing is paths. Other aspects pretty much boil down to ebuild
> > implementation details and can be solved in reasonable amount of code.
> 
> Because the problem I outlined in comment #43 is still valid for anyone with
> sys-devel/gcc:SLOT in their world file, which in my opinion should be
> everybody.

The same could be achieved via:

  =sys-devel/gcc-4.7*

while making it valid for any slotting models. It's one time switch with lasting benefit.

> Because I don't want people to have to remember to switch their
> active compiler on every minor version bump.

This is something that can be easily done via postinst -- auto-switching on minor upgrade. A bonus feature is that the user would be able to test the minor bump before having previous version removed.

> And because I think it's ugly.  This might sound dumb, but it doesn't fit my
> mental model of how gcc is maintained.  Right now each branch gets a SLOT.

I don't think slots are that much about maintenance as about making possible use more friendly. If Gentoo provides a way to have multiple minor versions of gcc installed and in use, I don't see why choice of slots should disallow that.

> > Masking it will not make the issues disappear. The ebuild will still be
> > calling 'use' in global scope and that's invalid.
> 
> Does the `use` call have side effects?  I thought the problem was that it's
> being used to set SLOT, violating metadata invariance.  If the problem is
> `use` in global scope then I agree masking won't help that.

Both are problems.

The main issue with 'use' is that it's completely undefined behavior. Some package managers will throw a big fat warning (pkgcore), some may completely error out.

Plus external tools that are only supposed to work with metadata (e.g. eix) may get real confused about functions they aren't supposed to encounter. Worst case I could imagine is something using 'multislot' implicitly because someone did dummy 'use() { :; }' to silence 'command not found'.

If we're working on this, I think we should focus on fixing both issues.

> > And users who need that will still hit most of the breakage involved. If you
> > really don't want to support it, then kill the support completely and handle
> > the fallout. Provide alternatives. Sweeping it under another carpet won't
> > improve things long-term.
> 
> Well, this bug was opened 7 years ago to find an alternative.  We've pretty
> much nailed down a USE flag is the wrong way to go about it.  What else can
> we look at?

Well, what has been proposed so far:

1. multislot-only or non-multislot-only gcc -- better for consistency, for rev-deps and stuff like that that ends up having to support two possible cases,

2. multislot gcc in an overlay or under different PN -- extra maintenance/sync burden, overlay may result in developers refusing to fix issues,

3. special environment variable to control that -- which fixes the 'use' problem but sweeps metadata variancy under the carpet,

4. some possible PMS/EAPI magic -- no agreement on any sane way to do that yet, and seems like a lot of effort for a few packages.
Comment 54 Ryan Hill (RETIRED) gentoo-dev 2014-04-18 09:39:32 UTC
(In reply to Michał Górny from comment #53)
> (In reply to Ryan Hill from comment #52)
> > Because the problem I outlined in comment #43 is still valid for anyone with
> > sys-devel/gcc:SLOT in their world file, which in my opinion should be
> > everybody.
> 
> The same could be achieved via:
> 
>   =sys-devel/gcc-4.7*
> 
> while making it valid for any slotting models. It's one time switch with
> lasting benefit.

Sure, but how many users go around manually editing their world files?  Both `emerge =gcc-4.7*` and `emerge gcc:4.7` put a SLOT atom in world.  That should work without manual intervention.  Manual intervention isn't friendly, remember?

Anyways, here's a possible solution.

We can use a hybrid approach.  Start with a major.minor SLOT (eg. 4.7).  When a new version is added, keep it at 4.7 and change the old version to a major.minor.micro SLOT (eg. 4.7.2).  Portage will still do the upgrade and remove the old version like normal (for both =sys-devel/gcc-4.7* and sys-devel/gcc:4.7 in the world file).  Because we're upgrading within the SLOT gcc-config will switch automatically to the new version like it should.  Portage seems smart enough not to remove the installed version if the user runs depclean before upgrading (at least not in my testing, but I have 14 other versions installed so I would appreciate an independent confirmation).  If someone wants an old version they can get it by explicitly emerging it with the full 3-part SLOT and it does all the right stuff wrt the world file.  I think that covers all my concerns.

One complication I can see is that if an ebuild with a major.minor SLOT is stabilized (which is always the case since we always stabilize the latest) it can't be changed to a 3-part SLOT later.  This means you wouldn't be able to install the latest stable and ~arch version together, which is a bit confusing.  I thought we might be able to do a special -r1 with a 3-part SLOT at stabilization time, but that runs right back into problem that later bumps won't get picked up.

Does this make any sense?
Comment 55 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-04-19 18:36:39 UTC
(In reply to Ryan Hill from comment #54)
> (In reply to Michał Górny from comment #53)
> > (In reply to Ryan Hill from comment #52)
> > > Because the problem I outlined in comment #43 is still valid for anyone with
> > > sys-devel/gcc:SLOT in their world file, which in my opinion should be
> > > everybody.
> > 
> > The same could be achieved via:
> > 
> >   =sys-devel/gcc-4.7*
> > 
> > while making it valid for any slotting models. It's one time switch with
> > lasting benefit.
> 
> Sure, but how many users go around manually editing their world files?  Both
> `emerge =gcc-4.7*` and `emerge gcc:4.7` put a SLOT atom in world.  That
> should work without manual intervention.  Manual intervention isn't
> friendly, remember?

Oh, that's some ugly behavior in portage ;/. Worse than that, it transforms atoms like '=gcc-4*' into pretty much random (and much more strict) slots.

> Anyways, here's a possible solution.
> 
> We can use a hybrid approach.  Start with a major.minor SLOT (eg. 4.7). 
> When a new version is added, keep it at 4.7 and change the old version to a
> major.minor.micro SLOT (eg. 4.7.2).  Portage will still do the upgrade and
> remove the old version like normal (for both =sys-devel/gcc-4.7* and
> sys-devel/gcc:4.7 in the world file).  Because we're upgrading within the
> SLOT gcc-config will switch automatically to the new version like it should.
> Portage seems smart enough not to remove the installed version if the user
> runs depclean before upgrading (at least not in my testing, but I have 14
> other versions installed so I would appreciate an independent confirmation).
> If someone wants an old version they can get it by explicitly emerging it
> with the full 3-part SLOT and it does all the right stuff wrt the world
> file.  I think that covers all my concerns.
> 
> One complication I can see is that if an ebuild with a major.minor SLOT is
> stabilized (which is always the case since we always stabilize the latest)
> it can't be changed to a 3-part SLOT later.  This means you wouldn't be able
> to install the latest stable and ~arch version together, which is a bit
> confusing.  I thought we might be able to do a special -r1 with a 3-part
> SLOT at stabilization time, but that runs right back into problem that later
> bumps won't get picked up.
> 
> Does this make any sense?

One thing you can't rely on is portage automagically switching SLOTs for installed package. For that, you need to use slotmoves. I think it could work but may be a bit overkill. And I don't know if slotmove won't change the dependency in @world -- that doesn't sound like something to rely on to be consistently implemented across PMs and not change in the future.

As far as stabilizations are concerned, I think using extra revisions could work. But that definitely requires more thinking, and I'm not sure how many extra rebuilds that will cause.

I'm afraid this already proves this solution to be too complex to be safe and sound. I suspect it will be very fragile, and hard to maintain in the future. However, I appreciate your effort and I will think about it a bit more.
Comment 56 Ryan Hill (RETIRED) gentoo-dev 2014-05-01 02:52:51 UTC
Actually, I'm relying on portage NOT switching SLOTs for the installed package.  That is the gimmick that makes it work, that the recorded installed SLOT is a two-digit SLOT, even though the package has now moved to a three-digit SLOT, and the upgrade path is preserved.  I didn't think of slotmove when I wrote the last comment, but I think it would undo the magic.

But yes, it is magic, and probably only works today by chance.

But even without the hinky upgrade stuff, I think the idea is still sound.  I think we could (eg.) move 4.6.0, 4.6.1, 4.6.2, and 4.6.3 into three-part SLOTs right now, leaving 4.6.4 as 4.6...  We know there will be no further 4.6 releases.
Comment 57 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-05-01 07:37:03 UTC
Doesn't this mean that people are going to have to rebuild old gcc version to get the three-part slot version?
Comment 58 Ryan Hill (RETIRED) gentoo-dev 2014-05-01 23:48:21 UTC
Yes, they have to.  It installs into a different path.
Comment 59 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-05-02 05:47:40 UTC
(In reply to Ryan Hill from comment #58)
> Yes, they have to.  It installs into a different path.

Well, that doesn't make any sense to me.

I specifically want multi-slotting to make gcc upgrades easier on me. To keep the old version for a few hours more so that I could upgrade all hosts and start switching them once last of them is updated. I'm perfectly happy with depclean removing the old version.

This solution means pretty much that I would have to replace the old version while installing new (breakage introduced already), then rebuild the old version just to remove it a few hours later.
Comment 60 Ryan Hill (RETIRED) gentoo-dev 2014-05-02 23:35:07 UTC
Oh I see.  That's not a use-case anyone has mentioned so far.  I've been thinking of the case where, if you want to test something against, say, version 4.6.1, you don't have to downgrade your currently installed 4.6.4 to 4.6.1, do your tests, then re-upgrade back to 4.6.4.

My solution doesn't support your use case, so I guess we're back to square one.
Comment 61 SpanKY gentoo-dev 2015-11-23 00:33:14 UTC
*** Bug 566562 has been marked as a duplicate of this bug. ***
Comment 62 SpanKY gentoo-dev 2015-11-23 09:15:05 UTC
*** Bug 566562 has been marked as a duplicate of this bug. ***
Comment 63 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2015-11-25 19:49:46 UTC
Well, we certainly aren't going to introduce circular dependency between USE flags and slots. I'm going to keep this open though for the value of discussion so far, and give space to a better idea.
Comment 64 SpanKY gentoo-dev 2016-03-16 14:19:23 UTC
*** Bug 507810 has been marked as a duplicate of this bug. ***
Comment 65 SpanKY gentoo-dev 2016-03-16 14:19:30 UTC
*** Bug 507814 has been marked as a duplicate of this bug. ***
Comment 66 SpanKY gentoo-dev 2016-03-30 19:08:39 UTC
*** Bug 434246 has been marked as a duplicate of this bug. ***
Comment 67 Ulrich Müller gentoo-dev 2018-02-04 13:01:31 UTC
Still no progress, after more than 10 years. Closing.