Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 558800 - sys-libs/ncurses: slot should change from 5 to 0 to follow main tree
Summary: sys-libs/ncurses: slot should change from 5 to 0 to follow main tree
Status: RESOLVED FIXED
Alias: None
Product: Gentoo/Alt
Classification: Unclassified
Component: Prefix Support (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Prefix
URL:
Whiteboard:
Keywords:
: 559882 (view as bug list)
Depends on: 519052
Blocks:
  Show dependency tree
 
Reported: 2015-08-26 09:57 UTC by Guilherme Amadio
Modified: 2016-01-07 20:28 UTC (History)
8 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
fixncurses.patch (fixncurses.patch,137.48 KB, patch)
2015-11-01 09:13 UTC, Greg Turner
Details | Diff
fixncurses-r2.patch (fixncurses-r2.patch,136.29 KB, patch)
2015-11-01 13:59 UTC, Greg Turner
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Guilherme Amadio gentoo-dev 2015-08-26 09:57:40 UTC
sys-libs/ncurses uses slot 0 and subslots in the main tree, so dependencies don't quite work properly in prefix. We should probably think about migrating prefix to use the same slot numbers, or start using ncurses from the main tree directly. I copied over ncurses-6.0 ebuild and despite the expected file collisions, the main tree ebuild builds fine for me.

There are about 100 ebuilds that depend on ncurses:0

repo/gentoo $ git grep "sys-libs/ncurses:0" | wc -l
102


Reproducible: Always
Comment 1 Fabian Groffen gentoo-dev 2015-08-26 18:53:34 UTC
Did you change anything?  Or does Clang do static linking on OSX for some reason?
Comment 2 Fabian Groffen gentoo-dev 2015-08-26 20:08:34 UTC
On the plus side, after some small fixes, it compiles and installs on Solaris, w00h00!  Just wondering is 6.0 is going to be acceptable enough for Prefix wrt outstanding bugs.
Comment 3 Guilherme Amadio gentoo-dev 2015-08-27 07:14:20 UTC
(In reply to Fabian Groffen from comment #1)
> Did you change anything?  Or does Clang do static linking on OSX for some
> reason?

No, I did not commit anything, I copied the ebuild for ncurses-6.0 to a local overlay on my prefix. But this was on linux, not OS X. I have no access to OS X right now to test this. I removed ncurses 5 and reinstalled.
I guess it was fine to fallback to system ncurses for me, but for other systems it might be tricky to upgrade. A new prefix bootstrap should probably just work, though.
Comment 4 Manfred Knick 2015-08-27 07:55:20 UTC
(In reply to Guilherme Amadio from comment #0)


$ cd /usr/portage/sys-libs/ncurses/

$ grep -i "SLOT=" *.ebuild


ncurses-5.9-r3.ebuild:SLOT="0"

ncurses-5.9-r4.ebuild:SLOT="0"

ncurses-5.9-r101.ebuild:SLOT="5/5"     <-----  ? "0/5" ?

ncurses-6.0.ebuild:SLOT="0/6"



from ncurses-5.9-r101.ebuild:

# The subslot reflects the SONAME.
......^^^....
Comment 5 Guilherme Amadio gentoo-dev 2015-08-27 08:05:27 UTC
(In reply to Manfred Knick from comment #4)
> (In reply to Guilherme Amadio from comment #0)
> 
> 
> $ cd /usr/portage/sys-libs/ncurses/
> 
> $ grep -i "SLOT=" *.ebuild
> 
> 
> ncurses-5.9-r3.ebuild:SLOT="0"
> 
> ncurses-5.9-r4.ebuild:SLOT="0"
> 
> ncurses-5.9-r101.ebuild:SLOT="5/5"     <-----  ? "0/5" ?
> 
> ncurses-6.0.ebuild:SLOT="0/6"
> 
> 
> 
> from ncurses-5.9-r101.ebuild:
> 
> # The subslot reflects the SONAME.
> ......^^^....

mic ~ $ cd ${EPREFIX}/usr/portage/sys-libs/ncurses/
mic ncurses $ grep -i "SLOT=" *.ebuild

ncurses-5.7-r7.ebuild:SLOT="5"
ncurses-5.9-r02.1.ebuild:SLOT="5"
ncurses-5.9-r1.ebuild:SLOT="5"
ncurses-5.9-r3.ebuild:SLOT="5"
ncurses-5.9-r4.ebuild:SLOT="5"
Comment 6 Manfred Knick 2015-08-27 08:53:06 UTC
Hint @ #558856 :

https://bugs.gentoo.org/show_bug.cgi?id=558856
Comment 7 Guilherme Amadio gentoo-dev 2015-08-27 09:06:05 UTC
(In reply to Manfred Knick from comment #6)
> Hint @ #558856 :
> 
> https://bugs.gentoo.org/show_bug.cgi?id=558856

Interesting. So we need to follow the migration path of the main tree by importing both the 5.9 and 6.0 ebuilds with the right slots and subslots, I guess.
Comment 8 Guilherme Amadio gentoo-dev 2015-09-07 16:19:59 UTC
*** Bug 559882 has been marked as a duplicate of this bug. ***
Comment 9 Guilherme Amadio gentoo-dev 2015-09-11 06:54:11 UTC
BTW, if you are on a prefix that needs upgrading, the prefix branch on my overlay at github.com/amadio/gentoo-overlay.git contains the newer ebuilds from the main tree that have worked well for me so far. I hope that soon we will just move prefix to use main tree ebuilds for ncurses.
Comment 10 Fabian Groffen gentoo-dev 2015-09-11 14:20:34 UTC
ok, are those the onces I committed, or did you make some changes?
Comment 11 Guilherme Amadio gentoo-dev 2015-09-11 15:34:34 UTC
(In reply to Fabian Groffen from comment #10)
> ok, are those the onces I committed, or did you make some changes?

My overlay contains verbatim copies of the ncurses ebuilds that moved to SLOT 0 in the main tree. If I don't use those, the dependency tree breaks from all ebuilds asking for SLOT=0 and only finding SLOT=5 ebuilds for ncurses in prefix. If you commited ebuilds in SLOT 0, they seem to not be reaching users yet (maybe a hg/git migration glitch).
Comment 12 Fabian Groffen gentoo-dev 2015-09-12 13:27:08 UTC
Right, prefix tree still has old ncurses ebuilds, I committed some patches to gx86 for 6.0, but didn't touch the 5.9s yet.  Therefore, the gx86 versions aren't available in the prefix tree.

I just need testing and stuff, and am not in the position right now to do so.  Maybe later next week.  I was just wondering if my changes were enough.  Tricky bit here is that ncurses is a base package, so platforms like AIX will have to work with the new ebuilds as well, which boils down to Haubi testing that path.
Comment 14 Greg Turner 2015-10-31 18:39:02 UTC
--- a/sys-libs/ncurses/ncurses-6.0-r1.ebuild
+++ b/sys-libs/ncurses/ncurses-6.0-r1.ebuild
@@ -227,7 +232,7 @@ multilib_src_install() {
        # Move main libraries into /.
        if multilib_is_native_abi ; then
                gen_usr_ldscript -a \
-                       "${NCURSES_TARGETS[@]}"
+                       "${NCURSES_TARGETS[@]}" \
                        $(use tinfo && usex unicode 'tinfow' '') \
                        $(usev tinfo)

looks like a no-brainer.  But I have a "brainer" coming up... :)
Comment 15 Fabian Groffen gentoo-dev 2015-11-01 09:06:54 UTC
(In reply to Gregory Turner from comment #14)
...
> looks like a no-brainer.  But I have a "brainer" coming up... :)

Yes, thanks, pushed now!
Comment 16 Greg Turner 2015-11-01 09:13:32 UTC
Created attachment 415862 [details, diff]
fixncurses.patch

Big ugly patch.  

Transforms sys-libs/ncurses from the current prefix-rsync state as of a couple days ago into "what-I-have" version of gx86-based respin WIP.

BTW, it's notable that, having at-least briefly audited all the ebuilds, I see no patches present in gx86, but absent or modified in prefix-rsync, nor any patches present in prefix-rsync but absent in gx86, which are actually consumed by any of the ebuilds in either tree.

That means, assuming I'm not missing something, one could without any practical effect nuke sys-libs/ncurses/files in prefix and replace it with an exact copy of sys-libs/ncurses/files from gentoo-x86 without changing any practical outcomes.

This patch does so, but also contains some more meaningful work; it represents the outcome of something like the following recipe*:

  mkdir -p /tmp/a/sys-libs
  mkdir -p /tmp/b/sys-libs
  cd ${prefix-rsync-portdir}/sys-libs
  cp -r ncurses /tmp/a/sys-libs/
  cd ${gx86-rsync-portdir}/sys-libs
  cp -r ncurses /tmp/b/sys-libs/
  cd /tmp/b/sys-libs/ncurses
  ...
  [various backports to the gx86 ebuilds (in "b") of prefix stuff ]
  cd /tmp
  diff -urpN a b
  
I have not rigorously audited this.  Most of the ebuilds haven't even been tested once (but 6.0-r1 works in the ~amd64-linux RAP bootstrap I'm working on)

When in doubt (I was, often) I erred on the side of presuming stuff was overlay-drift-bathwater rather than feature-branch-baby, mostly going by hunch and vague memory of the history of these ebuilds rather than any rigorous verification process.  Over the upcoming however-long-I-take-to-do-it I'll chase down the CVS bases of the prefix-rsync ebuilds and endeavour to ensure I didn't miss anything relevant.

Obv. needs more auditing and dust-up, but I suspect it's close.

The one half-assed test I've done so far was a success, which was either a good portent or a random luck-box leading to misguided confidence :)

--

* I actually did it in git.  The above no-vcs recipe just conveys the "gist" of what I did more clearly than a rigorously accurate description of the various steps I took in git, which involved various missteps/corrections and elaborate precautions that turned out to address non-problems.
Comment 17 Greg Turner 2015-11-01 13:59:58 UTC
Created attachment 415868 [details, diff]
fixncurses-r2.patch

(In reply to Fabian Groffen from comment #15)
> (In reply to Gregory Turner from comment #14)
> ...
> > looks like a no-brainer.  But I have a "brainer" coming up... :)
> 
> Yes, thanks, pushed now!

Here, then, is the exact same "brainer" content, only rebased against latest rsync.
Comment 18 Fabian Groffen gentoo-dev 2015-11-02 19:13:40 UTC
Question, do we actually need al the old ebuilds?
Comment 19 Greg Turner 2015-11-02 20:08:39 UTC
"note to housekeeping": just noticed bug #519052 substantially overlaps with what I've got in the patch... not sure which should depend on which, or if one should be marked as a dup; it's up to the respective bug-owners (I couldn't change those dependencies even if I wanted to if I'm not mistaken).
Comment 20 Greg Turner 2015-11-02 20:14:48 UTC
(In reply to Fabian Groffen from comment #18)
> Question, do we actually need al the old ebuilds?

I'm a bit fuzzy on some of the details but when the ncurses slot change happened, there was clearly much gnashing of teeth about it, and it took gx86 a few tries to get some sort of slot-move mess sorted out...

Let me try to figure out what all the fuss was about and get back to you on this once I have the facts straight in my head.
Comment 21 Greg Turner 2015-11-04 23:20:34 UTC
(In reply to Gregory Turner from comment #20)
> (In reply to Fabian Groffen from comment #18)
> > Question, do we actually need al the old ebuilds?
> 
ed: [I think there was]
> much gnashing of teeth
ed: [about the ncurses slot move or some such thing]

[snip!]

> Let me try to figure out what all the fuss was about and get back to you on
> this once I have the facts straight in my head.

OK, the story is told pretty well by clicking around on this page of gitweb (all the important stuff is above the fold).

  https://github.com/gentoo-mirror/gentoo/commits/a0072d7220178f6b9311e6307d549a04305fdafc/sys-libs/ncurses

(NB: the url above shows commits affecting gx86 sys-libs/ncurses as of November 3rd 2015; /not/ as of whenever future readers of this bug might be.  If some readers of this wanted the latter, they could find that at this URL, instead: https://github.com/gentoo-mirror/gentoo/commits/master/sys-libs/ncurses).

But... just in case somebody out there would rather get an executive summary, I'll try my best to produce one.  Just please take some grains of salt with this, as I'm no portage depsolving SME*.
 
Luckily, the blind /can/ lead the blind, perhaps, so long as everybody is mindful of what kind of stuff they may stepping in :P).

Hmm, I'm gonna post this now and make a second post with the promised executive summary soon just for purposes of preserving my sanity and avoiding losing all this text to some kind of browser crash or "I forgot" problem.

TO BE CONTINUED 

* On the contrary, I really don't "get it."   OK, ebuilds I kinda get.  But the interplay between those and the depcaches in various repo dirs, which may be present in various trees, in-vdb ebuilds, and so on... I have no clue.  Doesn't seem to be terribly well documented how all those things work.
Comment 22 Greg Turner 2015-11-05 18:58:40 UTC
(In reply to Gregory Turner from comment #21)
> (In reply to Gregory Turner from comment #20)
> > (In reply to Fabian Groffen from comment #18)
>
> > > Question, do we actually need al the old ebuilds?
> >
> > Let me try to figure out what all the fuss was about and get back to you on
> > this once I have the facts straight in my head.
> 
> OK, the story is told pretty well by clicking around on this page of gitweb
> (all the important stuff is above the fold).
> 
> https://github.com/gentoo-mirror/gentoo/commits/
> a0072d7220178f6b9311e6307d549a04305fdafc/sys-libs/ncurses
>
> TO BE CONTINUED

I'm just writing down everything that happened so I can get it straight in my head.  Then I'll try to summarize.

The TLDR version (Part I)
----------------------------------

Our story begins in early August, 2015.

Everyone and their dog's cousin was using the same major version of ncurses, 5.  There was peace and harmony in the cosmos.  Then, ncurses 6 was released, partly because the maintainers wanted an ABI with extensively versioned symbols.

Gentoo had an ebuild in-tree less than a week later.  The original approach taken was to put the new ebuild into SLOT "5/6".  In other words, ncurses was at this time not effectively a multi-SLOTted package -- you had to pick your ncurses version, and you only got one choice.

Of course this quickly led to unresolvable dependency problems for binary packages that depended on the old ncurses library by name.  Bug #557472 represented just such a problem, in particular for dropbox.

In response, ncurses-6 was "moved" into the "0" SLOT, and ncurses-5 (https://github.com/gentoo-mirror/gentoo/commit/051ad88caa127419c80d473021ab8909a2268789) left where it was in the "5" slot.  This alone did not enable multi-slot installation, however, because the SLOT=5 and SLOT=0 versions could not coexist in practice due to file collisions.

To fix that, a "compatibility package", called ncurses-5.9-r100, still in SLOT 5, which blocked installation of newer ncurses versions in SLOT 5 and excluded files such as man-pages and global headers which would cause collisions in multi-slot installs (https://github.com/gentoo-mirror/gentoo/commit/ec1cb36e722e62cf504f89ffdfc05ce87407ae3d).  This was subsequently moved to SLOT "5/5" and stripped down and bumped to -r101 (https://github.com/gentoo-mirror/gentoo/commit/8cc8ae51e676d082f5c785a7463e54773cf96714?diff=unified).

--

I think this is basically where TSHTF, subjectively speaking.  Around this time I attempted to do my standard "emerge -DuavN @world" and wound up with an attempt to upgrade to a slotted ncurses installation, half of which subsequently failed due to a file collision, leaving me with no working ncurses at all, and lots of broken executables.  Thankfully I was able to bail myself out using various processes I'd started before this... otherwise it might have been a real pain to fix.

Although, to be honest, looking at the changes, I don't quite get what was wrong.  Perhaps it has to do with leaving people's systems in a state where they installed a -6.0 ncurses package in SLOT=5, but now SLOT=5 refers to those compatibility packages.  The whole behavior of portage wrt installed packages and depsolving is pretty confusing to me as I mentioned above.

TO BE CONTINUED in ... "The TLDR Version (Part II)"
Comment 23 Greg Turner 2015-11-05 20:20:19 UTC
(In reply to Gregory Turner from comment #22)
>
> TO BE CONTINUED in ... "The TLDR Version (Part II)"

The TLDR Version (Part II)
------------------------------------

Thus far our story has been, perhaps, convoluted, but, I presume, basically comprehensible to normal Gentoo people.  Now, it starts to get pretty fucking weird.

So, when the original patch went in moving ncurses from SLOT=5 to SLOT=0, the following "slotmoves" went into profiles/updates/3Q-2015:

slotmove =sys-libs/ncurses-5.9-r3 5 0
slotmove =sys-libs/ncurses-5.9-r4 5 0
slotmove ~sys-libs/ncurses-6.0 5 0

What, you might ask, are slotmoves?... I may kind of know the answer, but only in rough layman's terms: slotmoves are humans broadcasting instructions to possibly-affected portage installations: "hey, all you portages, we took some packages in the tree and, without changing their names, changed the SLOT's that are assigned to them.  So, to keep everything consistent, please make the corresponding changes to your user's vdb (installed-package-metadata), but only do it once; then, forget all about it.

Now, I'm going to quote at length from bug #558856, because I don't think I fully understand it myself:

Title:

slot move of sys-devel/ncurses from 5 to 0 did not adjust vdb *DEPEND for ncurses atoms w/ subslot deps

Comment 0 (bkohler):

vdb entries did not get adjusted for packages using subslot deps.  For example:

/var/db/pkg/dev-db/mariadb-10.0.21/RDEPEND:>=dev-libs/openssl-1.0.0:0/0=[abi_x86_64(-)] sys-process/procps:0/0= dev-libs/libaio:0/0= >=sys-apps/sed-4 >=sys-apps/texinfo-4.7-r1 !dev-db/mariadb-native-client[mysqlcompat] >=dev-libs/openssl-1.0.0:0/0=[abi_x86_64(-)] >=sys-libs/zlib-1.2.3:0/0=[abi_x86_64(-)] sys-libs/ncurses:5/5=[abi_x86_64(-)] >=sys-libs/readline-4.1:0/0=[abi_x86_64(-)] !dev-db/mytop >=dev-libs/libpcre-8.35:3/3= !dev-db/mysql !dev-db/mariadb-galera !dev-db/percona-server !dev-db/mysql-cluster dev-db/mysql-init-scripts virtual/perl-Getopt-Long dev-perl/TermReadKey virtual/perl-Term-ANSIColor virtual/perl-Time-HiRes

This is wreaking havoc on users' systems when they go to update.  Autounmask suggests allowing unstable on ncurses-5.9-r101, but not on 6.0, the dep tree explodes.  I'm sure I and others will have more details to add but I wanted to get this bug open finally.  Been seeing problems in #gentoo all day.

Right now were are suggesting users force-rebuild all packages with a dep like sys-devel/ncurses:5 recorded in /var/db/pkg/*/*/RDEPEND, other options like a forced sed on /var/db/pkg seem a bit scary.

Comment 1 (axs):

(in an attachment)

#
# The following sed will rewrite the portage database (VDB)
# so that slot-operator dep records of packages emerged prior to
# the ncurses:5 -> ncurses:0 slot move will be changed
# to contain the appropriate new subslot value.
#
# WARNING WARNING WARNING
# This set is DIRECTLY MODIFYING the VDB, which is rarely a good
# idea.  The author is not responsible for any breakage that may
# occur. etc. etc. agree to hold indemnitable the author and so forth.
#

# backup VDB just in case
tar -jcf /tmp/vdb-backup.tar.bz2 /var/db/pkg

# change ncurses:5/5 to ncurses:0/0, ncurses:5/6 (or any others)
# to ncurses:0/6.
sed -i \
  -e 's#sys-libs/ncurses\([^:]*\):5/5#sys-libs/ncurses\1:0/0#g' \
  -e 's#sys-libs/ncurses\([^:]*\):5/#sys-libs/ncurses\1:0/#g' \
  /var/db/pkg/*/*/*DEPEND

.
.
.

Comment 7 (zmedico):

(In reply to Julian Ospald (hasufell) from comment #3)
> I think this might be a PMS bug/restriction.

In the case of portage, it simply doesn't try to update the *DEPEND atoms in /var/db/pkg because it's tricky to match atoms against atoms (since atoms are intended to match packages rather than other atoms). The relevant code is here:

https://gitweb.gentoo.org/proj/portage.git/tree/pym/portage/update.py?h=v2.2.20.1#n83

Note that if the slotmove atom does not specify a version, then portage will go ahead and update the *DEPEND atoms in /var/db/pkg.

.
.
.

---

and on it goes like that.  To me, I gotta be honest, it feels like that narrative device in the Peanuts movies, where the adults are talking, and their speech is replaced with a sound like "wah wah wah wah wah...."

I guess I've decided, personally, to mentally compartmentalize this in the same way I have the problem of "dark matter."  That is, I'm taking it on faith that, in the context of an intellectual framework I don't really understand, there is some kind of head-exploding discrepancy, with likely practical implications, that I need not trouble my pretty little head about, except to know that there is a discrepancy.

To fix this problem, several actions are taken:

https://github.com/gentoo-mirror/gentoo/commit/125fb7a6cdcde8fb1c7dd59edf87dbf276025ef4 does two things.  firstly, it creates '=sys-libs/ncurses-5.9-r5:0/5', which is ALMOST exactly like -r4, except (a) it's EAPI5 (I think this matters, but for reasons I don't understand), (b), it's in subslot "0/5" instead of "0", and (c) instead of blocking "!<=sys-libs/ncurses-5.9-r4:0", it blocks "!<=sys-libs/ncurses-5.9-r4:0".  Secondly, it creates "a dummy ebuild to bridge the old SLOT=5[/5] and the new SLOT=0/5", as "=sys-libs/ncurses-5.9-r99:5," and truly is a dummy package with no phase functions or inherits of any kind.  It's only semantic job seems to be to enforce this DEPEND:

  DEPEND="sys-libs/ncurses:0/5[ada?,cxx?,gpm?,static-libs?,tinfo?,unicode?]"

https://github.com/gentoo-mirror/gentoo/commit/5b9725757eaf3b7ec32ae854151183f222ba4189 marked 5.9-r5 and 5.9-r99 as stable for several ARCH'es.

https://github.com/gentoo-mirror/gentoo/commit/96470175e0316e0f3402cbce5a83838461e17f75 adds multilib-build support to 5.9-r99, so that the DEPEND requirement above passes through multilib ABI requirements.

https://github.com/gentoo-mirror/gentoo/commit/0f20b4ddc60c23a84ae918f31933d8e5f0b7d9eb
Comment 24 Greg Turner 2015-11-05 20:30:33 UTC
(In reply to Gregory Turner from comment #23)
> (In reply to Gregory Turner from comment #22)
> >
> > TO BE CONTINUED in ... "The TLDR Version (Part II)"
> 
> The TLDR Version (Part II)
> ------------------------------------

oops, premature post.  Continuing where I accidentally left off...

> https://github.com/gentoo-mirror/gentoo/commit/
> 0f20b4ddc60c23a84ae918f31933d8e5f0b7d9eb

Transforms sys-libs/ncurses-6.0 into a dummy package analogous to -5.9-r99, moving it into SLOT "5/6", where the old ncurses-6.0 was before the slot move., and adds sys-libs/ncurses-6.0-r1, which contains exactly the same ebuild as sys-libs/ncurses-6.0 (so exactly that git assigns them the same SHA1 id).

https://github.com/gentoo-mirror/gentoo/commit/9de81f6b6990908bfd9883c327371fe7815584c4 dinks around with some kind of DEPEND vs. RDEPEND subtleties in the new dummy ebuilds, presumably to make portage better-behaved.

This is where we stand as of this post.

TO BE CONTINUED, IN "Executive Summary"... :)
Comment 25 Greg Turner 2015-11-06 10:10:09 UTC
Executive Summary

nb: Not really an executive summary.  Better title: Analysis and Conclusions

Today, gx86 has the following ebuilds in-tree for sys-libs/ncurses.  Zero-deployed-content ebuilds meant only to cajole portage into various depsolving behaviors are marked with a "#"; ~everything-keyworded ebuilds are marked with a "~".

sys-libs/ncurses-5.9-r3:0
~sys-libs/ncurses-5.9-r4:0
sys-libs/ncurses-5.9-r5:0/5

#sys-libs/ncurses-5.9-r99:5/5
~sys-libs/ncurses-5.9-r101:5/5

#~sys-libs/ncurses-6.0:5/6

~sys-libs/ncurses-6.0-r1:0/6

Upstream, these all play some kind of meaningful unique semantic role, though of varying degrees of obscurity/importance.

In prefix, the big difference is, we never adopted upstream's slot-moves, and therefore, never corrupted people's vdb's.

Unfortunately, we already incorporated some of upstream's sub-slots.  We want to avoid duplicating gx86's mistake of slotmoving the places we have (see below).

In prefix, the -6.0{,-r1} ebuilds depsolve just like upstream's, and we pretty clearly want to keep both, since the dummy -6.0-r0 allows ebuilds with lingering SLOT=5/6 dependencies (and vdb's with generated 5/6 "effective" dependencies) to work correctly.

Meanwhile, prefix has the following -5.9 ebuilds:

sys-libs/ncurses-5.9-r101:5/5
sys-libs/ncurses-5.9-r4:5
sys-libs/ncurses-5.9-r99:5/5

5.9-r99 works analogously to -6.0 and is trivial to maintain, as it has no content and is an exact copy of upstream's (except we haven't stabled keywords as upstream has; presumably, we should).  It seems advisable to keep 5.9-r99 (and hence, we also should keep 5.9-r101, without which it'd be crazy to keep 5.9-r99).

That leaves just 5.9-r4.  Ours doesn't live in the same slot as gx86's.  So, from a depsolving perspective, our 5.9-r4 is more like upstream's 5.9-r3 (indeed it lacks the blocker that gx86 adds, moving from 5.9-r3 -> 5.9-r4).

Actually, that's a bit awkward.  We could exploit prefix's -r0a.b kludge, but that's not conducive to upstream-ability, and should probably be used only as a last resort.  What we could do, though, is slotmove 5.9-r4:5 to 5.9-r4:0, and add upstream's blocker.  Pretty sure this would avoid the problems upstream had, since we never had a sub-SLOTted ebuild there, in prefix.

I'm not if there's precedent for slotmoving something and dropping it from the tree in the same breath, but it can't be completely unacceptable, as this is effectively what happens anyhow if a slotmove has hung around in an out-of-date PORTDIR long enough for its target to fall out of the tree.

5.9-r4 is the only ebuild providing a ~ARCH-specific package meeting dependencies like "<sys-libs/ncurses-6.0:0/0", but it's hard for me to see how that matters in prefix.

The final question we should answer is -- I presume this is more-or-less what grobian's original question intended -- do we want to bring 5.9-r3 or 5.9-r5 back, for any reason (which my patch above does).

Let's assume we've slotmoved 5.9-r4 and now have the following subset of upstream's layout (assuming we didn't drop -5.9-r4 at the same moment we slotmoved it).

~sys-libs/ncurses-5.9-r4:0
#sys-libs/ncurses-5.9-r99:5/5
~sys-libs/ncurses-5.9-r101:5/5
#~sys-libs/ncurses-6.0:5/6
~sys-libs/ncurses-6.0-r1:0/6

We want to answer the following questions:

o Why does upstream have 5.9-r3, and does this reason apply to prefix?

I'm having trouble answering this.  See my footnote below.  My hunch is we don't need any analogous thing in prefix, but my continued inability to understand the full practical significance of subslot operators makes it difficult to opine confidently.  The whole thing seems to boil down to whether or not foo:0= would match the exact same things as foo:0/0=, or whether some other subslot might be matched by it, if foo is an EAPI4 package (assuming bar, a hypothetical package DEPENDing on it, is EAPI5)*.

Keeping it in my patch was, I suppose, a way of skirting around my lack of understanding, as I was, then, and remain, now, pretty sure that keeping all of the upstream packages in the exact same SLOTS as in gx86 would constitute a working, if not an economical, solution to the puzzle.

However, my hunch is, we don't need both 5.9-r3 and 5.9-r4, and in fact, we may not need either of them.  Part of this hinges on whether or not there is any longer any supported "stable prefix" configuration, or whether all prefixes are effectively unstable-only single-branch animals (at least, with respect to the full OS-ARCH prefix-specific keywords; another option would be to drop all the prefix keywords rely entirely on the gx86-style no-hyphen keywords, but then I think we have a stronger argument to duplicate the full compliment of ebuilds).

Come to think of it, some experiments with dummy packages in an overlay should suffice to disconfirm this null-hypothesis of superfluousness, if I'm wrong.  And would probably be worth doing, at this point, if only for my own education as an ebuild developer.

o Why does upstream have 5.9-r5 and, again, does any of that apply to prefix?

This says it all:

--- ncurses-5.9-r4.ebuild       2015-11-05 18:24:44.721446082 -0800
+++ ncurses-5.9-r5.ebuild       2015-11-05 18:24:44.721446082 -0800
@@ -2,7 +2,8 @@
 # Distributed under the terms of the GNU General Public License v2
 # $Id$
 
-EAPI="4"
+EAPI="5"
+
 inherit eutils flag-o-matic toolchain-funcs multilib-minimal
 
 MY_PV=${PV:0:3}
@@ -13,15 +14,16 @@ HOMEPAGE="https://www.gnu.org/software/n
 SRC_URI="mirror://gnu/ncurses/${MY_P}.tar.gz"
 
 LICENSE="MIT"
-SLOT="0"
-KEYWORDS="~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~m68k ~mips ~ppc ~ppc64 ~s390 ~sh ~sparc ~x86 ~amd64-fbsd ~sparc-fbsd ~x86-fbsd"
+# The subslot reflects the SONAME.
+SLOT="0/5"
+KEYWORDS="alpha amd64 arm arm64 hppa ia64 ~m68k ~mips ppc ppc64 s390 ~sh sparc x86 ~amd64-fbsd ~sparc-fbsd ~x86-fbsd"
 IUSE="ada +cxx debug doc gpm minimal profile static-libs tinfo trace unicode"
 
 DEPEND="gpm? ( sys-libs/gpm )"
 #      berkdb? ( sys-libs/db )"
 # Block the older ncurses that installed all files w/SLOT=5. #557472
 RDEPEND="${DEPEND}
-       !<=sys-libs/ncurses-5.9-r4:0
+       !<=sys-libs/ncurses-5.9-r4:5
        !<x11-terms/rxvt-unicode-9.06-r3
        abi_x86_32? (
                !<=app-emulation/emul-linux-x86-baselibs-20130224-r12

5.9-r5 is the only ebuild matching "<sys-libs/ncurses-6:0/5=", and the only stable ebuild matching "sys-libs/ncurses:0/5".  Basically 5.9-r5 does, today, in slot :0, what 5.9-r3 used to do in slot :5.  Therefore, I think it's necessary or at least advisable to pull it in.

*  ... and maybe other subslot operators -- after reading the PMS sub-slot spec dozens, if not hundreds of times, and having had it explained to me by more than one knowledgeable person, I still have trouble understanding what the fuck :slot= means.

I used to think I was just stupid, but I'm increasingly suspicious that, in fact, multiple conflicting interpretations of the EAPI5 PMS are equally correct.  But that's a different bug entirely -- though perhaps be a dependency of this one :D.


Epilogue

Wow, I have never typed so much crap into web-forms in my life.

All that work, and I still wasn't fully able to answer to grobian's question.  Kind of disappointing, but, at worst, it's helped me to cure some of my own cognitive dissonance and to zero-in on precisely where the gaps in my understanding lie.

I guess a true executive summary would be:

"I can't confidently say either way so for now the only think I can safely propose is to keep all the same semantics as upstream (except for their problematic slotmoves).  But there are ways to test this out that might be worth pursuing."
Comment 26 Fabian Groffen gentoo-dev 2015-11-07 16:42:44 UTC
Thanks for your extensive research.  I think I can answer some of your questions.

1. no we don't do stable keywords in Prefix (yay, one thing less to worry about)
2. our keywords are distinct from main tree keywords, there is no intention to merge (for it are different beasts)
3. 5.9-r5 I'm looking into it now, sounds like you figured out what they wanted with it
4. subslots, huuuuuu, nasty, a hack basically to make portage do rebuilds of packages that use the lib (slot) in question
5. :slot= should mean "have the same slot as my slot"
Comment 27 Fabian Groffen gentoo-dev 2015-11-07 16:59:33 UTC
In addition:
- ncurses-6 should be equal to gx86, I hope this one can become the delight in Prefix land wrt gx86 compatibility (probably wishful thinking here)
- I've just fixed the SLOT setting in 5.9 ebuild(s)
- I think we don't need -r3, because we don't have stable keywords

The only thing I don't quite get is this:

[ebuild     UD ] sys-libs/ncurses-5.9-r5:0/5::gentoo_prefix_svn [6.0-r1:0/6::gentoo_prefix_svn] USE="cxx unicode (-ada) -debug -doc (-gpm) -minimal -profile -static-libs -tinfo -trace (-test%) (-threads%)" 0 KiB

So it doesn't really see the slot as such, and I think our upgrade-path is foobar-ed or something.
Comment 28 Greg Turner 2015-11-07 17:47:46 UTC
(In reply to Fabian Groffen from comment #26)
> Thanks for your extensive research.  I think I can answer some of your
> questions.
> 
> 1. no we don't do stable keywords in Prefix (yay, one thing less to worry
> about)

Indeed.  I suspect this means we can slotmove/drop 5.9-r4 and folks needing legacy 5.9 ncurses will correctly upgrade to 5.9-r101.

> 2. our keywords are distinct from main tree keywords, there is no intention
> to merge (for it are different beasts)

ACK, also thanks for answering this in more detail on the -ml (I should probably ask some follow-up questions but I'll do so there).

> 3. 5.9-r5 I'm looking into it now, sounds like you figured out what they
> wanted with it

I wonder if we could drop this one, as well, given that there's no stable branch in prefix.  I guess not due to the fact that it's the only ebuild satisfying :0/5=.

I suspect it'd at worst be possible to gut it and convert it to another dummy package.  But I guess that'd be taking it too far, as it would just add one more prefix-specific gut-rehab ebuild to keep track of.

> 5. :slot= should mean "have the same slot as my slot"

Actually I don't think, formally speaking, a slot operator with those semantics exists, presumably, because that result can be achieved using the actual SLOT /variable/ in combination with the :slot= operator, i.e.,: "( foo/bar:${SLOT}= )".

In the PMS nomenclature I'm using above, :slot= is the thing that looks like: "( baz/quux:3= )" -- in other words, the syntax that's more or less replaced :slot (i.e.: "( dead/beef:3 )") in EAPI5+.

Where I think there is possibly an ambiguity in the spec is EAPI5's treatment of the :slot= operator.  In particular, in the (actually quite common) case of non-subslot-specific references to "partially" sub-slotted ebuilds, i.e,. "( sys-libs/ncurses:5= )".

It may be that all such references are ambiguous, or that only cases like ncurses, where some packages potentially matching the operator are EAPI4 and other packages potentially matching it are sub-slotted EAPI5, are ambiguous.  Or, I may be spinning my wheels; there may be no ambiguity.  I haven't quite decided yet :)

The practical question I need to answer for present purposes is not "is PMS ambiguous", but "what does portage do with, and what do developers mean by, these atoms right now?" which are knowable now, even if PMS is ambiguous.  However I still don't think I know :(

Seeing as I've already supplied this bug with an awful lot of content, I should probably find a more appropriate place than here to talk about the PMS, assuming the whole issue is not a PEBKAC on my part.  Maybe the -devel ml or something.
Comment 29 Greg Turner 2015-11-07 17:52:10 UTC
(In reply to Fabian Groffen from comment #27)
> In addition:
> - ncurses-6 should be equal to gx86, I hope this one can become the delight
> in Prefix land wrt gx86 compatibility (probably wishful thinking here)
> - I've just fixed the SLOT setting in 5.9 ebuild(s)
> - I think we don't need -r3, because we don't have stable keywords
> 
> The only thing I don't quite get is this:
> 
> [ebuild     UD ] sys-libs/ncurses-5.9-r5:0/5::gentoo_prefix_svn
> [6.0-r1:0/6::gentoo_prefix_svn] USE="cxx unicode (-ada) -debug -doc (-gpm)
> -minimal -profile -static-libs -tinfo -trace (-test%) (-threads%)" 0 KiB
> 
> So it doesn't really see the slot as such, and I think our upgrade-path is
> foobar-ed or something.

Interesting.  Didn't see that coming :)  I'll take a look this afternoon and see what I can figure out about it.
Comment 30 Greg Turner 2015-11-09 20:38:56 UTC
(In reply to Fabian Groffen from comment #27)
> The only thing I don't quite get is this:
> 
> [ebuild     UD ] sys-libs/ncurses-5.9-r5:0/5::gentoo_prefix_svn
> [6.0-r1:0/6::gentoo_prefix_svn] USE="cxx unicode (-ada) -debug -doc (-gpm)
> -minimal -profile -static-libs -tinfo -trace (-test%) (-threads%)" 0 KiB
> 
> So it doesn't really see the slot as such, and I think our upgrade-path is
> foobar-ed or something.

Well I couldn't repro in my prefix but (a) it's almost empty and (b) my prefix was formerly running ncurses from my overlay -- a more interesting test would be for me to revert ${PORTDIR}/sys-libs/ncurses to six months ago, emerge -1a, and then emerge --sync to see what happens in a world upgrade.

Any chance your prefixes' @world sets pull in any of the packages selected by:

  find "$(portageq envvar PORTDIR 2>/dev/null)" \
    -name '*.ebuild' -print0 | \
     xargs -0 grep -C2 -n --color=no \
        'ncurses[^[:space:]]*:./5'

?

In particular, I wonder about these:

  games-roguelike/adom
  dev-util/android-ndk

which have conditionals like:

  "( || ( sys-libs/ncurses:0/5 sys-libs/ncurses:5/5 ) )"

which list 0/5 first in their conditionals (iiuc portage always chooses the first option in such conditionals, regardless of how "deep" you ask it to depsolve, unless another option is already installed in the vdb).

If that's the reason it makes at least partial sense: 5.9-r5 is the only thing matching :0/5.

Alternatively, maybe it's because we never slotmove-d -r4.

If somehow an existing -5.9:5/5 atom in @world or vdb somewhere selected -r99 for some reason, this would select 5.9-r5 via its -5.9:0/5 dep.  Only problem with that hypothesis is, I fail to see why this wouldn't be solved by 5.9-r101, instead (indeed, this begs the question whether we need -r99 in prefix at all, given there's no stable branch).