Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 506276 (no-symlink-lib) - [TRACKER] SYMLINK_LIB=no / 17.1 profiles [FILE A SEPARATE BUG INSTEAD OF COMMENTING HERE!]
Summary: [TRACKER] SYMLINK_LIB=no / 17.1 profiles [FILE A SEPARATE BUG INSTEAD OF COMM...
Status: CONFIRMED
Alias: no-symlink-lib
Product: Gentoo Linux
Classification: Unclassified
Component: Profiles (show other bugs)
Hardware: All Linux
: Normal major with 2 votes (vote)
Assignee: Gentoo Toolchain Maintainers
URL:
Whiteboard:
Keywords: Tracker
Depends on: 589874 640184 640186 640456 643302 644538 644550 644850 645540 677374 680686 683796 685522 686750 687514 687552 687600 687686 687736 687746 688010 688012 688098 688174 688260 689312 689322 689326 690828 692512 692928 696106 697568 698304 699236 699242 699730 700378 701892 701912 702622 466892 486712 496940 503064 503624 504812 505694 505760 506596 508192 508906 510158 512338 513344 516240 516242 524018 529826 533482 533536 549998 552500 556650 560572 570304 601436 601746 627744 627746 628048 628810 640460 640686 640772 641826 642138 642368 642568 642574 642578 642598 642600 642734 643194 643322 643522 643632 643710 643812 644084 644734 645176 645240 646126 646958 647454 647456 648960 653024 659702 660406 662212 662252 662672 663008 664802 669494 675264 678762 678774 681608 686200 687254 687258 687264 687266 687666 687688 687884 688026 688028 688164 688316 688666 691462 693980 696978 699292 699296 701142
Blocks: metatracker
  Show dependency tree
 
Reported: 2014-03-30 20:29 UTC by Kai Krakow
Modified: 2019-12-13 08:23 UTC (History)
38 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
conversion script (multilib-conv.sh,2.80 KB, text/x-sh)
2014-03-31 23:16 UTC, Jonathan Callen (RETIRED)
Details
convert symlink systems (convert-symlink-lib.py,10.49 KB, text/plain)
2014-04-07 00:58 UTC, SpanKY
Details
convert symlink systems (convert-symlink-lib.py,13.11 KB, text/plain)
2014-05-06 20:21 UTC, SpanKY
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Kai Krakow 2014-03-30 20:29:48 UTC
While trying to debug systemd-212 with gdb or valgrind, symbols of systemd binaries cannot be found. They are installed in the wrong location.

I've symlinked /usr/lib/debug/usr/lib64/systemd to ../lib/systemd and then it works - symbols are found.

That means, it is wrong to install symbols to /usr/lib/debug/usr/lib/systemd because gdb and co evaluate the real path of systemd binaries to be within /usr/lib64/debug (after all, lib is only a symlink to lib64), and then they fail to find them in /usr/lib/debug/usr/lib64...

This seems to apply to many other packages, too. So maybe it would be best to place a symlink for /usr/lib/debug/{,usr/}lib to .../lib64, too? But I'm not sure how well that works if I'd try that now.

OTOH, the ebuild should probably install its files to /usr/lib64 and not through the symlinked directory.
Comment 1 Jeroen Roovers gentoo-dev 2014-03-30 22:56:27 UTC
[ You could use `gdb --symbols=/path/to/foo' instead of the symlink. I don't think valgrind has a similar option. ]
Comment 2 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-03-31 15:58:31 UTC
What gdb version? This looks like a dupe of bug #378537 to me,
Comment 3 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-03-31 16:00:04 UTC
Hm, or is this about the second 'lib' in the path and '/usr/lib64/debug' is just a mistake in the text?
Comment 4 Kai Krakow 2014-03-31 16:21:25 UTC
(In reply to Michał Górny from comment #2)
> What gdb version? This looks like a dupe of bug #378537 to me,

While it seems to describe my problem exactly, I think it is not exactly a dupe. From the description there I read package need to be fixed to install debuginfo into the right directory. Thus, this request is valid.

Does it make sense?
Comment 5 Kai Krakow 2014-03-31 16:30:02 UTC
(In reply to Kai Krakow from comment #4)
> (In reply to Michał Górny from comment #2)
> > What gdb version? This looks like a dupe of bug #378537 to me,
> 
> While it seems to describe my problem exactly, I think it is not exactly a
> dupe. From the description there I read package need to be fixed to install
> debuginfo into the right directory. Thus, this request is valid.
> 
> Does it make sense?

On a second thought: no, not a dupe.

The problem is not the base path /usr/lib/debug, it's with the paths folded into this directory:

/usr/lib/debug/usr/lib/systemd is wrong location, gdb does not look there for systemd debug symbols, it looks in /usr/lib/debug/usr/lib64/systemd.

As soon as I move the directory over and set a symlink for /usr/lib/debug/usr/(lib -> lib64) it starts working within gdb.

So, systemd ebuild should not install its debug symbols into /usr/lib/debug/usr/lib but into /usr/lib/debug/usr/lib64.

Since the /usr/lib/debug hierarchie replicates the rootfs hierarchie, I guess it should also have lib as a symlink to lib64 (which it doesn't currently).

So in the end I'm not sure who to blame: the systemd ebuild or the general fact that a /usr/lib/debug/usr/lib symlink never existed.
Comment 6 Mike Gilbert gentoo-dev 2014-03-31 16:31:08 UTC
(In reply to Kai Krakow from comment #0)
> This seems to apply to many other packages, too. So maybe it would be best
> to place a symlink for /usr/lib/debug/{,usr/}lib to .../lib64, too? But I'm
> not sure how well that works if I'd try that now.

I think this is probably the best solution given the way we symlink $prefix/lib. 

Maybe we could replicate the SYMLINK_LIB logic in the baselayout ebuild for /usr/lib/debug/lib and /usr/lib/debug/usr/lib?
Comment 7 Mike Gilbert gentoo-dev 2014-03-31 16:32:07 UTC
Adding base-system. Please see my question in the previous comment.
Comment 8 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-03-31 16:34:13 UTC
Reassigning to gdb maintainers to let them decide how to solve this.
Comment 9 SpanKY gentoo-dev 2014-03-31 20:47:00 UTC
i think we need to finally bite the bullet and kill SYMLINK_LIB.  i've killed it for all systems but x86 & ppc, and my own x86/ppc systems are running with it set to "no".

the trick is in the migration.  i took some notes in the past:
  http://dev.gentoo.org/~vapier/lib32-conversion
but we need to formalize this into an upgrader script.

at the very least, we should look at creating new 14.0 profiles which only have SYMLINK_LIB=no.
Comment 10 Mike Gilbert gentoo-dev 2014-03-31 21:13:11 UTC
I know jcallen has written a script.
Comment 11 Anthony Basile gentoo-dev 2014-03-31 21:49:35 UTC
(In reply to SpanKY from comment #9)
> at the very least, we should look at creating new 14.0 profiles which only
> have SYMLINK_LIB=no.

Can we do this as a first step?  I'm like 99% sure nothing will break for me, but I'm not 100% sure.
Comment 12 Jonathan Callen (RETIRED) gentoo-dev 2014-03-31 23:16:12 UTC
Created attachment 373992 [details]
conversion script

Attached is the script I wrote a while ago to document how I tend to do the conversions.  Note that this is very much a work-in-progess, and it may break your system horribly.
Comment 13 Kai Krakow 2014-04-01 10:51:36 UTC
(In reply to Michał Górny from comment #3)
> Hm, or is this about the second 'lib' in the path and '/usr/lib64/debug' is
> just a mistake in the text?

Yeah, mistake in the text. It should've read '/usr/lib/debug'...
Comment 14 Kai Krakow 2014-04-01 10:58:53 UTC
(In reply to Kai Krakow from comment #13)
> (In reply to Michał Górny from comment #3)
> > Hm, or is this about the second 'lib' in the path and '/usr/lib64/debug' is
> > just a mistake in the text?
> 
> Yeah, mistake in the text. It should've read '/usr/lib/debug'...

Argh... Sorry for the noise. Should've read '/usr/lib64/systemd' - otherwise the rest of the text does not make much sense because I think the following happens:

1. GDB evaluates /usr/lib/systemd/systemd-networkd to
   /usr/lib64/systemd/... using realpath function

2. GDB prepends '/usr/lib/debug' to find the debug symbols.

3. Result is '/usr/lib/debug/usr/lib64/systemd/...' but systemd debug symbols
   are in /usr/lib/debug/usr/lib/systemd/... and the second 'lib' is no symlink.

So my first intention is that the systemd ebuild installs to /usr/lib instead
of /usr/lib64.

The script outlined here probably won't fix the wrong locate of the second 'lib',
right? So I changed my system to move /usr/lib/debug/{,usr/}lib to ...lib64 and
place a symlink which fixed the issue for me.

I think the proposal outlined here will fix all those debug symbol problems for
new package installations but not for existing installations because while it
migrates the package db of portage, it won't touch the wrong installation in
/usr/lib/debug, right?
Comment 15 SpanKY gentoo-dev 2014-04-01 16:17:39 UTC
(In reply to Kai Krakow from comment #14)

your issue boils down to the lib->lib64 symlink.  once that is fixed, there is no longer an issue locating the symbols.
Comment 16 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-04-01 16:32:28 UTC
Do you feel like we oughtn't install a temporary symlink in /usr/lib/debug until we finally get rid of the SYMLINK_LIB cruft?

Maybe it would be wise to install such a thing in gdb for now.
Comment 17 William Hubbs gentoo-dev 2014-04-02 20:29:57 UTC
All,

I have a question about this as well.

FHS2 stated that for the amd64 architecture, the 64 bit libraries should
go in "lib64". However, FHS3 seems to remove this exception [1].

So, can we also do so and put the default abi libraries in the "lib"
directory and use "libxx" for the alternate abis?

[1] http://www.linuxbase.org/betaspecs/fhs/fhs.html
Comment 18 Mike Gilbert gentoo-dev 2014-04-02 20:57:16 UTC
(In reply to William Hubbs from comment #17)

Do any other distros actually do that?
Comment 19 Anthony Basile gentoo-dev 2014-04-02 22:04:00 UTC
(In reply to William Hubbs from comment #17)
> All,
> 
> I have a question about this as well.
> 
> FHS2 stated that for the amd64 architecture, the 64 bit libraries should
> go in "lib64". However, FHS3 seems to remove this exception [1].
> 
> So, can we also do so and put the default abi libraries in the "lib"
> directory and use "libxx" for the alternate abis?
> 
> [1] http://www.linuxbase.org/betaspecs/fhs/fhs.html

I don't think that would be a good idea.  In mips the standard is o32 in /lib, n32 in /lib32 and n64 in /lib64.  Usually n32 is taken as the default.  This goes back to the SGI days and I don't think we should arbitrarily change it.


(In reply to Mike Gilbert from comment #18)
> (In reply to William Hubbs from comment #17)
> 
> Do any other distros actually do that?

Multilib amd64 debian uses /lib32 for 32-bit, /lib for 64-bit but still has a /lib64 with only ld-linux-x86-64.so.2 -> /lib/x86_64-linux-gnu/ld-2.13.so.

However, multilib mips does the above.
Comment 20 William Hubbs gentoo-dev 2014-04-02 23:07:13 UTC
> (In reply to Mike Gilbert from comment #18)
> > (In reply to William Hubbs from comment #17)
> > 
> > Do any other distros actually do that?
> 
> Multilib amd64 debian uses /lib32 for 32-bit, /lib for 64-bit but still has
> a /lib64 with only ld-linux-x86-64.so.2 -> /lib/x86_64-linux-gnu/ld-2.13.so.
> 
> However, multilib mips does the above.

In that case, does multilib need to be able to set different default abi directories based on the architecture?
Comment 21 Anthony Basile gentoo-dev 2014-04-02 23:40:37 UTC
(In reply to William Hubbs from comment #20)
> > (In reply to Mike Gilbert from comment #18)
> > > (In reply to William Hubbs from comment #17)
> > > 
> > > Do any other distros actually do that?
> > 
> > Multilib amd64 debian uses /lib32 for 32-bit, /lib for 64-bit but still has
> > a /lib64 with only ld-linux-x86-64.so.2 -> /lib/x86_64-linux-gnu/ld-2.13.so.
> > 
> > However, multilib mips does the above.
> 
> In that case, does multilib need to be able to set different default abi
> directories based on the architecture?

Yes.  We should probably document our standards here by cross referencing ISA-ABI-libdir before we jump in.
Comment 22 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-04-03 06:03:28 UTC
(In reply to William Hubbs from comment #17)
> So, can we also do so and put the default abi libraries in the "lib"
> directory and use "libxx" for the alternate abis?

That sounds like a cheap trick to stop fixing the packages that doesn't respect libdir. Or at least stop getting QA warnings for them.
Comment 23 Anthony Basile gentoo-dev 2014-04-03 11:06:01 UTC
(In reply to Michał Górny from comment #22)
> (In reply to William Hubbs from comment #17)
> > So, can we also do so and put the default abi libraries in the "lib"
> > directory and use "libxx" for the alternate abis?
> 
> That sounds like a cheap trick to stop fixing the packages that doesn't
> respect libdir. Or at least stop getting QA warnings for them.

When working on the lemote (mips64el) I opened bugs against lots of these mostly involving xfce4 and desktop related packages.  It really didn't take that long, about 3-4 months.

But following through, we should do this right and follow the standards.
Comment 24 SpanKY gentoo-dev 2014-04-03 14:14:25 UTC
we already tried the route of forcing native abi to be /usr/lib (that's largely what the symlink was for, but it was also done on ppc64/s390x w/real dirs) and it just didn't really work out.  you need to hack gcc & binutils to use the right paths otherwise you continue to get funky behavior, and even then the ldso path is hardcoded for specific locations.  it's really not worth the time to try and go against it.

it also makes it a huge pita to try and take packages compiled on one system for one abi (e.g. ARCH=x86 ABI=x86) and try to run them on a different one (e.g. ARCH=amd64 ABI=x86) because now the libdirs are different.
Comment 25 SpanKY gentoo-dev 2014-04-03 14:15:12 UTC
(In reply to Michał Górny from comment #16)

i don't think it's really worth the effort, especially considering how few people have run into it, and how easy it is to workaround.
Comment 26 Anthony Basile gentoo-dev 2014-04-03 15:17:18 UTC
(In reply to SpanKY from comment #24)
> it also makes it a huge pita to try and take packages compiled on one system
> for one abi (e.g. ARCH=x86 ABI=x86) and try to run them on a different one
> (e.g. ARCH=amd64 ABI=x86) because now the libdirs are different.

I was thinking only of the harded coded paths in gcc and friends, but this is also a good point (and obviously related).  We want the INTERP path to be consistent across arches so that on amd64/x86 and x86/x86 we'd have readelf -l interpreter = /lib/ld-linux.so.2 otherwise we'd need sym link it.  All the more so with mips.
Comment 27 Joshua Kinard gentoo-dev 2014-04-04 23:49:36 UTC
(In reply to Anthony Basile from comment #19)
> (In reply to William Hubbs from comment #17)
> > All,
> > 
> > I have a question about this as well.
> > 
> > FHS2 stated that for the amd64 architecture, the 64 bit libraries should
> > go in "lib64". However, FHS3 seems to remove this exception [1].
> > 
> > So, can we also do so and put the default abi libraries in the "lib"
> > directory and use "libxx" for the alternate abis?
> > 
> > [1] http://www.linuxbase.org/betaspecs/fhs/fhs.html
> 
> I don't think that would be a good idea.  In mips the standard is o32 in
> /lib, n32 in /lib32 and n64 in /lib64.  Usually n32 is taken as the default.
> This goes back to the SGI days and I don't think we should arbitrarily
> change it.

Curious, since the 'x32' ABI being experimented with in x86_64 land, and since that appears to be very similar in purpose to MIPS' n32, would it adopt the same layout n a multilib environment?  Orig x86 in /lib, x32 in /lib32, and full x64 in /lib64?

I haven't tried x32 yet, so I don't know if the end goal is to have multilib functionality, or if it's eventually slated to utterly replace all 32-bit x86 code on 64-bit platforms.
Comment 28 Jonathan Callen (RETIRED) gentoo-dev 2014-04-05 01:48:40 UTC
(In reply to Joshua Kinard from comment #27)
> Curious, since the 'x32' ABI being experimented with in x86_64 land, and
> since that appears to be very similar in purpose to MIPS' n32, would it
> adopt the same layout n a multilib environment?  Orig x86 in /lib, x32 in
> /lib32, and full x64 in /lib64?
> 
> I haven't tried x32 yet, so I don't know if the end goal is to have multilib
> functionality, or if it's eventually slated to utterly replace all 32-bit
> x86 code on 64-bit platforms.

Because people used /lib32 for the original x86, /libx32 is used for x32 (and the dynamic linker is at /libx32/ld-linux-x32.so.2).
Comment 29 Samuli Suominen gentoo-dev 2014-04-05 03:20:59 UTC
(In reply to Anthony Basile from comment #19)
> (In reply to William Hubbs from comment #17)
> > All,
> > 
> > I have a question about this as well.
> > 
> > FHS2 stated that for the amd64 architecture, the 64 bit libraries should
> > go in "lib64". However, FHS3 seems to remove this exception [1].
> > 
> > So, can we also do so and put the default abi libraries in the "lib"
> > directory and use "libxx" for the alternate abis?
> > 
> > [1] http://www.linuxbase.org/betaspecs/fhs/fhs.html
> 
> I don't think that would be a good idea.  In mips the standard is o32 in
> /lib, n32 in /lib32 and n64 in /lib64.  Usually n32 is taken as the default.
> This goes back to the SGI days and I don't think we should arbitrarily
> change it.

That seems horrible inconsistent. Should move them like done for x32, put possible o32 to libo32, like x32 puts to libx32
Comment 30 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-04-05 07:35:59 UTC
I suggest checking what other distros do. If it's relatively sane, we may want to change our layout for binary package compatibility.
Comment 31 Joshua Kinard gentoo-dev 2014-04-05 08:36:05 UTC
(In reply to Samuli Suominen from comment #29)
> (In reply to Anthony Basile from comment #19)
> > (In reply to William Hubbs from comment #17)
> > > All,
> > > 
> > > I have a question about this as well.
> > > 
> > > FHS2 stated that for the amd64 architecture, the 64 bit libraries should
> > > go in "lib64". However, FHS3 seems to remove this exception [1].
> > > 
> > > So, can we also do so and put the default abi libraries in the "lib"
> > > directory and use "libxx" for the alternate abis?
> > > 
> > > [1] http://www.linuxbase.org/betaspecs/fhs/fhs.html
> > 
> > I don't think that would be a good idea.  In mips the standard is o32 in
> > /lib, n32 in /lib32 and n64 in /lib64.  Usually n32 is taken as the default.
> > This goes back to the SGI days and I don't think we should arbitrarily
> > change it.
> 
> That seems horrible inconsistent. Should move them like done for x32, put
> possible o32 to libo32, like x32 puts to libx32

Actually, I believe that's the exact way SGI did it back in the 1990's.  And SGI never was known for being the champion of consistency, except in the price tag of their machines.

o32 == "Old" 32-bit, IRIX 5.x and earlier.  Pure 32-bit only, equiv to x86.
n32 == "New" 32-bit, IRIX 6.x and up.  64-bit w/ 32-bit pointers.  Equiv to x32.
n64 == "New" 64-bit, IRIX 6.x and up.  Pure 64-bit, equiv x86_64.
o64 == "Old" 64-bit.  Doesn't really exist AFAIK.  Used only for compiling the Linux kernel on SGI MIPS64 systems because of oddities w/ ARCS, the PROM (bios).  Might've been used in IRIX, but for what, I don't know.

So using your example, in a triple-ABI MIPS setup, you'd want to keep o32 in /lib and put n32 in /libn32 (since n32 is largely what x32 is copying), and n64 into /lib64.

More details are in the N32 Handbook:
http://techpubs.sgi.com/library/tpl/cgi-bin/browse.cgi?coll=0650&db=bks&cmd=toc&pth=/SGI_Developer/Mpro_n32_ABI
Comment 32 Anthony Basile gentoo-dev 2014-04-05 12:52:29 UTC
(In reply to Michał Górny from comment #30)
> I suggest checking what other distros do. If it's relatively sane, we may
> want to change our layout for binary package compatibility.

Yes and no.  Let's go with patching binutils and gcc as little as possible.  Take a look at the paths coded in cc/config/<arch>/linux-<abi>.h  (or similar) for the hard coded paths.

My guess is that other distros already go with minimal patching.
Comment 33 Pacho Ramos gentoo-dev 2014-04-05 13:15:14 UTC
(CCing multilib as old emul packages are generated on a chroot still relying on SYMLINK_LIB=yes... I guess once x86 is migrated shouldn't be problematic... but not sure)
Comment 34 SpanKY gentoo-dev 2014-04-05 14:42:44 UTC
(In reply to Joshua Kinard from comment #27)

mips' libdirs are already correct afaik.  gcc uses:
 n32 lib32
 o32 lib
 n64 lib64
and that's the same as Gentoo.

(In reply to Pacho Ramos from comment #33)

yes & no ... this is why i made those commits to emul-linux-x86.eclass to handle SYMLINK_LIB all that long ago.  it mostly works.
Comment 35 Joshua Kinard gentoo-dev 2014-04-05 17:49:55 UTC
(In reply to SpanKY from comment #34)
> (In reply to Joshua Kinard from comment #27)
> 
> mips' libdirs are already correct afaik.  gcc uses:
>  n32 lib32
>  o32 lib
>  n64 lib64
> and that's the same as Gentoo.

Yeah, then going along w/ Anthony's approach of minimal patching, we don't want to mess with that.  I might throw together an x32 VM in a bit for my own curiosity to see how it works.
Comment 36 SpanKY gentoo-dev 2014-04-06 14:28:48 UTC
(In reply to Joshua Kinard from comment #35)

if you have an am64 box, just run the x32 stage3 in a chroot.  i also recently updated glibc so it can bootstrap itself on the fly, so if you have an amd64 box, you can change MULTILIB_ABIS to include x32 and then re-emerge glibc & gcc (in that order) to get the 3rd multilib.
Comment 37 William Hubbs gentoo-dev 2014-04-06 16:56:18 UTC
(In reply to SpanKY from comment #34)
> (In reply to Joshua Kinard from comment #27)
> 
> mips' libdirs are already correct afaik.  gcc uses:
>  n32 lib32
>  o32 lib
>  n64 lib64
> and that's the same as Gentoo.
> 

The most sane thing to do, thinking more about this, is probably to follow what gcc and friends do; that way patching is minimal.

What do they do for amd64?
Comment 38 Jonathan Callen (RETIRED) gentoo-dev 2014-04-06 19:06:43 UTC
(In reply to William Hubbs from comment #37)
> (In reply to SpanKY from comment #34)
> > (In reply to Joshua Kinard from comment #27)
> > 
> > mips' libdirs are already correct afaik.  gcc uses:
> >  n32 lib32
> >  o32 lib
> >  n64 lib64
> > and that's the same as Gentoo.
> > 
> 
> The most sane thing to do, thinking more about this, is probably to follow
> what gcc and friends do; that way patching is minimal.
> 
> What do they do for amd64?

GCC uses lib for x86, lib64 for x86_64, and libx32 for x32.
Comment 39 SpanKY gentoo-dev 2014-04-07 00:58:06 UTC
Created attachment 374432 [details]
convert symlink systems

i've rewritten the migration script in python and tested it on an amd64 stage3 and migrates everything correctly (afaict)

i'm in the process of landing this in Chromium OS to get some data about converting real systems over

it should be somewhat safe to re-run too ...
Comment 40 SpanKY gentoo-dev 2014-04-07 00:58:44 UTC
(In reply to Jonathan Callen from comment #38)

this is what amd64 systems are already using when you have SYMLINK_LIB=no
Comment 41 Ryan Hill (RETIRED) gentoo-dev 2014-04-07 03:33:22 UTC
I'm not sure what "minimal patching" other distros are doing.  I've never run into it AFAIR.  I'm pretty sure Fedora/Red Hat stick with the defaults and Debian/Ubuntu uses that multiarch madness they managed to get pushed in.
Comment 42 Anthony Basile gentoo-dev 2014-04-07 13:07:43 UTC
(In reply to SpanKY from comment #39)
> Created attachment 374432 [details]
> convert symlink systems
> 
> i've rewritten the migration script in python and tested it on an amd64
> stage3 and migrates everything correctly (afaict)
> 
> i'm in the process of landing this in Chromium OS to get some data about
> converting real systems over
> 
> it should be somewhat safe to re-run too ...

Will this script fail on systems using paludis because it depends on information in vdb which may not be supplied by paludis?  I've got bug #458866 open, with council support http://www.gentoo.org/proj/en/council/meeting-logs/20130910-summary.txt.  There the problem is NEEDED.ELF.2 which paludis people don't think is useful (sic!).  

Your script is different but I don't know paludis well enough to know if its missing even more info than just NEEDED.ELF.2 which may lead to breakage.  (Also I don't care enough to test because I don't like paludis, but I mention it paranthetically.)
Comment 43 SpanKY gentoo-dev 2014-04-07 18:05:40 UTC
(In reply to Anthony Basile from comment #42)

most likely, but i don't really care.  if the paludis peeps want to write the logic that involves directly munging their vdb equivalent, then they're certainly free to.  there is no portable way to reach into the inner vdb details and muck with data (by design).
Comment 44 Kai Krakow 2014-04-10 15:57:31 UTC
BTW: What about the original problem, read: Installing a symlink /usr/lib/debug/{usr/,}{lib64 -> lib} which would fix the problem for the time being?

And then you can have a lengthy descussion about what's the best way to go... ;-)

I'm feeling my bug report got hijacked.
Comment 45 Anthony Basile gentoo-dev 2014-04-14 12:12:20 UTC
(In reply to Kai Krakow from comment #44)
> BTW: What about the original problem, read: Installing a symlink
> /usr/lib/debug/{usr/,}{lib64 -> lib} which would fix the problem for the
> time being?
> 
> And then you can have a lengthy descussion about what's the best way to
> go... ;-)
> 
> I'm feeling my bug report got hijacked.

Not really.  We don't want a little patch job on a bigger problem.
Comment 46 SpanKY gentoo-dev 2014-04-14 22:43:20 UTC
(In reply to Kai Krakow from comment #44)

not really ... see comment #16 and comment #25
Comment 47 Alexander Tsoy 2014-04-24 14:10:00 UTC
(In reply to SpanKY from comment #39)
> Created attachment 374432 [details]
> convert symlink systems

Some issues:

- Incompatible with python3 (at least because raw_input() was renamed to input()). So maybe put /"usr/bin/python2" into shebang?

- Seems it doesn't move empty dirs:
$ ls -d /usr/lib64/systemd/*
/usr/lib64/systemd/system-shutdown  /usr/lib64/systemd/user-generators

- qcheck from stable portage-utils does not support "--root" option. I believe it's supported in newer versions, so a check of the `qcheck --version` output would be appropriate.
Comment 48 SpanKY gentoo-dev 2014-05-06 20:21:06 UTC
Created attachment 376516 [details]
convert symlink systems

fixes python3 raw_input handling

i don't follow systemd, so i can't comment on that.  the code will automatically trim empty dirs, so if systemd creates things in pkg_* and thus no package owns them, they probably get orphaned.

i've marked a newer portage-utils stable that supports --root.
Comment 49 Mike Gilbert gentoo-dev 2014-05-06 20:28:08 UTC
Regarding systemd, the empty directories are owned by the systemd package, but no files are installed below them.

% grep system-shutdown /var/db/pkg/sys-apps/systemd-212-r4/CONTENTS
dir /usr/lib/systemd/system-shutdown

% grep user-generators /var/db/pkg/sys-apps/systemd-212-r4/CONTENTS
dir /usr/lib/systemd/user-generators
Comment 50 SpanKY gentoo-dev 2014-05-06 20:51:05 UTC
(In reply to Mike Gilbert from comment #49)

owning an empty dir has no meaning.  it never really has.  you have to create files in there, and when you do, the script should migrate them fine.
Comment 51 Mike Gilbert gentoo-dev 2014-05-06 21:25:53 UTC
(In reply to SpanKY from comment #50)

The directories get created by the build system when we run make install. I think it is no great loss if the empty directories are not migrated.

Alexander: Did this cause any problems?
Comment 52 Alexander Tsoy 2014-05-06 22:49:30 UTC
(In reply to Mike Gilbert from comment #51)

No real problems. These directories just become orphans.

BTW, I just remember another problem with systemd. On systems with SYMLINK_LIB=yes symlinks in /etc/systemd/system points to /usr/lib64/* and after migration they become broken.
Comment 53 Mike Gilbert gentoo-dev 2014-05-07 01:17:56 UTC
(In reply to Alexander Tsoy from comment #52)
> No real problems. These directories just become orphans.

Hmm. I should think that we would remove the empty directories rather rather than leaving them under /usr/lib64.

> BTW, I just remember another problem with systemd. On systems with
> SYMLINK_LIB=yes symlinks in /etc/systemd/system points to /usr/lib64/* and
> after migration they become broken.

systemd ends up using just the basename of the symlink target, so the broken symlinks are not really an issue.
Comment 54 Alexander Tsoy 2014-05-07 07:16:27 UTC
(In reply to Mike Gilbert from comment #53)

> > BTW, I just remember another problem with systemd. On systems with
> > SYMLINK_LIB=yes symlinks in /etc/systemd/system points to /usr/lib64/* and
> > after migration they become broken.
> 
> systemd ends up using just the basename of the symlink target, so the broken
> symlinks are not really an issue.

This is true for symlinks inside *.target.wants directories. However some units have Alias= in [Install] section. For example:

$ tail -n2 /usr/lib/systemd/system/gdm.service          
[Install]
Alias=display-manager.service

$ ls -l /etc/systemd/system/display-manager.service 
lrwxrwxrwx 1 root root 35 Apr 25 03:09 /etc/systemd/system/display-manager.service -> /usr/lib/systemd/system/gdm.service

If display-manager.service symlink is broken, then gdm is not get started at boot.
Comment 55 Mike Gilbert gentoo-dev 2014-05-07 14:37:56 UTC
(In reply to Alexander Tsoy from comment #54)

Please go ahead and create a separate bug for this issue.
Comment 56 Steven Newbury 2014-05-13 07:15:08 UTC
(In reply to SpanKY from comment #9)
> i think we need to finally bite the bullet and kill SYMLINK_LIB.  i've
> killed it for all systems but x86 & ppc, and my own x86/ppc systems are
> running with it set to "no".
> 
> the trick is in the migration.  i took some notes in the past:
>   http://dev.gentoo.org/~vapier/lib32-conversion
> but we need to formalize this into an upgrader script.
> 
> at the very least, we should look at creating new 14.0 profiles which only
> have SYMLINK_LIB=no.

I've been using SYMLINK_LIB=no on a couple of amd64 systems for quite a while now.  I don't know if you've encountered this, but lately it seems a lot of ebuilds are using --with-<pkg>=/usr which triggers the typical autoconf snippet to always look in "/usr/lib" since it assumes that --with-<pkg>=PATH means PATH points to a directory containing the installed package with the libs in a "lib" subdir, rather than the system (ABI) libdir.

I can create bugs for each ebuild I've had to drop the "=/usr" path or modify the autoconf/configure code to fix, but if you're having no problems, maybe I'm missing something...?

I'm not the only one experiencing this and attempting to fix:
[dev-util/boost-build]
https://bugs.gentoo.org/show_bug.cgi?id=505694

A similar SYMLINK_LIB=no failure case https://bugs.gentoo.org/show_bug.cgi?id=503746, FIXED, but I find the Comment 3 troubling.  Is there no policy in place for this?  Shouldn't there be a meta-bug, or is this it??

There are also a small number of packages which do not correctly create their pkgconfig files (wrong libdir), for these I definitely will create new bugs, I'm currently carrying far too many fixes in my overlay!

For what it's worth, I would definitely like to see new profiles having SYMLINK_LIB=no, since it would hopefully mean getting these bugs fixed, and maintainers would test for this scenario before new breakage gets committed to the tree.
Comment 57 SpanKY gentoo-dev 2014-05-18 20:06:27 UTC
(In reply to Steven Newbury from comment #56)

the correct answer depends heavily on the package.  usually the autoconf logic is bad and needs patching.
Comment 58 William Hubbs gentoo-dev 2016-07-11 01:27:05 UTC
All,

can we please move this forward some how?

Thanks,

William
Comment 59 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2016-07-11 05:19:01 UTC
Someone would have to fix all the bugs that block this, and come up with a plan for the transition. I guess the only sane thing to do here is introduce new profiles, news item telling people how to upgrade and die in profile.bashrc if they switch to new profiles without upgrading first.

The migration script needs review as well. For a start, I can see that it's unnecessarily mangling make.conf.
Comment 60 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2016-11-04 16:19:36 UTC
FYI, I'm planning to work a bit on this soonish. I have an idea for a safer migration script. However, I'm still not sure how to handle the profile side best.
Comment 61 Steven Newbury 2016-12-21 13:51:24 UTC
(In reply to Michał Górny from comment #59)
> Someone would have to fix all the bugs that block this, and come up with a
> plan for the transition. I guess the only sane thing to do here is introduce
> new profiles, news item telling people how to upgrade and die in
> profile.bashrc if they switch to new profiles without upgrading first.
> 

I'm working my way through the bugs.  I've probably already fixed most of them in one of my overlays, but as I emerge new versions I'm making sure I provide the fix here.  Just fixed dev-lang/mujs.

The relevant overlays are here (x32 mostly contains x32 ports, while gx86-staging mostly contains multilibized versions of ebuilds) :

https://github.com/sjnewbury/gentoo-gx86-staging
https://github.com/sjnewbury/x32
Comment 62 William Hubbs gentoo-dev 2017-06-08 21:12:27 UTC
@toolchain:
What happened to this project?

I was thinking about us getting new profiles soon, and this might be a
time to do this?

William
Comment 63 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-12-05 14:13:04 UTC
Given that 17.0 profiles were released without this, I'm going to attempt to establish experimental 17.1 profiles with this for wider testing.
Comment 64 Larry the Git Cow gentoo-dev 2017-12-08 14:21:17 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=2dc384e97c73be555bec5cc3019ac89605477dce

commit 2dc384e97c73be555bec5cc3019ac89605477dce
Author:     Michał Górny <mgorny@gentoo.org>
AuthorDate: 2017-12-05 14:43:29 +0000
Commit:     Michał Górny <mgorny@gentoo.org>
CommitDate: 2017-12-08 14:21:07 +0000

    profiles: Add experimental SYMLINK_LIB=no 17.1 amd64 profiles
    
    Bug: https://bugs.gentoo.org/506276

 profiles/default/linux/amd64/17.1/desktop/eapi              |  1 +
 profiles/default/linux/amd64/17.1/desktop/gnome/eapi        |  1 +
 profiles/default/linux/amd64/17.1/desktop/gnome/parent      |  2 ++
 .../default/linux/amd64/17.1/desktop/gnome/systemd/eapi     |  1 +
 .../default/linux/amd64/17.1/desktop/gnome/systemd/parent   |  2 ++
 profiles/default/linux/amd64/17.1/desktop/parent            |  2 ++
 profiles/default/linux/amd64/17.1/desktop/plasma/eapi       |  1 +
 profiles/default/linux/amd64/17.1/desktop/plasma/parent     |  2 ++
 .../default/linux/amd64/17.1/desktop/plasma/systemd/eapi    |  1 +
 .../default/linux/amd64/17.1/desktop/plasma/systemd/parent  |  2 ++
 profiles/default/linux/amd64/17.1/developer/eapi            |  1 +
 profiles/default/linux/amd64/17.1/developer/make.defaults   |  7 +++++++
 profiles/default/linux/amd64/17.1/developer/parent          |  2 ++
 profiles/default/linux/amd64/17.1/eapi                      |  1 +
 profiles/default/linux/amd64/17.1/hardened/eapi             |  1 +
 profiles/default/linux/amd64/17.1/hardened/parent           |  2 ++
 profiles/default/linux/amd64/17.1/parent                    |  3 +++
 profiles/default/linux/amd64/17.1/selinux/eapi              |  1 +
 profiles/default/linux/amd64/17.1/selinux/parent            |  2 ++
 profiles/default/linux/amd64/17.1/systemd/eapi              |  1 +
 profiles/default/linux/amd64/17.1/systemd/parent            |  2 ++
 profiles/profiles.desc                                      | 13 +++++++++++++
 22 files changed, 51 insertions(+)}
Comment 65 Scott Short 2017-12-27 00:13:56 UTC
Followed instructions, switched to 17.1. That was painless. Thanks! ~amd64 no-multilib.
Comment 66 Leonid Kopylov 2017-12-27 04:44:19 UTC
Switched to 17.1, everything is smooth, default/linux/amd64/17.1/desktop/gnome/systemd
Comment 67 cryptopsy 2017-12-27 21:00:10 UTC
Switched to 17.1 according to the guide, now kernel panicks trying to load /usr/lib/systemd/systemd , which is an invalid symlink.
Comment 68 Mike Gilbert gentoo-dev 2017-12-27 22:51:20 UTC
(In reply to cryptopsy from comment #67)
> Switched to 17.1 according to the guide, now kernel panicks trying to load
> /usr/lib/systemd/systemd , which is an invalid symlink.

Where does the symlink point, and where is the systemd binary located?
Comment 69 david mattatall 2017-12-28 05:36:48 UTC
Portaudio failed to build when I ran the   emerge -1v /lib32 /usr/lib32

* Header files have changed between ABIs.
 * --- /tmp/portage/media-libs/portaudio-19_pre20140130/temp/.multilib_header_cksum	2017-12-27 21:33:38.079100213 -0800
 * +++ /tmp/portage/media-libs/portaudio-19_pre20140130/temp/.multilib_header_cksum.new	2017-12-27 21:33:38.343092820 -0800
 * @@ -12,6 +12,7 @@
 *  1155256988 2605 /tmp/portage/media-libs/portaudio-19_pre20140130/image/usr/include/portaudiocpp/StreamParameters.hxx
 *  2730295555 2704 /tmp/portage/media-libs/portaudio-19_pre20140130/image/usr/include/portaudiocpp/System.hxx
 *  454020832 2788 /tmp/portage/media-libs/portaudio-19_pre20140130/image/usr/include/portaudiocpp/Device.hxx
 * +1674912135 2805 /tmp/portage/media-libs/portaudio-19_pre20140130/image/usr/include/pa_jack.h
 *  2654482981 3201 /tmp/portage/media-libs/portaudio-19_pre20140130/image/usr/include/portaudiocpp/Exception.hxx
 *  2473272198 3422 /tmp/portage/media-libs/portaudio-19_pre20140130/image/usr/include/portaudiocpp/MemFunCallbackStream.hxx
 *  3396923871 3868 /tmp/portage/media-libs/portaudio-19_pre20140130/image/usr/include/pa_linux_alsa.h
 * ERROR: media-libs/portaudio-19_pre20140130::gentoo failed (install phase):
 *   Header checksum mismatch, aborting.
 * 
 * Call stack:
 *     ebuild.sh, line  124:  Called src_install
 *   environment, line 2964:  Called autotools-multilib_src_install
 *   environment, line  489:  Called multilib-minimal_src_install
 *   environment, line 2268:  Called multilib_foreach_abi 'multilib-minimal_abi_src_install'
 *   environment, line 2462:  Called multibuild_foreach_variant '_multilib_multibuild_wrapper' 'multilib-minimal_abi_src_install'
 *   environment, line 2155:  Called _multibuild_run '_multilib_multibuild_wrapper' 'multilib-minimal_abi_src_install'
 *   environment, line 2153:  Called _multilib_multibuild_wrapper 'multilib-minimal_abi_src_install'
 *   environment, line  449:  Called multilib-minimal_abi_src_install
 *   environment, line 2265:  Called multilib_check_headers
 *   environment, line 2324:  Called die
 * The specific snippet of code:
 *               die "Header checksum mismatch, aborting.";
 * 
 * If you need support, post the output of `emerge --info '=media-libs/portaudio-19_pre20140130::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=media-libs/portaudio-19_pre20140130::gentoo'`.
 * The complete build log is located at '/tmp/portage/media-libs/portaudio-19_pre20140130/temp/build.log'.
 * The ebuild environment file is located at '/tmp/portage/media-libs/portaudio-19_pre20140130/temp/environment'.
 * Working directory: '/tmp/portage/media-libs/portaudio-19_pre20140130/work/portaudio-abi_x86_64.amd64'
 * S: '/tmp/portage/media-libs/portaudio-19_pre20140130/work/portaudio'

>>> Failed to emerge media-libs/portaudio-19_pre20140130, Log file:

>>>  '/tmp/portage/media-libs/portaudio-19_pre20140130/temp/build.log'

 * Messages for package sys-apps/systemd-233-r6:

 *   CONFIG_FW_LOADER_USER_HELPER:	 should not be set. But it is.
 * Please check to make sure these options are set correctly.
 * Failure to do so may cause unexpected problems.

 * Messages for package media-libs/portaudio-19_pre20140130:

 * Header files have changed between ABIs.
 * --- /tmp/portage/media-libs/portaudio-19_pre20140130/temp/.multilib_header_cksum	2017-12-27 21:33:38.079100213 -0800
 * +++ /tmp/portage/media-libs/portaudio-19_pre20140130/temp/.multilib_header_cksum.new	2017-12-27 21:33:38.343092820 -0800
 * @@ -12,6 +12,7 @@
 *  1155256988 2605 /tmp/portage/media-libs/portaudio-19_pre20140130/image/usr/include/portaudiocpp/StreamParameters.hxx
 *  2730295555 2704 /tmp/portage/media-libs/portaudio-19_pre20140130/image/usr/include/portaudiocpp/System.hxx
 *  454020832 2788 /tmp/portage/media-libs/portaudio-19_pre20140130/image/usr/include/portaudiocpp/Device.hxx
 * +1674912135 2805 /tmp/portage/media-libs/portaudio-19_pre20140130/image/usr/include/pa_jack.h
 *  2654482981 3201 /tmp/portage/media-libs/portaudio-19_pre20140130/image/usr/include/portaudiocpp/Exception.hxx
 *  2473272198 3422 /tmp/portage/media-libs/portaudio-19_pre20140130/image/usr/include/portaudiocpp/MemFunCallbackStream.hxx
 *  3396923871 3868 /tmp/portage/media-libs/portaudio-19_pre20140130/image/usr/include/pa_linux_alsa.h
 * ERROR: media-libs/portaudio-19_pre20140130::gentoo failed (install phase):
 *   Header checksum mismatch, aborting.
 * 
 * Call stack:
 *     ebuild.sh, line  124:  Called src_install
 *   environment, line 2964:  Called autotools-multilib_src_install
 *   environment, line  489:  Called multilib-minimal_src_install
 *   environment, line 2268:  Called multilib_foreach_abi 'multilib-minimal_abi_src_install'
 *   environment, line 2462:  Called multibuild_foreach_variant '_multilib_multibuild_wrapper' 'multilib-minimal_abi_src_install'
 *   environment, line 2155:  Called _multibuild_run '_multilib_multibuild_wrapper' 'multilib-minimal_abi_src_install'
 *   environment, line 2153:  Called _multilib_multibuild_wrapper 'multilib-minimal_abi_src_install'
 *   environment, line  449:  Called multilib-minimal_abi_src_install
 *   environment, line 2265:  Called multilib_check_headers
 *   environment, line 2324:  Called die
 * The specific snippet of code:
 *               die "Header checksum mismatch, aborting.";
 * 
 * If you need support, post the output of `emerge --info '=media-libs/portaudio-19_pre20140130::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=media-libs/portaudio-19_pre20140130::gentoo'`.
 * The complete build log is located at '/tmp/portage/media-libs/portaudio-19_pre20140130/temp/build.log'.
 * The ebuild environment file is located at '/tmp/portage/media-libs/portaudio-19_pre20140130/temp/environment'.
 * Working directory: '/tmp/portage/media-libs/portaudio-19_pre20140130/work/portaudio-abi_x86_64.amd64'
 * S: '/tmp/portage/media-libs/portaudio-19_pre20140130/work/portaudio'
Comment 70 Mike Gilbert gentoo-dev 2017-12-28 16:43:33 UTC
(In reply to david mattatall from comment #69)

Please create separate bug reports for failing packages.

This is a tracker bug, so the chatter should be kept to a minimum.
Comment 71 Viktor Levin 2017-12-28 20:34:45 UTC
I faced one critical bug.

net-vpn/miredo stopped working on profile default/linux/amd64/17.1/systemd

Because it installs unit file miredo.service into /usr/lib64/systemd/system/
I think it should be installed to /lib/systemd/system/ instead.
Comment 72 Viktor Levin 2017-12-28 20:37:55 UTC
I created new bug for miredo: https://bugs.gentoo.org/642568
Comment 73 Ivan 2017-12-28 21:14:55 UTC
Hello.

Did everything by the book, but I ran into some troubles with some packages like libva, pango and even gcc-6.4.0. Some of them I've managed to solve.

I had problems with: 
x11-libs/libva 
media-libs/mesa 
x11-libs/libva-vdpau-driver 
x11-libs/libva-intel-driver 
media-libs/libepoxy 
x11-libs/pango 
gnome-base/librsvg 
x11-libs/gtk+ (both 3.22 and 2.24)
media-video/ffmpeg 
media-video/mediainfo 
=sys-devel/gcc-6.4.0

Re-emerging @world didn't help at all.
Changing active gcc from 7.2.0 to 6.4.0 didn't help too.

Most problems were solved by emerging packages with adding use flag "-vaapi" in make.conf.
After that I was able to rebuild almost everything from the list above (and then remove symlinks and rebuild again WITH "vaapi" use flag) with exeption to gcc-6.4.0 and mediainfo:

checking for wxWidgets version >= 2.6.0 (--unicode=yes --static=no)... no
WxWidgets not yet compiled, try to compile
File Shared/Project/WxWidgets/Compile.sh not found
configure: error: 
            wxWidgets must be installed on your system.

            Please check that wx-config is in path, the directory
            where wxWidgets libraries are installed (returned by
            'wx-config --libs' or 'wx-config --static --libs' command)
            is in LD_LIBRARY_PATH or equivalent variable and
            wxWidgets version is 2.6.0 or above, and the compilation
            is compatible with
            --unicode=yes --static=no
----

Should I file a separate bug or is it this one: https://bugs.gentoo.org/552500? (I am not that familiar to OS to understand it right)

**************

emerge --info:
Portage 2.3.19 (python 3.4.6-final-0, default/linux/amd64/17.1/desktop/plasma, gcc-7.2.0, glibc-2.26-r3, 4.14.8-gentoo-r1 x86_64)
=================================================================
System uname: Linux-4.14.8-gentoo-r1-x86_64-Intel-R-_Core-TM-_i5-2500K_CPU_@_3.30GHz-with-gentoo-2.4.1
KiB Mem:    16349556 total,   9793072 free
KiB Swap:          0 total,         0 free
Timestamp of repository gentoo: Thu, 28 Dec 2017 12:00:01 +0000
Head commit of repository gentoo: b7d7fd24bf095da0adddc41b4b07770f3e3497cb
sh bash 4.4_p12
ld GNU ld (Gentoo 2.29.1 p3) 2.29.1
app-shells/bash:          4.4_p12::gentoo
dev-lang/perl:            5.26.1-r1::gentoo
dev-lang/python:          2.7.14-r1::gentoo, 3.4.6-r1::gentoo
dev-util/cmake:           3.10.1::gentoo
dev-util/pkgconfig:       0.29.2::gentoo
sys-apps/baselayout:      2.4.1-r2::gentoo
sys-apps/openrc:          0.34.11::gentoo
sys-apps/sandbox:         2.12::gentoo
sys-devel/autoconf:       2.13::gentoo, 2.69-r4::gentoo
sys-devel/automake:       1.15.1-r1::gentoo
sys-devel/binutils:       2.29.1-r1::gentoo
sys-devel/gcc:            6.4.0::gentoo, 7.2.0::gentoo
sys-devel/gcc-config:     1.9.1::gentoo
sys-devel/libtool:        2.4.6-r4::gentoo
sys-devel/make:           4.2.1-r1::gentoo
sys-kernel/linux-headers: 4.13::gentoo (virtual/os-headers)
sys-libs/glibc:           2.26-r3::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000
    sync-rsync-extra-opts: 

my
    location: /usr/local/my
    masters: gentoo
    priority: 0

c2p-overlay
    location: /var/lib/layman/c2p-overlay
    masters: gentoo
    priority: 50

deadbeef-overlay
    location: /var/lib/layman/deadbeef-overlay
    masters: gentoo
    priority: 50

scrill
    location: /var/lib/layman/scrill
    masters: gentoo
    priority: 50

steam-overlay
    location: /var/lib/layman/steam-overlay
    masters: gentoo
    priority: 50

vortex
    location: /var/lib/layman/vortex
    masters: gentoo
    priority: 50

ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -mtune=native -O2 -pipe -fomit-frame-pointer"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=native -mtune=native -O2 -pipe -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--with-bdeps=y --quiet-build=n"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync metadata-transfer multilib-strict news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="ru_RU.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j5"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/tmp"
USE="X a52 aac acl acpi activities aes aften alsa amd64 amr ass audio avx bash-completion branding bzip2 cairo cdda cdr clang cleartype consolekit cracklib crypt css cups cxx dbus declarative device-mapper djvu dri dts dv dvd dvdr egl encode exif faac faad fam fat ffmpeg firefox firmware-loader flac fontconfig gd gif glamor gles gpm gzip iconv icu id3tag ieee1394 imagemagick ipv6 jit jpeg kde keymap ladspa lame lash lcms libass libnotify lm_sensors lzma lzo mad matroska midi mmx mmxext mng modules mp3 mp4 mpeg mplayer mtp mudflap multilib ncurses nls nptl ntfs nvidia ogg opengl openmp opus pam pango pcre pdf phonon pie plasma png policykit popcnt ppds pppd pulseaudio qml qt3support qt5 quicktime rar raw readline scanner sdl seccomp session sse sse2 sse3 sse4_1 sse4_2 ssl ssp ssse3 startup-notification svc svg sysfs tcpd theora threads tiff toolame truetype twolame udev udisks unicode upower usb vaapi vdpau vorbis vpx wav wavpack widgets win32codecs wxwidgets x264 x265 xattr xcb xcomposite xml xorg xscreensaver xulrunner xv xvfb xvid xvmc zip zlib" ABI_X86="64" ALSA_CARDS="emu10k1" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev" KERNEL="linux" L10N="ru en" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="ru en" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6 php7-0" POSTGRES_TARGETS="postgres9_5" PYTHON_SINGLE_TARGET="python3_4" PYTHON_TARGETS="python2_7 python3_4" RUBY_TARGETS="ruby22" USERLAND="GNU" VIDEO_CARDS="nvidia intel i965" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 74 Pavel Kozlov 2017-12-29 17:10:43 UTC
Hi guys!

I have received this in "eselect news" "2017-12-26-experimental-amd64-17-1-profiles". Have two questions:

 - It is said that I can test it in ~amd64 system, what if I have just amd64 (without "~"), is it a good idea to upgrate to 17.1? I didn't make an upgrade, my laptop is still on 13.0, I am thinking between 17.0 and 17.1

 - The first step is "1. Sync and upgrade your system to the newest package versions to reduce the risk of issues." - does it mean I have to run just "emerge --sync", is it enough, or does it mean to make "emerge -uDN world" too? Is it a good idea to upgrate 13.0 -> 17.1, or better to start with 17.0?
Comment 75 Kostas Kavourakis 2018-01-07 18:23:34 UTC
bug #643322 should be added to the list of blocks as the reporter did not do so themselves.
Comment 76 Olof Kindgren 2018-01-14 20:49:58 UTC
(In reply to Pavel Kozlov from comment #74)
> Hi guys!
> 
> I have received this in "eselect news"
> "2017-12-26-experimental-amd64-17-1-profiles". Have two questions:
> 
>  - It is said that I can test it in ~amd64 system, what if I have just amd64
> (without "~"), is it a good idea to upgrate to 17.1? I didn't make an
> upgrade, my laptop is still on 13.0, I am thinking between 17.0 and 17.1
> 
>  - The first step is "1. Sync and upgrade your system to the newest package
> versions to reduce the risk of issues." - does it mean I have to run just
> "emerge --sync", is it enough, or does it mean to make "emerge -uDN world"
> too? Is it a good idea to upgrate 13.0 -> 17.1, or better to start with 17.0?

I would very much like to know this as well. Are there packages that need to be promoted to stable first to make this work?
Comment 77 Kostas Kavourakis 2018-01-15 11:00:31 UTC
(In reply to Olof Kindgren from comment #76)
> (In reply to Pavel Kozlov from comment #74)
> > Hi guys!
> > 
> > I have received this in "eselect news"
> > "2017-12-26-experimental-amd64-17-1-profiles". Have two questions:
> > 
> >  - It is said that I can test it in ~amd64 system, what if I have just amd64
> > (without "~"), is it a good idea to upgrate to 17.1? I didn't make an
> > upgrade, my laptop is still on 13.0, I am thinking between 17.0 and 17.1
> > 
> >  - The first step is "1. Sync and upgrade your system to the newest package
> > versions to reduce the risk of issues." - does it mean I have to run just
> > "emerge --sync", is it enough, or does it mean to make "emerge -uDN world"
> > too? Is it a good idea to upgrate 13.0 -> 17.1, or better to start with 17.0?
> 
> I would very much like to know this as well. Are there packages that need to
> be promoted to stable first to make this work?

I performed the transition in my own system without running a full unstable OS, just a few ~amd64 packages for my own reasons. However, the system I am talking about is primarily a home server setup, so no fancy desktop environment and thousands of packages.
Comment 78 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2018-01-16 08:14:05 UTC
(In reply to Pavel Kozlov from comment #74)
>  - It is said that I can test it in ~amd64 system, what if I have just amd64
> (without "~"), is it a good idea to upgrate to 17.1? I didn't make an
> upgrade, my laptop is still on 13.0, I am thinking between 17.0 and 17.1
> 
>  - The first step is "1. Sync and upgrade your system to the newest package
> versions to reduce the risk of issues." - does it mean I have to run just
> "emerge --sync", is it enough, or does it mean to make "emerge -uDN world"
> too? Is it a good idea to upgrate 13.0 -> 17.1, or better to start with 17.0?

It probably isn't a good idea. The tool used to do the migration does not have stable keywords yet, and given there were some bugs found, the fixes for those bugs will also be ~amd64 for a while now.
Comment 79 Olof Kindgren 2018-01-16 10:06:01 UTC
(In reply to Michał Górny from comment #78)
> (In reply to Pavel Kozlov from comment #74)
> >  - It is said that I can test it in ~amd64 system, what if I have just amd64
> > (without "~"), is it a good idea to upgrate to 17.1? I didn't make an
> > upgrade, my laptop is still on 13.0, I am thinking between 17.0 and 17.1
> > 
> >  - The first step is "1. Sync and upgrade your system to the newest package
> > versions to reduce the risk of issues." - does it mean I have to run just
> > "emerge --sync", is it enough, or does it mean to make "emerge -uDN world"
> > too? Is it a good idea to upgrate 13.0 -> 17.1, or better to start with 17.0?
> 
> It probably isn't a good idea. The tool used to do the migration does not
> have stable keywords yet, and given there were some bugs found, the fixes
> for those bugs will also be ~amd64 for a while now.

Ok, so it sounds like it makes sense to wait at least until unsymlink-lib is stable. Will there be an announcement when it's time to upgrade stable systems?

Related question, will this give me 32-bit ctri.o and crtn.o in /usr/lib ? I have a commercial EDA tool that needs this, so I currently have to overwrite these files temporarily with 32-bit versions when running the program, which is a terrible hack
Comment 80 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2018-01-16 10:22:21 UTC
(In reply to Olof Kindgren from comment #79)
> Ok, so it sounds like it makes sense to wait at least until unsymlink-lib is
> stable. Will there be an announcement when it's time to upgrade stable
> systems?

Yes, there will be.

> Related question, will this give me 32-bit ctri.o and crtn.o in /usr/lib ? I
> have a commercial EDA tool that needs this, so I currently have to overwrite
> these files temporarily with 32-bit versions when running the program, which
> is a terrible hack

Yes, it will fix that.

And now please stop asking questions on this bug, there's a lot of people subscribed to it.
Comment 81 Alex 2018-02-01 10:43:44 UTC
Hi folks,
Tried to go from profile 16 to 30 as follow:
[16]  default/linux/amd64/17.0/desktop (stable)
[30]  default/linux/amd64/17.1/desktop (exp) *
migrate and finish went smoothly but the recompile not. After rebuilding gcc-7.3.0 I issued
emerge -1v /lib32 /usr/lib32 --keep-going
and this tried to rebuild more than 300 packages leaving about 120 with mistakes.
Actually only 3 packages (glibc, systemd and pam) were claiming /lib32 and among those systemd did not want to be rebuilt.
The second round
emerge -1v /lib32 /usr/lib32 --keep-going
left me with just 26 packages with mistakes without systemd among them. Here they are:

[ebuild   R    ] x11-libs/pango-1.40.14::gentoo  USE="X introspection {-test}" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] dev-cpp/cairomm-1.12.0-r1::gentoo  USE="X svg (-aqua) -doc" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] gnome-base/librsvg-2.40.18:2::gentoo  USE="introspection -tools -vala" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] dev-cpp/pangomm-2.40.1:1.4::gentoo  USE="-doc" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] x11-libs/pangox-compat-0.0.2-r1::gentoo  ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] x11-libs/gtk+-2.24.31-r1:2::gentoo  USE="cups introspection xinerama (-aqua) -examples {-test} -vim-syntax" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] x11-themes/gtk-engines-adwaita-3.22.3::gentoo  ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] gnome-base/libglade-2.6.4-r2:2.0::gentoo  USE="-static-libs {-test} -tools" ABI_X86="32 (64) (-x32)" PYTHON_SINGLE_TARGET="python2_7 (-pypy)" PYTHON_TARGETS="python2_7 (-pypy)" 0 KiB
[ebuild   R    ] dev-cpp/gtkmm-2.24.5:2.4::gentoo  USE="-doc -examples {-test}" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] x11-themes/gtk-engines-2.20.2-r2:2::gentoo  USE="-accessibility -lua" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] gnome-base/libgnomecanvas-2.30.3-r1::gentoo  USE="-glade {-test}" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] x11-themes/gtk-engines-murrine-0.98.2-r1::gentoo  USE="themes -animation-rtl" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] dev-cpp/libglademm-2.6.7-r2:2.4::gentoo  USE="-doc -examples" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] x11-libs/wxGTK-3.0.3:3.0::gentoo  USE="X gstreamer libnotify opengl sdl tiff (-aqua) -debug -doc" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] x11-libs/gtkglext-1.2.0-r4::gentoo  USE="-debug" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] x11-libs/libva-vdpau-driver-0.7.4-r4::gentoo  USE="opengl -debug" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] x11-libs/libva-1.7.3::gentoo  USE="X drm egl opengl vdpau wayland" ABI_X86="32 (64) (-x32)" VIDEO_CARDS="i965 intel nouveau nvidia -dummy" 0 KiB
[ebuild   R   ~] media-libs/mesa-18.0.0_rc3::gentoo  USE="classic dri3 egl gallium gbm gles2 llvm nptl opencl vaapi vdpau wayland xvmc -bindist -d3d9 -debug -gles1 -openmax -osmesa -pax_kernel -pic (-selinux) -unwind -valgrind -vulkan -xa" ABI_X86="32 (64) (-x32)" VIDEO_CARDS="i965 intel nouveau (-freedreno) -i915 (-imx) -r100 -r200 -r300 -r600 -radeon -radeonsi (-vc4) -virgl (-vivante) -vmware" 0 KiB
[ebuild   R    ] media-libs/libepoxy-1.4.2::gentoo  USE="X {-test}" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] x11-libs/libva-intel-driver-1.7.3::gentoo  USE="X drm wayland" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] media-libs/libsdl2-2.0.4::gentoo  USE="X alsa dbus gles joystick opengl pulseaudio sound threads udev video wayland xinerama (-altivec) (-custom-cflags) (-fusionsound) -haptic -nas -oss -static-libs -tslib -xscreensaver" ABI_X86="32 (64) (-x32)" CPU_FLAGS_X86="mmx sse sse2 -3dnow" 0 KiB
[ebuild   R    ] x11-libs/gtk+-3.22.19:3::gentoo  USE="X colord cups introspection wayland xinerama (-aqua) -broadway -cloudprint -examples {-test} -vim-syntax" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] media-libs/gst-plugins-bad-1.12.3:1.0::gentoo  USE="X bzip2 egl gtk introspection nls opengl orc wayland -gles2 {-test} -vcd -vnc" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] dev-cpp/gtkmm-3.22.2:3.0::gentoo  USE="X wayland (-aqua) -doc {-test}" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] media-libs/libcanberra-0.30-r5::gentoo  USE="alsa gnome gstreamer gtk gtk3 pulseaudio sound udev -oss -tdb" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] media-plugins/gst-plugins-vaapi-1.12.3:1.0::gentoo  USE="X drm egl opengl wayland" ABI_X86="32 (64) (-x32)" 0 KiB

No further attempt to re-emerge succeeded for neither of the packages.

Well, I have a rich set of USE flags as I need to test my projects in xfce, mate, gnome and kde. Surely I can live with symlinks but it would be nicer to improve the situation.

Mostly emerges die upon the configure phase with some dependencies missing. Many want to see pango (first in my list) but it does not emerge.
Particular question: why???
pango claims that
cairo.h
is missing but cairo itself is installed and can itself be rebuilt. I tried to go to an unstable cairo but no luck here again.
I do not understand in this particular case: if cairo.h exists in its usual place (not related to lib-s at all, of course) why some other build cannot locate it (especially given it was all fine before)? Now, what should happen? New version of cairo or a new version of pango? And what exactly to improve. How changing the location of libs could affect the detection of includes?

Confused
Comment 82 Alexander Tsoy 2018-02-01 11:54:52 UTC
(In reply to Alex from comment #81)
Please use search. This has already been reported here: bug 645540
Comment 83 Alex 2018-02-03 08:27:54 UTC
(In reply to Alexander Tsoy from comment #82)
> (In reply to Alex from comment #81)
> Please use search. This has already been reported here: bug 645540

Man, I will try better, but still ...

In comment #81 I stopped with a set of packages which declined to be rebuilt. I fixed this for all but one by first disabling some keywords and second rebuilding the world with
emerge -e world
The most notable change - I removed abi_x64_32 flag which I had globally on my make.conf This is different from Ivan in comment #73 who disabled somewhat other flag AND originally had no x64_32 enabled (as his emerge --info shows).

Still, the behaviour was strange. Upon running the world rebuild all packages were rebuilt smoothly. However, even after this the command
emerge -1vp /usr/lib32
was showing 12 packages. Checking those packages with
equery f PKGNAME
clearly showed files explicitly in /usr/lib32/...
But, what the magic, the subsequent rebuilding of those 12 with the command
emerge -1v /usr/lib32
eliminated almost all apart from mesa.
mesa still claims 
/usr/lib32
/usr/lib32/libGL.so.1 -> libGL.so.1.2.0
/usr/lib32/libGL.so.1.2.0
/usr/lib32/libglapi.so.0 -> libglapi.so.0.0.0
/usr/lib32/libglapi.so.0.0.0

What is it? BUG of mesa or of somewhat else? Logically I do not understand why rebuilding the world kept files into /usr/lib32 and an explicit emerge call with /usr/lib32 made files of most packages placed differently. I assume it is somehow weirdly encoded (and perhaps ambiguously) in emerge.
Second, after this emerge bugs me with preserved rebuilds. It wants me to rebuild android-ndk (and 2 others), which I did but on the next run it wants to do the same again.
I know what blocks these preserved rebuilds. on the way of rebuilding the world I did ^C on chromium (too long) and continued without it. I believe chromium claims those libs in preserved rebuild but emerge is silent about chromium, it only shows android-ndk (and 2 others).

Is it an artefact of migration or some bug in emerge/portage. My portage is stable. Should it be unstable for a safer migration?

Finally I did not try (yet) to restore the 32-ABI. It was demanded for some 16 packages which I did separately. Those packages are NOT those who blocked rebuilding in /usr/lib32 initially, they are not those who bug me with preserved rebuilds and they do not claim anything in /usr/lib32
Comment 84 Alex 2018-02-03 11:48:29 UTC
My puzzle of mesa was resolved. My
eselect opencl
was pointing to mesa and not to nvidia. This seems have blocked the files relocation. I changed
eselect opencl
to nvidia and re-merged mesa. Now portage has removed the link. Finally.
Interesting question here: what to do if there is opencl and there is only mesa as its choice ??? Like on a bit older laptop I have where only the intel gpu is present.
Comment 85 Alex 2018-02-05 00:51:43 UTC
As the final chord I have rebuilt everything wit ABI 32 enabled.

It seems that all around it is emerge/portage who messes things up. The migration is smooth by design as I now see but the emerge algorithms are doing wrong. For example:
# emerge -e world
did not claim for me some required packages.
After full migration and deletion of /usr/lib32 with ABI 32 turned off the nvidia driver claimed some 32bit libs in /usr/lib and this created some circular dependencies making portage thinking that there are dozens of preserved libs. Portage suggested to me to reinstall nvidia-drivers and after a re-install the story repeated in cycle.
Well, restoring the ABI 32 keyword and recompiling the stuff I removed the annoying preserved rebuilds notification but not all re-installs were computed.

SO, the lesson is: it is better to migrate without any ABI 32 package.

The question to devs: given have migrated but still have lib32 symlink and given two commands:

emerge -1v /usr/lib32 [will include some 12 or more packages like gtk+ ...]
emerge -1v gtk+ gtkmm pango pangomm ... (about 12 for me)

WHY the first command ends up with files installed using /usr/lib as the path and the second command ends up (being successful after many dances around) using /usr/lib32 in the path. WHY???
Comment 86 Matt Turner gentoo-dev 2018-02-05 04:10:26 UTC
Please stop commenting in this bug (as the title of the bug says IN ALL CAPS).
Comment 87 Matthew Thode ( prometheanfire ) archtester Gentoo Infrastructure gentoo-dev Security 2018-03-30 05:02:41 UTC
Does get_libdir not work at all?  If it does work https://bugs.gentoo.org/601746 may not need to be depended on.