Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 753743 - app-portage/unsymlink-lib fails in Prefix
Summary: app-portage/unsymlink-lib fails in Prefix
Status: CONFIRMED
Alias: None
Product: Gentoo/Alt
Classification: Unclassified
Component: Prefix Support (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Michał Górny
URL:
Whiteboard:
Keywords: NeedPatch
Depends on:
Blocks:
 
Reported: 2020-11-09 17:12 UTC by Steven Trogdon
Modified: 2020-12-22 12:39 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Steven Trogdon 2020-11-09 17:12:42 UTC
I'm attempting to upgrade my profile

$ eselect profile show
Current /storage/strogdon/gentoo-rap/etc/portage/make.profile symlink:
  default/linux/amd64/17.0/no-multilib/prefix/kernel-3.2+

to 

  default/linux/amd64/17.1/no-multilib/prefix/kernel-3.2+

$ eselect profile list
Available profile symlink targets:
  [1]   prefix/linux/amd64 (exp)
  [2]   default/linux/amd64/17.0/no-multilib/prefix/kernel-3.2+ (dev) *
  [3]   default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.32+ (exp)
  [4]   default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.16+ (exp)
  [5]   default/linux/amd64/17.1/no-multilib/prefix/kernel-3.2+ (dev)
  [6]   default/linux/amd64/17.1/no-multilib/prefix/kernel-2.6.32+ (exp)
  [7]   default/linux/amd64/17.1/no-multilib/prefix/kernel-2.6.16+ (exp)

The required upgrade instructions involve using unsymlink-lib and the first step fails with:

$ unsymlink-lib --analyze
[Running as unprivileged user, results will not be saved]
Traceback (most recent call last):
  File "/storage/strogdon/gentoo-rap/usr/lib/python-exec/python3.7/unsymlink-lib", line 830, in <module>
    main()
  File "/storage/strogdon/gentoo-rap/usr/lib/python-exec/python3.7/unsymlink-lib", line 701, in main
    if os.path.samefile(lib64_dir, usr_lib64_dir):
  File "/storage/strogdon/gentoo-rap/usr/lib/python3.7/genericpath.py", line 101, in samefile
    s2 = os.stat(f2)
FileNotFoundError: [Errno 2] No such file or directory: b'/usr/lib64'

It appears unsymlink-lib looks for root privileges

$ unsymlink-lib --migrate --pretend
usage: unsymlink-lib [-h] [-p] [--root ROOT] [--analyze] [--migrate]
                     [--rollback] [--finish] [--force-rollback]
                     [--resume-finish] [-P PREFIX] [--hardlink]
unsymlink-lib: error: Requested action requires root privileges

It doesn't appear that unsymlink-lib can be used to make the profile upgrade.
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-11-09 17:33:24 UTC
Indeed, I didn't account for unprivileged installs.  Could you try removing this condition and seeing if --prefix option works at all?
Comment 2 Fabian Groffen gentoo-dev 2020-11-09 17:44:13 UTC
Is this necessary at all on Prefix?
Comment 3 Mike Gilbert gentoo-dev 2020-11-09 18:43:31 UTC
After skimming through the unsymlink-lib code, I think the necessary calling convention would be "unsymlink-lib --root EROOT", replacing EROOT with the prefix path.

That should cause the script to look for ${EROOT}/lib, ${EROOT}/usr/lib, etc.
Comment 4 Mike Gilbert gentoo-dev 2020-11-09 19:05:46 UTC
(In reply to Fabian Groffen from comment #2)
> Is this necessary at all on Prefix?

It would allow people to migrate from SYMLINK_LIB=yes (17.0) to SYMLINK_LIB=no (17.1) without doing a fresh prefix install.
Comment 5 Steven Trogdon 2020-11-09 19:06:32 UTC
(In reply to Mike Gilbert from comment #3)
> After skimming through the unsymlink-lib code, I think the necessary calling
> convention would be "unsymlink-lib --root EROOT", replacing EROOT with the
> prefix path.
> 
> That should cause the script to look for ${EROOT}/lib, ${EROOT}/usr/lib, etc.

unsymlink-lib --root /storage/strogdon/gentoo-rap --analyze

does give meaningful output. My prefix is /storage/strogdon/gentoo-rap. This terminates with

Everything looks good from here. However, you need to rerun
the process as root to confirm.

But root privilege is still needed:

unsymlink-lib --root /storage/strogdon/gentoo-rap --migrate --pretend
usage: unsymlink-lib [-h] [-p] [--root ROOT] [--analyze] [--migrate]
                     [--rollback] [--finish] [--force-rollback]
                     [--resume-finish] [-P PREFIX] [--hardlink]
unsymlink-lib: error: Requested action requires root privileges
Comment 6 Mike Gilbert gentoo-dev 2020-11-09 19:24:43 UTC
If you feel like being a guinea pig, remove this code from the script and give it a try.

https://github.com/mgorny/unsymlink-lib/blob/v18/unsymlink-lib#L682-L686
Comment 7 Mike Gilbert gentoo-dev 2020-11-09 19:26:11 UTC
Looks like you would also need to adjust this section:

https://github.com/mgorny/unsymlink-lib/blob/v18/unsymlink-lib#L725-L740
Comment 8 Steven Trogdon 2020-11-09 20:28:36 UTC
Removing those lines I have

$ unsymlink-lib --root /storage/strogdon/gentoo-rap --analyze

terminates with

directories that will be moved to /storage/strogdon/gentoo-rap/usr/local/lib/:
        (+ 0 files)

directories whose contents will be split between /storage/strogdon/gentoo-rap/usr/local/lib/ and /storage/strogdon/gentoo-rap/usr/local/lib64/:


Warning: no lib32 paths found. This is fine if you are running no-multilib,
otherwise this is suspicious.

This seems OK. And then

$ unsymlink-lib --root /storage/strogdon/gentoo-rap --migrate --pretend
State file could not be loaded. Did you run --analyze?

So something isn't saved from the --analyze step.
Comment 9 Steven Trogdon 2020-11-09 20:30:47 UTC
Perhaps the m.save_state() is needed.
Comment 10 Mike Gilbert gentoo-dev 2020-11-09 20:36:19 UTC
(In reply to Steven Trogdon from comment #9)
> Perhaps the m.save_state() is needed.

Right.
Comment 11 Steven Trogdon 2020-11-09 20:48:43 UTC
Retaining just

        m.save_state()
        log('')
        log('')
        log('The state has been saved and the migration is ready to proceed.')
        log('To initiate it, please run:')
        log('')
        log('\t{} --migrate', sys.argv[0])
        log('')
        log('Please do not perform any changes to the system at this point.')
        log('If you performed any changes, please rerun the analysis.')

in 

https://github.com/mgorny/unsymlink-lib/blob/v18/unsymlink-lib#L725-L740

seems to work.
Comment 12 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-11-09 21:40:16 UTC
Thank you.  I'm adding tests for prefix scenarios as we speak, and should be able to make a release in the next hour or so.
Comment 13 Larry the Git Cow gentoo-dev 2020-11-09 22:08:12 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=10f7e8061b7f780e7ba008281e1bd920b5ab5c8a

commit 10f7e8061b7f780e7ba008281e1bd920b5ab5c8a
Author:     Michał Górny <mgorny@gentoo.org>
AuthorDate: 2020-11-09 22:06:52 +0000
Commit:     Michał Górny <mgorny@gentoo.org>
CommitDate: 2020-11-09 22:08:07 +0000

    app-portage/unsymlink-lib: Bump to v19
    
    Closes: https://bugs.gentoo.org/753743
    Signed-off-by: Michał Górny <mgorny@gentoo.org>

 app-portage/unsymlink-lib/Manifest                |  1 +
 app-portage/unsymlink-lib/unsymlink-lib-19.ebuild | 29 +++++++++++++++++++++++
 2 files changed, 30 insertions(+)
Comment 14 Benda Xu gentoo-dev 2020-11-09 23:36:26 UTC
Thanks guys.

@Fabian, I regard keeping Prefix directory structures in sync with Gentoo vanilla gives least surprise to our users.
Comment 15 Steven Trogdon 2020-11-10 01:46:55 UTC
I thought I would comment here before opening another bug. I followed the procedure to upgrade to the 17.1 profile using --root EPREFIX and --unprivileged where needed. After restarting prefix, executing the --finish step and eselecting the 17.1 profile my gcc is apparently broken. Wnen rebuilding gcc-10.2.0 I have

/storage/strogdon/gentoo-rap/var/tmp/portage/sys-devel/gcc-10.2.0-r3/work/gcc-10.2.0/configure --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --prefix=/storage/strogdon/gentoo-rap/usr --bindir=/storage/strogdon/gentoo-rap/usr/x86_64-pc-linux-gnu/gcc-bin/10.2.0 --includedir=/storage/strogdon/gentoo-rap/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/include --datadir=/storage/strogdon/gentoo-rap/usr/share/gcc-data/x86_64-pc-linux-gnu/10.2.0 --mandir=/storage/strogdon/gentoo-rap/usr/share/gcc-data/x86_64-pc-linux-gnu/10.2.0/man --infodir=/storage/strogdon/gentoo-rap/usr/share/gcc-data/x86_64-pc-linux-gnu/10.2.0/info --with-gxx-include-dir=/storage/strogdon/gentoo-rap/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/include/g++-v10 --with-python-dir=/share/gcc-data/x86_64-pc-linux-gnu/10.2.0/python --enable-languages=c,c++,fortran --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion=Gentoo 10.2.0-r3 p4 --disable-esp --enable-libstdcxx-time --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --disable-multilib --with-multilib-list=m64 --disable-fixed-point --enable-targets=all --enable-libgomp --disable-libssp --disable-libada --disable-systemtap --enable-vtable-verify --without-zstd --enable-lto --without-isl --enable-default-pie --enable-default-ssp --with-sysroot=/storage/strogdon/gentoo-rap
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking target system type... x86_64-pc-linux-gnu
checking for a BSD-compatible install... /storage/strogdon/gentoo-rap/usr/bin/install -c
checking whether ln works... yes
checking whether ln -s works... yes
checking for a sed that does not truncate output... /storage/strogdon/gentoo-rap/bin/sed
checking for gawk... gawk
checking for libatomic support... yes
checking for libitm support... yes
checking for libsanitizer support... yes
checking for libvtv support... yes
checking for libhsail-rt support... yes
checking for libphobos support... yes
checking for x86_64-pc-linux-gnu-gcc... x86_64-pc-linux-gnu-gcc
checking whether the C compiler works... no
configure: error: in `/storage/strogdon/gentoo-rap/var/tmp/portage/sys-devel/gcc-10.2.0-r3/work/build':
configure: error: C compiler cannot create executables
See `config.log' for more details
Comment 16 Steven Trogdon 2020-11-10 01:49:11 UTC
This is apparently the problem. From config.log

configure:4483: x86_64-pc-linux-gnu-gcc    conftest.c  >&5
/storage/strogdon/gentoo-rap/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/as: error while loading shared libraries: libopcodes-2.30.0.so: cannot open shared object file: No such file or directory
Comment 17 Steven Trogdon 2020-11-10 02:04:40 UTC
It appears that binutils wasn't moved from EPREFIX/usr/lib to EPREFIX/usr/lib64

ls /storage/strogdon/gentoo-rap/usr/lib/binutils
x86_64-pc-linux-gnu
Comment 18 Fabian Groffen gentoo-dev 2020-11-10 07:58:55 UTC
(In reply to Benda Xu from comment #14)
> Thanks guys.
> 
> @Fabian, I regard keeping Prefix directory structures in sync with Gentoo
> vanilla gives least surprise to our users.

that's fine, but the problem is that people are breaking their systems at this point :(
Comment 19 Benda Xu gentoo-dev 2020-12-22 03:09:24 UTC
I have met problems on prefix installations

unsymlink-lib --root /opt/gentoo --analyze  gives

orphan dirs/files (not owned by any package) that will be moved to /opt/gentoo/usr/lib/:
        CLHEP-2.4.1.3
        .....
        .....

But those directories are owned by packages.

# equery b CLHEP-2.4.1.3
 * Searching for CLHEP-2.4.1.3 ... 
sci-physics/clhep-2.4.1.3-r2 (/opt/gentoo/usr/lib64/CLHEP-2.4.1.3)


The analyze result is not correct.
Comment 20 Benda Xu gentoo-dev 2020-12-22 12:39:12 UTC
Patch sent.  I made a successful migration to 17.1 prefix profile.