Bug 868291 – sys-apps/merge-usr keywording

• Status: RESOLVED FIXED
• Product: Gentoo Linux

[Description Mike Gilbert 2022-09-03 16:00:28 UTC]

This package installs a script that converts /bin, /sbin, and /lib into symlinks pointing at /usr/bin and /usr/lib.

Required testing:

1. Set up a chroot environment, preferably with sys-apps/systemd installed.
2. Run merge-usr. This may emit some warnings.
3. Make sure basic programs still function.
4. Check for broken symlinks in /usr/bin, /usr/lib (adjust libdirs for your arch).
     eg. find /usr/bin /usr/lib /usr/lib64 -xtype l

Optional testing:

5. Switch to a "merged-usr" profile.
     eg. default/linux/amd64/17.1/systemd/merged-usr
6. Run emerge --newuse or emerge --changed-use to rebuild packages with USE="-split-usr".

[Comment 1 Arthur Zamarin 2022-09-03 16:49:56 UTC]

s390 & sparc done

all 6 stages were done on respective devboxes in systemd container

[Comment 2 Arthur Zamarin 2022-09-03 18:14:04 UTC]

arm & arm64 done

all 6 stages were done on kamaji.arm in systemd container used for all the arch testing

[addCollapseLink(3, 1, 'Toggle comment display'); Comment 3 Jakov Smolić 2022-09-03 21:29:42 UTC Comment hidden (obsolete)]

ppc64 done

[addCollapseLink(4, 1, 'Toggle comment display'); Comment 4 Jakov Smolić 2022-09-03 21:29:44 UTC Comment hidden (obsolete)]

ppc done

[Comment 5 Sam James 2022-09-04 00:31:20 UTC]

hppa done.

[Comment 6 Yixun Lan 2022-09-04 12:43:36 UTC]

got this on riscv machine, use: profiles/default/linux/riscv/20.0/rv64gc/lp64d/systemd
 # merge-usr 
WARNING:__main__:Skipping symlink '/bin/awk'
WARNING:__main__:Skipping symlink '/sbin/iptables-xml'
WARNING:__main__:Skipping symlink '/sbin/ss'
ERROR:__main__:Conflict for symlink '/lib64/lp64d': [Errno 17] File exists: '/usr/lib64'
ERROR:__main__:Leaving '/lib64' as a directory due to prior errors


# file /lib64/lp64d
/lib64/lp64d: symbolic link to .
# file /usr/lib64
/usr/lib64: directory

[Comment 7 Mike Gilbert 2022-09-04 14:09:21 UTC]

(In reply to Yixun Lan from comment #6)

The problem is that both /lib64/lp64d and /usr/lib64/lp64d exist as symlinks, and they technically point at different targets.

/lib64/lp64d -> . (/lib64)
/usr/lib64/lp64d -> . (/usr/lib64)

These will actually be the same location after /lib64 is converted to a symlink, but the script is not smart enough to figure that out.

As a workaround, you could remove /lib64/lp64d an rerun the script.

I'll need to put some thought into how to handle it automatically.

[Comment 8 Arthur Zamarin 2022-09-04 18:18:56 UTC]

ia64 done

[Comment 9 Arthur Zamarin 2022-09-04 18:18:57 UTC]

ppc done

[Comment 10 Arthur Zamarin 2022-09-04 18:18:58 UTC]

ppc64 done

[Comment 11 James Le Cuirot 2022-09-10 15:30:14 UTC]

m68k done (after migrating to systemd)

I encountered a couple of non-platform-specific issues though.

For some reason, /usr/lib/libtinfo.so.6 was not a symlink, so this was raised as a conflict. I don't know why this was. It hadn't been modified since 2018, so I removed it and all was well.

The script does not cope well with NFS on root. It could not remove the /tmpXXXX directories due to lingering .nfsXXXX files that reported "Device or resource busy". These files were copies of running processes like bash, udev, and agetty. This explains why it happens: http://nfs.sourceforge.net/#faq_d2. I was able to complete the process under qemu-user (as opposed to qemu-system), but it would be good to address this, if possible.

[Comment 12 James Le Cuirot 2022-09-10 15:40:57 UTC]

Obviously I meant root on NFS. :)

[Comment 13 Mike Gilbert 2022-09-10 16:11:00 UTC]

(In reply to James Le Cuirot from comment #11)

Could you provide the backtrace, or at least the exception type? I could catch it and warn instead of erroring.

[Comment 14 James Le Cuirot 2022-09-10 20:57:47 UTC]

(In reply to Mike Gilbert from comment #13)
> 
> Could you provide the backtrace, or at least the exception type? I could
> catch it and warn instead of erroring.

I was able to trigger it again by recreating /bin with symlinks, except for bash, which I moved from /usr/bin, and then rebooting.

Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.9/merge-usr", line 247, in <module>
    sys.exit(main())
  File "/usr/lib/python-exec/python3.9/merge-usr", line 242, in main
    if not mu.run():
  File "/usr/lib/python-exec/python3.9/merge-usr", line 208, in run
    shutil.rmtree(tmp)
  File "/usr/lib/python3.9/shutil.py", line 734, in rmtree
    _rmtree_safe_fd(fd, path, onerror)
  File "/usr/lib/python3.9/shutil.py", line 690, in _rmtree_safe_fd
    onerror(os.unlink, fullname, sys.exc_info())
  File "/usr/lib/python3.9/shutil.py", line 688, in _rmtree_safe_fd
    os.unlink(entry.name, dir_fd=topfd)
OSError: [Errno 16] Device or resource busy: '.nfs000000000000a82100000001'

If you simply move bash, it's fine. It's deleting it that's the problem. You could ignore the error. The .nfsXXXX files will disappear after rebooting, but the tmpXXXX directories won't. It's not the end of the world.

If you want to avoid that, you'd either need to detect that each file is in use before you delete it, or just move it to /usr instead of deleting it. You'd be overwriting the file you just copied/hardlinked to /usr, but I don't see the harm in that. There's a tiny chance the /usr version will also be in use by something else by that point, but that will clean itself up.

Unfortunately, NFS isn't smart enough to figure out that you're not really deleting the last instance of that inode, as you hardlinked it earlier, and it could therefore use that hardlink instead of creating an .nfsXXXX file.

[Comment 15 Mike Gilbert 2022-09-10 23:09:55 UTC]

(In reply to Yixun Lan from comment #6)

I just pushed a new version (1) which should resolve the problem on riscv. Please test again.

[Comment 16 Mike Gilbert 2022-09-10 23:19:11 UTC]

(In reply to James Le Cuirot from comment #14)

I added a try/catch so that the shutil.rmtree() call is nonfatal.

Regarding your suggested change: Moving/renaming the files from /bin to /usr/bin directly does not work if /bin and /usr/bin live on separate filesystems. It also makes the migration less "atomic" in that /bin/foo goes missing for a short period of time.

[Comment 17 James Le Cuirot 2022-09-11 09:18:20 UTC]

(In reply to Mike Gilbert from comment #16)
> 
> I added a try/catch so that the shutil.rmtree() call is nonfatal.

I can confirm that works better now.

> It also makes the migration less "atomic" in that /bin/foo goes
> missing for a short period of time.

Looking at the code again, I see what you mean now. I forgot you were renaming the symlink over the original, instead of just deleting the original.

[Comment 18 Yixun Lan 2022-09-14 02:00:10 UTC]

(In reply to Mike Gilbert from comment #15)
> (In reply to Yixun Lan from comment #6)
> 
> I just pushed a new version (1) which should resolve the problem on riscv.
> Please test again.

tested on riscv machine, and it works fine, thanks!

[Comment 19 Yixun Lan 2022-09-14 02:35:59 UTC]

riscv done

[Comment 20 matoro 2022-09-21 13:23:36 UTC]

On default/linux/alpha/17.0/systemd after merge-usr:

# find /usr/bin /usr/lib /usr/lib64 -xtype l | xargs file
/usr/lib/sysctl.d/99-sysctl.conf: broken symbolic link to ../../../etc/sysctl.conf

Everything else looks good including switching to default/linux/alpha/17.0/systemd/merged-usr

[Comment 21 Mike Gilbert 2022-09-25 23:11:19 UTC]

(In reply to matoro from comment #20)
> On default/linux/alpha/17.0/systemd after merge-usr:
> 
> # find /usr/bin /usr/lib /usr/lib64 -xtype l | xargs file
> /usr/lib/sysctl.d/99-sysctl.conf: broken symbolic link to
> ../../../etc/sysctl.conf

This is expected/ok.

[Comment 22 matoro 2022-09-26 13:59:43 UTC]

alpha done

[Comment 23 Larry the Git Cow 2022-10-07 08:47:46 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=f1352f205c2259f5ab376cf974cac52f761788f0

commit f1352f205c2259f5ab376cf974cac52f761788f0
Author:     WANG Xuerui <[e-mail]>
AuthorDate: 2022-10-07 08:28:47 +0000
Commit:     WANG Xuerui <[e-mail]>
CommitDate: 2022-10-07 08:29:22 +0000

    sys-apps/merge-usr: keyword 1 for ~loong
    
    Tested on bare-matal installation, works okay.
    
    Bug: https://bugs.gentoo.org/868291
    Signed-off-by: WANG Xuerui <[e-mail]>

 sys-apps/merge-usr/merge-usr-1.ebuild | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[Comment 24 WANG Xuerui 2022-10-07 08:49:28 UTC]

loong done

[Comment 25 jeremy mills 2022-10-24 02:22:40 UTC]

worked well here on a live system fully ~amd64 keyworded. i even started it backwards. i didnt know about the script so i did the profile change first. emerge bailed on rebuilding systemd. (because i hadnt ran this script yet) the error message eventually brought me here. ran the script with no issues really. ran --dry run first. it was complaining about some iptables symlinks. ran emerge -C iptables (probally not thge proper choice but i was already gonna do a reinstall for other reasons so it didnt matter if it broke the system...it didnt though i just ran emerge iptables after the script ran successfully). i let emerge finish updating for the new profile. checked for broken symlinks. all good. checked for basic functionality...all good. rebuilt the nvidia kernel module (i read awhile back there was some issue with the symlinks being wrong) all good there. the fact that i didnt even do things in the proper order and it was still successful should be a testament to the safeguards that are in place to prevent a broken system as well as the quailty of the script. very nicely done. thats about all i have to add. im off to run a new stage 3 install and test that route but i doubt any problems will crop up. happy compiling ladies (if any are here) and GENTOOmen :)

[Comment 26 immolo 2023-11-20 20:17:04 UTC]

I've been testing on mips and m68k and found no issues (as expected).

[Comment 27 matoro 2023-11-29 04:27:11 UTC]

mips done

all arches done