Bug 816141 – app-portage/portage-utils-0.92: can not rename vdb cross-device after otherwise successful merge

• Product: Gentoo Linux
• Status: RESOLVED FIXED

[Description Barnabás Virágh 2021-10-03 20:42:40 UTC]

POSIX rename() doesn't work cross-device, when after a successful merge wants to move vdb folder to .../var/db/pkg.

It is problematic two fold:
- if --root is specified, it has a good chance it's on a different device mounted
- if /var/tmp is on tmpfs to speed up portage

The proposed solution (in patch) in case of errno==EXDEV, it falls back invoking mv command (can be provided by busybox, which availability was checked at init time), which implements copy-delete for this scenario.

Reproducible: Always

Steps to Reproduce:
1. mount /dev/sdb2 /mnt
2. qmerge -Kv --root /mnt sys-block/parted
Actual Results:  
qmerge: failed to move 'vdb' to '/mnt/var/db/pkg/sys-block/parted-3.4': Cross-device link

Expected Results:  
/mnt/var/db/pkg/sys-block/parted-3.4 is created and populated, no warning displayed in case of the destination, or PORTAGE_TMPDIR is on a different device.

A possible workaround would be to set PORTAGE_TMPDIR to point into the --root device, which only works, if there is enough free space available on that disk.

[Comment 1 Barnabás Virágh 2021-10-03 20:43:26 UTC]

Created attachment 742848 [details, diff]
vdb dir move cross-device patch

[Comment 2 Fabian Groffen 2021-10-04 06:40:18 UTC]

We have libq/copy_file, so I'd rather jam that up with scandir or something to do the move without having to spawn an external process.

Perhaps overkill, but I'm trying to get rid of most of those system spawns.

[Comment 3 Barnabás Virágh 2021-10-04 07:46:30 UTC]

I like the idea, even better than my easy quick fix solution.

[Comment 4 Fabian Groffen 2021-10-04 07:54:22 UTC]

I think qmerge at some point does a recursive move already to merge image to live, so it probably is a single file at a time at that point.

[Comment 5 Sam James 2021-11-10 05:33:11 UTC]

Ah, I hit this recently when helping a user in #gentoo with --root, but somehow didn't find this bug.

Their system had become totally hosed and we needed to install a series of rescue packages but we hit a snag (this), I think.

[Comment 6 Larry the Git Cow 2021-12-26 13:59:53 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=ed8a0a8694de1a83a29041104de162ff63e5f8c6

commit ed8a0a8694de1a83a29041104de162ff63e5f8c6
Author:     Fabian Groffen <[e-mail]>
AuthorDate: 2021-12-26 13:55:24 +0000
Commit:     Fabian Groffen <[e-mail]>
CommitDate: 2021-12-26 13:55:24 +0000

    qmerge: support cross-device merges also for VDB #816141
    
    As originally suggested by Barnabás Virágh, but in this implementation
    using move_file, which is now shared code between pkg merge and the
    putting the vdb in place.
    
    Bug: https://bugs.gentoo.org/816141
    Signed-off-by: Fabian Groffen <[e-mail]>

 qmerge.c | 115 ++++++++++++++++++++-------------------------------------------
 1 file changed, 37 insertions(+), 78 deletions(-)

[Comment 7 Larry the Git Cow 2021-12-26 14:30:39 UTC]

The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=aec1262e6d8a12033789f6d80b431e9838293496

commit aec1262e6d8a12033789f6d80b431e9838293496
Author:     Fabian Groffen <[e-mail]>
AuthorDate: 2021-12-26 14:30:05 +0000
Commit:     Fabian Groffen <[e-mail]>
CommitDate: 2021-12-26 14:30:37 +0000

    app-portage/portage-utils-0.93.2: bump for qlop -Ev regr, fix #816141
    
    Closes: https://bugs.gentoo.org/816141
    Package-Manager: Portage-3.0.28, Repoman-3.0.3
    Signed-off-by: Fabian Groffen <[e-mail]>

 app-portage/portage-utils/Manifest                                      | 2 +-
 .../{portage-utils-0.93.1.ebuild => portage-utils-0.93.2.ebuild}        | 0
 2 files changed, 1 insertion(+), 1 deletion(-)

[Comment 8 Sam James 2021-12-27 06:45:58 UTC]

Thank you both. Huge help for recovering broken boxes!

[Comment 9 Fabian Groffen 2021-12-27 08:26:54 UTC]

sorry for being slow to deliver on this one ...

I tested this by forcing tmp to be on another FS here, and confirmed the copy+unlink path kicked in.  Any confirmation on setups where this was/is an issue would be welcome though.