from lib/portage/util/movefile.py: def movefile(...): """... Move is atomic.""" but then it does: if destexists: if stat.S_ISLNK(dstat[stat.ST_MODE]): try: >> os.unlink(dest) destexists = 0 ... if stat.S_ISLNK(sstat[stat.ST_MODE]): try: target = os.readlink(src) if mysettings and "D" in mysettings and target.startswith(mysettings["D"]): target = target[len(mysettings["D"]) - 1 :] if destexists and not stat.S_ISDIR(dstat[stat.ST_MODE]): >> os.unlink(dest) try: if selinux_enabled: >> selinux.symlink(target, dest, src) else: >> os.symlink(target, dest) that is by definition not atomic. when merging a package, there's a small window where the symlink doesn't exist, so anyone using that symlink (e.g. a symlink to a program) can break. it's a small window, but on loaded systems running a lot of things in parallel, it's not that small. we're seeing this flake in CrOS builders on a semi-regular basis. bug 563614 might be an example of this that i misattributed to binutils-config. it's not clear why this code is deleting the symlink in the first place. POSIX says that rename supports moving symlinks over symlinks, symlinks over files, and files over symlinks: https://pubs.opengroup.org/onlinepubs/9699919799/functions/rename.html so this code should try os.rename first like it does with files, and if that fails (e.g. EXDEV), fallback to the same method as files where it creates a unique name in the same fs and then renames from there.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=fa82725ba03ffa5edf0b82db7307c87fc97e83f2 commit fa82725ba03ffa5edf0b82db7307c87fc97e83f2 Author: Mike Frysinger <vapier@chromium.org> AuthorDate: 2022-03-30 06:32:07 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-04-15 02:46:19 +0000 movefile: merge symlinks atomically Since POSIX allows renaming symlinks with symlinks, use that to get atomic updates. This is faster and less flaky to parallel processes: removing a live symlink and then recreating it leaves a small window where other things might try to use it but fail. [sam: cherry-picked from chromiumos' third_party/portage_tool repo] (cherry picked from commit cherry-pick 80ed2dac95db81acac8043e6685d0a853a08d268) Bug: https://bugs.gentoo.org/836400 Signed-off-by: Sam James <sam@gentoo.org> Closes: https://github.com/gentoo/portage/pull/816 Signed-off-by: Sam James <sam@gentoo.org> lib/portage/util/movefile.py | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-)
