This suggests that run_exitfuncs _exithandlers should prevent inheritance via fork: (In reply to Sam James from bug 937888 comment #5) > With that applied, on `emerge -v1 portage` (run twice to ensure using > patched code), I get this at the end when cleaning up: > ``` > > * GNU info directory index is up-to-date. > > * IMPORTANT: 9 news items need reading for repository 'gentoo'. > * Use eselect news read to view new items. > > > ==================================== > Error in portage.process.run_exitfuncs > > File "/usr/lib/python3.13/site-packages/portage/process.py", line 227, in > run_exitfuncs > func(*targs, **kargs) > ~~~~^^^^^^^^^^^^^^^^^ > File "/usr/lib/python3.13/shutil.py", line 763, in rmtree > _rmtree_safe_fd(stack, onexc) > ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^ > File "/usr/lib/python3.13/shutil.py", line 704, in _rmtree_safe_fd > onexc(func, path, err) > ~~~~~^^^^^^^^^^^^^^^^^ > File "/usr/lib/python3.13/shutil.py", line 665, in _rmtree_safe_fd > orig_st = os.lstat(name, dir_fd=dirfd) > [Errno 2] No such file or directory: > '/var/tmp/portage/._portage_reinstall_.nu7j61p8' > ==================================== > > Exception ignored in atexit callback <function run_exitfuncs at > 0x7f364a39f7e0>: > Traceback (most recent call last): > File "/usr/lib/python3.13/site-packages/portage/process.py", line 235, in > run_exitfuncs > raise exc_info[0](exc_info[1]).with_traceback(exc_info[2]) > File "/usr/lib/python3.13/site-packages/portage/process.py", line 227, in > run_exitfuncs > func(*targs, **kargs) > File "/usr/lib/python3.13/shutil.py", line 763, in rmtree > _rmtree_safe_fd(stack, onexc) > File "/usr/lib/python3.13/shutil.py", line 704, in _rmtree_safe_fd > onexc(func, path, err) > File "/usr/lib/python3.13/shutil.py", line 665, in _rmtree_safe_fd > orig_st = os.lstat(name, dir_fd=dirfd) > FileNotFoundError: [Errno 2] No such file or directory: > '/var/tmp/portage/._portage_reinstall_.nu7j61p8' > ```
Also, we should replay any relevant atexit_register calls for the new process after fork, to re-register hooks like _thread_weakrefs_atexit and close_portdbapi_caches.
We've only observed the FileNotFoundError from bug 937888 comment #5 since python3.13 because for https://github.com/python/cpython/issues/83856 they fixed atexit hooks in subprocesses, and chose not to backport this fix to older python. This is good news in the sense that we can stop honoring atexit handlers inherited via fork, and the result will only change the status quo for python3.13.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=cfd767cd35f5affd3b61b665b0f8814fe2de24c4 commit cfd767cd35f5affd3b61b665b0f8814fe2de24c4 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2024-08-14 05:30:42 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2024-08-14 15:22:05 +0000 run_exitfuncs: Drop hooks inherited via fork Drop hooks inherited via fork because they can trigger redundant actions as shown in bug 937891. Note that atexit hooks only work after fork since issue 83856 was fixed in Python 3.13. Bug: https://bugs.gentoo.org/937891 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/portage/process.py | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
(In reply to Sam James from bug 937888 comment #10) > /usr/bin/python3.13: can't open file > '/var/tmp/portage/._portage_reinstall_.oubn2hj3/bin/pid-ns-init': [Errno 2] > No such file or directory > /usr/bin/python3.13: can't open file > '/var/tmp/portage/._portage_reinstall_.oubn2hj3/bin/pid-ns-init': [Errno 2] > No such file or directory Errors like these can be triggered by an inherited atexit hook running much earlier than the original hook was intended to run, when a child process exits.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=02d0e00a1ba811b39140d10e17488f7fc3916534 commit 02d0e00a1ba811b39140d10e17488f7fc3916534 Author: Sam James <sam@gentoo.org> AuthorDate: 2024-09-11 01:30:10 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2024-09-11 01:30:30 +0000 sys-apps/portage: add 3.0.66 Closes: https://bugs.gentoo.org/435066 Closes: https://bugs.gentoo.org/907061 Closes: https://bugs.gentoo.org/910560 Closes: https://bugs.gentoo.org/933433 Closes: https://bugs.gentoo.org/934220 Closes: https://bugs.gentoo.org/934514 Closes: https://bugs.gentoo.org/934784 Closes: https://bugs.gentoo.org/935830 Closes: https://bugs.gentoo.org/936273 Closes: https://bugs.gentoo.org/937384 Closes: https://bugs.gentoo.org/937485 Closes: https://bugs.gentoo.org/937740 Closes: https://bugs.gentoo.org/937888 Closes: https://bugs.gentoo.org/937891 Closes: https://bugs.gentoo.org/938127 Closes: https://bugs.gentoo.org/933499 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.66.ebuild | 227 +++++++++++++++++++++++++++++++++ 2 files changed, 228 insertions(+)
