Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 937888 - asyncio.exceptions.CancelledError crash
Summary: asyncio.exceptions.CancelledError crash
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on: 933499
Blocks:
  Show dependency tree
 
Reported: 2024-08-13 22:35 UTC by Sam James
Modified: 2024-09-11 01:30 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
/var/log/portage/build/sys-libs/libutempter-1.2.1:20240813-225442.log (file_937888.txt,206.88 KB, text/plain)
2024-08-13 22:58 UTC, Sam James
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-08-13 22:35:00 UTC
```
[...]
>>> Running pre-merge checks for sys-fs/bees-0.10-r1                                                                                                                  22:49:10 [1056/9983]
/usr/bin/python3.13: can't open file '/var/tmp/portage/._portage_reinstall_.a4tdj7bx/bin/pid-ns-init': [Errno 2] No such file or directory
/usr/bin/python3.13: can't open file '/var/tmp/portage/._portage_reinstall_.a4tdj7bx/bin/pid-ns-init': [Errno 2] No such file or directory
/bin/bash: line 1: /var/tmp/portage/._portage_reinstall_.a4tdj7bx/bin/misc-functions.sh: No such file or directory
>>> Failed to emerge sys-fs/bees-0.10-r1, Log file:
>>>  '/var/log/portage/build/sys-fs/bees-0.10-r1:20240813-214858.log'
*** Resuming merge...

These are the packages that would be merged, in order:

Calculating dependencies... done!
Dependency resolution took 10.23 s.
[...]
Total: 977 packages (1 upgrade, 976 reinstalls), Size of downloads: 0 KiB
>>> Emerging (1 of 977) sys-apps/keyutils-1.6.3-r1::gentoo
>>> Failed to emerge sys-apps/keyutils-1.6.3-r1
>>> Jobs: 0 of 977 complete, 1 failed               Load avg: 2.8, 13.0, 9.4[ERROR] Exception in callback _EbuildFetcherProcess._async_uri_map.<locals>.cache_result() at /usr/lib/python3.13/site-packages/_emerge/EbuildFetcher.py:373
handle: <Handle _EbuildFetcherProcess._async_uri_map.<locals>.cache_result() at /usr/lib/python3.13/site-packages/_emerge/EbuildFetcher.py:373>
Traceback (most recent call last):
  File "/usr/lib/python3.13/asyncio/events.py", line 89, in _run
    self._context.run(self._callback, *self._args)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/_emerge/EbuildFetcher.py", line 375, in cache_result
    self._uri_map = result.result()
                    ~~~~~~~~~~~~~^^
asyncio.exceptions.CancelledError
xargs: emerge: terminated by signal 15

```
Comment 1 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-08-13 22:36:52 UTC
$ emerge --info
Portage 3.0.65-23+gcb0c09d8c-dirty (python 3.13.0-candidate-1, default/linux/amd64/23.0/hardened, gcc-15, glibc-2.40, 6.6.45 x86_64)
=================================================================
System uname: Linux-6.6.45-x86_64-AMD_Ryzen_9_3950X_16-Core_Processor-with-glibc2.40
KiB Mem:    65742504 total,  10943476 free
KiB Swap:   16662520 total,  13408128 free
Timestamp of repository gentoo: Tue, 13 Aug 2024 19:33:59 +0000
Head commit of repository gentoo: 3b5832d688dd41864480043836025186187af3f7

Timestamp of repository kde: Mon, 12 Aug 2024 14:33:56 +0000
Head commit of repository kde: 12bfe624fbf922717c00ac3239386709a23284c8

Timestamp of repository qt: Sat, 10 Aug 2024 14:21:49 +0000
Head commit of repository qt: 8d355f09248f70e10df8d6f1fbfa29eeba6751cc

Timestamp of repository sam_c: Sat, 10 Aug 2024 14:22:07 +0000
Head commit of repository sam_c: bd23e12121cca7bce2c3821592b97e057d0184ba

sh dash 0.5.12
ld GNU ld (Gentoo 9999 p1) 2.43.50.20240812
ccache version 4.10.2 [disabled]
app-misc/pax-utils:        1.3.7::gentoo
app-shells/bash:           5.3_alpha_p20240730::gentoo
dev-build/autoconf:        2.13-r8::gentoo, 2.69-r9::gentoo, 2.72-r1::gentoo, 9999::gentoo
dev-build/automake:        1.16.5-r2::gentoo, 1.17-r1::gentoo
dev-build/cmake:           3.30.2::gentoo
dev-build/libtool:         2.5.0::gentoo
dev-build/make:            4.4.1-r1::gentoo
dev-build/meson:           1.5.1::gentoo
dev-java/java-config:      2.3.4::gentoo
dev-lang/perl:             5.40.0::gentoo
dev-lang/python:           3.10.14_p2::gentoo, 3.11.9_p1::gentoo, 3.12.5::gentoo, 3.13.0_rc1_p1::gentoo
dev-lang/rust-bin:         1.79.0::gentoo
dev-util/ccache:           4.10.2::gentoo
sys-apps/baselayout:       2.15::gentoo
sys-apps/sandbox:          2.39::gentoo
sys-apps/systemd:          256.4::gentoo
sys-devel/binutils:        2.38-r2::gentoo, 2.39-r5::gentoo, 2.40-r9::gentoo, 2.41-r5::gentoo, 2.42-r2::gentoo, 2.43.9999::gentoo, 9999::gentoo
sys-devel/binutils-config: 5.5.2::gentoo
sys-devel/clang:           17.0.6::gentoo, 18.1.8::gentoo
sys-devel/gcc:             10.5.0::gentoo, 11.5.0::gentoo, 12.4.1_p20240808::gentoo, 13.3.1_p20240809::gentoo, 14.2.1_p20240810::gentoo, 15.0.9999::gentoo
sys-devel/gcc-config:      2.11::gentoo
sys-devel/lld:             18.1.8::gentoo
sys-devel/llvm:            17.0.6-r2::gentoo, 18.1.8-r4::gentoo
sys-kernel/linux-headers:  6.10::gentoo (virtual/os-headers)
sys-libs/glibc:            2.40::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/gentoo.git
    priority: -1000
    eclass-overrides: sam_c
    volatile: False
    sync-git-clone-extra-opts: -b stable -c gc.reflogExpire=0 -c gc.reflogExpireUnreachable=0 -c gc.rerereresolved=0 -c gc.rerereunresolved=0 -c gc.pruneExpire=now
    sync-git-verify-commit-signature: yes

crossdev
    location: /var/db/repos/crossdev
    masters: gentoo
    eclass-overrides: sam_c
    volatile: False

kde
    location: /var/db/repos/kde
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/kde.git
    masters: gentoo
    eclass-overrides: sam_c
    volatile: False

local
    location: /var/db/repos/local
    masters: gentoo
    eclass-overrides: sam_c
    volatile: False

qt
    location: /var/db/repos/qt
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/qt.git
    masters: gentoo
    eclass-overrides: sam_c
    volatile: False

sam_c
    location: /var/db/repos/sam_c
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/sam_c.git
    masters: gentoo
    eclass-overrides: sam_c
    volatile: False

Installed sets: @sam-emacs, @sam-fonts, @sam-fortune
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="@FREE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O3 -march=native -mtls-dialect=gnu2 -flto=jobserver -fno-semantic-interposition -pipe -fcf-protection=none -fdiagnostics-color=always -fdiagnostics-urls=never -frecord-gcc-switches -Wa,-O2 -Wa,-mtune=znver2  -Wstrict-aliasing -Wfree-nonheap-object -Werror=lto-type-mismatch -Werror=strict-aliasing -Werror=odr -Wstrict-aliasing -Wfree-nonheap-object -Werror=lto-type-mismatch -Werror=strict-aliasing -Werror=odr -Wbuiltin-declaration-mismatch"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O3 -march=native -mtls-dialect=gnu2 -flto=jobserver -fno-semantic-interposition -pipe -fcf-protection=none -fdiagnostics-color=always -fdiagnostics-urls=never -frecord-gcc-switches -Wa,-O2 -Wa,-mtune=znver2  -Wstrict-aliasing -Wfree-nonheap-object -Werror=lto-type-mismatch -Werror=strict-aliasing -Werror=odr"
DISTDIR="/var/cache/distfiles"
EMERGE_DEFAULT_OPTS="--keep-going --with-bdeps=y --complete-graph --deep --dynamic-deps=n --changed-deps=n --usepkg=y --implicit-system-deps=n --usepkg-exclude sys-fs/zfs --usepkg-exclude sys-fs/zfs-kmod --jobs=4 --load-average 32"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-O3 -march=native -mtls-dialect=gnu2 -flto=jobserver -fno-semantic-interposition -pipe -fcf-protection=none -fdiagnostics-color=always -fdiagnostics-urls=never -frecord-gcc-switches -Wa,-O2 -Wa,-mtune=znver2  -Wstrict-aliasing -Wfree-nonheap-object -Werror=lto-type-mismatch -Werror=strict-aliasing -Werror=odr"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg clean-logs config-protect-if-modified distlocks fixlafiles ipc-sandbox merge-wait mount-sandbox multilib-strict network-sandbox news parallel-fetch parallel-install pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms split-log strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O3 -march=native -mtls-dialect=gnu2 -flto=jobserver -fno-semantic-interposition -pipe -fcf-protection=none -fdiagnostics-color=always -fdiagnostics-urls=never -frecord-gcc-switches -Wa,-O2 -Wa,-mtune=znver2  -Wstrict-aliasing -Wfree-nonheap-object -Werror=lto-type-mismatch -Werror=strict-aliasing -Werror=odr"
GENTOO_MIRRORS="https://www.mirrorservice.org/sites/distfiles.gentoo.org/ https://mirror.bytemark.co.uk/gentoo/ https://mirrors.soeasyto.com/distfiles.gentoo.org/"
LANG="en_GB.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs -flto=jobserver -fuse-ld=mold -Wl,--defsym=__gentoo_check_ldflags__=0"
LEX="reflex"
LINGUAS="en en_GB"
MAKEOPTS="-j32 -l32"
PKGDIR="/var/cache/binpkgs"
PORTAGE_BUNZIP2_COMMAND="lbunzip2"
PORTAGE_COMPRESS="xz"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
RUSTFLAGS="-C target-cpu=native -C link-arg=-Wl,-z,pack-relative-relocs"
SHELL="/bin/bash"
USE="X a52 aac acl acpi activities aes alsa amd64 avx avx2 bash-completion bluetooth branding bzip2 cairo caps cdda cdr clang crypt dbus declarative dist-kernel dri dts dvd dvdr editorconfig emacs encode exif f16c filecaps firewalld flac fma3 freetype gdbm gif gmp gpm graphite gtk gui hardened harfbuzz hunspell iconv icu ipv6 jit jpeg kde kdesu kf6compat kwallet lcms libnotify libtirpc lto mad mmx mmxext mng modules-compress mp3 mp4 mpeg multilib ncurses nftables nls ogg opengl openmp pam pango pclmul pcre pdf pgo pic pie pipewire plasma png policykit popcnt ppds pulseaudio qml qt5 rdrand readline screencast sdl seccomp semantic-desktop sha sound spell sse sse2 sse3 sse4_1 sse4_2 sse4a ssl ssp ssse3 startup-notification svg system-av1 system-binutils system-boost system-bootstrap system-cairo system-clang system-digest system-ffmpeg system-harfbuzz system-heimdal system-icu system-jpeg system-leveldb system-libevent system-libs system-libvpx system-libyaml system-llvm system-lz4 system-mitkrb5 system-sqlite system-ssl system-tbb system-uulib system-webp system-zlib systemd test-rust tiff truetype udev udisks unicode upower usb valgrind verify-sig vorbis vulkan wayland widgets x264 xattr xcb xft xml xtpax xv xvid zlib zsh-completion" ABI_X86="32 64" ADA_TARGET="gcc_12" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en en-GB" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LLVM_TARGETS="AArch64 AMDGPU ARM AVR BPF Hexagon Lanai MSP430 Mips NVPTX PowerPC RISCV Sparc SystemZ WebAssembly X86 XCore" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-2" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_12" PYTHON_TARGETS="python3_12 pypy3 python3_10 python3_11 python3_13" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="aarch64 aarch64_be alpha armeb hppa i386 m68k ppc ppc64 ppc64le sparc sparc64 x86_64" RUBY_TARGETS="ruby31 ruby32" VIDEO_CARDS="amdgpu radeonsi radeon" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account"
YACC="byacc"
Unset:  ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, SIZE, STRINGS, STRIP, YFLAGS
Comment 2 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-08-13 22:44:42 UTC
I hit this when running:
```
for path in /var/db/pkg/*/[!-]*/NEEDED; do
        path=${path%/NEEDED}
        if read -r slot < "$path/SLOT"; then
                printf '%s:%s\n' "${path#/var/db/pkg/}" "$slot"
        else
                printf >&2 'Failed to read SLOT for %s\n' "$contents"
        fi
done \
| xargs -rd '\n' qatom -F '%{CATEGORY}/%{PN}:%[SLOT]' \
| awk -F'[/:]' '$2 !~ /-bin$/' \
| grep -v sys-devel/gcc \
| grep -v cross-*/ \
| grep -v sys-kernel/gentoo-kernel \
| xargs -ord '\n' emerge --usepkg=n -v1 "$@"
```

Trying to reproduce it now
Comment 3 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-08-13 22:58:43 UTC
Created attachment 900082 [details]
/var/log/portage/build/sys-libs/libutempter-1.2.1:20240813-225442.log

An example failure log (note that it contains failures for other packages too)
Comment 4 Zac Medico gentoo-dev 2024-08-13 23:09:39 UTC
(In reply to Sam James from comment #0)
> handle: <Handle _EbuildFetcherProcess._async_uri_map.<locals>.cache_result()
> at /usr/lib/python3.13/site-packages/_emerge/EbuildFetcher.py:373>
> Traceback (most recent call last):
>   File "/usr/lib/python3.13/asyncio/events.py", line 89, in _run
>     self._context.run(self._callback, *self._args)
>     ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "/usr/lib/python3.13/site-packages/_emerge/EbuildFetcher.py", line
> 375, in cache_result
>     self._uri_map = result.result()
>                     ~~~~~~~~~~~~~^^
> asyncio.exceptions.CancelledError
> xargs: emerge: terminated by signal 15

It's normal for CancelledError to propagate up the stack like this, but in this case we don't want that. It's not caught here because CancelledError derives from BaseException rather than Exception:

diff --git a/lib/_emerge/EbuildFetcher.py b/lib/_emerge/EbuildFetcher.py
index 994271236c..422d7c0a06 100644
--- a/lib/_emerge/EbuildFetcher.py
+++ b/lib/_emerge/EbuildFetcher.py
@@ -373,7 +373,7 @@ class _EbuildFetcherProcess(ForkProcess):
         def cache_result(result):
             try:
                 self._uri_map = result.result()
-            except Exception:
+            except (CancelledError, Exception):
                 # The caller handles this when it retrieves the result.
                 pass
Comment 5 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-08-13 23:18:47 UTC
With that applied, on `emerge -v1 portage` (run twice to ensure using patched code), I get this at the end when cleaning up:
```

 * GNU info directory index is up-to-date.

 * IMPORTANT: 9 news items need reading for repository 'gentoo'.
 * Use eselect news read to view new items.


====================================
Error in portage.process.run_exitfuncs

  File "/usr/lib/python3.13/site-packages/portage/process.py", line 227, in run_exitfuncs
    func(*targs, **kargs)
    ~~~~^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/shutil.py", line 763, in rmtree
    _rmtree_safe_fd(stack, onexc)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/shutil.py", line 704, in _rmtree_safe_fd
    onexc(func, path, err)
    ~~~~~^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/shutil.py", line 665, in _rmtree_safe_fd
    orig_st = os.lstat(name, dir_fd=dirfd)
[Errno 2] No such file or directory: '/var/tmp/portage/._portage_reinstall_.nu7j61p8'
====================================

Exception ignored in atexit callback <function run_exitfuncs at 0x7f364a39f7e0>:
Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/portage/process.py", line 235, in run_exitfuncs
    raise exc_info[0](exc_info[1]).with_traceback(exc_info[2])
  File "/usr/lib/python3.13/site-packages/portage/process.py", line 227, in run_exitfuncs
    func(*targs, **kargs)
  File "/usr/lib/python3.13/shutil.py", line 763, in rmtree
    _rmtree_safe_fd(stack, onexc)
  File "/usr/lib/python3.13/shutil.py", line 704, in _rmtree_safe_fd
    onexc(func, path, err)
  File "/usr/lib/python3.13/shutil.py", line 665, in _rmtree_safe_fd
    orig_st = os.lstat(name, dir_fd=dirfd)
FileNotFoundError: [Errno 2] No such file or directory: '/var/tmp/portage/._portage_reinstall_.nu7j61p8'
```
Comment 6 Zac Medico gentoo-dev 2024-08-13 23:27:14 UTC
The EbuildFetcher was likely cancelled for the --keep-going resume, like when I discovered bug 936273. We just need to block any resulting CancelledError from propagating up the stack.

(In reply to Sam James from comment #5)
> With that applied, on `emerge -v1 portage` (run twice to ensure using
> patched code), I get this at the end when cleaning up:
> ```
> 
>  * GNU info directory index is up-to-date.
> 
>  * IMPORTANT: 9 news items need reading for repository 'gentoo'.
>  * Use eselect news read to view new items.
> 
> 
> ====================================
> Error in portage.process.run_exitfuncs
> 
>   File "/usr/lib/python3.13/site-packages/portage/process.py", line 227, in
> run_exitfuncs
>     func(*targs, **kargs)
>     ~~~~^^^^^^^^^^^^^^^^^
>   File "/usr/lib/python3.13/shutil.py", line 763, in rmtree
>     _rmtree_safe_fd(stack, onexc)
>     ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
>   File "/usr/lib/python3.13/shutil.py", line 704, in _rmtree_safe_fd
>     onexc(func, path, err)
>     ~~~~~^^^^^^^^^^^^^^^^^
>   File "/usr/lib/python3.13/shutil.py", line 665, in _rmtree_safe_fd
>     orig_st = os.lstat(name, dir_fd=dirfd)
> [Errno 2] No such file or directory:
> '/var/tmp/portage/._portage_reinstall_.nu7j61p8'
> ====================================
> 
> Exception ignored in atexit callback <function run_exitfuncs at
> 0x7f364a39f7e0>:
> Traceback (most recent call last):
>   File "/usr/lib/python3.13/site-packages/portage/process.py", line 235, in
> run_exitfuncs
>     raise exc_info[0](exc_info[1]).with_traceback(exc_info[2])
>   File "/usr/lib/python3.13/site-packages/portage/process.py", line 227, in
> run_exitfuncs
>     func(*targs, **kargs)
>   File "/usr/lib/python3.13/shutil.py", line 763, in rmtree
>     _rmtree_safe_fd(stack, onexc)
>   File "/usr/lib/python3.13/shutil.py", line 704, in _rmtree_safe_fd
>     onexc(func, path, err)
>   File "/usr/lib/python3.13/shutil.py", line 665, in _rmtree_safe_fd
>     orig_st = os.lstat(name, dir_fd=dirfd)
> FileNotFoundError: [Errno 2] No such file or directory:
> '/var/tmp/portage/._portage_reinstall_.nu7j61p8'
> ```

This looks safe to suppress with something like this:

--- a/lib/portage/package/ebuild/doebuild.py
+++ b/lib/portage/package/ebuild/doebuild.py
@@ -3238,0 +3239,7 @@ def _merge_unicode_error(errors):
+def _quiet_rmtree(path):
+       try:
+               shutil.rmtree(path)
+       except FileNotFoundError:
+               pass
+
+
@@ -3262 +3269 @@ def _prepare_self_update(settings):
-    portage.process.atexit_register(shutil.rmtree, base_path_tmp)
+    portage.process.atexit_register(_quiet_rmtree, base_path_tmp)
Comment 7 Zac Medico gentoo-dev 2024-08-13 23:57:25 UTC
(In reply to Sam James from comment #5)
It looks like a possible fork inheritance issue so I opened bug 937891.
Comment 8 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-08-14 00:27:33 UTC
Things seem happy so far with the two patches...
Comment 9 Zac Medico gentoo-dev 2024-08-14 05:47:10 UTC
(In reply to Sam James from comment #8)
> Things seem happy so far with the two patches...

The _quiet_rmtree patch should not be needed when https://github.com/gentoo/portage/pull/1370 fixes bug 937891.

The CancelledError patch looks good to merge, since suppressing it there is the correct thing to do. We return the cancelled result, so the caller will certainly be informed of the cancellation regardless of this suppression.
Comment 10 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-08-14 09:52:19 UTC
I still see the pid-ns crashes:

```
[... lots of merge-wait Emerging ... ]
>>> Emerging (62 of 1109) app-arch/cpio-2.15::gentoo
>>> Installing (1 of 1109) app-admin/entr-5.6::gentoo
>>> Installing (3 of 1109) app-arch/ncompress-5.0-r1::gentoo
>>> Installing (2 of 1109) app-admin/pwgen-2.08-r2::gentoo
>>> Installing (4 of 1109) app-arch/unrar-7.0.9::gentoo
>>> Completed (1 of 1109) app-admin/entr-5.6::gentoo
>>> Installing (5 of 1109) app-crypt/argon2-20190702-r1::gentoo
>>> Completed (5 of 1109) app-crypt/argon2-20190702-r1::gentoo
>>> Failed to install app-crypt/argon2-20190702-r1, Log file:
>>>  '/var/log/portage/build/app-crypt/argon2-20190702-r1:20240814-094243.log'
>>> Installing (8 of 1109) app-portage/cpuid2cpuflags-14::gentoo
>>> Completed (3 of 1109) app-arch/ncompress-5.0-r1::gentoo
>>> Installing (9 of 1109) app-shells/quoter-4.2::gentoo
>>> Completed (8 of 1109) app-portage/cpuid2cpuflags-14::gentoo
>>> Failed to install app-portage/cpuid2cpuflags-14, Log file:
```

and argon looks like:
```
strip: x86_64-pc-linux-gnu-strip --strip-unneeded -N __gentoo_check_ldflags__ -R .comment -R .GCC.command.line -R .note.gnu.gold-version
   /usr/lib64/libargon2.so.1
   /usr/bin/argon2
>>> Done.
/usr/bin/python3.13: can't open file '/var/tmp/portage/._portage_reinstall_.oubn2hj3/bin/pid-ns-init': [Errno 2] No such file or directory
/usr/bin/python3.13: can't open file '/var/tmp/portage/._portage_reinstall_.oubn2hj3/bin/pid-ns-init': [Errno 2] No such file or directory
 * The ebuild phase 'instprep' has exited unexpectedly. This type of
 * behavior is known to be triggered by things such as failed variable
 * assignments (bug #190128) or bad substitution errors (bug #200313).
 * Normally, before exiting, bash should have displayed an error message
 * above. If bash did not produce an error message above, it's possible
 * that the ebuild has called `exit` when it should have called `die`
 * instead. This behavior may also be triggered by a corrupt bash binary or
 * a hardware problem such as memory or cpu malfunction. If the problem is
 * not reproducible or it appears to occur randomly, then it is likely to
 * be triggered by a hardware problem. If you suspect a hardware problem
 * then you should try some basic hardware diagnostics such as memtest.
 * Please do not report this as a bug unless it is consistently
 * reproducible and you are sure that your bash binary and hardware are
 * functioning properly.
/bin/bash: line 1: /var/tmp/portage/._portage_reinstall_.oubn2hj3/bin/misc-functions.sh: No such file or directory
 * The ebuild phase 'die_hooks' has exited unexpectedly. This type of
 * behavior is known to be triggered by things such as failed variable
 * assignments (bug #190128) or bad substitution errors (bug #200313).
 * Normally, before exiting, bash should have displayed an error message
 * above. If bash did not produce an error message above, it's possible
 * that the ebuild has called `exit` when it should have called `die`
 * instead. This behavior may also be triggered by a corrupt bash binary or
 * a hardware problem such as memory or cpu malfunction. If the problem is
 * not reproducible or it appears to occur randomly, then it is likely to
 * be triggered by a hardware problem. If you suspect a hardware problem
 * then you should try some basic hardware diagnostics such as memtest.
 * Please do not report this as a bug unless it is consistently
 * reproducible and you are sure that your bash binary and hardware are
 * functioning properly.
!!! instprep failed
[...]
```
Comment 11 Larry the Git Cow gentoo-dev 2024-08-14 15:17:27 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=74e29110d86a55ac74fdde3e1b79506b3fb695c2

commit 74e29110d86a55ac74fdde3e1b79506b3fb695c2
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2024-08-14 05:49:30 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2024-08-14 15:08:12 +0000

    _EbuildFetcherProcess: Suppress CancelledError
    
    Suppress CancelledError when attempting to cache the
    result in the _async_uri_map method. The cancelled
    result is returned for the caller to handle.
    
    Bug: https://bugs.gentoo.org/937888
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 lib/_emerge/EbuildFetcher.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Comment 12 Larry the Git Cow gentoo-dev 2024-08-14 16:05:16 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=e4f0fa4a316aba2cf2b1a47c9d2974710a1d8bd3

commit e4f0fa4a316aba2cf2b1a47c9d2974710a1d8bd3
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2024-08-14 16:03:52 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2024-08-14 16:04:34 +0000

    _EbuildFetcherProcess: Suppress asyncio.CancelledError
    
    Fixes: 74e29110d86a ("_EbuildFetcherProcess: Suppress CancelledError")
    Bug: https://bugs.gentoo.org/937888
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 lib/_emerge/EbuildFetcher.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Comment 13 Zac Medico gentoo-dev 2024-08-14 16:31:31 UTC
(In reply to Sam James from comment #10)
> I still see the pid-ns crashes:
> 
> ```
> [... lots of merge-wait Emerging ... ]
> >>> Emerging (62 of 1109) app-arch/cpio-2.15::gentoo
> >>> Installing (1 of 1109) app-admin/entr-5.6::gentoo
> >>> Installing (3 of 1109) app-arch/ncompress-5.0-r1::gentoo
> >>> Installing (2 of 1109) app-admin/pwgen-2.08-r2::gentoo
> >>> Installing (4 of 1109) app-arch/unrar-7.0.9::gentoo
> >>> Completed (1 of 1109) app-admin/entr-5.6::gentoo
> >>> Installing (5 of 1109) app-crypt/argon2-20190702-r1::gentoo
> >>> Completed (5 of 1109) app-crypt/argon2-20190702-r1::gentoo
> >>> Failed to install app-crypt/argon2-20190702-r1, Log file:
> >>>  '/var/log/portage/build/app-crypt/argon2-20190702-r1:20240814-094243.log'
> >>> Installing (8 of 1109) app-portage/cpuid2cpuflags-14::gentoo
> >>> Completed (3 of 1109) app-arch/ncompress-5.0-r1::gentoo
> >>> Installing (9 of 1109) app-shells/quoter-4.2::gentoo
> >>> Completed (8 of 1109) app-portage/cpuid2cpuflags-14::gentoo
> >>> Failed to install app-portage/cpuid2cpuflags-14, Log file:
> ```
> 
> and argon looks like:
> ```
> strip: x86_64-pc-linux-gnu-strip --strip-unneeded -N
> __gentoo_check_ldflags__ -R .comment -R .GCC.command.line -R
> .note.gnu.gold-version
>    /usr/lib64/libargon2.so.1
>    /usr/bin/argon2
> >>> Done.
> /usr/bin/python3.13: can't open file
> '/var/tmp/portage/._portage_reinstall_.oubn2hj3/bin/pid-ns-init': [Errno 2]
> No such file or directory
> /usr/bin/python3.13: can't open file
> '/var/tmp/portage/._portage_reinstall_.oubn2hj3/bin/pid-ns-init': [Errno 2]
> No such file or directory


This looks like it was due to premature atexit invocations due to bug  bug 937891 that https://github.com/gentoo/portage/pull/1370 would prevent.
Comment 14 Larry the Git Cow gentoo-dev 2024-09-11 01:30:53 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=02d0e00a1ba811b39140d10e17488f7fc3916534

commit 02d0e00a1ba811b39140d10e17488f7fc3916534
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2024-09-11 01:30:10 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-09-11 01:30:30 +0000

    sys-apps/portage: add 3.0.66
    
    Closes: https://bugs.gentoo.org/435066
    Closes: https://bugs.gentoo.org/907061
    Closes: https://bugs.gentoo.org/910560
    Closes: https://bugs.gentoo.org/933433
    Closes: https://bugs.gentoo.org/934220
    Closes: https://bugs.gentoo.org/934514
    Closes: https://bugs.gentoo.org/934784
    Closes: https://bugs.gentoo.org/935830
    Closes: https://bugs.gentoo.org/936273
    Closes: https://bugs.gentoo.org/937384
    Closes: https://bugs.gentoo.org/937485
    Closes: https://bugs.gentoo.org/937740
    Closes: https://bugs.gentoo.org/937888
    Closes: https://bugs.gentoo.org/937891
    Closes: https://bugs.gentoo.org/938127
    Closes: https://bugs.gentoo.org/933499
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-3.0.66.ebuild | 227 +++++++++++++++++++++++++++++++++
 2 files changed, 228 insertions(+)