| Summary: | sys-apps/portage-9999: regression/backtrace in pretend fetch | ||
|---|---|---|---|
| Product: | Portage Development | Reporter: | Greg Kubaryk <greg.kubaryk> |
| Component: | Core - Interface (emerge) | Assignee: | Portage team <dev-portage> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | zmedico |
| Priority: | Normal | Keywords: | InVCS |
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| See Also: |
https://bugs.gentoo.org/show_bug.cgi?id=702154 https://github.com/gentoo/portage/pull/1262 https://bugs.gentoo.org/show_bug.cgi?id=657360 https://bugs.gentoo.org/show_bug.cgi?id=601252 |
||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | |||
| Bug Blocks: | 916566 | ||
$ emerge --info
Portage 3.0.61_p59 (python 3.11.8-final-0, default/linux/amd64/17.1/desktop, gcc-13, glibc-2.38-r10, 6.7.2 x86_64)
=================================================================
System uname: Linux-6.7.2-x86_64-AMD_Ryzen_7_4700U_with_Radeon_Graphics-with-glibc2.38
KiB Mem: 15737392 total, 1289368 free
KiB Swap: 33554428 total, 33093884 free
Timestamp of repository gentoo: Sat, 10 Feb 2024 04:30:01 +0000
Head commit of repository gentoo: c0dd4c19405172c7132c084618c69d6fa747850e
Timestamp of repository trolltoo: Sat, 27 Jan 2024 17:35:16 +0000
Head commit of repository trolltoo: 60a3e46723683bc3f5dfffc66119c7329491f417
sh bash 5.2_p26
ld GNU ld (Gentoo 2.41 p5) 2.41.0
ccache version 4.9.1 [disabled]
app-misc/pax-utils: 1.3.7::gentoo
app-shells/bash: 5.2_p26::gentoo
dev-build/autoconf: 2.13-r8::gentoo, 2.72-r1::gentoo
dev-build/automake: 1.16.5-r2::gentoo
dev-build/cmake: 3.28.3::gentoo
dev-build/libtool: 2.4.7-r2::gentoo
dev-build/make: 4.4.1-r1::gentoo
dev-build/meson: 1.3.1-r1::gentoo
dev-lang/perl: 5.38.2-r1::gentoo
dev-lang/python: 3.11.8::gentoo, 3.12.2::gentoo
dev-lang/rust-bin: 1.74.1::gentoo
dev-util/ccache: 4.9.1::gentoo
sys-apps/baselayout: 2.14-r2::gentoo
sys-apps/openrc: 0.53::gentoo
sys-apps/sandbox: 2.38::gentoo
sys-devel/binutils: 2.41-r5::gentoo
sys-devel/binutils-config: 5.5::gentoo
sys-devel/clang: 17.0.6::gentoo
sys-devel/gcc: 13.2.1_p20240113-r1::gentoo
sys-devel/gcc-config: 2.11::gentoo
sys-devel/lld: 17.0.6::gentoo
sys-devel/llvm: 17.0.6::gentoo
sys-kernel/linux-headers: 6.7::gentoo (virtual/os-headers)
sys-libs/glibc: 2.38-r10::gentoo
Repositories:
gentoo
location: /var/db/repos/gentoo
sync-type: rsync
sync-uri: rsync://192.168.1.151/gentoo-portage
priority: -1000
volatile: False
sync-rsync-verify-metamanifest: no
sync-rsync-verify-jobs: 1
sync-rsync-extra-opts:
sync-rsync-verify-max-age: 3
meow
location: /var/db/repos/meow
masters: gentoo
volatile: True
trolltoo
location: /var/db/repos/trolltoo
sync-type: git
sync-uri: https://github.com/gentoo-mirror/trolltoo.git
masters: gentoo
volatile: False
Binary Repositories:
purr
sync-uri: http://192.168.5.55:8080
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="@FREE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=native"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe -march=native"
DISTDIR="/var/cache/distfiles"
EMERGE_DEFAULT_OPTS="--verbose-slot-rebuilds=n"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-O2 -pipe -march=native"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance binpkg-request-signature binpkg-signing buildpkg buildpkg-live config-protect-if-modified distlocks fixlafiles ipc-sandbox merge-sync merge-wait multilib-strict network-sandbox news parallel-fetch parallel-install pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe -march=native"
GENTOO_MIRRORS="https://kurly.net/gentoo-dist/ http://gentoo.osuosl.org/ http://mirrors.rit.edu/gentoo/ http://mirror.csclub.uwaterloo.ca/gentoo-distfiles/"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LEX="flex"
MAKEOPTS="-j8"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
SHELL="/bin/bash"
USE="X a52 aac acl acpi alsa amd64 bluetooth branding bzip2 cairo cdda cdr cli crypt cups dbus dist-kernel dri dts dvd dvdr elogind encode exif flac fortran gdbm gif gpm gtk gui iconv icu ipv6 jpeg lcms libnotify libtirpc mad mng mp3 mp4 mpeg multilib ncurses nls offensive ogg opengl openmp pam pango pcre pdf png policykit ppds pulseaudio qt5 readline sdl seccomp sixel sound spell split-usr ssl startup-notification svg test-rust tiff truetype udev udisks unicode upower usb vaapi vdpau vorbis vulkan wxwidgets x264 xattr xcb xft xml xv xvid zlib" ABI_X86="64" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" GRUB_PLATFORMS="efi-64" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-1" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_11" RUBY_TARGETS="ruby31" VIDEO_CARDS="amdgpu radeonsi radeon" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account"
Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
Hmm, I'll see what I can do to fix this. It's expected for synchronous spawn calls to fail if the event loop is already running since this commit: https://gitweb.gentoo.org/proj/portage.git/commit/?id=3e10368e52ecead86b75478ca448ef0f59333e3e commit 3e10368e52ecead86b75478ca448ef0f59333e3e Author: Zac Medico <zmedico@gentoo.org> Date: 2024-02-03 20:34:45 -0800 process.spawn: Use _start_proc for returnpid=False This essentially completes the implementation of bug 916566, eliminating os.fork() usage when "spawn" becomes the default multiprocessing start method. Bug: https://bugs.gentoo.org/916566 Signed-off-by: Zac Medico <zmedico@gentoo.org> The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=67663e586320347e604727aef006bfe027bcd470 commit 67663e586320347e604727aef006bfe027bcd470 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2024-02-12 02:10:52 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2024-02-12 13:57:42 +0000 EbuildBuild: Execute EbuildFetchonly in subprocess Execute EbuildFetchonly in a subprocess since it needs to run the event loop itself (even for pretend mode since it may need to fetch mirror layouts as reported in bug 702154). Also remove obsolete loop.is_running() case in doebuild related to bug 601252, since it will fail if a mirror layout file needs to be fetched, so we should just assume that the event loop is not running. Set fetched = 2 for listonly (pretend) mode in the fetch function so that EbuildFetchonly will properly report success for this mode. Pass ebuild_path to EbuildFetchonly, and also remove setcpv call from EbuildFetchonly since it's called earlier in EbuildBuild. Bug: https://bugs.gentoo.org/924287 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/_emerge/EbuildBuild.py | 39 +++++++++++++++++----------------- lib/_emerge/EbuildFetchonly.py | 10 +++------ lib/portage/package/ebuild/doebuild.py | 30 ++++++++------------------ lib/portage/package/ebuild/fetch.py | 1 + 4 files changed, 33 insertions(+), 47 deletions(-) |
Don't remember exact commit, but it would've been within the last 24h or so. $ emerge -pvfO =sys-libs/timezone-data-2023d These are the packages that would be fetched, in order: [ERROR] Exception in callback AsynchronousTask._exit_listener_cb(<bound method...7fe3187fa200>>) handle: <Handle AsynchronousTask._exit_listener_cb(<bound method...7fe3187fa200>>)> Traceback (most recent call last): File "/usr/lib/python3.11/asyncio/events.py", line 84, in _run self._context.run(self._callback, *self._args) File "/usr/lib/python3.11/site-packages/_emerge/AsynchronousTask.py", line 209, in _exit_listener_cb listener(self) File "/usr/lib/python3.11/site-packages/_emerge/EbuildBuild.py", line 123, in _start_with_metadata self._prefetch_exit(prefetcher) File "/usr/lib/python3.11/site-packages/_emerge/EbuildBuild.py", line 160, in _prefetch_exit retval = fetcher.execute() ^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/_emerge/EbuildFetchonly.py", line 23, in execute rval = portage.doebuild( ^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/portage/package/ebuild/doebuild.py", line 1339, in doebuild success = fetch( ^^^^^^ File "/usr/lib/python3.11/site-packages/portage/package/ebuild/fetch.py", line 1557, in fetch loc = loc() ^^^^^ File "/usr/lib/python3.11/site-packages/portage/package/ebuild/fetch.py", line 711, in get_mirror_url elif fetch( ^^^^^^ File "/usr/lib/python3.11/site-packages/portage/package/ebuild/fetch.py", line 1122, in fetch _ensure_distdir(mysettings, mysettings["DISTDIR"]) File "/usr/lib/python3.11/site-packages/portage/package/ebuild/fetch.py", line 243, in _ensure_distdir if _userpriv_test_write_file(settings, write_test_file): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/portage/package/ebuild/fetch.py", line 210, in _userpriv_test_write_file returncode = _spawn_fetch(settings, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/portage/package/ebuild/fetch.py", line 172, in _spawn_fetch rval = spawn_func(args, env=env, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/portage/process.py", line 700, in spawn retval = loop.run_until_complete(pid.wait()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/portage/util/_eventloop/asyncio_event_loop.py", line 149, in _run_until_complete return self._loop.run_until_complete(future) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/asyncio/base_events.py", line 630, in run_until_complete self._check_running() File "/usr/lib/python3.11/asyncio/base_events.py", line 589, in _check_running raise RuntimeError('This event loop is already running') RuntimeError: This event loop is already running