Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 924287 - sys-apps/portage-9999: regression/backtrace in pretend fetch
Summary: sys-apps/portage-9999: regression/backtrace in pretend fetch
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core - Interface (emerge) (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on:
Blocks: 916566
  Show dependency tree
 
Reported: 2024-02-11 08:34 UTC by Greg Kubaryk
Modified: 2024-02-13 18:34 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Greg Kubaryk 2024-02-11 08:34:42 UTC
Don't remember exact commit, but it would've been within the last 24h or so.

$ emerge -pvfO =sys-libs/timezone-data-2023d

These are the packages that would be fetched, in order:


[ERROR] Exception in callback AsynchronousTask._exit_listener_cb(<bound method...7fe3187fa200>>)
handle: <Handle AsynchronousTask._exit_listener_cb(<bound method...7fe3187fa200>>)>
Traceback (most recent call last):
  File "/usr/lib/python3.11/asyncio/events.py", line 84, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/lib/python3.11/site-packages/_emerge/AsynchronousTask.py", line 209, in _exit_listener_cb
    listener(self)
  File "/usr/lib/python3.11/site-packages/_emerge/EbuildBuild.py", line 123, in _start_with_metadata
    self._prefetch_exit(prefetcher)
  File "/usr/lib/python3.11/site-packages/_emerge/EbuildBuild.py", line 160, in _prefetch_exit
    retval = fetcher.execute()
             ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/_emerge/EbuildFetchonly.py", line 23, in execute
    rval = portage.doebuild(
           ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/portage/package/ebuild/doebuild.py", line 1339, in doebuild
    success = fetch(
              ^^^^^^
  File "/usr/lib/python3.11/site-packages/portage/package/ebuild/fetch.py", line 1557, in fetch
    loc = loc()
          ^^^^^
  File "/usr/lib/python3.11/site-packages/portage/package/ebuild/fetch.py", line 711, in get_mirror_url
    elif fetch(
         ^^^^^^
  File "/usr/lib/python3.11/site-packages/portage/package/ebuild/fetch.py", line 1122, in fetch
    _ensure_distdir(mysettings, mysettings["DISTDIR"])
  File "/usr/lib/python3.11/site-packages/portage/package/ebuild/fetch.py", line 243, in _ensure_distdir
    if _userpriv_test_write_file(settings, write_test_file):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/portage/package/ebuild/fetch.py", line 210, in _userpriv_test_write_file
    returncode = _spawn_fetch(settings, args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/portage/package/ebuild/fetch.py", line 172, in _spawn_fetch
    rval = spawn_func(args, env=env, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/portage/process.py", line 700, in spawn
    retval = loop.run_until_complete(pid.wait())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/portage/util/_eventloop/asyncio_event_loop.py", line 149, in _run_until_complete
    return self._loop.run_until_complete(future)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 630, in run_until_complete
    self._check_running()
  File "/usr/lib/python3.11/asyncio/base_events.py", line 589, in _check_running
    raise RuntimeError('This event loop is already running')
RuntimeError: This event loop is already running
Comment 1 Greg Kubaryk 2024-02-11 08:35:38 UTC
$ emerge --info
Portage 3.0.61_p59 (python 3.11.8-final-0, default/linux/amd64/17.1/desktop, gcc-13, glibc-2.38-r10, 6.7.2 x86_64)
=================================================================
System uname: Linux-6.7.2-x86_64-AMD_Ryzen_7_4700U_with_Radeon_Graphics-with-glibc2.38
KiB Mem:    15737392 total,   1289368 free
KiB Swap:   33554428 total,  33093884 free
Timestamp of repository gentoo: Sat, 10 Feb 2024 04:30:01 +0000
Head commit of repository gentoo: c0dd4c19405172c7132c084618c69d6fa747850e
Timestamp of repository trolltoo: Sat, 27 Jan 2024 17:35:16 +0000
Head commit of repository trolltoo: 60a3e46723683bc3f5dfffc66119c7329491f417

sh bash 5.2_p26
ld GNU ld (Gentoo 2.41 p5) 2.41.0
ccache version 4.9.1 [disabled]
app-misc/pax-utils:        1.3.7::gentoo
app-shells/bash:           5.2_p26::gentoo
dev-build/autoconf:        2.13-r8::gentoo, 2.72-r1::gentoo
dev-build/automake:        1.16.5-r2::gentoo
dev-build/cmake:           3.28.3::gentoo
dev-build/libtool:         2.4.7-r2::gentoo
dev-build/make:            4.4.1-r1::gentoo
dev-build/meson:           1.3.1-r1::gentoo
dev-lang/perl:             5.38.2-r1::gentoo
dev-lang/python:           3.11.8::gentoo, 3.12.2::gentoo
dev-lang/rust-bin:         1.74.1::gentoo
dev-util/ccache:           4.9.1::gentoo
sys-apps/baselayout:       2.14-r2::gentoo
sys-apps/openrc:           0.53::gentoo
sys-apps/sandbox:          2.38::gentoo
sys-devel/binutils:        2.41-r5::gentoo
sys-devel/binutils-config: 5.5::gentoo
sys-devel/clang:           17.0.6::gentoo
sys-devel/gcc:             13.2.1_p20240113-r1::gentoo
sys-devel/gcc-config:      2.11::gentoo
sys-devel/lld:             17.0.6::gentoo
sys-devel/llvm:            17.0.6::gentoo
sys-kernel/linux-headers:  6.7::gentoo (virtual/os-headers)
sys-libs/glibc:            2.38-r10::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: rsync
    sync-uri: rsync://192.168.1.151/gentoo-portage
    priority: -1000
    volatile: False
    sync-rsync-verify-metamanifest: no
    sync-rsync-verify-jobs: 1
    sync-rsync-extra-opts: 
    sync-rsync-verify-max-age: 3

meow
    location: /var/db/repos/meow
    masters: gentoo
    volatile: True

trolltoo
    location: /var/db/repos/trolltoo
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/trolltoo.git
    masters: gentoo
    volatile: False

Binary Repositories:

purr
    sync-uri: http://192.168.5.55:8080

ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="@FREE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=native"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe -march=native"
DISTDIR="/var/cache/distfiles"
EMERGE_DEFAULT_OPTS="--verbose-slot-rebuilds=n"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-O2 -pipe -march=native"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance binpkg-request-signature binpkg-signing buildpkg buildpkg-live config-protect-if-modified distlocks fixlafiles ipc-sandbox merge-sync merge-wait multilib-strict network-sandbox news parallel-fetch parallel-install pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe -march=native"
GENTOO_MIRRORS="https://kurly.net/gentoo-dist/ http://gentoo.osuosl.org/ http://mirrors.rit.edu/gentoo/ http://mirror.csclub.uwaterloo.ca/gentoo-distfiles/"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LEX="flex"
MAKEOPTS="-j8"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
SHELL="/bin/bash"
USE="X a52 aac acl acpi alsa amd64 bluetooth branding bzip2 cairo cdda cdr cli crypt cups dbus dist-kernel dri dts dvd dvdr elogind encode exif flac fortran gdbm gif gpm gtk gui iconv icu ipv6 jpeg lcms libnotify libtirpc mad mng mp3 mp4 mpeg multilib ncurses nls offensive ogg opengl openmp pam pango pcre pdf png policykit ppds pulseaudio qt5 readline sdl seccomp sixel sound spell split-usr ssl startup-notification svg test-rust tiff truetype udev udisks unicode upower usb vaapi vdpau vorbis vulkan wxwidgets x264 xattr xcb xft xml xv xvid zlib" ABI_X86="64" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" GRUB_PLATFORMS="efi-64" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-1" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_11" RUBY_TARGETS="ruby31" VIDEO_CARDS="amdgpu radeonsi radeon" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account"
Unset:  ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
Comment 2 Zac Medico gentoo-dev 2024-02-11 22:40:47 UTC
Hmm, I'll see what I can do to fix this. It's expected for synchronous spawn calls to fail if the event loop is already running since this commit:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=3e10368e52ecead86b75478ca448ef0f59333e3e

commit 3e10368e52ecead86b75478ca448ef0f59333e3e
Author: Zac Medico <zmedico@gentoo.org>
Date:   2024-02-03 20:34:45 -0800

    process.spawn: Use _start_proc for returnpid=False
    
    This essentially completes the implementation of bug 916566,
    eliminating os.fork() usage when "spawn" becomes the default
    multiprocessing start method.
    
    Bug: https://bugs.gentoo.org/916566
    Signed-off-by: Zac Medico <zmedico@gentoo.org>
Comment 3 Larry the Git Cow gentoo-dev 2024-02-12 13:58:46 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=67663e586320347e604727aef006bfe027bcd470

commit 67663e586320347e604727aef006bfe027bcd470
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2024-02-12 02:10:52 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2024-02-12 13:57:42 +0000

    EbuildBuild: Execute EbuildFetchonly in subprocess
    
    Execute EbuildFetchonly in a subprocess since it needs to
    run the event loop itself (even for pretend mode since it
    may need to fetch mirror layouts as reported in bug 702154).
    
    Also remove obsolete loop.is_running() case in doebuild
    related to bug 601252, since it will fail if a mirror
    layout file needs to be fetched, so we should just assume
    that the event loop is not running.
    
    Set fetched = 2 for listonly (pretend) mode in the fetch
    function so that EbuildFetchonly will properly report
    success for this mode.
    
    Pass ebuild_path to EbuildFetchonly, and also remove
    setcpv call from EbuildFetchonly since it's called earlier
    in EbuildBuild.
    
    Bug: https://bugs.gentoo.org/924287
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 lib/_emerge/EbuildBuild.py             | 39 +++++++++++++++++-----------------
 lib/_emerge/EbuildFetchonly.py         | 10 +++------
 lib/portage/package/ebuild/doebuild.py | 30 ++++++++------------------
 lib/portage/package/ebuild/fetch.py    |  1 +
 4 files changed, 33 insertions(+), 47 deletions(-)