Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 922038 - sys-apps/portage-3.0.61-r1 raises KeyError when asking to --fetchonly
Summary: sys-apps/portage-3.0.61-r1 raises KeyError when asking to --fetchonly
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
: 924060 (view as bug list)
Depends on: 921380
Blocks: 377365
  Show dependency tree
 
Reported: 2024-01-13 11:55 UTC by Sebastian Pucilowski
Modified: 2024-02-22 07:24 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sebastian Pucilowski 2024-01-13 11:55:17 UTC
As I'm travelling with a gentoo laptop, I don't have time to emerge gcc/llvm/clang:


gentoo ~ # emerge -DupN world

These are the packages that would be merged, in order:

Calculating dependencies... done!
Dependency resolution took 12.61 s (backtrack: 3/20).

[ebuild   R    ] sys-devel/gcc-13.2.1_p20230826  USE="(-nptl%*)"
[ebuild  NS    ] sys-devel/llvm-17.0.6 [16.0.6] USE="-debuginfod%"
[ebuild  NS    ] sys-devel/llvm-toolchain-symlinks-17 [16-r1]
[ebuild     U  ] sys-devel/llvmgold-17 [16]
[ebuild  NS    ] sys-devel/clang-17.0.6 [16.0.6]
[ebuild  NS    ] sys-devel/clang-toolchain-symlinks-17 [16-r2]
[ebuild  NS    ] sys-libs/compiler-rt-17.0.6 [16.0.6]
[ebuild  NS    ] sys-libs/compiler-rt-sanitizers-17.0.6 [16.0.6]
[ebuild  NS    ] sys-devel/clang-runtime-17.0.6 [16.0.6]

WARNING: One or more updates/rebuilds have been skipped due to a dependency conflict:

gui-libs/wlroots:0

  (gui-libs/wlroots-0.17.1:0/17::gentoo, ebuild scheduled for merge) USE="X drm libinput session vulkan -liftoff -tinywl -x11-backend -xcb-errors" ABI_X86="(64)" conflicts with
    >=gui-libs/wlroots-0.16:0/16=[X] required by (gui-wm/sway-1.8.1-r2:0/0::gentoo, installed) USE="X man swaybar swaynag -grimshot -tray -wallpapers" ABI_X86="(64)"
                           ^^^^^^
    <gui-libs/wlroots-0.17:=[X] required by (gui-wm/sway-1.8.1-r2:0/0::gentoo, installed) USE="X man swaybar swaynag -grimshot -tray -wallpapers" ABI_X86="(64)"
    ^                 ^^^^
    <gui-libs/wlroots-0.17:0/16=[X] required by (gui-wm/sway-1.8.1-r2:0/0::gentoo, installed) USE="X man swaybar swaynag -grimshot -tray -wallpapers" ABI_X86="(64)"
    ^


I do have time to grab the source files to compile later overnight at my hotel room, however:


gentoo ~ # emerge -DupN world -f

These are the packages that would be fetched, in order:

Calculating dependencies... done!
Dependency resolution took 12.76 s.

Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.11/emerge", line 57, in main
    retval = emerge_main()
             ^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/_emerge/main.py", line 1308, in emerge_main
    return run_action(emerge_config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/_emerge/actions.py", line 4014, in run_action
    retval = action_build(emerge_config, spinner=spinner)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/_emerge/actions.py", line 410, in action_build
    success, mydepgraph, favorites = backtrack_depgraph(
                                     ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/_emerge/depgraph.py", line 11734, in backtrack_depgraph
    success, mydepgraph, favorites, backtracked, max_retries = _backtrack_depgraph(
                                                               ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/_emerge/depgraph.py", line 11788, in _backtrack_depgraph
    success, favorites = mydepgraph.select_files(myfiles)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/_emerge/depgraph.py", line 4760, in select_files
    return self._select_files(args)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/_emerge/depgraph.py", line 5194, in _select_files
    return self._resolve(myfavorites)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/_emerge/depgraph.py", line 5494, in _resolve
    and self._eliminate_rebuilds()
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/_emerge/depgraph.py", line 3780, in _eliminate_rebuilds
    priorities = self._dynamic_config.digraph.nodes[pkg][1][parent][:]
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
KeyError: <Package ('installed', '/', 'app-text/poppler-23.12.0', 'nomerge', 'installed')>


Reproducible: Always

Steps to Reproduce:
1. emerge -DupN world -f


Expected Results:  
The usual busywork of sourcing build files and populating /var/cache/distfiles

gentoo ~ # emerge --info
Portage 3.0.61 (python 3.11.7-final-0, default/linux/amd64/17.1/desktop/systemd/merged-usr, gcc-13, glibc-2.38-r9, 6.6.10-arch1-1 x86_64)
=================================================================
System uname: Linux-6.6.10-arch1-1-x86_64-11th_Gen_Intel-R-_Core-TM-_i9-11950H_@_2.60GHz-with-glibc2.38
KiB Mem:    65545736 total,  59356492 free
KiB Swap:   67108860 total,  67108860 free
Timestamp of repository gentoo: Sat, 13 Jan 2024 11:33:10 +0000
Head commit of repository gentoo: 23dee2d3fb5fd68c4f2a0413a00304c8c92b9158

sh bash 5.1_p16-r6
ld GNU ld (Gentoo 2.41 p4) 2.41.0
app-misc/pax-utils:        1.3.5::gentoo
app-shells/bash:           5.1_p16-r6::gentoo
dev-lang/perl:             5.38.2-r1::gentoo
dev-lang/python:           3.11.7::gentoo, 3.12.1_p1::gentoo
dev-lang/rust:             1.74.1::gentoo
dev-util/cmake:            3.27.9::gentoo
dev-util/meson:            1.3.0-r2::gentoo
sys-apps/baselayout:       2.14-r1::gentoo
sys-apps/sandbox:          2.38::gentoo
sys-apps/systemd:          255.2-r1::gentoo
sys-devel/autoconf:        2.71-r6::gentoo
sys-devel/automake:        1.16.5-r1::gentoo
sys-devel/binutils:        2.41-r3::gentoo
sys-devel/binutils-config: 5.5::gentoo
sys-devel/clang:           16.0.6::gentoo
sys-devel/gcc:             13.2.1_p20230826::gentoo
sys-devel/gcc-config:      2.11::gentoo
sys-devel/libtool:         2.4.7-r1::gentoo
sys-devel/llvm:            16.0.6::gentoo
sys-devel/make:            4.4.1-r1::gentoo
sys-kernel/linux-headers:  6.1::gentoo (virtual/os-headers)
sys-libs/glibc:            2.38-r9::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/gentoo
    priority: -1000
    volatile: False

Installed sets: @desktop-stack, @dev-stack, @system-stack
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=native"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d"
CXXFLAGS="-O2 -pipe -march=native"
DISTDIR="/var/cache/distfiles"
EMERGE_DEFAULT_OPTS="--quiet-build=y --keep-going"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-O2 -pipe -march=native"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe -march=native"
GENTOO_MIRRORS="http://ftp.kaist.ac.kr/pub/gentoo http://ftp.daum.net/gentoo http://ftp.lanet.kr/pub/gentoo"
LANG="en_AU.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LEX="flex"
LINGUAS="en_AU en_GB en de_CH de"
MAKEOPTS="-j17"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
SHELL="/bin/bash"
USE="X a52 aac acl acpi alsa amd64 bash-completion bluetooth boost branding btrfs bzip2 cairo cdr cli crypt dbus dri dts dvdr encode exif flac fortran gdbm gif gles2 gnuplot gpm graphite gtk gui harfbuzz iconv icu ipv6 jit jpeg lcms libaom libnotify libtirpc lto mad mng mp3 mp4 mpeg mtp multilib ncurses nls numpy ogg opengl openmp pam pango pcre pdf pgo png policykit ppds readline sdl seccomp sound spell ssl startup-notification svg symlink systemd test-rust threads tiff truetype udev udisks unicode upower usb vaapi vdpau verify-sig vorbis vulkan wayland webp wxwidgets x264 x265 xattr xcb xft xv zlib zstd" ABI_X86="64" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 avx512f avx512dq avx512cd avx512bw avx512vl avx512vbmi f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en-AU en-GB en de-CH de" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-1" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_11" RUBY_TARGETS="ruby31" VIDEO_CARDS="intel i965 iris" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account"
Unset:  ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS

gentoo /etc/portage/sets # grep . *
desktop-stack:#~ x11-drivers/nvidia-drivers
desktop-stack:#media-libs/libva
desktop-stack:media-libs/libva-intel-media-driver
desktop-stack:media-video/libva-utils
desktop-stack:media-libs/vulkan-loader
desktop-stack:#dev-util/vulkan-headers
desktop-stack:#~ x11-misc/vdpauinfo
desktop-stack:gui-wm/sway
desktop-stack:gui-libs/wlroots
desktop-stack:dev-libs/wayland-protocols
desktop-stack:#x11-libs/gtk+
desktop-stack:#gui-libs/libadwaita
desktop-stack:#x11-themes/adwaita-icon-theme
desktop-stack:x11-terms/alacritty
desktop-stack:media-video/ffmpeg
desktop-stack:media-video/mpv
desktop-stack:#media-libs/libplacebo
desktop-stack:#~ media-gfx/gimp
desktop-stack:media-gfx/inkscape
desktop-stack:net-p2p/transmission-remote-gtk
desktop-stack:net-misc/yt-dlp
desktop-stack:media-fonts/ubuntu-font-family
dev-stack:app-misc/tmux
dev-stack:dev-vcs/git
dev-stack:sys-process/parallel
dev-stack:sys-fs/inotify-tools
dev-stack:#~ media-libs/flac
dev-stack:#~ media-libs/mutagen
science-stack:#~ dev-python/numpy lapack
science-stack:#~ dev-python/scipy
science-stack:#~ dev-python/pandas
science-stack:# For project work
science-stack:#~ dev-python/networkx
science-stack:#~ dev-python/openpyxl
science-stack:#~ dev-python/matplotlib -wxwidgets gtk3
science-stack:#app-text/texlive
science-stack:#dev-texlive/texlive-latexextra
science-stack:#dev-tex/latexdiff
science-stack:#~ media-gfx/inkscape -spell
science-stack:#media-gfx/povray -X
science-stack:# Don't forget to +latex later
science-stack:#~ sci-visualization/gnuplot cairo -X -qt5 -wxwidgets
system-stack:sys-kernel/gentoo-kernel-bin
system-stack:#virtual/dist-kernel
system-stack:sys-apps/systemd
system-stack:sys-kernel/linux-firmware
system-stack:sys-firmware/intel-microcode
system-stack:sys-firmware/sof-firmware
system-stack:app-portage/gentoolkit
system-stack:sys-block/parted
system-stack:sys-fs/btrfs-progs
system-stack:sys-fs/dosfstools
system-stack:sys-fs/exfatprogs
system-stack:net-wireless/iw
system-stack:net-wireless/iwd
system-stack:#dev-libs/ell
system-stack:sys-apps/pciutils
system-stack:sys-apps/usbutils
system-stack:net-vpn/wireguard-tools
system-stack:net-fs/cifs-utils
system-stack:#~ sys-power/intel-undervolt
system-stack:#~ sys-power/powertop
system-stack:app-editors/vim
system-stack:app-vim/gentoo-syntax
system-stack:app-shells/bash-completion
system-stack:sys-apps/smartmontools
Comment 1 Zac Medico gentoo-dev 2024-01-15 01:09:42 UTC
This KeyError should not happen because it indicates a inconsistent/corrupt state. Maybe the _solve_non_slot_operator_slot_conflicts method could corrupt the state like this, though it's not obvious how that would happen.

The --fetchonly option causes the _accept_blocker_conflicts method to return False, which prevents some backtracking, and explains how it can get into a different state than without --fetchonly.
Comment 2 Zac Medico gentoo-dev 2024-01-15 01:15:02 UTC
The backtracking which --fetchonly suppresses could hide this sort of inconsistent/corrupt state, since backtracking causes this state to be discarded.
Comment 3 Zac Medico gentoo-dev 2024-01-15 02:01:42 UTC
It could be that _eliminate_rebuilds does itself corrupt the state, since it may have modified the parent in a previous iteration of the loop, if both parent and child were in the @__auto_slot_operator_replace_installed__ set.
Comment 4 Zac Medico gentoo-dev 2024-01-15 07:04:57 UTC
I think we need to align the _accept_blocker_conflicts backtracking behavior with the normal behavior, since it's very annoying to get different results here.
Comment 5 Larry the Git Cow gentoo-dev 2024-01-16 16:03:16 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=a5078e5774932103d4ad367de4c6bf130a6da34f

commit a5078e5774932103d4ad367de4c6bf130a6da34f
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2024-01-15 23:10:03 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2024-01-16 16:01:53 +0000

    emerge: backtrack consistently regardless of --fetchonly
    
    Make the _accept_blocker_conflicts method always return
    True when backtracking is enabled, so that backtracking
    results will be identical regardless of options that
    _accept_blocker_conflicts treats specially. This way,
    conflicts will only be accepted when backtracking is
    disabled or all backtracking tries have been exhausted.
    Make --nodeps imply --backtrack=0, since backtracking
    is only useful with dependencies.
    
    Make _eliminate_rebuilds return early if there are
    slot conflicts, since the merge list is invalid anyway,
    and its possible that state inconsistencies could
    trigger unexpected exceptions as in bug 922038. Make
    the KeyError from bug 922038 a warning, and include
    relevant information to help trace the inconsistency
    back to the _eliminate_rebuilds method or some other
    source like  _solve_non_slot_operator_slot_conflicts.
    
    Bug: https://bugs.gentoo.org/161422
    Bug: https://bugs.gentoo.org/607252
    Bug: https://bugs.gentoo.org/675748
    Bug: https://bugs.gentoo.org/922038
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 lib/_emerge/depgraph.py | 63 +++++++++++++++++++++++++++++++++++++++++++++----
 man/emerge.1            |  5 ++--
 2 files changed, 61 insertions(+), 7 deletions(-)
Comment 6 Zac Medico gentoo-dev 2024-02-08 15:39:16 UTC
*** Bug 924060 has been marked as a duplicate of this bug. ***
Comment 7 Larry the Git Cow gentoo-dev 2024-02-22 07:24:09 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=77c44c46194922509bc4f2b5cfc099412a560a69

commit 77c44c46194922509bc4f2b5cfc099412a560a69
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2024-02-22 07:23:40 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-02-22 07:23:50 +0000

    sys-apps/portage: add 3.0.62
    
    Closes: https://bugs.gentoo.org/663324
    Closes: https://bugs.gentoo.org/728046
    Closes: https://bugs.gentoo.org/891137
    Closes: https://bugs.gentoo.org/906368
    Closes: https://bugs.gentoo.org/916566
    Closes: https://bugs.gentoo.org/921170
    Closes: https://bugs.gentoo.org/921208
    Closes: https://bugs.gentoo.org/921400
    Closes: https://bugs.gentoo.org/922038
    Closes: https://bugs.gentoo.org/922142
    Closes: https://bugs.gentoo.org/923368
    Closes: https://bugs.gentoo.org/923750
    Closes: https://bugs.gentoo.org/923841
    Closes: https://bugs.gentoo.org/923852
    Closes: https://bugs.gentoo.org/923854
    Closes: https://bugs.gentoo.org/924192
    Closes: https://bugs.gentoo.org/924273
    Closes: https://bugs.gentoo.org/924585
    Closes: https://bugs.gentoo.org/921380
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-3.0.62.ebuild | 246 +++++++++++++++++++++++++++++++++
 2 files changed, 247 insertions(+)