In a prefix install, sys-apps/portage-3.0.49-r2 fails to install. Output (with ECLASS_DEBUG_OUTPUT=on): >>> Installing (1 of 1) sys-apps/portage-3.0.49-r2::gentoo * checking 5501 files for package collisions >>> Merging sys-apps/portage-3.0.49-r2 to / debug: python_setup: entering function, parameters: debug: _python_validate_useflags: entering function, parameters: debug: _python_verify_patterns: entering function, parameters: debug: _python_export: entering function, parameters: python3_11 EPYTHON PYTHON debug: _python_export: implementation: python3.11 debug: _python_export: EPYTHON = python3.11 debug: _python_export: PYTHON = /usr/bin/python3.11 debug: _python_wrapper_setup: entering function, parameters: * Using python3.11 in global scope debug: python_get_sitedir: entering function, parameters: debug: _python_export: entering function, parameters: PYTHON_SITEDIR debug: _python_export: implementation: python3.11 debug: _python_export: PYTHON_SITEDIR = /usr/local/lib/python3.11/site-packages * ERROR: sys-apps/portage-3.0.49-r2::gentoo failed (preinst phase): * /home/marienz/gentoo/var/tmp/portage/sys-apps/portage-3.0.49-r2/image/usr/local/lib/python3.11/site-packages: No such directory * * Call stack: * ebuild.sh, line 136: Called pkg_preinst * environment, line 3908: Called die * The specific snippet of code: * [[ -d ${D}${sitedir} ]] || die "${D}${sitedir}: No such directory"; "PYTHON = /usr/bin/python3.11" looks wrong: that's the system Python, not Gentoo's (which is in /home/marienz/gentoo/usr/bin/python3.11). It works if I revert https://gitweb.gentoo.org/repo/gentoo.git/commit/eclass?id=9e4dc068a3066434bfbfa3a0f979ec5d6d8020eb (making python-utils-r1 use EPREFIX again if BROOT is unset). ECLASS_DEBUG_OUTPUT shows PYTHON is set correctly while building, but not in pkg_preinst (will attach full build.log to confirm). And https://projects.gentoo.org/pms/7/pms.html says BROOT is only "legal in" src_*, so I think this is a problem in the eclass. Reproducible: Always Steps to Reproduce: 1. emerge -va1 portage (using prefix) Actual Results: * /home/marienz/gentoo/var/tmp/portage/sys-apps/portage-3.0.49-r2/image/usr/local/lib/python3.11/site-packages: No such directory Expected Results: portage (re)installs % emerge -pqv '=sys-apps/portage-3.0.49-r2::gentoo' [ebuild R ] sys-apps/portage-3.0.49-r2 USE="(ipc) rsync-verify xattr -apidoc -build -debug -doc -gentoo-dev (-native-extensions) (-selinux) -test" PYTHON_TARGETS="pypy3 python3_11 -python3_10 -python3_12" % emerge --info '=sys-apps/portage-3.0.49-r2::gentoo' Portage 3.0.49 (python 3.11.4-final-0, default/linux/amd64/17.1/no-multilib/prefix/kernel-3.2+, gcc-13, glibc-2.37-r3, 6.4.7-200.fc38.x86_64 x86_64) ================================================================= System Settings ================================================================= System uname: Linux-6.4.7-200.fc38.x86_64-x86_64-AMD_Ryzen_9_5900X_12-Core_Processor-with-glibc2.37 KiB Mem: 65722372 total, 52552568 free KiB Swap: 8388604 total, 8388604 free Timestamp of repository gentoo: Sun, 06 Aug 2023 03:15:01 +0000 Head commit of repository gentoo: caeddb0b2ac532e9aa7cb9f4d2de39f5ff075fbb sh bash 5.2_p15-r6 ld GNU ld (Gentoo 2.40 p5) 2.40.0 app-misc/pax-utils: 1.3.7::gentoo app-shells/bash: 5.2_p15-r6::gentoo dev-lang/perl: 5.38.0::gentoo dev-lang/python: 3.11.4::gentoo, 3.12.0_beta4::gentoo dev-lang/rust: 1.70.0::gentoo dev-util/cmake: 3.26.4-r1::gentoo dev-util/meson: 1.1.1::gentoo sys-apps/baselayout: 2.14::gentoo sys-apps/sandbox: 2.36::gentoo sys-devel/autoconf: 2.71-r6::gentoo sys-devel/automake: 1.16.5-r1::gentoo sys-devel/binutils: 2.40-r5::gentoo sys-devel/binutils-config: 5.5::gentoo sys-devel/gcc: 13.1.1_p20230527::gentoo sys-devel/gcc-config: 2.11::gentoo sys-devel/libtool: 2.4.7-r1::gentoo sys-devel/llvm: 16.0.6::gentoo sys-devel/make: 4.4.1-r1::gentoo sys-kernel/linux-headers: 6.4::gentoo (virtual/os-headers) sys-libs/glibc: 2.37-r3::gentoo Repositories: gentoo location: /home/marienz/gentoo/var/db/repos/gentoo sync-type: rsync sync-uri: rsync://minotaur/gentoo-portage priority: -1000 volatile: True sync-rsync-extra-opts: sync-rsync-verify-metamanifest: yes sync-rsync-verify-jobs: 1 sync-rsync-verify-max-age: 24 marienz location: /home/marienz/portage masters: gentoo volatile: True ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="@FREE" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -ggdb -O2 -pipe -march=native" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /home/marienz/gentoo/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /home/marienz/gentoo/etc/env.d" CONFIG_SHELL="/home/marienz/gentoo/bin/bash" CXXFLAGS="-O2 -pipe -ggdb -O2 -pipe -march=native" DISTDIR="/home/marienz/gentoo/var/cache/distfiles" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live collision-protect compressdebug config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sfperms splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans unprivileged xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://mirror.internode.on.net/pub/gentoo/" LANG="en_US.UTF-8" LDFLAGS="-Wl,--hash-style=gnu -Wl,-O1 -Wl,--as-needed" LEX="flex" MAKEOPTS="-j22" PKGDIR="/home/marienz/gentoo/var/cache/binpkgs" PORTAGE_COMPRESS="zstd" PORTAGE_CONFIGROOT="/home/marienz/gentoo/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/home/marienz/gentoo/var/tmp" RUSTFLAGS="" SHELL="/home/marienz/gentoo/bin/zsh" USE="acl adns amd64 avahi btrfs bzip2 canberra caps cdio chromecast clang cli clutter corefonts crypt dav1d dbus default-gold device-mapper distinct-l dos dot dri dri3 drm dvi efi egl expat ffmpeg fontconfig fuse g3dvl gbm geckodriver gegl gflags ghcbootstrap gles2 gmp gnuefi gold google gpg hostonly http http2 iconv icu idn imap inotify ipv6 irc json lame libass libdrm libffi libidn2 libinput libkms libtirpc libvisual libxml2 llvm llvm-shared-libs lm-sensors lto lua lzma maildir mtp multimedia ncurses nfsidmap nls nptl numpy openmp opus pcre32 pgo playlist postscript prefix prelink preview-latex projectm python python3 qml readline realtime s3tc seccomp semantic-desktop sip smtp speex spice split-usr sqlite ssl svc system-bootstrap system-harfbuzz system-icu system-jflex system-jpeg system-jsoncpp system-llvm sysv-utils test-rust theora toolkit-scroll-bars udisks2 uefi unicode unicode3 unwind urwid usbredir user-session vaapi vala vpx vulkan wayland webkit webp widevine widgets xattr xcomposite xetex xft xinerama xml xmp xwayland zlib zsh-completion zstd" ABI_X86="64" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en en-GB en-US nl fy" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-1" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_11 pypy3" QEMU_SOFTMMU_TARGETS="x86_64" RUBY_TARGETS="ruby26" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, SIZE, STRINGS, STRIP, YACC, YFLAGS ================================================================= Package Settings ================================================================= sys-apps/portage-3.0.49-r2::gentoo was built with the following: USE="(ipc) (prefix) rsync-verify xattr -apidoc -build -debug -doc -gentoo-dev (-native-extensions) (-selinux) -test" PYTHON_TARGETS="pypy3 python3_11 -python3_10 -python3_12" FEATURES="ipc-sandbox fixlafiles compressdebug distlocks binpkg-logs unknown-features-warn splitdebug merge-sync unmerge-orphans sfperms config-protect-if-modified assume-digests network-sandbox unmerge-logs binpkg-dostrip binpkg-docompress binpkg-multi-instance strict ebuild-locks pid-sandbox multilib-strict qa-unresolved-soname-deps parallel-fetch protect-owned xattr buildpkg-live news collision-protect unprivileged preserve-libs"
Created attachment 867189 [details] build.log (with ECLASS_DEBUG_OUTPUT=on)
This got added to PMS very recently wrt bug #911574, but looks like portage indeed does not set it (yet), so guess portage needs an update if that's effective for old EAPIs too (does feel a bit awkward without being in a new EAPI, considering we now don't know if the PM supports it or not and can't rely on this). Either way, will need a more immediate workaround in the eclass.
(In reply to Ionen Wolkens from comment #2) > Either way, will need a more immediate workaround in the eclass. Or portage ebuild? Thinking about it, unsure if calling python_setup in pkg_preinst is considered valid. CC'ing eclass maintainers & committers for now anyhow.
Ech, sorry, I didn't realise it wasn't available in all phases. The problem is the old behaviour was also buggy, so this requires a Portage fix either way.
Here's a fix. Please could you try it. I don't have a regular prefix handy, only a cross prefix, where this problem doesn't apply.
On a side-note, after seeing this bug, ulm did mention: 04:28 <@ulm> ionen: re bug 911797 comment #2, this was added to PMS under the assumption that portage would set BROOT globally 04:29 <@ulm> now I wonder if it should be reverted in PMS, and either restricted to EAPI 8 or postponed to EAPI 9 Also, even if changed in portage, the eclass remains an issue given it's still going to be used by older portage anyway when people update.
Hmmm, that is true. In theory, you could base PYTHON on the PATH, but I fear that would break other things. I've realised that the original bug I was trying to fix was actually the same issue from the other side. It was using EPREFIX when it was supposed to use the blank BROOT, but BROOT was totally unset.
Thinking that through some more, I guess I could revert my change once the Portage fix is merged. The cross-prefix case is very much less common and is still broken for sys-apps/portage anyway.
As I read the EAPI 8 spec (https://projects.gentoo.org/pms/8/pms.html#x1-720008.1), I would conclude from tables 8.1 and 8.2 that BROOT should be defined in the pkg_*inst and pkg_*rm phases if the EAPI supports IDEPEND: - Table 8.1 says that IDEPEND should be satisfied for pkg_preinst, pkg_postinst, pkg_prerm, pkg_postrm (and _only_ these phases). - Table 8.2 lists ${BROOT} for IDEPEND. Updating the BROOT entry in the variables table was effectively overlooked when we added IDEPEND to PMS, leaving the spec in an (IMHO) inconsistent state.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=cfa3149797c2204f3a156ec2909c444baa640a6b commit cfa3149797c2204f3a156ec2909c444baa640a6b Author: James Le Cuirot <chewi@gentoo.org> AuthorDate: 2023-08-06 09:40:02 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-08-06 22:28:50 +0000 Define BROOT in pkg_* phases following PMS change Closes: https://bugs.gentoo.org/911797 Bug: https://bugs.gentoo.org/911574 Signed-off-by: James Le Cuirot <chewi@gentoo.org> Closes: https://github.com/gentoo/portage/pull/1077 Signed-off-by: Sam James <sam@gentoo.org> NEWS | 3 +++ bin/phase-helpers.sh | 4 ++-- lib/portage/package/ebuild/config.py | 2 +- 3 files changed, 6 insertions(+), 3 deletions(-)
I can confirm that the fix works for my prefix install (by dropping a patch without the NEWS changes into etc/portage/patches and reinstalling sys-apps/portage-3.0.49-r2) but also (as expected) that I had to temporarily re-revert the python-utils-r1.eclass change so I could install portage with that patch. Reopening, the portage half of this is fixed but prefix users still need that eclass change reverted or the portage ebuild's pkg_preinst changed so they can upgrade.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=01e4a638c3decc74baa52975043bb4af89ae2c70 commit 01e4a638c3decc74baa52975043bb4af89ae2c70 Author: James Le Cuirot <chewi@gentoo.org> AuthorDate: 2023-08-07 11:59:34 +0000 Commit: James Le Cuirot <chewi@gentoo.org> CommitDate: 2023-08-07 12:09:48 +0000 Revert "python-utils-r1.eclass: Remove old EAPI hack for exporting PYTHON" This reverts commit 9e4dc068a3066434bfbfa3a0f979ec5d6d8020eb. It fixed cross-prefix but inadvertently broke regular prefix. Both were ultimately affected by the same PMS + Portage bug where BROOT was not set in pkg_preinst. This has now been fixed, but regular prefix users need to be able to update Portage without hitting the issue. Cross-prefix users are safe because the BROOT Portage can be updated without hitting the issue. Closes: https://bugs.gentoo.org/911797 Signed-off-by: James Le Cuirot <chewi@gentoo.org> eclass/python-utils-r1.eclass | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
(In reply to Larry the Git Cow from comment #10) > The bug has been closed via the following commit(s): > > https://gitweb.gentoo.org/proj/portage.git/commit/ > ?id=cfa3149797c2204f3a156ec2909c444baa640a6b > > commit cfa3149797c2204f3a156ec2909c444baa640a6b > Author: James Le Cuirot <chewi@gentoo.org> > AuthorDate: 2023-08-06 09:40:02 +0000 > Commit: Sam James <sam@gentoo.org> > CommitDate: 2023-08-06 22:28:50 +0000 > > Define BROOT in pkg_* phases following PMS change > > Closes: https://bugs.gentoo.org/911797 > Bug: https://bugs.gentoo.org/911574 > Signed-off-by: James Le Cuirot <chewi@gentoo.org> > Closes: https://github.com/gentoo/portage/pull/1077 > Signed-off-by: Sam James <sam@gentoo.org> > > NEWS | 3 +++ > bin/phase-helpers.sh | 4 ++-- > lib/portage/package/ebuild/config.py | 2 +- > 3 files changed, 6 insertions(+), 3 deletions(-) Maybe this should be adjusted to behave differently for EAPIs that do not support IDEPEND, since PMS was also adjusted as noted in bug 911574 comment #6.
(In reply to Zac Medico from comment #13) > Maybe this should be adjusted to behave differently for EAPIs that do not > support IDEPEND, since PMS was also adjusted as noted in bug 911574 comment > #6. Not necessarily, because PMS says that "variables that are not meaningful in a given phase or in global scope may be unset or set to any value." Probably it is cleaner to be more strict, so this patch restores the previous behaviour for EAPI 7: https://github.com/gentoo/portage/pull/1078 Please review if the logic is correct.
(In reply to Zac Medico from comment #13) > > Maybe this should be adjusted to behave differently for EAPIs that do not > support IDEPEND, since PMS was also adjusted as noted in bug 911574 comment > #6. I asked ulm to not merge the change to do that, as it would break EAPI 7 ebuilds that use python-utils-r1.eclass for the cross-prefix case. It's a niche case, but gaining some interest, and I'm trying to make it work better.
