I have an environment where I'm running many identical instances of the same gentoo system on a local network with a binhost. However, updating those systems using emerge -1gKuDp @world leads to only a subset of packages scheduled to be installed compared to issuing the command emerge -1uDp @world As of this bug, for example, I get the following results: ``` bluedragon ~ # emerge -1gKuDp @world Local copy of remote index is up-to-date and will be used. Local copy of remote index is up-to-date and will be used. These are the packages that would be merged, in order: Calculating dependencies... done! ``` compare to ``` bluedragon ~ # emerge -1uDp @world These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild U ] dev-python/mako-1.2.1 [1.2.0] ``` I did run emaint --fix binhost on the binhost and checked that it was correct. I also checked the copy of the Package file on one of the clients /var/cache/edb/binhost/192.168.3.13/Packages and it is in fact identical to what is on the binhost. This file shows that mako-1.2.1 is available for update, and yet unexpectedly it is not scheduled for installation within the @world set. However, if explicitly name the package, it is scheduled! ``` bluedragon ~ # emerge -1gKuDp mako Local copy of remote index is up-to-date and will be used. Local copy of remote index is up-to-date and will be used. These are the packages that would be merged, in order: Calculating dependencies... done! [binary U ] dev-python/mako-1.2.1-1 [1.2.0] ``` There appears to be something wrong with how @world is calculated when -K is given. I also tested -k with no change in behavior. Reproducible: Always
Please provide emerge --info from both systems.
This is `emerge --info` on the binhost ortage 3.0.30 (python 3.10.5-final-0, default/linux/amd64/17.0/musl, gcc-11.3.0, musl-1.2.3, 5.15.52-gentoo-x86_64 x86_64) ================================================================= System uname: Linux-5.15.52-gentoo-x86_64-x86_64-Intel-R-_Core-TM-_i7_CPU_920_@_2.67GHz-with-libc KiB Mem: 12251892 total, 11802684 free KiB Swap: 8388600 total, 8388600 free sh bash 5.1_p16-r1 ld GNU ld (Gentoo 2.38 p4) 2.38 app-misc/pax-utils: 1.3.4::gentoo app-shells/bash: 5.1_p16-r1::gentoo dev-lang/perl: 5.34.1-r3::gentoo dev-lang/python: 3.10.5::gentoo dev-lang/rust: 1.62.1::gentoo dev-util/cmake: 3.22.4::gentoo dev-util/meson: 0.62.2::gentoo sys-apps/baselayout: 2.8::gentoo sys-apps/openrc: 0.44.10::gentoo sys-apps/sandbox: 2.29::gentoo sys-devel/autoconf: 2.71-r1::gentoo sys-devel/automake: 1.16.5::gentoo sys-devel/binutils: 2.38-r2::gentoo sys-devel/binutils-config: 5.4.1::gentoo sys-devel/gcc: 11.3.0::gentoo sys-devel/gcc-config: 2.5-r1::gentoo sys-devel/libtool: 2.4.7::gentoo sys-devel/llvm: 14.0.6-r2::gentoo sys-devel/make: 4.3::gentoo sys-kernel/linux-headers: 5.15-r3::gentoo (virtual/os-headers) sys-libs/musl: 1.2.3::gentoo Repositories: gentoo location: /var/db/repos/gentoo sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 sync-rsync-verify-jobs: 1 sync-rsync-extra-opts: sync-rsync-verify-metamanifest: yes sync-rsync-verify-max-age: 24 musl location: /var/lib/layman/musl masters: gentoo priority: 50 ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="@FREE linux-fw-redistributable no-source-code" CBUILD="x86_64-gentoo-linux-musl" CFLAGS="-O2 -pipe -fno-diagnostics-color" CHOST="x86_64-gentoo-linux-musl" CONFIG_PROTECT="/etc /etc/grs/systems.conf /usr/lib/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -pipe -fno-diagnostics-color" DISTDIR="/var/cache/distfiles" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" INSTALL_MASK="charset.alias /usr/share/locale/locale.alias" LANG="C.UTF8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j4" PKGDIR="/var/cache/binpkgs" PORTAGE_BINHOST="http://192.168.3.13/" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" SHELL="/bin/bash" USE="acl amd64 bzip2 cli crypt dri fortran iconv ipv6 libglvnd libtirpc ncurses nls nptl openmp pam pcre readline seccomp split-usr ssl unicode xattr zlib" ABI_X86="64" ADA_TARGET="gnat_2020" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="musl" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-4 php8-0" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_10" PYTHON_TARGETS="python3_10" RUBY_TARGETS="ruby27" USERLAND="GNU" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, LC_ALL, LD, LEX, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
This is `emerge --info` on one of the instances pulling from the binhost Portage 3.0.30 (python 3.10.5-final-0, default/linux/amd64/17.0/musl, gcc-11.3.0, musl-1.2.3, 5.15.52-gentoo-x86_64 x86_64) ================================================================= System uname: Linux-5.15.52-gentoo-x86_64-x86_64-AMD_FX-tm-8350_Eight-Core_Processor-with-libc KiB Mem: 4023360 total, 3444304 free KiB Swap: 3407868 total, 3407868 free Head commit of repository gentoo: 45f27e6d51513385dcba2e5d13d4a4ef49a34dd7 sh bash 5.1_p16-r1 ld GNU ld (Gentoo 2.38 p4) 2.38 app-misc/pax-utils: 1.3.4::gentoo app-shells/bash: 5.1_p16-r1::gentoo dev-lang/perl: 5.34.1-r3::gentoo dev-lang/python: 3.10.5::gentoo dev-lang/rust: 1.62.1::gentoo dev-util/cmake: 3.22.4::gentoo dev-util/meson: 0.62.2::gentoo sys-apps/baselayout: 2.8::gentoo sys-apps/openrc: 0.44.10::gentoo sys-apps/sandbox: 2.29::gentoo sys-devel/autoconf: 2.71-r1::gentoo sys-devel/automake: 1.16.5::gentoo sys-devel/binutils: 2.38-r2::gentoo sys-devel/binutils-config: 5.4.1::gentoo sys-devel/gcc: 11.3.0::gentoo sys-devel/gcc-config: 2.5-r1::gentoo sys-devel/libtool: 2.4.7::gentoo sys-devel/llvm: 14.0.6-r2::gentoo sys-devel/make: 4.3::gentoo sys-kernel/linux-headers: 5.15-r3::gentoo (virtual/os-headers) sys-libs/musl: 1.2.3::gentoo Repositories: gentoo location: /var/db/repos/gentoo sync-type: git sync-uri: https://gitweb.gentoo.org/repo/gentoo priority: -1000 musl location: /var/lib/layman/musl masters: gentoo priority: 50 ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="@FREE linux-fw-redistributable no-source-code" CBUILD="x86_64-gentoo-linux-musl" CFLAGS="-O2 -pipe -fno-diagnostics-color" CHOST="x86_64-gentoo-linux-musl" CONFIG_PROTECT="/etc /etc/grs/systems.conf /usr/lib/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -pipe -fno-diagnostics-color" DISTDIR="/var/cache/distfiles" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" INSTALL_MASK="charset.alias /usr/share/locale/locale.alias" LANG="C.UTF8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j4" PKGDIR="/var/cache/binpkgs" PORTAGE_BINHOST="http://192.168.3.13/" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" SHELL="/bin/bash" USE="acl amd64 bzip2 cli crypt dri fortran iconv ipv6 libglvnd libtirpc ncurses nls nptl openmp pam pcre readline seccomp split-usr ssl unicode xattr zlib" ABI_X86="64" ADA_TARGET="gnat_2020" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="musl" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-4 php8-0" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_10" PYTHON_TARGETS="python3_10" RUBY_TARGETS="ruby27" USERLAND="GNU" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, LC_ALL, LD, LEX, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
To help you read these, here's the output to the diff between the two # diff emerge-info-binhost.txt emerge-info-client.txt 3,5c3,7 < System uname: Linux-5.15.52-gentoo-x86_64-x86_64-Intel-R-_Core-TM-_i7_CPU_920_@_2.67GHz-with-libc < KiB Mem: 12251892 total, 11802684 free < KiB Swap: 8388600 total, 8388600 free --- > System uname: Linux-5.15.52-gentoo-x86_64-x86_64-AMD_FX-tm-8350_Eight-Core_Processor-with-libc > KiB Mem: 4023360 total, 3424272 free > KiB Swap: 3407868 total, 3407868 free > Head commit of repository gentoo: 45f27e6d51513385dcba2e5d13d4a4ef49a34dd7 > 33,34c35,36 < sync-type: rsync < sync-uri: rsync://rsync.gentoo.org/gentoo-portage --- > sync-type: git > sync-uri: https://gitweb.gentoo.org/repo/gentoo 36,39d37 < sync-rsync-verify-jobs: 1 < sync-rsync-extra-opts: < sync-rsync-verify-metamanifest: yes < sync-rsync-verify-max-age: 24
You need to use --with-bdeps=y with -K to pull in the optional build time dependencies.
(In reply to Zac Medico from comment #5) > You need to use --with-bdeps=y with -K to pull in the optional build time > dependencies. Thank you Zac! Confirmed --- see below. The question now is, did this behavior change in the last few years? Or am I just mis-remembering what I expected the behavior to be? Also should we document this? Currently `man emerge` doesn't say anything about this in --usepkgonly. If you don't think documentation is necessary, feel free to close INVALID. bluedragon ~ # emerge -1gKuDq @world -p Local copy of remote index is up-to-date and will be used. bluedragon ~ # emerge -1gKuDq --with-bdeps=y @world -p Local copy of remote index is up-to-date and will be used. [binary U ] dev-python/mako-1.2.1-1 [1.2.0]
(In reply to Anthony Basile from comment #6) > Thank you Zac! Confirmed --- see below. The question now is, did this > behavior change in the last few years? Or am I just mis-remembering what I > expected the behavior to be? It's been this way for a very long time (see bug 274454). > Also should we document this? Currently `man > emerge` doesn't say anything about this in --usepkgonly. > > If you don't think documentation is necessary, feel free to close INVALID. Yeah, it would be nice to add a note to the --usepkgonly documentation about --with-bdeps.
(In reply to Zac Medico from comment #7) > > Yeah, it would be nice to add a note to the --usepkgonly documentation about > --with-bdeps. I'll do a PR tomorrow. There's already documentation under --with-bdeps about it (ionen pointed that out to me), but nothing under --usepkgonly which is where the reader is going to go for binpkg info and miss this point.
Note that --with-bdeps=y is relevant even without --usepkg, since it pulls in optional build time dependencies for installed packages.
(In reply to Zac Medico from comment #9) > Note that --with-bdeps=y is relevant even without --usepkg, since it pulls > in optional build time dependencies for installed packages. Yeah, I get that, but the problem is that, naively, one might expect that emerge -1uDp @world and emerge -1gKuDp @world would calculated the same packages for installation given that -K means use binpkgs instead of building locally. So -k and -K don't *just* mean prefer/insist on binpkgs over local builds, but additionally imply --with-bdeps=n. Which makes sense since you don't need the bdeps with binpkgs, but like I said, naively one might expect the behaviour I stated. I'll will add language to clarify that under -k and -K documentation. For completeness, the -g just says use the PORTAGE_BINHOST for the binhost and doesn't effect the behaviour in question.
This commit is relevant: emerge: auto-enable --with-bdeps if --usepkg is not enabled (bug 598444) https://gitweb.gentoo.org/proj/portage.git/commit/?id=852c729bdef3d4c2e2d459a43dc21f0a05dfa2ba
(In reply to Zac Medico from comment #11) > This commit is relevant: > > emerge: auto-enable --with-bdeps if --usepkg is not enabled (bug 598444) > > https://gitweb.gentoo.org/proj/portage.git/commit/ > ?id=852c729bdef3d4c2e2d459a43dc21f0a05dfa2ba Thanks for the history. I agree with that design for the reason stated. Just a one liner under -k/-K to make that behaviour explicit in the documentation would be helpful.
(In reply to Anthony Basile from comment #12) > (In reply to Zac Medico from comment #11) > > This commit is relevant: > > > > emerge: auto-enable --with-bdeps if --usepkg is not enabled (bug 598444) > > > > https://gitweb.gentoo.org/proj/portage.git/commit/ > > ?id=852c729bdef3d4c2e2d459a43dc21f0a05dfa2ba > > Thanks for the history. I agree with that design for the reason stated. > Just a one liner under -k/-K to make that behaviour explicit in the > documentation would be helpful. I add a short blurb under -k/-K. I'm not sure its needed under -g / -G since those sections refer back to -k/-K. Again someone interested in bin pkgs might look only under -k/-K and miss the default behavior stated elsewhere. https://github.com/gentoo/portage/pull/886
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=b7c3656f8c356b20675abf90df37b24ab8b8eeb5 commit b7c3656f8c356b20675abf90df37b24ab8b8eeb5 Author: Anthony G. Basile <blueness@gentoo.org> AuthorDate: 2022-08-10 12:57:23 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2022-08-12 16:05:47 +0000 man/emerge.1: add clarification to -k and -K Both -k and -K imply --with-bdeps=n. While this is already noted under the documentation for --with-bdeps, a brief reminder under the documentation for -k and -K is useful. A user interested in binary packages might look under -k and -K first and miss the default behavior with respect to build time dependencies. Bug: https://bugs.gentoo.org/863422 Closes: https://github.com/gentoo/portage/pull/886 Signed-off-by: Anthony G. Basile <blueness@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org> man/emerge.1 | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=328977ffff1aa2d13cd3d8f7b5cb01eea6d634f5 commit 328977ffff1aa2d13cd3d8f7b5cb01eea6d634f5 Author: Sam James <sam@gentoo.org> AuthorDate: 2022-08-23 00:38:45 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-08-23 00:39:03 +0000 sys-apps/portage: add 3.0.35 Closes: https://bugs.gentoo.org/859439 Closes: https://bugs.gentoo.org/672916 Closes: https://bugs.gentoo.org/858791 Closes: https://bugs.gentoo.org/862600 Closes: https://bugs.gentoo.org/862606 Closes: https://bugs.gentoo.org/863422 Closes: https://bugs.gentoo.org/864382 Closes: https://bugs.gentoo.org/865635 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.35.ebuild | 273 +++++++++++++++++++++++++++++++++ 2 files changed, 274 insertions(+)
