Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 940120 - sys-apps/portage-3.0.66.1 breaks emerge --sync with sync-type=webrsync
Summary: sys-apps/portage-3.0.66.1 breaks emerge --sync with sync-type=webrsync
Status: IN_PROGRESS
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core - Interface (emerge) (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on: 939444
Blocks:
  Show dependency tree
 
Reported: 2024-09-23 02:01 UTC by Hank Leininger
Modified: 2024-10-09 03:39 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Hank Leininger 2024-09-23 02:01:34 UTC
sys-apps/portage-3.0.66.1 errors with:

# emerge --sync
>>> Syncing repository 'gentoo' into '/usr/portage'...
 * ERROR: /:: failed:
 *   Do not call emerge-webrsync directly, instead call emerge --sync or emaint sync.
 * 
 * If you need support, post the output of `emerge --info '=/::'`,
 * the complete build log and the output of `emerge -pqv '=/::'`.
 * Working directory: '/root'
!!! emerge-webrsync error in /usr/portage

emaint sync errors in the same way. 

With sys-apps/portage-3.0.65-r1, emerge --sync works just fine.

This is with:

# cat /etc/portage/repos.conf/gentoo.conf 
[DEFAULT]
main-repo = gentoo

[gentoo]
location = /usr/portage
sync-type = webrsync
sync-webrsync-verify-signature = true
sync-uri = rsync://rsync.us.gentoo.org/gentoo-portage

I have not actually tried setting sync-type != webrsync, but I suspect git works fine or this would have been reported already.
Comment 1 Greg Kubaryk 2024-09-23 02:53:11 UTC
got an emerge --info for us?
Comment 2 Hank Leininger 2024-09-23 06:09:14 UTC
(In reply to Greg Kubaryk from comment #1)
> got an emerge --info for us?

Oh, sure. I've already masked the new portage on most boxes and rolled back but was able to trivially reproduce on a system that hasn't updated in a while; here is its emerge --info:

Portage 3.0.66.1 (python 3.11.9-final-0, default/linux/amd64/23.0/split-usr/hardened, gcc-13, glibc-2.39-r9, 6.6.31-gentoo x86_64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-6.6.31-gentoo-x86_64-AMD_Ryzen_9_PRO_7940HS_w-_Radeon_780M_Graphics-with-glibc2.39
KiB Mem:    64523968 total,   8590632 free
KiB Swap:   64001020 total,  63994876 free
Timestamp of repository gentoo: Mon, 23 Sep 2024 00:45:00 +0000
sh bash 5.2_p26-r5
ld GNU ld (Gentoo 2.42 p3) 2.42.0
ccache version 4.10 [disabled]
app-misc/pax-utils:        1.3.7::gentoo
app-shells/bash:           5.2_p26-r5::gentoo
dev-build/autoconf:        2.13-r8::gentoo, 2.71-r7::gentoo, 2.72-r1::gentoo
dev-build/automake:        1.16.5-r2::gentoo
dev-build/cmake:           3.29.3::gentoo
dev-build/libtool:         2.4.7-r4::gentoo
dev-build/make:            4.4.1-r1::gentoo
dev-build/meson:           1.4.1::gentoo
dev-java/java-config:      2.3.4::gentoo
dev-lang/perl:             5.40.0::gentoo
dev-lang/python:           3.11.9-r1::gentoo, 3.12.4_p1::gentoo
dev-lang/rust:             1.77.1::gentoo
dev-util/ccache:           4.10-r1::gentoo
sys-apps/baselayout:       2.15::gentoo
sys-apps/openrc:           0.54.1::gentoo
sys-apps/sandbox:          2.39::gentoo
sys-devel/binutils:        2.42-r1::gentoo
sys-devel/binutils-config: 5.5::gentoo
sys-devel/clang:           17.0.6::gentoo, 18.1.8::gentoo
sys-devel/gcc:             13.3.1_p20240614::gentoo
sys-devel/gcc-config:      2.11::gentoo
sys-devel/lld:             17.0.6::gentoo, 18.1.8::gentoo
sys-devel/llvm:            17.0.6::gentoo, 18.1.8::gentoo
sys-kernel/linux-headers:  6.9::gentoo (virtual/os-headers)
sys-libs/glibc:            2.39-r9::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: webrsync
    sync-uri: rsync://rsync.us.gentoo.org/gentoo-portage
    priority: -1000
    volatile: True
    sync-webrsync-verify-signature: true

Installed sets: 
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=core2 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /etc/stunnel/stunnel.conf /usr/lib64/libreoffice/program/sofficerc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=core2 -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync merge-wait multilib-strict network-sandbox news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms splitdebug strict strict-keepdir unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync warn-on-large-env xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://gentoo.osuosl.org/ http://lug.mtu.edu/gentoo/ http://gentoo.ussg.indiana.edu/ http://mirrors.rit.edu/gentoo/"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs"
LEX="flex"
LINGUAS="en en_US"
MAKEOPTS="-j16"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
SHELL="/bin/bash"
USE="X acl aes-ni alsa amd64 bluetooth bzip2 caps cet chroot crypt dbus elogind filecaps gdbm hardened iconv ipv6 libtirpc logrotate loop-aes mmx multilib ncurses nls openmp pam pcre pic pie pulseaudio readline seccomp split-usr sse sse2 ssl ssp suid syslog test-rust truetype udev unicode xattr xinerama xtpax zlib" ABI_X86="64" ADA_TARGET="gcc_12" APACHE2_MODULES="alias auth_basic authn_core authn_file authz_core authz_host authz_owner authz_user authz_groupfile cache deflate dir env expires filter headers include log_config logio mime mime_magic negotiation rewrite setenvif socache_shmcb vhost_alias unixd cgi" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 avx512_bf16 avx512_bitalg avx512_vbmi2 avx512_vnni avx512_vpopcntdq avx512bw avx512cd avx512dq avx512f avx512ifma avx512vbmi avx512vl f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 vpclmulqdq" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" GUILE_SINGLE_TARGET="3-0" GUILE_TARGETS="3-0" INPUT_DEVICES="evdev synaptics" KERNEL="linux" L10N="en en-US en_US" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-2" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_11" RUBY_TARGETS="ruby31" VIDEO_CARDS="amdgpu radeonsi" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account"
Unset:  ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS

=================================================================
                        Package Settings
=================================================================

sys-apps/portage-3.0.66.1::gentoo was built with the following:
USE="gentoo-dev (ipc) native-extensions rsync-verify (xattr) -apidoc -build -doc (-selinux) -test" ABI_X86="(64)" PYTHON_TARGETS="python3_11 -pypy3 -python3_10 -python3_12 -python3_13"
Comment 3 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-09-23 06:14:08 UTC
The only change in that period is:

commit f99eb91227918d5bf0cc531e78bd597c73b7ce1f
Author: Kerin Millar <kfm@plushkava.net>
Date:   Wed Jul 24 17:09:10 2024 +0100

    emerge-webrsync: actually honour the sync-webrsync-verify-signature attribute

    The repo_has_webrsync_verify variable was always being set as the empty
    string, despite later being compared to 1 in the arithmetic context.
    Have it work in the way that was originally intended. This also
    eliminates a superfluous use of the has() function, which was the reason
    for my inspecting the code in the first place.

    Also, apply the exact same fix to emerge-delta-webrsync.

    Fixes: b5cd549e07f3d835cbe04e5b718cdd47d7ee69f5
    Fixes: 0347637fe5033d3d8eb4fdafa1e86de171934819
    Signed-off-by: Kerin Millar <kfm@plushkava.net>
    Signed-off-by: Sam James <sam@gentoo.org>

Have I mentioned how much I'm a fan of the dual emerge-webrsync thing? :)

Looking...
Comment 4 Hank Leininger 2024-09-23 21:06:20 UTC
(In reply to Sam James from comment #3)
> The only change in that period is:

Maybe this will make more sense then =) I get the same issue with portage-3.0.66. So it wasn't 66 -> 66.1, but 65-r1 -> 66 where the issue was introduced.
Comment 5 Matt Jolly gentoo-dev 2024-10-01 07:37:54 UTC
Reported on IRC but this completely breaks syncing from web tarballs:

```
$ emerge-webrsync  
 * ERROR: /:: failed:
 *   Do not call emerge-webrsync directly, instead call emerge --sync or emaint sync.

$ doas emerge --sync
. . .
>>> Syncing repository 'gentoo' into '/var/db/repos/gentoo'...
 * ERROR: /:: failed:
 *   Do not call emerge-webrsync directly, instead call emerge --sync or emaint sync.

$ doas emaint sync
. . .
>>> Syncing repository 'gentoo' into '/var/db/repos/gentoo'...
 * ERROR: /:: failed:
 *   Do not call emerge-webrsync directly, instead call emerge --sync or emaint sync.

```
Comment 6 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-10-01 10:43:50 UTC
Sorry for the wait, I've just been trying to convince myself what the actual intention was.

I'll revert that commit for now (later today as I'm going to bed and don't want to break things) and add that to the ebuild, then reapply it & figure out what I actually want this to do.
Comment 7 Larry the Git Cow gentoo-dev 2024-10-02 05:06:36 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=123a7be533161b14cacc4aa98708fbf1027ecce1

commit 123a7be533161b14cacc4aa98708fbf1027ecce1
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2024-10-02 05:02:24 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-10-02 05:05:43 +0000

    Revert "emerge-webrsync: actually honour the sync-webrsync-verify-signature attribute"
    
    This reverts commit f99eb91227918d5bf0cc531e78bd597c73b7ce1f.
    
    Revert for now as we understand the status quo better, and this
    breaks syncing w/ emerge-webrsync. I'll come back to this but I'd
    rather work with a known quantity for now than rush a fix.
    
    Bug: https://bugs.gentoo.org/940120
    Signed-off-by: Sam James <sam@gentoo.org>

 bin/emerge-webrsync        | 9 ++++-----
 misc/emerge-delta-webrsync | 9 ++++-----
 2 files changed, 8 insertions(+), 10 deletions(-)
Comment 8 Larry the Git Cow gentoo-dev 2024-10-02 05:10:32 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=86f421c9b96a893d9ddf9c8c4065906475c3dcae

commit 86f421c9b96a893d9ddf9c8c4065906475c3dcae
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2024-10-02 05:08:52 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-10-02 05:09:56 +0000

    sys-apps/portage: backport emerge-webrsync fix
    
    As I said in the revert commit message, just pick the safe option for now
    until I can be sure about the proper fix, as emerge-webrsync has been
    brittle in the past.
    
    Note that the important thing is that verification works and is on still,
    i.e. the commit being reverted doesn't disable that.
    
    Bug: https://bugs.gentoo.org/940120
    Signed-off-by: Sam James <sam@gentoo.org>

 .../portage/files/portage-3.0.66.1-webrsync.patch  |  76 +++++++
 sys-apps/portage/portage-3.0.66.1-r1.ebuild        | 231 +++++++++++++++++++++
 2 files changed, 307 insertions(+)
Comment 9 Matt Jolly gentoo-dev 2024-10-09 03:39:29 UTC
Can confirm that 3.0.66.1-r1 can successfully sync. \o/