Created attachment 779471 [details] build.log ====================================================================== ERROR: test_whirlpool (portage.tests.util.test_checksum.ChecksumTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/var/tmp/portage/sys-apps/portage-3.0.30-r5/work/portage-3.0.30-pypy3/lib/portage/tests/__init__.py", line 248, in run testMethod() File "/var/tmp/portage/sys-apps/portage-3.0.30-r5/work/portage-3.0.30-pypy3/lib/portage/tests/util/test_checksum.py", line 62, in test_whirlpool checksum_str(b"", "WHIRLPOOL"), File "/var/tmp/portage/sys-apps/portage-3.0.30-r5/work/portage-3.0.30-pypy3/lib/portage/checksum.py", line 643, in checksum_str return hashfunc_map[hashname].checksum_str(data) File "/var/tmp/portage/sys-apps/portage-3.0.30-r5/work/portage-3.0.30-pypy3/lib/portage/checksum.py", line 76, in checksum_str checksum.update(data) File "/var/tmp/portage/sys-apps/portage-3.0.30-r5/work/portage-3.0.30-pypy3/lib/portage/util/whirlpool.py", line 48, in update WhirlpoolAdd(arg, len(arg) * 8, self.ctx) File "/var/tmp/portage/sys-apps/portage-3.0.30-r5/work/portage-3.0.30-pypy3/lib/portage/util/whirlpool.py", line 2222, in WhirlpoolAdd b = (source[sourcePos] << sourceGap) & 0xFF IndexError: byte index out of range ---- Portage 3.0.30 (python 3.10.4-final-0, default/linux/amd64/17.1/hardened, gcc-12.1.1, glibc-2.35-r5, 5.15.40-gentoo-dist-hardened x86_64) ================================================================= System uname: Linux-5.15.40-gentoo-dist-hardened-x86_64-AMD_Ryzen_9_3950X_16-Core_Processor-with-glibc2.35 KiB Mem: 65831164 total, 4067960 free KiB Swap: 25067512 total, 25058296 free Timestamp of repository gentoo: Thu, 19 May 2022 03:18:00 +0000 Head commit of repository gentoo: 46615b6b6973db16d1943baeb1c2335c5a6aad51 Timestamp of repository kde: Wed, 18 May 2022 12:47:53 +0000 Head commit of repository kde: db3a6d11f3516e4a0d7b675ece4c746bbaef5b86 Timestamp of repository qt: Tue, 17 May 2022 03:23:13 +0000 Head commit of repository qt: 30f48af7ee824186fb168e30a4c06c43531cdba4 Timestamp of repository sam_c: Wed, 18 May 2022 11:48:05 +0000 Head commit of repository sam_c: ff26ee30d6429a0c3814acfde7596af28d0ff7b8 Timestamp of repository steam-overlay: Tue, 17 May 2022 03:23:15 +0000 Head commit of repository steam-overlay: d9dd07b19a17994512155d551807a1e0fdf3e9b7 sh dash 0.5.11.5 ld GNU ld (Gentoo 2.38 p4) 2.38 ccache version 4.6 [disabled] app-misc/pax-utils: 1.3.4::gentoo app-shells/bash: 5.1_p16::gentoo dev-java/java-config: 2.3.1::gentoo dev-lang/perl: 5.34.1-r3::gentoo dev-lang/python: 2.7.18_p15::gentoo, 3.8.13_p2::gentoo, 3.9.13::gentoo, 3.10.4_p1::gentoo, 3.11.0_beta1-r2::gentoo dev-lang/rust-bin: 1.60.0::gentoo dev-util/ccache: 4.6-r1::gentoo dev-util/cmake: 3.23.1::gentoo dev-util/meson: 0.62.1::gentoo sys-apps/baselayout: 2.8::gentoo sys-apps/sandbox: 2.29::gentoo sys-apps/systemd: 251_rc3::gentoo sys-devel/autoconf: 2.13-r1::gentoo, 2.71-r1::gentoo sys-devel/automake: 1.16.5::gentoo sys-devel/binutils: 2.38-r2::gentoo sys-devel/binutils-config: 5.4.1::gentoo sys-devel/clang: 13.0.1::gentoo, 14.0.3::gentoo sys-devel/gcc: 9.4.0::gentoo, 10.3.1_p20211126::gentoo, 11.3.0::gentoo, 12.1.1_pre20220514::gentoo sys-devel/gcc-config: 2.5-r1::gentoo sys-devel/libtool: 2.4.7::gentoo sys-devel/lld: 14.0.3::gentoo sys-devel/llvm: 13.0.1::gentoo, 14.0.3::gentoo sys-devel/make: 4.3::gentoo sys-kernel/linux-headers: 5.17-r1::gentoo (virtual/os-headers) sys-libs/glibc: 2.35-r5::gentoo Repositories: gentoo location: /var/db/repos/gentoo sync-type: git sync-uri: https://github.com/gentoo-mirror/gentoo.git priority: -1000 sync-git-clone-extra-opts: -b stable -c gc.reflogExpire=0 -c gc.reflogExpireUnreachable=0 -c gc.rerereresolved=0 -c gc.rerereunresolved=0 -c gc.pruneExpire=now sync-git-verify-commit-signature: yes crossdev location: /var/db/repos/crossdev masters: gentoo kde location: /var/db/repos/kde sync-type: git sync-uri: https://github.com/gentoo-mirror/kde.git masters: gentoo qt location: /var/db/repos/qt sync-type: git sync-uri: https://github.com/gentoo-mirror/qt.git masters: gentoo sam_c location: /var/db/repos/sam_c sync-type: git sync-uri: https://github.com/gentoo-mirror/sam_c.git masters: gentoo steam-overlay location: /var/db/repos/steam-overlay sync-type: git sync-uri: https://github.com/gentoo-mirror/steam-overlay.git masters: gentoo test location: /var/db/repos/test masters: gentoo ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="@FREE" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -march=native -fdiagnostics-color=always -frecord-gcc-switches" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-O2 -pipe -march=native -fdiagnostics-color=always -frecord-gcc-switches -D_GLIBCXX_ASSERTIONS" DISTDIR="/var/cache/distfiles" EMERGE_DEFAULT_OPTS="--keep-going --with-bdeps=y --complete-graph --deep --dynamic-deps=n --changed-deps=n --usepkg-exclude sys-fs/zfs --usepkg-exclude sys-fs/zfs-kmod --usepkg=y --implicit-system-deps=n --jobs=2 --load-average 32" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe -march=native -fdiagnostics-color=always -frecord-gcc-switches" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg-live cgroup clean-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox mount-sandbox multilib-strict network-sandbox news parallel-fetch parallel-install pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms split-log strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe -march=native -fdiagnostics-color=always -frecord-gcc-switches" GENTOO_MIRRORS="http://mirror.bytemark.co.uk/gentoo/ http://www.mirrorservice.org/sites/distfiles.gentoo.org/ http://mirrors.soeasyto.com/distfiles.gentoo.org/" LANG="en_GB.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--defsym=__gentoo_check_ldflags__=0" LINGUAS="en en_GB" PKGDIR="/var/cache/binpkgs" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" SHELL="/bin/bash" USE="PIC X a52 aac acl acpi activities aes alsa amd64 avx avx2 bash-completion bluetooth branding bzip2 cairo caps cdda cdr clang crypt dbus declarative dist-kernel dri dts dvd dvdr emacs encode exif f16c filecaps firewalld flac fma3 freetype gif gmp gpm graphite gtk gui hardened harfbuzz hunspell iconv icu ipv6 jit jpeg kde kdesu kwallet lcms libglvnd libnotify libtirpc llvm-libunwind mad mmx mmxext mng mp3 mp4 mpeg multilib ncurses nftables nls nptl ogg opengl openmp pam pango pclmul pcre pdf pgo pie pipewire plasma png policykit popcnt ppds pulseaudio qml qt5 rdrand readline screencast sdl seccomp semantic-desktop sha spell sse sse2 sse3 sse4_1 sse4_2 sse4a ssl ssp ssse3 startup-notification svg system-av1 system-binutils system-boost system-bootstrap system-cairo system-clang system-digest system-ffmpeg system-harfbuzz system-heimdal system-icu system-jpeg system-leveldb system-libevent system-libs system-libvpx system-libyaml system-lz4 system-mitkrb5 system-sqlite system-ssl system-tbb system-uulib system-webp system-zlib systemd threads tiff truetype udev udisks unicode upower usb verify-sig vorbis vulkan wayland widgets x264 xattr xcb xml xtpax xv xvid zfs zlib zsh-completion" ABI_X86="32 64" ADA_TARGET="gnat_2020" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en en-GB" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LLVM_TARGETS="AArch64 AMDGPU ARM AVR BPF Hexagon Lanai MSP430 Mips NVPTX PowerPC RISCV Sparc SystemZ WebAssembly X86 XCore" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-4 php8-0" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_10" PYTHON_TARGETS="python3_9 pypy3 python3_10 python3_11 python3_8" QEMU_USER_TARGETS="aarch64 hppa ppc64le sparc64" RUBY_TARGETS="ruby30" USERLAND="GNU" VIDEO_CARDS="amdgpu radeonsi radeon" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LEX, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, MAKEOPTS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
(Using local overlay for portage ebuild just for libselinux not supporting py3.11 yet, shouldn't be relevant here.)
As discussed on IRC, my guess is that the code path with the local whirlpool fallback hasn't been tested in years (because hashlib supplied one prior to openssl-3.x).
I think I managed to reproduce: * Check out tag portage-3.0.30 * apply the patch listed at the bottom * run './runtests lib/portage/tests/util/test_checksum.py' and you should get the following error: ====================================================================== ERROR: test_whirlpool (portage.tests.util.test_checksum.ChecksumTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/tj/sources/gentoo/portage/lib/portage/tests/__init__.py", line 248, in run testMethod() File "/home/tj/sources/gentoo/portage/lib/portage/tests/util/test_checksum.py", line 62, in test_whirlpool checksum_str(b"", "WHIRLPOOL"), File "/home/tj/sources/gentoo/portage/lib/portage/checksum.py", line 643, in checksum_str return hashfunc_map[hashname].checksum_str(data) File "/home/tj/sources/gentoo/portage/lib/portage/checksum.py", line 76, in checksum_str checksum.update(data) File "/home/tj/sources/gentoo/portage/lib/portage/util/whirlpool.py", line 49, in update WhirlpoolAdd(arg, len(arg) * 8, self.ctx) File "/home/tj/sources/gentoo/portage/lib/portage/util/whirlpool.py", line 2223, in WhirlpoolAdd b = (source[sourcePos] << sourceGap) & 0xFF IndexError: index out of range (once for each python version installed, I think) From d90fdd3beb2cb6ba33fdbed5d5e3b4a01a59f4ae Mon Sep 17 00:00:00 2001 From: Thomas Bracht Laumann Jespersen <t@laumann.xyz> Date: Thu, 19 May 2022 10:56:02 +0200 Subject: [PATCH] Force used of bundled whirlpool --- lib/portage/checksum.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/portage/checksum.py b/lib/portage/checksum.py index c19a50df9..d0f294b36 100644 --- a/lib/portage/checksum.py +++ b/lib/portage/checksum.py @@ -110,7 +110,7 @@ _generate_hash_function("SHA256", hashlib.sha256, origin="hashlib") _generate_hash_function("SHA512", hashlib.sha512, origin="hashlib") for local_name, hash_name in ( ("RMD160", "ripemd160"), - ("WHIRLPOOL", "whirlpool"), + #("WHIRLPOOL", "whirlpool"), # available since Python 3.6 ("BLAKE2B", "blake2b"), ("BLAKE2S", "blake2s"), @@ -270,7 +270,7 @@ if "RMD160" not in hashfunc_map or "WHIRLPOOL" not in hashfunc_map: for local_name, hash_name in ( ("RMD160", "RIPEMD160"), - ("WHIRLPOOL", "WHIRLPOOL"), + #("WHIRLPOOL", "WHIRLPOOL"), ): if local_name not in hashfunc_map and hasattr( mhash, "MHASH_%s" % hash_name -- 2.35.1
PR: https://github.com/gentoo/portage/pull/832
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=1f33f97b9e9c132d77d586d10bfe6ba0d3123050 commit 1f33f97b9e9c132d77d586d10bfe6ba0d3123050 Author: Thomas Bracht Laumann Jespersen <t@laumann.xyz> AuthorDate: 2022-05-19 09:50:05 +0000 Commit: Michał Górny <mgorny@gentoo.org> CommitDate: 2022-05-21 10:11:50 +0000 lib/portage/util: fix bundled whirlpool on empty bytestring input The WhirlpoolAdd function did not consider zero-length input, so calls to update(b'') would produce out-of-bounds errors. This was not covered by any tests, because the constructor implicitly skipped the call to update on zero-length input. Add check for zero-length input to WhirlpoolAdd, and have the Whirlpool constructor skip calling update() only if arg is None. Closes: https://bugs.gentoo.org/846389 Signed-off-by: Thomas Bracht Laumann Jespersen <t@laumann.xyz> Closes: https://github.com/gentoo/portage/pull/832 Signed-off-by: Michał Górny <mgorny@gentoo.org> lib/portage/util/whirlpool.py | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=b0e43d6bc73cb63642404e661b073d6183fb9aa8 commit b0e43d6bc73cb63642404e661b073d6183fb9aa8 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-03-20 03:37:26 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-03-21 02:30:24 +0000 checksum: add further comments on existing fallback logic for RIPEMD/WHIRLPOOL Bug: https://github.com/python/cpython/issues/91257 Bug: https://github.com/python/cpython/issues/92876 Bug: https://bugs.gentoo.org/846389 Signed-off-by: Sam James <sam@gentoo.org> lib/portage/checksum.py | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
