After today's astonishingly problem-free update & rebuild for openssl-3.0.7 I just tried to update a custom ebuild by running 'ebuild bla.ebuld manifest'. This used to work yesterday and now fails, with the process running endlessly. I suspect the openssl update and the related python rebuilds; analysis below. Reproducible: Always Steps to Reproduce: 1. update to openssl-3.0.7-r1 (~amd64) 2. try to update an ebuild manifest Actual Results: emerge hangs endlessly Expected Results: successful manifest creation
Some ideas: - Could you try sending SIGUSR1 to Portage? It should drop into pdb and then we can at least get a backtrace. - What repository are you trying to create a Manifest in? Specifically which hashes are required by its metadata/layout.conf? We had trouble with Whirlpool before: bug 846389. - What entries are in the Manifest? (Interested in which hash types and general entry types specifically)
Investigation yielded the following: - the endlessly running emerge instance has the following frames (thanks nostrip + perf): 28.80% libpython3.11.so.1.0 [.] _PyEval_EvalFrameDefault 6.37% libpython3.11.so.1.0 [.] x_divrem 6.03% libpython3.11.so.1.0 [.] long_bitwise 4.58% libpython3.11.so.1.0 [.] binary_op1 4.20% libpython3.11.so.1.0 [.] long_rshift1 4.18% libpython3.11.so.1.0 [.] arena_map_is_used 4.13% libpython3.11.so.1.0 [.] _PyLong_New 3.87% libpython3.11.so.1.0 [.] l_mod 3.59% libpython3.11.so.1.0 [.] _PyObject_Free 3.08% libpython3.11.so.1.0 [.] _PyObject_Malloc 2.89% libpython3.11.so.1.0 [.] maybe_small_long 2.04% libpython3.11.so.1.0 [.] long_mod 1.90% libpython3.11.so.1.0 [.] _Py_NewReference 1.89% libpython3.11.so.1.0 [.] long_rshift 1.57% libpython3.11.so.1.0 [.] long_and 1.41% libpython3.11.so.1.0 [.] PyObject_Malloc 1.16% libpython3.11.so.1.0 [.] divmod_shift 1.15% libpython3.11.so.1.0 [.] PyLong_AsSsize_t 1.10% libpython3.11.so.1.0 [.] long_xor 0.94% libpython3.11.so.1.0 [.] _Py_NewReference@plt 0.93% libpython3.11.so.1.0 [.] PyNumber_Xor 0.82% libpython3.11.so.1.0 [.] PyNumber_Rshift 0.78% libpython3.11.so.1.0 [.] PyErr_CheckSignals 0.74% libpython3.11.so.1.0 [.] PyObject_Malloc@plt 0.74% libpython3.11.so.1.0 [.] _PyFrame_Clear 0.70% libpython3.11.so.1.0 [.] PyObject_Free 0.62% libpython3.11.so.1.0 [.] PyNumber_And 0.55% libpython3.11.so.1.0 [.] _PyErr_CheckSignalsTstate 0.54% libpython3.11.so.1.0 [.] object_dealloc 0.53% libpython3.11.so.1.0 [.] PyNumber_Remainder 0.53% libpython3.11.so.1.0 [.] PyThread_get_thread_ident 0.46% libpython3.11.so.1.0 [.] _Py_Dealloc@plt 0.39% libpython3.11.so.1.0 [.] _PyLong_Add 0.35% libpython3.11.so.1.0 [.] _PyLong_FromMedium 0.29% libpython3.11.so.1.0 [.] long_lshift1 0.25% libc.so.6 [.] __GI___pthread_self 0.22% libpython3.11.so.1.0 [.] _PyLong_Subtract 0.20% libpython3.11.so.1.0 [.] _PyFrame_Push 0.19% libpython3.11.so.1.0 [.] PyLong_FromLong 0.18% libpython3.11.so.1.0 [.] _Py_Dealloc 0.18% libpython3.11.so.1.0 [.] 0x00000000000f90d0 0.17% [kernel] [k] read_hpet 0.16% libpython3.11.so.1.0 [.] PyNumber_AsSsize_t Yeah, no idea what it is doing other than allocating/deallocating stuff and parsing numbers? - it happens with both python-3.11.1 and 3.10.9 - python was built with gcc-12.2.1_p20221210, so suspect is a recent gcc update and python being rebuilt with it. I rebuilt python-3.11 with clang and the problem is still there, so the last gcc update is probably not the cause. - it happens on both my old SandyBridge box and my Zen2 laptop, so behaviour is consistent (both are mostly consistent with each other, so that's good) - kernel 6.1.0 everywhere which has not shown any other problems so far (everything else has been working fine since yesterday). I'm open to suggestions because this has me stumped. :(
Cheers sam - - the repo is my local overlay where I've just tried to revbump a local version of prometheus-bin, nothing special - the Manifest: cat Manifest AUX prometheus.confd 43 SHA256 940f1a3ad08f0eb370951b1a42c29a9217cba9ae2f5cb6cc675bdec1d51e8309 SHA512 37468e897cd34b50b0edc65ada3f0c2e05459c3d96bb7f82d58b6de5bf10a127f1bf7912af93d83d94e3ea7287cfeb2311e36535f291d01a11d152a0b438591c WHIRLPOOL b8bdd0bc16fde3fb99cf8939427781a1b250df7e4a607fb8c780f75d502e7d2aa76aef6a1b4c082d3aa32f0a9a21b3b39df72fcf5e6d26592df43661f8d08169 AUX prometheus.initd 929 SHA256 01e2d326421b96986453765ac6615ab5f4be91b540dcf5799cfec6e844c90a07 SHA512 504122eb8ed19beac81987b77eb78bba9b36b2ef15f28b1a542804271046e06363baf464f3c303982f647eba65fbc2b5459235f56625e1ff3fed9834937cd0d2 WHIRLPOOL d990a218d4c8737e3703745bc28b7d06c6ab4e5e4433d37bbf8e034ae376b722fa16e7f227c9b2b74f4910edcbc41c7f1e88e74023756fb0a278c54ed7960b70 AUX prometheus.service 534 SHA256 01b8498b5bf4379210cdc0bc40210e4142a6a249fcebb6a9a841aa8e6c8834f6 SHA512 84eaffba13c030dc7b2b9dcdf4c96eb12ea5ca16e569ea4f9d4647b1bf912c00826f12cabf02cd97c0e2e76646ab9f179429a92bc32d49ba33c6360ec3cfce8f WHIRLPOOL 2e2b84591ffadb92845fec0b884bf86ce963585cdfde76df0071b86c5038534b9060f0d56b5b040d661f9de471ce817c5cd4a848eaa197f18c9fe35d0ed20ec0 DIST prometheus-2.40.6.linux-amd64.tar.gz 87797143 SHA256 a14b049b2f0f3b1ea0251a34fe8a01a835e6ba494c5bb5ea4014104eba7b956a SHA512 008c489a525a92d94ecc2b9a16859964e07ecf838a47fa7e18b687793b82f3a316136b4dadeabb0c426cdc8c0830265f8e4a3a13e4b7235a3d5738ba15910b09 WHIRLPOOL 1a038e4b9db98fd1c8b72d9cec899299c05cb5f5ab86a1d1144937c47a72db88c28823ee082dddb04ec3dc8620d8ea7329f1b791d30085761dc9dfffcb0381e8 EBUILD prometheus-bin-2.40.6.ebuild 1569 SHA256 af910f1d779131a0c364436aab7de57b572cbfb42cbb153077d79726b523b73f SHA512 c1a43c6a1d587f5ce70fbc41c5e5d89bd804c9edbc75b17d7767beb852ab82d6746c6e1a4c33680198e964d833eaf6603eccf6855fd75d667a22d686abb11dd4 WHIRLPOOL 9f46ab950d28d1eed9f269bdd0b91622aa6bd36169fec39da4f99228945cd084ffce8da2799b9350c5e33577c73e466ae2008fa43c245e74e6ee204bac2f8a3e MISC metadata.xml 350 SHA256 7ff90fe8a4b59aff98c7d3b2caa526b92680c313541ff545d156920b3a9664f3 SHA512 8766671859085c659b8f33678bad7d75009e49b66764a7db1157b4ab644e648f9678551100c21b834ff97723b3020f4d7968a12d79867cbcd5dc323ed4fe0256 WHIRLPOOL 5bfd8f69015b69f76c8579875c867f280d78412f55f2766621c6295ac05474e8f8f3642bec4f2261a27fcbdc9ba5620bdda582d46fedc6a294724e04b2941ead - it also hangs when I remove the Manifest and try to recreate from scratch - I did the USR1 thing and got: holger>ebuild prometheus-bin-2.40.7.ebuild manifest >>> Creating Manifest for /home/holger/Projects/portage/app-metrics/prometheus-bin --Return-- > /usr/lib/python-exec/python3.11/ebuild(31)debug_signal()->None -> pdb.set_trace() (Pdb) bt /usr/lib/python-exec/python3.11/ebuild(402)<module>() -> a = portage.doebuild( /usr/lib/python3.11/site-packages/portage/package/ebuild/doebuild.py(1400)doebuild() -> return not digestgen(mysettings=mysettings, myportdb=mydbapi) /usr/lib/python3.11/site-packages/portage/package/ebuild/digestgen.py(191)digestgen() -> mf.create( /usr/lib/python3.11/site-packages/portage/manifest.py(576)create() -> self.fhashdict["DIST"][f] = perform_multiple_checksums( /usr/lib/python3.11/site-packages/portage/checksum.py(615)perform_multiple_checksums() -> rVal[x] = perform_checksum(filename, x, calc_prelink)[0] /usr/lib/python3.11/site-packages/portage/checksum.py(576)perform_checksum() -> myhash, mysize = hashfunc_map[hashname].checksum_file(myfilename) /usr/lib/python3.11/site-packages/portage/checksum.py(93)checksum_file() -> checksum.update(data) /usr/lib/python3.11/site-packages/portage/util/whirlpool.py(46)update() -> WhirlpoolAdd(arg, len(arg) * 8, self.ctx) /usr/lib/python3.11/site-packages/portage/util/whirlpool.py(2212)WhirlpoolAdd() -> processBuffer(ctx) /usr/lib/python3.11/site-packages/portage/util/whirlpool.py(2320)processBuffer() -> L[0] = CDo(state, 0, 7, 6, 5, 4, 3, 2, 1) ^ K[0] /usr/lib/python3.11/site-packages/portage/util/whirlpool.py(2280)CDo() -> ^ C7[((buf[a7] >> 0) % 0x100000000) & 0xFF] > /usr/lib/python-exec/python3.11/ebuild(31)debug_signal()->None -> pdb.set_trace() "What fresh hell is this?"
My overlay's layout.conf says: manifest-hashes = SHA256 SHA512 WHIRLPOOL I removed the WHIRLPOOL and it immediately passes \o/
Excellent! (Not excellent, but we got to the bottom of it quickly at least...) The frustrating thing is, it turns out that the Whirlpool implementation in Portage was pretty much never used because < openssl 3 included it by default, but then openssl 3 shifts it into the "legacy" provider which has to be explicitly loaded. so I'm guessing the fix for bug 846389 wasn't enough. If it is that, there's a fun second question about how your case is different from the test which started passing :)
(In reply to Sam James from comment #5) > The frustrating thing is, it turns out that the Whirlpool implementation in > Portage was pretty much never used because < openssl 3 included it by > default, but then openssl 3 shifts it into the "legacy" provider which has > to be explicitly loaded. This is hilarious because I spent almost an hour this morning trying to make the legacy providers work and couldn't (because apache and an old self-signed cert and aargh). In the end I just gave up, created a new cert and moved on. > so I'm guessing the fix for bug 846389 wasn't enough. If it is that, there's > a fun second question about how your case is different from the test which > started passing :) Should it then not work with removed Manifest? I guess I just keep WHIRLPOOL disabled in my overlay for now.
Hi, I'd like to see if I can reproduce this (so far no luck). Can I ask for a copy of the ebuild you were using? If I'm reading the stack trace right, it should be possible to get this behaviour by just calling checksum_file() in from checksum.py. Until now the only impression I got was that the bundled whirlpool is significantly slower than hashlib's.
(In reply to Thomas Bracht Laumann Jespersen from comment #7) > Hi, I'd like to see if I can reproduce this (so far no luck). Can I ask for > a copy of the ebuild you were using? Sure, it's nothing secret. Can be downloaded from: https://github.com/hhoffstaette/portage/blob/master/app-metrics/prometheus-bin/prometheus-bin-2.40.7.ebuild Good luck!
I don't see the version of sys-apps/portage mentioned anywhere. Could you please provide emerge --info for completeness?
(In reply to Mike Gilbert from comment #9) > I don't see the version of sys-apps/portage mentioned anywhere. Could you > please provide emerge --info for completeness? It's all ~amd64: $emerge --info Portage 3.0.41 (python 3.11.1-final-0, default/linux/amd64/17.1/desktop, gcc-12, glibc-2.36-r6, 6.1.0 x86_64) ================================================================= System uname: Linux-6.1.0-x86_64-AMD_Ryzen_7_PRO_4750U_with_Radeon_Graphics-with-glibc2.36 KiB Mem: 32219764 total, 21317428 free KiB Swap: 1048572 total, 1048572 free Timestamp of repository gentoo: Thu, 15 Dec 2022 15:47:02 +0000 Head commit of repository gentoo: 6d819ea42a4a04cc7ea22f8b0c8591f765d45c5a sh bash 5.2_p15 ld GNU ld (Gentoo 2.39 p5) 2.39.0 ccache version 4.7.4 [disabled] app-misc/pax-utils: 1.3.5::gentoo app-shells/bash: 5.2_p15::gentoo dev-java/java-config: 2.3.1::gentoo dev-lang/perl: 5.36.0-r1::gentoo dev-lang/python: 3.10.9::gentoo, 3.11.1::gentoo dev-lang/rust-bin: 1.65.0::gentoo dev-util/ccache: 4.7.4::gentoo dev-util/cmake: 3.25.1::gentoo dev-util/meson: 0.64.1::gentoo sys-apps/baselayout: 2.9::gentoo sys-apps/openrc: 0.45.2-r2::gentoo sys-apps/sandbox: 2.29::gentoo sys-devel/autoconf: 2.71-r5::gentoo sys-devel/automake: 1.16.5::gentoo sys-devel/binutils: 2.39-r4::gentoo sys-devel/binutils-config: 5.4.1::gentoo sys-devel/clang: 14.0.6-r1::gentoo, 15.0.6::gentoo sys-devel/gcc: 12.2.1_p20221210::gentoo sys-devel/gcc-config: 2.8::gentoo sys-devel/libtool: 2.4.7-r1::gentoo sys-devel/lld: 15.0.6::gentoo sys-devel/llvm: 14.0.6-r2::gentoo, 15.0.6::gentoo sys-devel/make: 4.4::gentoo sys-kernel/linux-headers: 6.0::gentoo (virtual/os-headers) sys-libs/glibc: 2.36-r6::gentoo Repositories: gentoo location: /var/db/repos/gentoo sync-type: git sync-uri: https://github.com/gentoo-mirror/gentoo.git priority: -1000 sync-git-pull-extra-opts: --quiet hhoffstaette location: /home/holger/Projects/portage masters: gentoo priority: 100 ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-pipe -march=native -fno-semantic-interposition -O2" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/maven-3.8/conf" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-pipe -march=native -fno-semantic-interposition -O2" DISTDIR="/var/cache/distfiles" EMERGE_DEFAULT_OPTS="--binpkg-respect-use=y --deep --keep-going --nospinner --quiet-build --with-bdeps=y" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news noinfo parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS=" http://mirror.eu.oneandone.net/linux/distributions/gentoo/gentoo/ http://mirror.netcologne.de/gentoo/ http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/ http://ftp.halifax.rwth-aachen.de/gentoo/ http://ftp.fau.de/gentoo http://ftp-stud.hs-esslingen.de/pub/Mirrors/gentoo/" INSTALL_MASK="/usr/share/gtk-doc /usr/share/locale" LANG="C.utf8" LDFLAGS="-Wl,-O1,--as-needed,-z,now" LEX="flex" LINGUAS="en" MAKEOPTS="-j8 -l8" PKGDIR="/var/cache/distfiles/pkg" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/tmp" SHELL="/bin/zsh" USE="X a52 aac acl acpi alsa amd64 branding btrfs bzip2 cacert cairo caps cli crypt cups dbus dri dts elogind encode exif flac gif gtk gui http2 hwloc iconv icu io-uring ipv6 jpeg lcms libglvnd libnotify libtirpc lm-sensors logrotate lz4 mad maxminddb mng mp3 mp4 mpeg natspec ncurses nptl nscd ogg opengl openmp pam pango pcre pdf png policykit ppds pulseaudio qt5 readline savedconfig seccomp split-usr ssl startup-notification svg system-bootstrap system-sqlite system-zlib test-rust tiff truetype udev udisks unicode upower urandom usb vaapi vdpau vorbis wxwidgets x264 xattr xcb xml xv xvid xxhash zlib zsh-completion zstd" ABI_X86="64" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev" KERNEL="linux" L10N="en" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="luajit" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-4 php8-0" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_11" RUBY_TARGETS="ruby27" USERLAND="GNU" VIDEO_CARDS="amdgpu radeonsi" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, LC_ALL, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
I can reproduce the "hang" locally with the Portage master branch. However, I think it is just very slow, and your distfile is fairly large. For reference, it took 45 seconds for Portage to generate a WHIRLPOOL hash for coreutils-9.1.tar.xz, which has a size of 5.5 MB. prometheus-2.40.7.linux-amd64.tar.gz is 84 MB; it would likely take several minutes to complete a WHIRLPOOL hash on the same hardware.
I took my system over 11 minutes to compute the WHIRLPOOL hash for prometheus-2.40.7.linux-amd64.tar.gz. > % time ebuild prometheus-bin-2.40.7.ebuild digest > Appending /home/floppym/tmp/foo to PORTDIR_OVERLAY... > >>> Creating Manifest for /home/floppym/tmp/foo/app-metrics/prometheus-bin > ebuild prometheus-bin-2.40.7.ebuild digest 685.41s user 0.21s system 99% cpu 11:26.20 total
Wow! I let it run for a minute or two while debugging but never thought it would be *this* slow. If the python fallback implementation is so slow then maybe it should just be removed? Maybe the dynamic loading of providers would work (see https://wiki.openssl.org/index.php/OpenSSL_3.0#Providers) but I don't know if that works when the legacy providers are not explicitly configured, which probably makes it a no-go. Anyway I simply resorted to BLAKE/SHA512 which is what the mainline gentoo repo uses and both are plenty fast. Maybe it's just time to kick WHIRLPOOL out, I only found all this by accident.
(In reply to Holger Hoffstätte from comment #13) > Maybe it's just time to kick WHIRLPOOL out, I only found all this by accident. Yeah, it's probably not worth spending effort to continue supporting an obsolete hash that isn't used in the gentoo repo anyway.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=cf6b3f9482248782ee2468ce4d550768e0f5d87f commit cf6b3f9482248782ee2468ce4d550768e0f5d87f Author: Sam James <sam@gentoo.org> AuthorDate: 2022-12-29 19:36:29 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-29 19:36:29 +0000 NEWS: update for Whirlpool fixes Bug: https://bugs.gentoo.org/885909 Signed-off-by: Sam James <sam@gentoo.org> NEWS | 13 +++++++++++++ 1 file changed, 13 insertions(+)
commit cf6b3f9482248782ee2468ce4d550768e0f5d87f (HEAD -> master, origin/master, origin/HEAD) Author: Sam James <sam@gentoo.org> Date: Thu Dec 29 19:36:29 2022 +0000 NEWS: update for Whirlpool fixes Bug: https://bugs.gentoo.org/885909 Signed-off-by: Sam James <sam@gentoo.org> commit 45f2da735a473d8b800861f97d7be1ba7850187f Author: Michał Górny <mgorny@gentoo.org> Date: Thu Dec 29 18:04:24 2022 +0100 portage.util: Add C implementation of Whirlpool hash Add an extension module providing Whirlpool hash based on the reference C implementation. This is necessary since Python hashlib no longer provides Whirlpool by default with OpenSSL 3, and the Python implementation used by Portage is unusably slow. Signed-off-by: Michał Górny <mgorny@gentoo.org> Closes: https://github.com/gentoo/portage/pull/966 Signed-off-by: Sam James <sam@gentoo.org>
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=1d78c5d85797399b0dbd0fe885d04b7f50fc3a60 commit 1d78c5d85797399b0dbd0fe885d04b7f50fc3a60 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-01-02 05:31:32 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-01-02 05:31:42 +0000 sys-apps/portage: add 3.0.43 Closes: https://bugs.gentoo.org/888487 Closes: https://bugs.gentoo.org/887817 Closes: https://bugs.gentoo.org/885909 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.43.ebuild | 292 +++++++++++++++++++++++++++++++++ 2 files changed, 293 insertions(+)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=992cf6e5ecbc8de428792fe27bb821d04e03097c commit 992cf6e5ecbc8de428792fe27bb821d04e03097c Author: Sam James <sam@gentoo.org> AuthorDate: 2023-03-21 02:29:34 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-03-21 02:30:24 +0000 NEWS: update for checksum cleanups Bug: https://bugs.gentoo.org/597736 Bug: https://bugs.gentoo.org/615620 Bug: https://bugs.gentoo.org/885909 Bug: https://bugs.gentoo.org/888884 Signed-off-by: Sam James <sam@gentoo.org> NEWS | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) https://gitweb.gentoo.org/proj/portage.git/commit/?id=2d76645750ec5ee12a2eadbdc866ed0e4310e443 commit 2d76645750ec5ee12a2eadbdc866ed0e4310e443 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-03-20 03:30:35 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-03-21 02:30:24 +0000 checksum: update WHIRLPOOL comment for provider preference Since 45f2da735a473d8b800861f97d7be1ba7850187f, we use a bundled C implementation before falling back to the bundled pure Python impl. Bug: https://bugs.gentoo.org/885909 Signed-off-by: Sam James <sam@gentoo.org> lib/portage/checksum.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) https://gitweb.gentoo.org/proj/portage.git/commit/?id=377a31434ae51b6e6465ba48b5132eb200a8cf40 commit 377a31434ae51b6e6465ba48b5132eb200a8cf40 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-03-20 03:16:37 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-03-21 02:30:23 +0000 checksum: drop STREEBOG{256,512} & pygost support For STREEBOG{256,512}, we had three options that we "supported": 1. using pygost; pygost is unpackaged in Gentoo and appears to, as of March 2023, been yanked from pypi too. Hence not only can we test it on the distribution where we do our development (Gentoo), but we also can't acquire it for CI easily either. 2. using pygcrypt; No longer in Gentoo and the codepath for this has been disabled since 2017(!) because of https://bugs.gentoo.org/615620. 3. a slow fallback implementation. This is not something we can rely on anyway, but this is especially wrong given the issues we've had recently with WHIRLPOOL. See https://bugs.gentoo.org/885909. As for whether we should support STREEBOG at all: - Per above, we don't have a decent implementation for it available. - We're not testing it very well. - Jugging obscure hash algorithms has become a pain because some of them have ended up being marked "legacy" or removed from OpenSSL by default. - I don't think we should have hash algorithms supported just for the sake of it. We're not in the business of supporting every single hash algorithm that a library with Python bindings out there provides. Bug: https://bugs.gentoo.org/597736 Bug: https://bugs.gentoo.org/615620 Bug: https://bugs.gentoo.org/885909 Bug: https://bugs.gentoo.org/888884 Signed-off-by: Sam James <sam@gentoo.org> lib/portage/checksum.py | 25 +------------------------ lib/portage/tests/util/test_checksum.py | 26 -------------------------- tox.ini | 1 - 3 files changed, 1 insertion(+), 51 deletions(-)
