INFO:root:Manifest timestamp: 2018-02-01 22:38:16 UTC INFO:root:Valid OpenPGP signature found: INFO:root:- primary key: DCD05B71EAB94199527F44ACDB6B8C1F96D8BF6D INFO:root:- subkey: E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250 INFO:root:- timestamp: 2018-02-01 22:38:16 UTC INFO:root:Verifying /var/cache/portage/gentoo... ERROR:root:Manifest mismatch for games-puzzle/numptyphysics/Manifest BLAKE2B: expected: ad556c1e81ba4ca422332b6162717826e81f4610fe847a13bceb3bcf09b2c8830e7e0241275f7a22cc006a91718da01a2e51812c89ca5ffbf19483f8c2776540, have: 7664a2fbee3f601774678f34c41d0c682f04877802ccb594d7fb9f68516c8ee13230f62d4aeaa3dd9be4f2fb2b8e778e0de00d5697ccfa06bb3b99a882310229
Server was: >>> Syncing repository 'gentoo' into '/var/cache/portage/gentoo'... >>> Starting rsync with rsync://[2a01:4f8:140:5123::2]/gentoo-portage... >>> Checking server timestamp ... This Gentoo Linux Portage tree mirror is provided by amiconsult GmbH in Karlsruhe, Germany. Please direct any questions regarding this system to Timo Rothweiler <tr.bgo@gmx.de>. System data: Link speed: 100 MBit/s (full duplex) CPU: Intel Core i7 920 RAM: 8 GiB Max Connections: 50 IP: 88.198.69.178 Location: Nuremberg, Germany
And this server: >>> Syncing repository 'gentoo' into '/var/cache/portage/gentoo'... >>> Starting rsync with rsync://[2a03:b0c0:3:d0::c3:f001]/gentoo-portage... >>> Checking server timestamp ... rsync8.de.gentoo.org IPv4: 46.101.221.22 IPv6: 2a03:b0c0:3:d0::c3:f001 Contact: rsync@moonitor.org Ships this broken Manifest: INFO:root:Manifest timestamp: 2018-02-01 22:38:16 UTC INFO:root:Valid OpenPGP signature found: INFO:root:- primary key: DCD05B71EAB94199527F44ACDB6B8C1F96D8BF6D INFO:root:- subkey: E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250 INFO:root:- timestamp: 2018-02-01 22:38:16 UTC INFO:root:Verifying /var/cache/portage/gentoo... ERROR:root:Manifest mismatch for dev-texlive/texlive-pstricks/Manifest BLAKE2B: expected: 1fc1fc5dc928b6e9c55ae58a273975ff509022e4b1577cb60d9d354d9df1d0c22d8932dc9c9aaaa639afc12c851c4177ace3892517408a6eef8e2d2ac74f3f65, have: 701f736bb95a409761ae4d4df66b209356a88ecd640376377fd2f8ec407250b4e506b847ca2a5c6ef57bc8315398f80c2e5216794e278c2158aab931c48db465
This time I hit the first server again (comment #1), but a different Manifest did not check out: INFO:root:Manifest timestamp: 2018-02-01 22:38:16 UTC INFO:root:Valid OpenPGP signature found: INFO:root:- primary key: DCD05B71EAB94199527F44ACDB6B8C1F96D8BF6D INFO:root:- subkey: E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250 INFO:root:- timestamp: 2018-02-01 22:38:16 UTC INFO:root:Verifying /var/cache/portage/gentoo... ERROR:root:Manifest mismatch for dev-haskell/keys/Manifest BLAKE2B: expected: 71c82a3d6722836b19f0469e1ee6f19f21c23c360431c82e1ed566fda531ca018902dc95022d79c788fc21df6c1d5618fae06d598105cdd0774cb4f7d385c049, have: 53abb7445ba7eff813c2a9405fe9ddc756eb69193dccb0a2d304d48cca6e30b9eeb85e117d4beaf27dd68d9315c587f3c7de49553cf0b01c440ddfe8479c7797
>>> Starting rsync with rsync://[2a03:b0c0:3:d0::c3:f001]/gentoo-portage... >>> Checking server timestamp ... rsync8.de.gentoo.org IPv4: 46.101.221.22 IPv6: 2a03:b0c0:3:d0::c3:f001 Contact: rsync@moonitor.org [...] INFO:root:Refreshing keys from keyserver... INFO:root:Keys refreshed. INFO:root:Manifest timestamp: 2018-02-02 19:38:18 UTC INFO:root:Valid OpenPGP signature found: INFO:root:- primary key: DCD05B71EAB94199527F44ACDB6B8C1F96D8BF6D INFO:root:- subkey: E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250 INFO:root:- timestamp: 2018-02-02 19:38:18 UTC INFO:root:Verifying /var/cache/portage/gentoo... ERROR:root:Manifest mismatch for dev-texlive/texlive-metapost/Manifest BLAKE2B: expected: 053af954a38163c20e37335692e5396ca7a6fbdabc7bb02a69ba3b2698104b56f42c660ecde36a8d01781c60a11b51ac580d68b3ab33cc0fa57908ca54f8500b, have: f7ab0aa4b2406ef69df334a1f9dc316068a039ad9c49d88c97270a10bbc4d46cab47317dac6e002815c90f2a94d6cd49d91fb4d12d07864a9adeb4088305ac46
I now went from: sync-uri = rsync://rsync.de.gentoo.org/gentoo-portage To: sync-uri = rsync://rsync.gentoo.org/gentoo-portage But the issue affects those, too: >>> Syncing repository 'gentoo' into '/var/cache/portage/gentoo'... >>> Starting rsync with rsync://[2a01:90:200:10::1a]/gentoo-portage... >>> Checking server timestamp ... Welcome to starling.gentoo.org / rsync.gentoo.org Server Address : 81.91.253.252, 2a01:90:200:10::1a Contact Name : mirror-admin@gentoo.org Hardware : 2 x Intel(R) Xeon(R) CPU E5-2470 0 @ 2.30GHz, 3948MB RAM Sponsor : Qube Managed Services Limited, Zurich, Switzerland, EU [...] INFO:root:Refreshing keys from keyserver... INFO:root:Keys refreshed. INFO:root:Manifest timestamp: 2018-02-02 20:08:19 UTC INFO:root:Valid OpenPGP signature found: INFO:root:- primary key: DCD05B71EAB94199527F44ACDB6B8C1F96D8BF6D INFO:root:- subkey: E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250 INFO:root:- timestamp: 2018-02-02 20:08:19 UTC INFO:root:Verifying /var/cache/portage/gentoo... ERROR:root:Manifest mismatch for net-analyzer/raddump/Manifest BLAKE2B: expected: 0bd9a50e8b3ff713699affdc2ad5fcd67f7ba628fe6ddea7d96ddc92715d55ffbc25bcfa42876895105860493be7b1f3c2148e925b216984187d745a857d1dcc, have: a2a3b5da06410826eceb77a4f671bacd6b6c841ad6335f846bb58d35116707f747453e8b3dd4e1f8402f11e8fb8f6c674987ffb7e4ba910ac9fe515b61f0193f
Also, sometimes, it just hangs at "Verifying /var/cache/portage/gentoo..." forever. | | `-eix-sync,22506 /usr/bin/eix-sync | | `-emerge,22602 -b /usr/lib/python-exec/pypy/emerge --sync | | `-emerge,22626 -b /usr/lib/python-exec/pypy/emerge --sync | | `-gemato,23581 /usr/lib/python-exec/pypy/gemato verify -s /var/cache/portage/gentoo -K... | | |-gemato,23641 /usr/lib/python-exec/pypy/gemato verify -s /var/cache/portage/gentoo -K... | | |-gemato,23642 /usr/lib/python-exec/pypy/gemato verify -s /var/cache/portage/gentoo -K... | | |-gemato,23643 /usr/lib/python-exec/pypy/gemato verify -s /var/cache/portage/gentoo -K... | | |-gemato,23644 /usr/lib/python-exec/pypy/gemato verify -s /var/cache/portage/gentoo -K... | | `-{gemato},23646 root 23581 0.0 1.2 436056 186620 pts/1 SNl+ 21:42 0:02 /usr/bin/pypy /usr/lib/python-exec/pypy/gemato verify -s /var/cache/portage/gentoo -K /var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg root 23641 0.0 0.4 382200 75928 pts/1 SN+ 21:42 0:02 /usr/bin/pypy /usr/lib/python-exec/pypy/gemato verify -s /var/cache/portage/gentoo -K /var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg root 23642 0.0 0.5 392744 86068 pts/1 SN+ 21:42 0:02 /usr/bin/pypy /usr/lib/python-exec/pypy/gemato verify -s /var/cache/portage/gentoo -K /var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg root 23643 0.0 0.6 413276 106808 pts/1 SN+ 21:42 0:02 /usr/bin/pypy /usr/lib/python-exec/pypy/gemato verify -s /var/cache/portage/gentoo -K /var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg root 23644 0.0 0.5 385720 79592 pts/1 SN+ 21:42 0:02 /usr/bin/pypy /usr/lib/python-exec/pypy/gemato verify -s /var/cache/portage/gentoo -K /var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg
Portage 2.3.24 (python 2.7.13-final-42, default/linux/amd64/17.1/desktop/plasma/systemd, gcc-7.2.0, glibc-2.26-r5, 4.15.0-gentoo x86_64) ================================================================= System Settings ================================================================= System uname: Linux-4.15.0-gentoo-x86_64-AMD_A10-7800_Radeon_R7,_12_Compute_Cores_4C+8G-with-gentoo-2.4.1 KiB Mem: 15270076 total, 4583280 free KiB Swap: 0 total, 0 free Timestamp of repository gentoo: Sat, 03 Feb 2018 12:15:01 +0000 Head commit of repository gentoo: 41e870d20e85d86e61567a5afcbe996df1cef45c Head commit of repository flatpak-overlay: 74797c725c9eedd81acb7bcf357f1489d52fdb97 Head commit of repository riscv: 0cbfcd44cd9f216f4c4b27fbf3b09801f42c758f Timestamp of repository sage-on-gentoo: Fri, 02 Feb 2018 01:45:43 +0000 Head commit of repository sage-on-gentoo: 8f2d0d64fad51c13702d276fbd20e48bce239cb2 Timestamp of repository science: Mon, 29 Jan 2018 15:58:50 +0000 Head commit of repository science: ad5397ae722100552ebf55e8e4b2d5abf3bba8f4 Timestamp of repository steam-overlay: Thu, 01 Feb 2018 00:31:14 +0000 Head commit of repository steam-overlay: 84234b6f9cf78b7bff34616105e46fca3767e442 sh bash 4.4_p18 ld GNU ld (Gentoo 2.29.1 p3) 2.29.1 ccache version 3.3.4 [disabled] app-shells/bash: 4.4_p18::gentoo dev-java/java-config: 2.2.0-r3::gentoo dev-lang/perl: 5.26.1-r1::gentoo dev-lang/python: 2.7.14-r101::sage-on-gentoo, 3.5.4-r1::gentoo dev-util/ccache: 3.3.4-r1::gentoo dev-util/cmake: 3.10.2::gentoo dev-util/pkgconfig: 0.29.2::gentoo sys-apps/baselayout: 2.4.1-r2::gentoo sys-apps/sandbox: 2.12::gentoo sys-devel/autoconf: 2.13::gentoo, 2.69-r4::gentoo sys-devel/automake: 1.11.6-r2::gentoo, 1.15.1-r1::gentoo sys-devel/binutils: 2.29.1-r1::gentoo, 2.30::gentoo sys-devel/gcc: 7.2.0-r1::gentoo, 7.3.0::gentoo sys-devel/gcc-config: 1.9.1::gentoo sys-devel/libtool: 2.4.6-r4::gentoo sys-devel/make: 4.2.1-r1::gentoo sys-kernel/linux-headers: 4.15::gentoo (virtual/os-headers) sys-libs/glibc: 2.26-r5::gentoo Repositories: gentoo location: /var/cache/portage/gentoo sync-type: rsync sync-uri: rsync://rsync.de.gentoo.org/gentoo-portage priority: -1000 sync-rsync-extra-opts: sync-rsync-verify-metamanifest: yes flatpak-overlay location: /var/db/repos/flatpak-overlay sync-type: git sync-uri: git://github.com/fosero/flatpak-overlay.git masters: gentoo riscv location: /var/db/repos/riscv sync-type: git sync-uri: https://github.com/palmer-dabbelt/riscv-gentoo.git masters: gentoo sage-on-gentoo location: /var/db/repos/sage-on-gentoo sync-type: git sync-uri: https://github.com/gentoo-mirror/sage-on-gentoo.git masters: gentoo science science location: /var/db/repos/science sync-type: git sync-uri: https://github.com/gentoo-mirror/science.git masters: gentoo steam-overlay location: /var/db/repos/steam-overlay sync-type: git sync-uri: https://github.com/gentoo-mirror/steam-overlay.git masters: gentoo local location: /var/cache/portage/local masters: gentoo riscv priority: 100 ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-pipe -O2 -march=bdver3" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /etc/grs/systems.conf /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php7.2/ext-active/ /etc/php/cgi-php7.2/ext-active/ /etc/php/cli-php7.2/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-pipe -O2 -march=bdver3" DISTDIR="/var/cache/portage/distfiles" EMERGE_DEFAULT_OPTS="--keep-going --nospinner --verbose-conflicts" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs buildsyspkg cgroup compressdebug config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch parallel-install preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://ftp.spline.inf.fu-berlin.de/mirrors/gentoo/ http://ftp-stud.hs-esslingen.de/pub/Mirrors/gentoo/ http://distfiles.gentoo.org" LANG="en_GB.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu" LINGUAS="de en en_GB ar fa tr" MAKEOPTS="-j4" PKGDIR="/var/cache/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="X a52 aac aacplus aacs acl acpi activities alsa amd64 appindicator appstream archive audit avahi ayatana bash-completion bdplus berkdb bluetooth bluray branding brotli bs2b bzip2 cairo caps cdda cddb cdio cdr celt chromaprint cjk clang cli colord colorio conntrack crypt cups cxx d3d9 dbus declarative dirac djvu dri dts dvb dvd dvdr egl emboss encode epub exif fam fax fbcon ffmpeg fftw fish-completion fits flac fontconfig fortran fribidi ftdi gdbm geoclue geolocation gif git glamor gles gmp google googledrive gpg gpm gps graphicsmagick gstreamer gtk gtk3 harfbuzz hdf5 ibus iconv icu idn imlib inotify introspection ipv6 jemalloc jpeg jpeg2k kde kipi kwallet ladspa latex lcms ldap libidn2 libinput libnotify libproxy libsecret libsoxr lv2 lz4 lzma lzo mad markdown mercurial metis mjpeg mng mobi modemmanager modplug modules mp3 mp4 mpeg mplayer mtp multilib mysql ncurses netlink networkmanager nftables nls nptl office ofono ogg openal opencl openconnect opencv openexr opengl openh264 openmax openmp openvpn opus pam pango pcap pch pcre pcre2 pdf phonon plasma png policykit postscript ppds prison pulseaudio python qml qt3support qt4 qt5 raw readline rtmp samba scanner schroedinger sctp sdl seccomp semantic-desktop share snappy sparse speech speex spell ssl startup-notification svg systemd tbb tcpd teamd telepathy tga theora threads tiff timezone truetype tslib udev udisks unicode unwind upnp upnp-av upower usb v4l v4l2 vaapi vdpau vorbis vpnc vpx vulkan wavpack wayland webp widgets x264 x265 xattr xcb xcomposite xinerama xkb xml xmp xrandr xscreensaver xv xvid xwayland xz zeroconf zeromq zlib" ABI_X86="64" ALSA_CARDS="hda-intel" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_EXPERIMENTAL_FEATURES="stage" CALLIGRA_FEATURES="words sheets karbon plan" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx f16c fma3 fma4 mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 xop" ELIBC="glibc" ENLIGHTENMENT_MODULES="*" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64" INPUT_DEVICES="joystick libinput" KERNEL="linux" L10N="de en en-GB ar fa tr ja ko zh zh-CN zh-TW" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LIRC_DEVICES="devinput" LUA_TARGET="lua5-2" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6 php7-0" POSTGRES_TARGETS="postgres9_5" PYTHON_SINGLE_TARGET="python3_5" PYTHON_TARGETS="python2_7 python3_5 pypy pypy3" RUBY_TARGETS="ruby22 ruby23" STEAMGAMES="dirt_rally dont_starve portal source_engine te120 trine2 witcher2" USERLAND="GNU" VIDEO_CARDS="amdgpu virgl" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS ================================================================= Package Settings ================================================================= app-portage/gemato-11.0::gentoo was built with the following: USE="blake2 bzip2 gpg lzma -sha3 -test" ABI_X86="(64)" PYTHON_TARGETS="pypy python2_7 python3_5 -python3_4 -python3_6" sys-apps/portage-2.3.24::gentoo was built with the following: USE="(ipc) native-extensions rsync-verify xattr -build -doc -epydoc (-selinux)" ABI_X86="(64)" PYTHON_TARGETS="pypy python2_7 python3_5 -python3_4 -python3_6" dev-python/pyblake2-1.1.0::gentoo was built with the following: USE="" ABI_X86="(64)" PYTHON_TARGETS="pypy python2_7 python3_5 -python3_4 -python3_6"
(In reply to Dennis Schridde from comment #6) > Also, sometimes, it just hangs at "Verifying /var/cache/portage/gentoo..." > forever. In such cases I get backtraces like: INFO:root:Refreshing keys from keyserver... INFO:root:Keys refreshed. INFO:root:Manifest timestamp: 2018-02-05 07:08:19 UTC INFO:root:Valid OpenPGP signature found: INFO:root:- primary key: DCD05B71EAB94199527F44ACDB6B8C1F96D8BF6D INFO:root:- subkey: E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250 INFO:root:- timestamp: 2018-02-05 07:08:19 UTC INFO:root:Verifying /var/cache/portage/gentoo... ^CProcess PoolWorker-7: Traceback (most recent call last): Process PoolWorker-6: Traceback (most recent call last): Exiting on signal 2 Process PoolWorker-5: File "/usr/lib64/pypy/lib-python/2.7/multiprocessing/process.py", line 258, in _bootstrap File "/usr/lib64/pypy/lib-python/2.7/multiprocessing/process.py", line 258, in _bootstrap Traceback (most recent call last): File "/usr/lib64/pypy/lib-python/2.7/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib64/pypy/lib-python/2.7/multiprocessing/process.py", line 114, in run self._target(*self._args, **self._kwargs) File "/usr/lib64/pypy/lib-python/2.7/multiprocessing/pool.py", line 102, in worker self.run() File "/usr/lib64/pypy/lib-python/2.7/multiprocessing/process.py", line 114, in run self.run() File "/usr/lib64/pypy/lib-python/2.7/multiprocessing/process.py", line 114, in run self._target(*self._args, **self._kwargs) File "/usr/lib64/pypy/lib-python/2.7/multiprocessing/pool.py", line 117, in worker put((job, i, result)) File "/usr/lib64/pypy/lib-python/2.7/multiprocessing/queues.py", line 390, in put task = get() wacquire() KeyboardInterrupt File "/usr/lib64/pypy/lib-python/2.7/multiprocessing/queues.py", line 376, in get racquire() KeyboardInterrupt Traceback (most recent call last): File "/usr/lib/python-exec/pypy/gemato", line 11, in <module> load_entry_point('gemato==11.0', 'console_scripts', 'gemato')() self._target(*self._args, **self._kwargs) File "/usr/lib64/pypy/lib-python/2.7/multiprocessing/pool.py", line 102, in worker task = get() File "/usr/lib64/pypy/site-packages/gemato/cli.py", line 358, in setuptools_main sys.exit(main(sys.argv)) File "/usr/lib64/pypy/site-packages/gemato/cli.py", line 350, in main return vals.func(vals, argp) File "/usr/lib64/pypy/lib-python/2.7/multiprocessing/queues.py", line 376, in get racquire() File "/usr/lib64/pypy/site-packages/gemato/cli.py", line 92, in do_verify ret &= m.assert_directory_verifies(relpath, **kwargs) File "/usr/lib64/pypy/site-packages/gemato/recursiveloader.py", line 586, in assert_directory_verifies KeyboardInterrupt entry_dict = self.get_file_entry_dict(path) File "/usr/lib64/pypy/site-packages/gemato/recursiveloader.py", line 519, in get_file_entry_dict verify=verify_manifests) File "/usr/lib64/pypy/site-packages/gemato/recursiveloader.py", line 406, in load_manifests_for_path pool.terminate() File "/usr/lib64/pypy/lib-python/2.7/multiprocessing/pool.py", line 465, in terminate self._terminate() File "/usr/lib64/pypy/lib-python/2.7/multiprocessing/util.py", line 207, in __call__ res = self._callback(*self._args, **self._kwargs) File "/usr/lib64/pypy/lib-python/2.7/multiprocessing/pool.py", line 500, in _terminate_pool outqueue.put(None) # sentinel File "/usr/lib64/pypy/lib-python/2.7/multiprocessing/queues.py", line 390, in put wacquire() KeyboardInterrupt
This affects much more than just dev-texlive/texlive-metapost and the affected package appears to vary between runs. (Or many are broken and Portage just non-deterministically bails out on one of them.)
not sure whom to assign to, but it's definitely not texlive-related; sending to portage team as this seems to have to do with the signature verification
I've got a successful verification of rsync://rsync8.de.gentoo.org/gentoo-portage here: INFO:root:Refreshing keys from keyserver... INFO:root:Keys refreshed. INFO:root:Manifest timestamp: 2018-02-06 19:08:24 UTC INFO:root:Valid OpenPGP signature found: INFO:root:- primary key: DCD05B71EAB94199527F44ACDB6B8C1F96D8BF6D INFO:root:- subkey: E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250 INFO:root:- timestamp: 2018-02-06 19:08:24 UTC I'm using dev-python/pyblake2-1.1.0. Please post the output of this command: python -c 'import portage.checksum; print(portage.checksum.hashorigin_map["BLAKE2B"])'
(In reply to Zac Medico from comment #11) > Please post the output of this command: > > python -c 'import portage.checksum; > print(portage.checksum.hashorigin_map["BLAKE2B"])' $ python -c 'import portage.checksum; print(portage.checksum.hashorigin_map["BLAKE2B"])' pyblake2
What version of dev-python/pyblake2 do you have?
(In reply to Zac Medico from comment #13) > What version of dev-python/pyblake2 do you have? (In reply to Dennis Schridde from comment #7) > dev-python/pyblake2-1.1.0::gentoo was built with the following: > USE="" ABI_X86="(64)" PYTHON_TARGETS="pypy python2_7 python3_5 -python3_4 > -python3_6"
Please attach one of the Manifest files that failed to verify, and include the verification error message for that Manifest.
Created attachment 518272 [details] Manifest (net-analyzer/nagios-check_mysql_health) Most recently: INFO:root:Refreshing keys from keyserver... INFO:root:Keys refreshed. INFO:root:Manifest timestamp: 2018-02-06 20:08:20 UTC INFO:root:Valid OpenPGP signature found: INFO:root:- primary key: DCD05B71EAB94199527F44ACDB6B8C1F96D8BF6D INFO:root:- subkey: E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250 INFO:root:- timestamp: 2018-02-06 20:08:20 UTC INFO:root:Verifying /var/cache/portage/gentoo... ERROR:root:Manifest mismatch for net-analyzer/nagios-check_mysql_health/Manifest BLAKE2B: expected: 38673b2d92e2a7f666268844d26cf85ae13a4094e35156a7ad79511a2af79f4ef948b34afd1e570454196ade6761d7b90c0ca2332aa330df5ef2ebd55078396e, have: dcb599eecfda691b8509bb5cd26f40f069f719e516b2a5fbfcbd1bcbdaac96bedf87913256932110bff621f93bba715615c9610bb3f4e9796503d71af95d9047 $ zgrep nagios-check_mysql_health /var/cache/portage/gentoo/net-analyzer/Manifest.gz MANIFEST nagios-check_mysql_health/Manifest 1581 BLAKE2B 38673b2d92e2a7f666268844d26cf85ae13a4094e35156a7ad79511a2af79f4ef948b34afd1e570454196ade6761d7b90c0ca2332aa330df5ef2ebd55078396e SHA512 10add6b34ba16b6ed2691fdeeecf2feaf486315b139a50baa51d7cbda8a5adadd77eb991d9510c13dd2d081bc892270603b81a136f069267bb07fe407975ee32
(In reply to Dennis Schridde from comment #16) > Created attachment 518272 [details] > Manifest (net-analyzer/nagios-check_mysql_health) The digest of this Manifest is correct. It looks like your installed instance of dev-python/pyblake2-1.1.0 has calculated an incorrect digest. You can use a command like this to double check the result and see if it's reproducible: python -c ' import portage.checksum print(portage.checksum.perform_multiple_checksums("/var/cache/portage/gentoo/net-analyzer/nagios-check_mysql_health/Manifest", hashes=["BLAKE2B"])) '
You can also use 'b2sum' to get another checksum for comparison.
CFLAGS="-pipe -O2 -march=bdver3" Try dropping the -march and recompile pyblake2, I for one wouldn't be surprised if it changes things, as I had similar problems in the past.
Also, you could start by building pyblake2 with FEATURES=test.
Failure of the day: INFO:root:Refreshing keys from keyserver... INFO:root:Keys refreshed. INFO:root:Manifest timestamp: 2018-02-07 07:38:22 UTC INFO:root:Valid OpenPGP signature found: INFO:root:- primary key: DCD05B71EAB94199527F44ACDB6B8C1F96D8BF6D INFO:root:- subkey: E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250 INFO:root:- timestamp: 2018-02-07 07:38:22 UTC INFO:root:Verifying /var/cache/portage/gentoo... ERROR:root:Manifest mismatch for dev-haskell/transformers/Manifest BLAKE2B: expected: f15c3d2e9c11376de9820cc12e6c8e8b339402c773a80e0da00eea9f4005d16cf73253cc4b5587db5b02c0c91eab01cb841eb39185aab719e7836319e26b87f2, have: d6b69d9c45bdc38bf6af352ae2964cb043785f6ed93d21cbefae1eabbdb127ab1090c7db115e13ac2cfa546c4ad2638c453348a5e5edb7a5c9da67edc5e80fdc $ python -c ' import portage.checksum print(portage.checksum.perform_multiple_checksums("/var/cache/portage/gentoo/dev-haskell/transformers/Manifest", hashes=["BLAKE2B"])) ' {'BLAKE2B': 'f15c3d2e9c11376de9820cc12e6c8e8b339402c773a80e0da00eea9f4005d16cf73253cc4b5587db5b02c0c91eab01cb841eb39185aab719e7836319e26b87f2'} This answer is consistent over 1k runs. $ b2sum /var/cache/portage/gentoo/dev-haskell/transformers/Manifest f15c3d2e9c11376de9820cc12e6c8e8b339402c773a80e0da00eea9f4005d16cf73253cc4b5587db5b02c0c91eab01cb841eb39185aab719e7836319e26b87f2 /var/cache/portage/gentoo/dev-haskell/transformers/Manifest This is also consistent over 1k runs. Rebuilding dev-python/pyblake2-1.1.0 with FEATURES=test succeeds (all tests pass), but eix-sync afterwards fails again with a verification error. Rebuilding dev-python/pyblake2-1.1.0 without C(XX)FLAGS results in the same behaviour (though for the first time it hit metadata/md5-cache/dev-haskell/Manifest.gz instead of a package manifest).
Ok, this looks really weird. So it's either gemato somehow getting the wrong hashes, or using the wrong backend (but it shouldn't have another backend FWICS). 1. Are you using btrfs? 2. Try to recheck it manually (but don't --sync, or wait a while between --sync and testing): $ gemato verify -K /var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg \ -s /var/cache/portage/gentoo 3. Try rebuilding gemato with FEATURES=test.
Created attachment 518500 [details] eix-sync.conf (In reply to Michał Górny from comment #22) > Ok, this looks really weird. So it's either gemato somehow getting the wrong > hashes, or using the wrong backend (but it shouldn't have another backend > FWICS). > > 1. Are you using btrfs? Yes, but not for /var/cache/portage/gentoo. During `emerge --sync` this is tmpfs and during normal operation it is squashfs (and it does not seem to make a difference which of the two). If that seems confusing, please see the attached eix-sync.conf > 2. Try to recheck it manually (but don't --sync, or wait a while between > --sync and testing): > > $ gemato verify -K /var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg > \ > -s /var/cache/portage/gentoo After mounting the squashfs image to /var/cache/portage/gentoo: # gemato verify -K /var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg -s /var/cache/portage/gentoo INFO:root:Refreshing keys from keyserver... INFO:root:Keys refreshed. INFO:root:Manifest timestamp: 2018-01-31 19:08:17 UTC INFO:root:Valid OpenPGP signature found: INFO:root:- primary key: DCD05B71EAB94199527F44ACDB6B8C1F96D8BF6D INFO:root:- subkey: E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250 INFO:root:- timestamp: 2018-01-31 19:08:17 UTC INFO:root:Verifying /var/cache/portage/gentoo... ERROR:root:Manifest mismatch for dev-haskell/simple-sendfile/Manifest BLAKE2B: expected: 4038d5ff321594e8be8f7b60ef89d042e4fd72749c230e27f8ba920b0cd7bef2ebf49c123be7dbdac22f9e25a3270aa1cbcf7850c6451c2db23bdaff40fb873b, have: 89fe0b615def2677f7e821284d61d870bfb84ab2f5feab0db40ca10aa973d37aa620d3485e11c50c10facfbea9d2098e319b7f89fae5d7b58e35da78ec1bbc0c # gemato verify -K /var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg -s /var/cache/portage/gentoo INFO:root:Refreshing keys from keyserver... INFO:root:Keys refreshed. INFO:root:Manifest timestamp: 2018-01-31 19:08:17 UTC INFO:root:Valid OpenPGP signature found: INFO:root:- primary key: DCD05B71EAB94199527F44ACDB6B8C1F96D8BF6D INFO:root:- subkey: E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250 INFO:root:- timestamp: 2018-01-31 19:08:17 UTC INFO:root:Verifying /var/cache/portage/gentoo... ERROR:root:Manifest mismatch for media-video/ffmpeg2theora/Manifest BLAKE2B: expected: 7dc07265a6b3c587686d26619e68c33a79829cef1dca614051df0f9580fd078961db8ce532feb38f049afd9d1a5e01efb01a0498772af65e99386126ac8cb988, have: 596ca04fe0406563781826eb39d23f25833d2949282d753e089bbf430690fc5951c30860debeb2de25ccddd4b63c22a05f5a5541c7e78c340baf6c3fc8f5aaee # gemato verify -K /var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg -s /var/cache/portage/gentoo INFO:root:Refreshing keys from keyserver... INFO:root:Keys refreshed. INFO:root:Manifest timestamp: 2018-01-31 19:08:17 UTC INFO:root:Valid OpenPGP signature found: INFO:root:- primary key: DCD05B71EAB94199527F44ACDB6B8C1F96D8BF6D INFO:root:- subkey: E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250 INFO:root:- timestamp: 2018-01-31 19:08:17 UTC INFO:root:Verifying /var/cache/portage/gentoo... ERROR:root:Manifest mismatch for dev-texlive/texlive-langother/Manifest BLAKE2B: expected: f8b9f01548bfe293e85f4d9c6ef8a901e0e9907a61960baf18a12246ddba6fb17304bb0ce717b77e1b6470c48f18696960d660840c5a557350049bd21fd7d261, have: 5fce986255a9f56218f488dd99131f506537eb704d372f6d1ef0ea2b4422ae7c786f8a7394d6cbe0be208be6834e54a1087b626bc3785ad8aa1afe48a79d2558 > 3. Try rebuilding gemato with FEATURES=test. Tests finished successfully, but the resulting binary still reports Manifest mismatches.
Please note that file verification order is undefined, so different errors are possible. Does the gemato verification happen after rsync on tmpfs, or after squashfs step? Could you do 'diff -dupR' between tmpfs and the squashfs created from it? I think your process is doing something wrong. I don't know eix-sync, and I would advise against using it. It's a horrible piece of bad quality code that nobody even tried auditing.
Created attachment 518538 [details] emaint-sync (eix-sync replacement script) (In reply to Michał Górny from comment #24) > Does the gemato verification happen after rsync on tmpfs, or after squashfs > step? Right after rsync on tmpfs, before creating squashfs. (eix-sync calls `emerge --sync` or `emaint sync` between ! and @ steps.) > Could you do 'diff -dupR' between tmpfs and the squashfs created from > it? There are no differences. > I don't know eix-sync, and I would advise against using it. It's a horrible > piece of bad quality code that nobody even tried auditing. I created attached script (and attached postsync.d scripts) that I will use instead of eix-sync in the future. They should do exactly what a call to `eix-sync` had done before.
Created attachment 518540 [details] 80-egencache-update-local.sh (postsync.d script)
Created attachment 518542 [details] 90-pack-gentoo.sh (postsync.d script)
Are you still hitting this? I just had an another idea (that something may be mangling line endings) but this doesn't seem to give your checksums either. Could you ping me on IRC so we could try to debug this more actively?
Portage 2.3.24 (python 2.7.13-final-42, default/linux/amd64/17.1/desktop/plasma/systemd, gcc-7.3.0, glibc-2.26-r5, 4.15.1-gentoo x86_64) ================================================================= System Settings ================================================================= System uname: Linux-4.15.1-gentoo-x86_64-AMD_A10-7800_Radeon_R7,_12_Compute_Cores_4C+8G-with-gentoo-2.4.1 KiB Mem: 15270072 total, 4637844 free KiB Swap: 0 total, 0 free Timestamp of repository gentoo: Sun, 11 Feb 2018 19:45:01 +0000 Head commit of repository gentoo: 811e64445c4c55cc76aac97bd1e52d060608a5eb Head commit of repository flatpak-overlay: 74797c725c9eedd81acb7bcf357f1489d52fdb97 Head commit of repository riscv: 72482aea643cab3512adffa4865016904330af5f Timestamp of repository sage-on-gentoo: Fri, 09 Feb 2018 08:51:44 +0000 Head commit of repository sage-on-gentoo: 1a3d326f07c2c348f7b658e264ecf47fe837648f Timestamp of repository science: Fri, 09 Feb 2018 23:25:47 +0000 Head commit of repository science: 7d62e55fab1c22bd3722d6343d95a49ffe97d2ec Timestamp of repository steam-overlay: Sun, 11 Feb 2018 18:15:19 +0000 Head commit of repository steam-overlay: 62b535d189274ce98f9cac980428de36cc51a704 sh bash 4.4_p19 ld GNU ld (Gentoo 2.30 p1) 2.30.0 ccache version 3.3.4 [disabled] app-shells/bash: 4.4_p19::gentoo dev-java/java-config: 2.2.0-r3::gentoo dev-lang/perl: 5.26.1-r1::gentoo dev-lang/python: 2.7.14-r101::sage-on-gentoo, 3.5.4-r1::gentoo dev-util/ccache: 3.3.4-r1::gentoo dev-util/cmake: 3.10.2::gentoo dev-util/pkgconfig: 0.29.2::gentoo sys-apps/baselayout: 2.4.1-r2::gentoo sys-apps/sandbox: 2.12::gentoo sys-devel/autoconf: 2.13::gentoo, 2.69-r4::gentoo sys-devel/automake: 1.11.6-r2::gentoo, 1.15.1-r1::gentoo sys-devel/binutils: 2.30::gentoo sys-devel/gcc: 7.3.0::riscv sys-devel/gcc-config: 1.9.1::gentoo sys-devel/libtool: 2.4.6-r4::gentoo sys-devel/make: 4.2.1-r1::gentoo sys-kernel/linux-headers: 4.15::gentoo (virtual/os-headers) sys-libs/glibc: 2.26-r5::gentoo Repositories: gentoo location: /var/cache/portage/gentoo sync-type: rsync sync-uri: rsync://rsync.de.gentoo.org/gentoo-portage priority: -1000 sync-rsync-extra-opts: sync-rsync-verify-metamanifest: yes flatpak-overlay location: /var/db/repos/flatpak-overlay sync-type: git sync-uri: git://github.com/fosero/flatpak-overlay.git masters: gentoo riscv location: /var/db/repos/riscv sync-type: git sync-uri: https://github.com/palmer-dabbelt/riscv-gentoo.git masters: gentoo sage-on-gentoo location: /var/db/repos/sage-on-gentoo sync-type: git sync-uri: https://github.com/gentoo-mirror/sage-on-gentoo.git masters: gentoo science science location: /var/db/repos/science sync-type: git sync-uri: https://github.com/gentoo-mirror/science.git masters: gentoo steam-overlay location: /var/db/repos/steam-overlay sync-type: git sync-uri: https://github.com/gentoo-mirror/steam-overlay.git masters: gentoo local location: /var/cache/portage/local masters: gentoo riscv priority: 100 ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-pipe -O2 -march=bdver3" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /etc/grs/systems.conf /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php7.2/ext-active/ /etc/php/cgi-php7.2/ext-active/ /etc/php/cli-php7.2/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-pipe -O2 -march=bdver3" DISTDIR="/var/cache/portage/distfiles" EMERGE_DEFAULT_OPTS="--keep-going --nospinner --verbose-conflicts" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs buildsyspkg cgroup compressdebug config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch parallel-install preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://ftp.spline.inf.fu-berlin.de/mirrors/gentoo/ http://ftp-stud.hs-esslingen.de/pub/Mirrors/gentoo/ http://distfiles.gentoo.org" LANG="en_GB.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu" LINGUAS="de en en_GB ar fa tr" MAKEOPTS="-j4" PKGDIR="/var/cache/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="X a52 aac aacplus aacs acl acpi activities alsa amd64 appindicator appstream archive audit avahi ayatana bash-completion bdplus berkdb bluetooth bluray branding brotli bs2b bzip2 cairo caps cdda cddb cdio cdr celt chromaprint cjk clang cli colord colorio conntrack crypt cups cxx d3d9 dbus declarative dirac djvu dri dts dvb dvd dvdr egl emboss encode epub exif fam fax fbcon ffmpeg fftw fish-completion fits flac fontconfig fortran fribidi ftdi gdbm geoclue geolocation gif git glamor gles gmp google googledrive gpg gpm gps graphicsmagick gstreamer gtk gtk3 harfbuzz hdf5 ibus iconv icu idn imlib inotify introspection ipv6 jemalloc jpeg jpeg2k kde kipi kwallet ladspa latex lcms ldap libidn2 libinput libnotify libproxy libsecret libsoxr lv2 lz4 lzma lzo mad markdown mercurial metis mjpeg mng mobi modemmanager modplug modules mp3 mp4 mpeg mplayer mtp multilib mysql ncurses netlink networkmanager nftables nls nptl office ofono ogg openal opencl openconnect opencv openexr opengl openh264 openmax openmp openvpn opus pam pango pcap pch pcre pcre2 pdf phonon plasma png policykit postscript ppds prison pulseaudio python qml qt3support qt4 qt5 raw readline rtmp samba scanner schroedinger sctp sdl seccomp semantic-desktop share snappy sparse speech speex spell ssl startup-notification svg systemd tbb tcpd teamd telepathy tga theora threads tiff timezone truetype tslib udev udisks unicode unwind upnp upnp-av upower usb v4l v4l2 vaapi vdpau vorbis vpnc vpx vulkan wavpack wayland webp widgets x264 x265 xattr xcb xcomposite xinerama xkb xml xmp xrandr xscreensaver xv xvid xwayland xz zeroconf zeromq zlib" ABI_X86="64" ALSA_CARDS="hda-intel" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_EXPERIMENTAL_FEATURES="stage" CALLIGRA_FEATURES="words sheets karbon plan" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx f16c fma3 fma4 mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 xop" ELIBC="glibc" ENLIGHTENMENT_MODULES="*" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64" INPUT_DEVICES="joystick libinput" KERNEL="linux" L10N="de en en-GB ar fa tr ja ko zh zh-CN zh-TW" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LIRC_DEVICES="devinput" LUA_TARGET="lua5-2" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6 php7-0" POSTGRES_TARGETS="postgres9_5" PYTHON_SINGLE_TARGET="python3_5" PYTHON_TARGETS="python2_7 python3_5 pypy pypy3" RUBY_TARGETS="ruby22 ruby23" STEAMGAMES="dirt_rally dont_starve portal source_engine te120 trine2 witcher2" USERLAND="GNU" VIDEO_CARDS="amdgpu virgl" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS ================================================================= Package Settings ================================================================= dev-python/pypy-5.10.0::gentoo was built with the following: USE="bzip2 gdbm jit ncurses sqlite -libressl -low-memory (-sandbox) -test -tk" ABI_X86="(64)" CPU_FLAGS_X86="(-sse2)" dev-python/pypy3-5.10.1::gentoo was built with the following: USE="bzip2 gdbm jit ncurses sqlite -libressl -low-memory (-sandbox) -tk" ABI_X86="(64)"
Ok, so we debugged this a bit with Dennis and it looks like an issue specific to PyPy (sorry that I've missed that you're using PyPy originally). Curious enough, it happens only with large input (that's why tests don't fail). I'm going to take a closer look but I'm afraid it's a PyPy bug.
It's definitely related to multiprocessing. Curious enough, it doesn't seem to affect PyPy3 (you need pyblake2 from git to test with it). I need to investigate more, it might be related to how objects are serialized. I may also provide an option to disable multiprocessing as a temporary solution. I already wanted to do it in the past, especially for small inputs. In this case, we could disable it unconditionally when using PyPy.
I've been able to establish so far that the problem is read()/read1() misfiring. In particular, if I replace the reading loop with a single read() call reading the whole file, it works. However, if I call read() at least twice, at some point it misfires. I wasn't able to establish exactly how yet but I suspect it's not advancing the file pointer correctly and rereading part of the file instead of returning EOF.
Trying to figure it out, I've came with a good, simple test case. it turns out multiprocessing is not needed at all, and it's enough to run pyblake2 from file in a loop with enough iterations.
Upon further investigation, it seems to be a problem in how buffers are retrieved by C extensions. I've linked an isolated test case to the upstream bug and FWICS both the C and the Python code is correct, so this is a bug in PyPy. Given that we can't expect a fix to be deployed in Gentoo anytime soon, I've added a workaround to the code that reads small-ish (<1 MiB) files in one read() call instead of in portions. It fixes the issue for me.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=40e5ce570d21987604088c0a30c4071bc7efe7e5 commit 40e5ce570d21987604088c0a30c4071bc7efe7e5 Author: Michał Górny <mgorny@gentoo.org> AuthorDate: 2018-02-13 11:19:02 +0000 Commit: Michał Górny <mgorny@gentoo.org> CommitDate: 2018-02-13 11:37:36 +0000 app-portage/gemato: Bump to 11.2 Bump to 11.2. The new version improves Python 2 compatibility, adds a workaround for PyPy misbehavior and adds a new utility commands that could be useful to debug verification failures. Closes: https://bugs.gentoo.org/646368 app-portage/gemato/Manifest | 1 + app-portage/gemato/gemato-11.2.ebuild | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 35 insertions(+)
