I think the problem is that it's pickling a _pkg_str here, so we should coerce that to a plain string: lib/portage/_emirrordist/FetchTask.py: self.config.distfiles_db[self.distfile] = self.cpv > ---------- Forwarded message ---------- > From: "(Cron Daemon)" <root@gentoo.org> > To: infra-gmirror@gentoo.org > Cc: > Bcc: > Date: 21 Jan 2021 22:53:58 +0000 > Subject: Cron <root@dipper> /usr/local/bin/pidlock -s sync-distfiles /usr/bin/timeout -k 2h 1h /usr/local/bin/mastermirror/sync-distfiles.sh > [ERROR] Exception in callback AsynchronousTask._exit_listener_cb(<bound method...7fea0fe8ac30>>) > handle: <Handle AsynchronousTask._exit_listener_cb(<bound method...7fea0fe8ac30>>)> > Traceback (most recent call last): > File "/usr/lib/python3.7/shelve.py", line 111, in __getitem__ > value = self.cache[key] > KeyError: 'libinput-1.16.3.tar.xz' > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): > File "/usr/lib/python3.7/asyncio/events.py", line 88, in _run > self._context.run(self._callback, *self._args) > File "/usr/lib/python3.7/site-packages/_emerge/AsynchronousTask.py", line 201, in _exit_listener_cb > listener(self) > File "/usr/lib/python3.7/site-packages/portage/_emirrordist/MirrorDistTask.py", line 53, in _fetch_exit > self._start_task(deletion, self._deletion_exit) > File "/usr/lib/python3.7/site-packages/_emerge/CompositeTask.py", line 113, in _start_task > task.start() > File "/usr/lib/python3.7/site-packages/_emerge/AsynchronousTask.py", line 30, in start > self._start() > File "/usr/lib/python3.7/site-packages/portage/util/_async/AsyncScheduler.py", line 90, in _start > self._schedule() > File "/usr/lib/python3.7/site-packages/_emerge/PollScheduler.py", line 152, in _schedule > self._schedule_tasks() > File "/usr/lib/python3.7/site-packages/portage/util/_async/AsyncScheduler.py", line 66, in _schedule_tasks > task.start() > File "/usr/lib/python3.7/site-packages/_emerge/AsynchronousTask.py", line 30, in start > self._start() > File "/usr/lib/python3.7/site-packages/portage/_emirrordist/DeletionTask.py", line 34, in _start > self._delete_links() > File "/usr/lib/python3.7/site-packages/portage/_emirrordist/DeletionTask.py", line 114, in _delete_links > self._success() > File "/usr/lib/python3.7/site-packages/portage/_emirrordist/DeletionTask.py", line 123, in _success > cpv = self.config.distfiles_db.get(self.distfile, cpv) > File "/usr/lib/python3.7/shelve.py", line 106, in get > return self[key] > File "/usr/lib/python3.7/shelve.py", line 114, in __getitem__ > value = Unpickler(f).load() > TypeError: __new__() missing 1 required positional argument: 'show_deprecated_warning' > /usr/local/bin/mastermirror/sync-distfiles.sh: line 57: 32248 Terminated /usr/bin/emirrordist --distfiles=${DATADIR}/distfiles/ --delete --jobs=10 --repo=gentoo --deletion-delay=${DELAY} --failure-log=${LOGDIR}/failure.log --success-log=${LOGDIR}/success.log --scheduled-deletion-log=${LOGDIR}/deletion.log --deletion-db=${LOGDIR}/deletion-db.gdbm --distfiles-db=${LOGDIR}/distfile-db.gdbm --recycle-db=${LOGDIR}/recycle-db.gdbm --recycle-dir=${DATADIR}/distfiles-archive/ --temp-dir=${DATADIR}/tmp/ --whitelist-from=${DATADIR}/tmp/whitelist-master.txt --distfiles-local=${DATADIR}/distfiles-local --symlinks --layout-conf=/usr/local/bin/mastermirror/layout.conf.new --mirror
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=8b5f9678200964c3cbb2eb06b58ebfa138fbac45 commit 8b5f9678200964c3cbb2eb06b58ebfa138fbac45 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2021-01-22 01:00:16 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2021-01-22 01:11:35 +0000 emirrordist: prevent distfiles_db _pkg_str pickle problems When storing a cpv as a shelve value, convert _pkg_str to str in order to prevent pickle problems. Bug: https://bugs.gentoo.org/766459 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/portage/_emirrordist/FetchTask.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
For python2 this issue would have been avoided by the ShelveUnicodeWrapper (removed with other python2 support in https://gitweb.gentoo.org/proj/portage.git/commit/?id=8aaf817db51f0369095b0f7d93cdbecdfc32c869).
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=6b98e103aa15da331d647a8f65a45bb3bb4e3197 commit 6b98e103aa15da331d647a8f65a45bb3bb4e3197 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2021-02-22 13:46:04 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2021-02-22 13:54:29 +0000 sys-apps/portage: Bump to version 3.0.15 #715112 default enable FEATURES=binpkg-multi-instance #756778 content-hash distfiles layout #766459 emirrordist: prevent distfiles_db _pkg_str pickle problems #766767 emaint --fix merges: add -y, --yes option #766773 emerge: disable --autounmask-license by default #767913 portage.getpid: call os.getpid() lazily #770712 PopenProcess: use call_soon for _async_waipid in _start #771549 prevent USE="${USE} ..." misbehavior Bug: https://bugs.gentoo.org/766117 Bug: https://bugs.gentoo.org/715112 Bug: https://bugs.gentoo.org/756778 Bug: https://bugs.gentoo.org/766459 Bug: https://bugs.gentoo.org/766767 Bug: https://bugs.gentoo.org/766773 Bug: https://bugs.gentoo.org/767913 Bug: https://bugs.gentoo.org/770712 Bug: https://bugs.gentoo.org/771549 Package-Manager: Portage-3.0.15, Repoman-3.0.2 Signed-off-by: Zac Medico <zmedico@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.15.ebuild | 268 +++++++++++++++++++++++++++++++++ 2 files changed, 269 insertions(+)
