After I fixed bug 916106 and bug 916108, I found that vardbapi.unpack_contents contains an unpicklable _io.FileIO object that triggers this error with the multiprocessing spawn start method: > Traceback (most recent call last): > File "/usr/lib/python3.10/asyncio/events.py", line 80, in _run > self._context.run(self._callback, *self._args) > File "portage/lib/_emerge/AsynchronousTask.py", line 209, in _exit_listener_cb > listener(self) > File "portage/lib/_emerge/Binpkg.py", line 431, in _unpack_contents_exit > unpack_contents.future.result() > File "portage/lib/portage/dbapi/bintree.py", line 391, in unpack_contents > await add_pkg._db.unpack_contents(pkg, dest_dir, loop=loop) > File "portage/lib/portage/dbapi/vartree.py", line 1101, in unpack_contents > excluded_config_files = await loop.run_in_executor( > File "/usr/lib/python3.10/asyncio/base_events.py", line 821, in run_in_executor > executor.submit(func, *args), loop=self) > File "portage/lib/portage/util/futures/executor/fork.py", line 46, in submit > self._schedule() > File "portage/lib/portage/util/futures/executor/fork.py", line 62, in _schedule > proc.start() > File "portage/lib/_emerge/AsynchronousTask.py", line 34, in start > self._start() > File "portage/lib/portage/util/_async/AsyncFunction.py", line 35, in _start > ForkProcess._start(self) > File "portage/lib/portage/util/_async/ForkProcess.py", line 61, in _start > retval = self._spawn(self.args, fd_pipes=self.fd_pipes) > File "portage/lib/portage/util/_async/ForkProcess.py", line 189, in _spawn > self._proc.start() > File "/usr/lib/python3.10/multiprocessing/process.py", line 121, in start > self._popen = self._Popen(self) > File "/usr/lib/python3.10/multiprocessing/context.py", line 224, in _Popen > return _default_context.get_context().Process._Popen(process_obj) > File "/usr/lib/python3.10/multiprocessing/context.py", line 288, in _Popen > return Popen(process_obj) > File "/usr/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 32, in __init__ > super().__init__(process_obj) > File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 19, in __init__ > self._launch(process_obj) > File "/usr/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 47, in _launch > reduction.dump(process_obj, fp) > File "/usr/lib/python3.10/multiprocessing/reduction.py", line 60, in dump > ForkingPickler(file, protocol).dump(obj) > TypeError: cannot pickle '_io.FileIO' object
It looks like it's the pw_file variable here: > if binpkg_format == "xpak": > tar_cmd = ("tar", "-x", "--xattrs", "--xattrs-include=*", "-C", dest_dir) > pr, pw = os.pipe() > proc = await asyncio.create_subprocess_exec(*tar_cmd, stdin=pr) > os.close(pr) > with os.fdopen(pw, "wb", 0) as pw_file: > excluded_config_files = await loop.run_in_executor( > ForkExecutor(loop=loop), > functools.partial( > self._dblink(cpv).quickpkg, > pw_file, > include_config=opts.include_config == "y", > include_unmodified_config=opts.include_unmodified_config == "y", > ), > )
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=0b9fbe389db3c3b9e625e1f5f526b3a921d2e0ce commit 0b9fbe389db3c3b9e625e1f5f526b3a921d2e0ce Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2023-10-22 19:57:16 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2023-10-22 19:58:20 +0000 vardbapi.unpack_contents: Use multiprocessing.Pipe for spawn compat Bug: https://bugs.gentoo.org/916112 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/portage/dbapi/vartree.py | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=2b9128b401def05cac46f9e6d66048e5a0d888b8 commit 2b9128b401def05cac46f9e6d66048e5a0d888b8 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-10-25 05:48:31 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-10-25 05:48:40 +0000 sys-apps/portage: add 3.0.54 Closes: https://bugs.gentoo.org/905869 Closes: https://bugs.gentoo.org/915551 Closes: https://bugs.gentoo.org/915896 Closes: https://bugs.gentoo.org/916106 Closes: https://bugs.gentoo.org/916108 Closes: https://bugs.gentoo.org/916112 Closes: https://bugs.gentoo.org/916116 Closes: https://bugs.gentoo.org/916139 Closes: https://bugs.gentoo.org/916141 Closes: https://bugs.gentoo.org/916142 Closes: https://bugs.gentoo.org/916149 Closes: https://bugs.gentoo.org/916182 Closes: https://bugs.gentoo.org/916231 Closes: https://bugs.gentoo.org/916235 Closes: https://bugs.gentoo.org/916240 Closes: https://bugs.gentoo.org/916242 Closes: https://bugs.gentoo.org/916245 Closes: https://bugs.gentoo.org/916248 Closes: https://bugs.gentoo.org/899898 Closes: https://bugs.gentoo.org/906129 Closes: https://bugs.gentoo.org/906156 Closes: https://bugs.gentoo.org/916031 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.54.ebuild | 238 +++++++++++++++++++++++++++++++++ 2 files changed, 239 insertions(+)
