Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 916139 - sys-apps/portage: doebuild local _fetch_subprocess function is incompatible with multiprocessing spawn start method
Summary: sys-apps/portage: doebuild local _fetch_subprocess function is incompatible w...
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on: 916031
Blocks: 914876
  Show dependency tree
 
Reported: 2023-10-22 21:16 UTC by Zac Medico
Modified: 2023-10-25 05:49 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Zac Medico gentoo-dev 2023-10-22 21:16:54 UTC
After I fixed bug 916106, bug 916108, and bug 916112, I found that the doebuild local _fetch_subprocess function is incompatible with multiprocessing spawn start method:

> Traceback (most recent call last):
>   File "portage/bin/ebuild", line 409, in <module>
>     a = portage.doebuild(
>         ^^^^^^^^^^^^^^^^^
>   File "portage/lib/portage/package/ebuild/doebuild.py", line 1336, in doebuild
>     loop.run_in_executor(
>   File "/usr/lib/python3.12/asyncio/base_events.py", line 840, in run_in_executor
>     executor.submit(func, *args), loop=self)
>     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "portage/lib/portage/util/futures/executor/fork.py", line 46, in submit
>     self._schedule()
>   File "portage/lib/portage/util/futures/executor/fork.py", line 62, in _schedule
>     proc.start()
>   File "portage/lib/_emerge/AsynchronousTask.py", line 34, in start
>     self._start()
>   File "portage/lib/portage/util/_async/AsyncFunction.py", line 35, in _start
>     ForkProcess._start(self)
>   File "portage/lib/portage/util/_async/ForkProcess.py", line 61, in _start
>     retval = self._spawn(self.args, fd_pipes=self.fd_pipes)
>              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "portage/lib/portage/util/_async/ForkProcess.py", line 189, in _spawn
>     self._proc.start()
>   File "/usr/lib/python3.12/multiprocessing/process.py", line 121, in start
>     self._popen = self._Popen(self)
>                   ^^^^^^^^^^^^^^^^^
>   File "/usr/lib/python3.12/multiprocessing/context.py", line 224, in _Popen
>     return _default_context.get_context().Process._Popen(process_obj)
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "/usr/lib/python3.12/multiprocessing/context.py", line 289, in _Popen
>     return Popen(process_obj)
>            ^^^^^^^^^^^^^^^^^^
>   File "/usr/lib/python3.12/multiprocessing/popen_spawn_posix.py", line 32, in __init__
>     super().__init__(process_obj)
>   File "/usr/lib/python3.12/multiprocessing/popen_fork.py", line 19, in __init__
>     self._launch(process_obj)
>   File "/usr/lib/python3.12/multiprocessing/popen_spawn_posix.py", line 47, in _launch
>     reduction.dump(process_obj, fp)
>   File "/usr/lib/python3.12/multiprocessing/reduction.py", line 60, in dump
>     ForkingPickler(file, protocol).dump(obj)
> AttributeError: Can't pickle local object 'doebuild.<locals>._fetch_subprocess'
Comment 1 Larry the Git Cow gentoo-dev 2023-10-22 22:46:23 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=e9d6782ca18b2f30d97b4859419bf7da4da01bab

commit e9d6782ca18b2f30d97b4859419bf7da4da01bab
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2023-10-22 21:24:47 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2023-10-22 22:45:02 +0000

    doebuild: Make _fetch_subprocess compatible with spawn
    
    Before it failed like this with the multiprocessing spawn
    start method:
    
    AttributeError: Can't pickle local object 'doebuild.<locals>._fetch_subprocess'
    
    Bug: https://bugs.gentoo.org/916139
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 lib/portage/package/ebuild/doebuild.py | 32 +++++++++++++++++---------------
 1 file changed, 17 insertions(+), 15 deletions(-)
Comment 2 Larry the Git Cow gentoo-dev 2023-10-25 05:49:18 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=2b9128b401def05cac46f9e6d66048e5a0d888b8

commit 2b9128b401def05cac46f9e6d66048e5a0d888b8
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2023-10-25 05:48:31 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2023-10-25 05:48:40 +0000

    sys-apps/portage: add 3.0.54
    
    Closes: https://bugs.gentoo.org/905869
    Closes: https://bugs.gentoo.org/915551
    Closes: https://bugs.gentoo.org/915896
    Closes: https://bugs.gentoo.org/916106
    Closes: https://bugs.gentoo.org/916108
    Closes: https://bugs.gentoo.org/916112
    Closes: https://bugs.gentoo.org/916116
    Closes: https://bugs.gentoo.org/916139
    Closes: https://bugs.gentoo.org/916141
    Closes: https://bugs.gentoo.org/916142
    Closes: https://bugs.gentoo.org/916149
    Closes: https://bugs.gentoo.org/916182
    Closes: https://bugs.gentoo.org/916231
    Closes: https://bugs.gentoo.org/916235
    Closes: https://bugs.gentoo.org/916240
    Closes: https://bugs.gentoo.org/916242
    Closes: https://bugs.gentoo.org/916245
    Closes: https://bugs.gentoo.org/916248
    Closes: https://bugs.gentoo.org/899898
    Closes: https://bugs.gentoo.org/906129
    Closes: https://bugs.gentoo.org/906156
    Closes: https://bugs.gentoo.org/916031
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-3.0.54.ebuild | 238 +++++++++++++++++++++++++++++++++
 2 files changed, 239 insertions(+)