Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 916235

Summary: sys-apps/portage: dbapi._iuse_implicit_cnstr not compatible with multiprocessing spawn
Product: Portage Development Reporter: Zac Medico <zmedico>
Component: CoreAssignee: Portage team <dev-portage>
Status: RESOLVED FIXED    
Severity: normal Keywords: InVCS
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://github.com/gentoo/portage/pull/1160
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on: 916031    
Bug Blocks: 914876    

Description Zac Medico gentoo-dev 2023-10-24 17:24:58 UTC
The test lib/portage/tests/emerge/test_emerge_blocker_file_collision.py fails when bin/emerge is patched like this:

multiprocessing.set_start_method("spawn", force=True)

> Traceback (most recent call last):                                                                                                                                                                                 
>   File "/usr/lib/python3.12/asyncio/events.py", line 84, in _run                                                                                                                                        [139/1245]
>     self._context.run(self._callback, *self._args)                                                                                                                                                                 
>   File "portage/lib/_emerge/AsynchronousTask.py", line 209, in _exit_listener_cb                                                              
>     listener(self)                                                                                                                                                                                                 
>   File "portage/lib/_emerge/Scheduler.py", line 1495, in _build_exit                                                                          
>     self._task_queues.merge.add(merge)                                                                                                                                                                             
>   File "portage/lib/_emerge/SequentialTaskQueue.py", line 22, in add                                                                          
>     self.schedule()                                                                                                                                                                                                
>   File "portage/lib/_emerge/SequentialTaskQueue.py", line 44, in schedule                                                                     
>     task.start()                                                                                                                                                                                                   
>   File "portage/lib/_emerge/AsynchronousTask.py", line 34, in start                                                                           
>     self._start()                                                                                                                                                                                                  
>   File "portage/lib/_emerge/PackageMerge.py", line 57, in _start                                                                              
>     self._start_task(task, self._install_exit)
>   File "portage/lib/_emerge/CompositeTask.py", line 111, in _start_task
>     task.start()
>   File "portage/lib/_emerge/AsynchronousTask.py", line 34, in start
>     self._start()
>   File "portage/lib/_emerge/EbuildMerge.py", line 58, in _start
>     self._start_task(merge_task, self._merge_exit)
>   File "portage/lib/_emerge/CompositeTask.py", line 111, in _start_task
>     task.start()
>   File "portage/lib/_emerge/AsynchronousTask.py", line 34, in start
>     self._start()
>   File "portage/lib/portage/dbapi/_MergeProcess.py", line 78, in _start
>     super()._start()
>   File "portage/lib/portage/util/_async/ForkProcess.py", line 61, in _start
>     retval = self._spawn(self.args, fd_pipes=self.fd_pipes)
>              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "portage/lib/portage/dbapi/_MergeProcess.py", line 208, in _spawn
>     pids = super()._spawn(args, fd_pipes, **kwargs)
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "portage/lib/portage/util/_async/ForkProcess.py", line 189, in _spawn
>     self._proc.start()
>   File "/usr/lib/python3.12/multiprocessing/process.py", line 121, in start
>     self._popen = self._Popen(self)
>                   ^^^^^^^^^^^^^^^^^
>   File "/usr/lib/python3.12/multiprocessing/context.py", line 224, in _Popen
>     return _default_context.get_context().Process._Popen(process_obj)
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "/usr/lib/python3.12/multiprocessing/context.py", line 289, in _Popen
>     return Popen(process_obj)
>            ^^^^^^^^^^^^^^^^^^
>   File "/usr/lib/python3.12/multiprocessing/popen_spawn_posix.py", line 32, in __init__
>     super().__init__(process_obj)
>   File "/usr/lib/python3.12/multiprocessing/popen_fork.py", line 19, in __init__
>     self._launch(process_obj)
>   File "/usr/lib/python3.12/multiprocessing/popen_spawn_posix.py", line 47, in _launch
>     reduction.dump(process_obj, fp)
>   File "/usr/lib/python3.12/multiprocessing/reduction.py", line 60, in dump
>     ForkingPickler(file, protocol).dump(obj)
> AttributeError: Can't pickle local object 'dbapi._iuse_implicit_cnstr.<locals>.<lambda>'
Comment 1 Larry the Git Cow gentoo-dev 2023-10-24 18:37:29 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=7a43de7a8ce8dc6836d998cb87a5135ba8f0bc2c

commit 7a43de7a8ce8dc6836d998cb87a5135ba8f0bc2c
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2023-10-24 18:07:53 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2023-10-24 18:08:23 +0000

    dbapi: Make _iuse_implicit_cnstr picklable for spawn compat
    
    Bug: https://bugs.gentoo.org/916235
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 lib/portage/dbapi/__init__.py | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)
Comment 2 Larry the Git Cow gentoo-dev 2023-10-25 05:49:10 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=2b9128b401def05cac46f9e6d66048e5a0d888b8

commit 2b9128b401def05cac46f9e6d66048e5a0d888b8
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2023-10-25 05:48:31 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2023-10-25 05:48:40 +0000

    sys-apps/portage: add 3.0.54
    
    Closes: https://bugs.gentoo.org/905869
    Closes: https://bugs.gentoo.org/915551
    Closes: https://bugs.gentoo.org/915896
    Closes: https://bugs.gentoo.org/916106
    Closes: https://bugs.gentoo.org/916108
    Closes: https://bugs.gentoo.org/916112
    Closes: https://bugs.gentoo.org/916116
    Closes: https://bugs.gentoo.org/916139
    Closes: https://bugs.gentoo.org/916141
    Closes: https://bugs.gentoo.org/916142
    Closes: https://bugs.gentoo.org/916149
    Closes: https://bugs.gentoo.org/916182
    Closes: https://bugs.gentoo.org/916231
    Closes: https://bugs.gentoo.org/916235
    Closes: https://bugs.gentoo.org/916240
    Closes: https://bugs.gentoo.org/916242
    Closes: https://bugs.gentoo.org/916245
    Closes: https://bugs.gentoo.org/916248
    Closes: https://bugs.gentoo.org/899898
    Closes: https://bugs.gentoo.org/906129
    Closes: https://bugs.gentoo.org/906156
    Closes: https://bugs.gentoo.org/916031
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-3.0.54.ebuild | 238 +++++++++++++++++++++++++++++++++
 2 files changed, 239 insertions(+)