Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 924369 - sys-apps/portage: signal.signal(signal.SIGUSR1, debug_signal) in conftest.py triggers "ValueError: signal only works in main thread of the main interpreter"
Summary: sys-apps/portage: signal.signal(signal.SIGUSR1, debug_signal) in conftest.py ...
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Portage team
URL: https://github.com/pytest-dev/pytest-...
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-02-12 14:25 UTC by Zac Medico
Modified: 2024-05-12 22:29 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Zac Medico gentoo-dev 2024-02-12 14:25:59 UTC
I've only seen this error trigger once (didn't trigger after simple retry), but anyway it seems like we should not be setting a signal handler in global module scope, unprotected by __name__ == "__main__":

https://github.com/gentoo/portage/actions/runs/7872826586/job/21478884435

> ==================================== ERRORS ====================================
> ______________________ ERROR collecting lib/portage/tests ______________________
> /opt/hostedtoolcache/Python/3.13.0-alpha.3/x64/lib/python3.13/importlib/__init__.py:88: in import_module
>     return _bootstrap._gcd_import(name[level:], package, level)
>         level      = 0
>         name       = 'portage.tests.conftest'
>         package    = None
> <frozen importlib._bootstrap>:1387: in _gcd_import
>     ???
>         level      = 0
>         name       = 'portage.tests.conftest'
>         package    = None
> <frozen importlib._bootstrap>:1360: in _find_and_load
>     ???
>         import_    = <function _gcd_import at 0x7f4b2f97c360>
>         module     = <object object at 0x7f4b2f928060>
>         name       = 'portage.tests.conftest'
> <frozen importlib._bootstrap>:1331: in _find_and_load_unlocked
>     ???
>         child      = 'conftest'
>         import_    = <function _gcd_import at 0x7f4b2f97c360>
>         name       = 'portage.tests.conftest'
>         parent     = 'portage.tests'
>         parent_module = <module 'portage.tests' from '/home/runner/work/portage/portage/lib/portage/tests/__init__.py'>
>         parent_spec = ModuleSpec(name='portage.tests', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7f4b27f24530>, origin='/home/runner/work/portage/portage/lib/portage/tests/__init__.py', submodule_search_locations=['/home/runner/work/portage/portage/lib/portage/tests'])
>         path       = ['/home/runner/work/portage/portage/lib/portage/tests']
>         spec       = ModuleSpec(name='portage.tests.conftest', loader=<_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7f4b2c1ab3b0>, origin='/home/runner/work/portage/portage/lib/portage/tests/conftest.py')
> <frozen importlib._bootstrap>:935: in _load_unlocked
>     ???
>         module     = <module 'portage.tests.conftest' from '/home/runner/work/portage/portage/lib/portage/tests/conftest.py'>
>         spec       = ModuleSpec(name='portage.tests.conftest', loader=<_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7f4b2c1ab3b0>, origin='/home/runner/work/portage/portage/lib/portage/tests/conftest.py')
> /opt/hostedtoolcache/Python/3.13.0-alpha.3/x64/lib/python3.13/site-packages/_pytest/assertion/rewrite.py:175: in exec_module
>     exec(co, module.__dict__)
>         cache_dir  = PosixPath('/home/runner/work/portage/portage/lib/portage/tests/__pycache__')
>         cache_name = 'conftest.cpython-313-pytest-8.0.0.pyc'
>         co         = <code object <module> at 0x7f4b27f9dfb0, file "/home/runner/work/portage/portage/lib/portage/tests/conftest.py", line 1>
>         fn         = PosixPath('/home/runner/work/portage/portage/lib/portage/tests/conftest.py')
>         module     = <module 'portage.tests.conftest' from '/home/runner/work/portage/portage/lib/portage/tests/conftest.py'>
>         ok         = True
>         pyc        = PosixPath('/home/runner/work/portage/portage/lib/portage/tests/__pycache__/conftest.cpython-313-pytest-8.0.0.pyc')
>         self       = <_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7f4b2c1ab3b0>
>         state      = <_pytest.assertion.AssertionState object at 0x7f4b2c18d0d0>
>         write      = True
> /home/runner/work/portage/portage/lib/portage/tests/conftest.py:27: in <module>
>     signal.signal(signal.SIGUSR1, debug_signal)
>         PORTAGE_BIN_PATH = '/home/runner/work/portage/portage/bin'
>         __builtins__ = <builtins>
>         __cached__ = '/home/runner/work/portage/portage/lib/portage/tests/__pycache__/conftest.cpython-313.pyc'
>         __doc__    = None
>         __file__   = '/home/runner/work/portage/portage/lib/portage/tests/conftest.py'
>         __loader__ = <_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7f4b2c1ab3b0>
>         __name__   = 'portage.tests.conftest'
>         __package__ = 'portage.tests'
>         __spec__   = ModuleSpec(name='portage.tests.conftest', loader=<_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7f4b2c1ab3b0>, origin='/home/runner/work/portage/portage/lib/portage/tests/conftest.py')
>         debug_signal = <function debug_signal at 0x7f4b27c82660>
>         global_event_loop = <function _safe_loop at 0x7f4b27c83ec0>
>         grp        = <module 'grp' from '/opt/hostedtoolcache/Python/3.13.0-alpha.3/x64/lib/python3.13/lib-dynload/grp.cpython-313-x86_64-linux-gnu.so'>
>         os         = <module 'os' (frozen)>
>         osp        = <module 'posixpath' (frozen)>
>         portage    = <module 'portage' from '/home/runner/work/portage/portage/lib/portage/__init__.py'>
>         pwd        = <module 'pwd' (built-in)>
>         pytest     = <module 'pytest' from '/opt/hostedtoolcache/Python/3.13.0-alpha.3/x64/lib/python3.13/site-packages/pytest/__init__.py'>
>         shutil     = <module 'shutil' from '/opt/hostedtoolcache/Python/3.13.0-alpha.3/x64/lib/python3.13/shutil.py'>
>         signal     = <module 'signal' from '/opt/hostedtoolcache/Python/3.13.0-alpha.3/x64/lib/python3.13/signal.py'>
>         sys        = <module 'sys' (built-in)>
>         tempfile   = <module 'tempfile' from '/opt/hostedtoolcache/Python/3.13.0-alpha.3/x64/lib/python3.13/tempfile.py'>
> /opt/hostedtoolcache/Python/3.13.0-alpha.3/x64/lib/python3.13/signal.py:58: in signal
>     handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))
> E   ValueError: signal only works in main thread of the main interpreter
>         handler    = <function debug_signal at 0x7f4b27c82660>
>         signalnum  = <Signals.SIGUSR1: 10>

Prior to https://gitweb.gentoo.org/proj/portage.git/commit/?id=4ba1cd651c648a691665359e36eee71c61fc370e the call was also unprotected by __name__ == "__main__", though it was in an entrypoint script rather than a module.
Comment 1 Zac Medico gentoo-dev 2024-02-13 18:32:49 UTC
I also observed this error locally when I ran the unit tests in a python3.9 virtualenv since I didn't have pytest[python_targets_python3_9] installed on gentoo.
Comment 2 Zac Medico gentoo-dev 2024-02-16 07:11:25 UTC
This appears to be an instance of pytest-xdist / execnet running tests in a non-main thread as reported in https://github.com/pytest-dev/pytest-xdist/issues/620 and https://github.com/pytest-dev/execnet/issues/96.

I've proposed a solution in https://github.com/pytest-dev/execnet/pull/243.
Comment 3 Zac Medico gentoo-dev 2024-02-25 22:02:31 UTC
They've merged https://github.com/pytest-dev/execnet/pull/243 but it's not tagged yet, and I've sent https://github.com/pytest-dev/pytest-xdist/pull/1027 to make pytest-xdist use it.
Comment 4 Zac Medico gentoo-dev 2024-05-12 22:29:36 UTC
It's fixed in pytest-xdist-3.6.1 with these:

* https://github.com/pytest-dev/pytest-xdist/pull/1027
* https://github.com/pytest-dev/pytest-xdist/pull/1072

And we have it in ~arch now:

commit d3bc1a8050a110abd7a0b208bed6a7879114e5e4
Author: Michał Górny <mgorny@gentoo.org>
Date:   2024-04-29 04:26:05 +0200

    dev-python/pytest-xdist: Bump to 3.6.1
    
    Signed-off-by: Michał Górny <mgorny@gentoo.org>