Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 934220 - sys-apps/portage: allow GIL to be disabled in whirlpool C extension
Summary: sys-apps/portage: allow GIL to be disabled in whirlpool C extension
Status: IN_PROGRESS
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All All
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on: 933499
Blocks:
  Show dependency tree
 
Reported: 2024-06-13 21:07 UTC by Zac Medico
Modified: 2024-06-28 18:18 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Zac Medico gentoo-dev 2024-06-13 21:07:17 UTC
In 3.13 python extensions need to declare support for GIL features, for example if they don't declare Py_MOD_GIL_NOT_USED then it will cause the GIL to be enabled even when python was launched in free-threaded mode:

https://docs.python.org/3.13/whatsnew/3.13.html#free-threaded-cpython

We can use a patch with ifdef like this to simultaneously support older python:

diff --git a/src/portage_util__whirlpool.c b/src/portage_util__whirlpool.c
index 6c9421e56b..be779f44fd 100644
--- a/src/portage_util__whirlpool.c
+++ b/src/portage_util__whirlpool.c
@@ -1112,11 +1112,22 @@ static PyTypeObject WhirlpoolType = {
     .tp_methods = Whirlpool_methods,
 };
 
+static PyModuleDef_Slot _whirlpoolmodule_slots[] = {
+#ifdef Py_MOD_PER_INTERPRETER_GIL_SUPPORTED
+    {Py_mod_multiple_interpreters, Py_MOD_PER_INTERPRETER_GIL_SUPPORTED},
+#endif
+#ifdef Py_MOD_GIL_NOT_USED
+    {Py_mod_gil, Py_MOD_GIL_NOT_USED},
+#endif
+    {0, NULL},
+};
+
 static PyModuleDef moduledef = {
     PyModuleDef_HEAD_INIT,
     .m_name = "_whirlpool",
     .m_doc = "Reference Whirlpool implementation",
     .m_size = -1,
+    .m_slots = _whirlpoolmodule_slots,
 };
 
 PyMODINIT_FUNC
Comment 1 Zac Medico gentoo-dev 2024-06-16 20:30:10 UTC
There's a PyUnstable_Module_SetGIL() function we can use with single-phase init modules, but it's an unstable API so it seems better to migrate to multi-phase init:

https://docs.python.org/3.13/c-api/module.html#c.PyUnstable_Module_SetGIL
Comment 2 Larry the Git Cow gentoo-dev 2024-06-28 18:16:38 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=918f4a4aa82f47291deac496d13c2761f306e01f

commit 918f4a4aa82f47291deac496d13c2761f306e01f
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2024-06-16 19:44:37 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2024-06-28 18:15:38 +0000

    Allow GIL to be disabled in whirlpool C extension
    
    In 3.13 python extensions need to declare support for GIL features, for
    example if they don't declare Py_MOD_GIL_NOT_USED then it will cause the
    GIL to be enabled even when python was launched in free-threaded mode.
    
    This requires "multi-phase initialization" because Py_mod_create is
    incompatible with m_slots. There's a PyUnstable_Module_SetGIL() function
    that can be used with single-phase init, but it's an unstable API, so
    it's better to use multi-phase init. There's no need to use
    PyModule_GetState() because the whirlpool module has no mutable state.
    
    Bug: https://bugs.gentoo.org/934220
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 src/portage_util__whirlpool.c | 46 +++++++++++++++++++++++++++----------------
 1 file changed, 29 insertions(+), 17 deletions(-)