Since python3.7, the asyncio.get_running_loop() and asyncio.run(coro) functions are the preferred ways to access and run event loops. The asyncio.get_running_loop() function makes it possible to avoid passing around event loop instances in order to accommodate loops running in non-main threads (see the fix for bug 737698). For python3.6 compatibility, we can add shims for these functions in the portage.util.futures.asyncio module.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=631bedffe29124d693de3b539fc908d9feec1420 commit 631bedffe29124d693de3b539fc908d9feec1420 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2021-09-19 22:05:38 -0700 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2021-09-19 22:08:47 -0700 _safe_loop: fix python3.10 DeprecationWarning DeprecationWarning: There is no current event loop Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/portage/util/futures/_asyncio/__init__.py | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=fbaaa4a733aaadc2744b656527756ac4e2b7ab58 commit fbaaa4a733aaadc2744b656527756ac4e2b7ab58 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2024-02-22 06:47:33 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2024-02-22 07:28:38 +0000 socks5: Use real asyncio.run Use real asyncio.run to demonstrate that it is compatible with portage internals. Since the socks5 ProxyManager uses the process.spawn function, the internal _running_loop function needs to return the correct loop for use in the wait method of MultiprocessingProcess, or else it will lead to Future "attached to a different loop" errors. Bug: https://bugs.gentoo.org/761538 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/portage/tests/util/test_socks5.py | 45 +++++++++++++++++++---------------- lib/portage/util/socks5.py | 30 ++++++++++++++++------- 2 files changed, 46 insertions(+), 29 deletions(-) https://gitweb.gentoo.org/proj/portage.git/commit/?id=d718cea94a180042b2285698b2c19113c5d25987 commit d718cea94a180042b2285698b2c19113c5d25987 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2024-02-22 06:41:49 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2024-02-22 07:28:38 +0000 _get_running_loop: Support real asyncio.run When called via the real asyncio.run implementation, wrap the running asyncio loop. Otherwise, it's not possible to call portage libraries via the real asyncio.run without triggering Future "attached to a different loop" errors. Bug: https://bugs.gentoo.org/761538 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/portage/util/futures/_asyncio/__init__.py | 28 +++++++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-)
(In reply to Zac Medico from comment #0) > The asyncio.get_running_loop() function makes it possible to avoid passing > around event loop instances in order to accommodate loops running in > non-main threads (see the fix for bug 737698). In https://github.com/gentoo/portage/pull/1368 for bug 937740, we have a special variation of this "avoid loop passing" theme, where the _thread_weakrefs_atexit hook makes an adjustment so that global_event_loop() will return the appropriate (about to close) loop while run_coroutine_exitfuncs is executing.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=e97acd3c62ff02eb41ff643e75eb5e07c27717f8 commit e97acd3c62ff02eb41ff643e75eb5e07c27717f8 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2024-08-18 14:59:46 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2024-08-18 15:46:45 +0000 _wrap_loop: Prevent redundant AsyncioEventLoop instances Ultimately the loop arguments that necessitate the _wrap_loop function can be removed, because our aim since bug 761538 should be to eliminate them. Meanwhile, we don't want _wrap_loop to return redundant AsyncioEventLoop instances if we can easily prevent it. Therefore, use _safe_loop(create=False) to look up the AsyncioEventLoop instance associated with the current thread, and avoid creating redundant instances. This serves to guard against garbage collection of AsyncioEventLoop instances which may have _coroutine_exithandlers added by the atexit_register function since commit cb0c09d8cecb from bug 937740. If _safe_loop(create=False) fails to associate a loop with the current thread, raise an AssertionError for portage internal API consumers. It's not known whether external API consumers will trigger this case, so if it happens then emit a UserWarning and return a temporary AsyncioEventLoop instance. Fixes: cb0c09d8cecb ("Support coroutine exitfuncs for non-main loops") Bug: https://bugs.gentoo.org/938127 Bug: https://bugs.gentoo.org/937740 Bug: https://bugs.gentoo.org/761538 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/portage/util/futures/_asyncio/__init__.py | 43 +++++++++++++++++++++++---- 1 file changed, 38 insertions(+), 5 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=ee17cbd807ba976491e4c657be8aa9b9a29fe059 commit ee17cbd807ba976491e4c657be8aa9b9a29fe059 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2024-08-31 19:06:25 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2024-08-31 19:06:25 +0000 _safe_loop: Discard wrapped asyncio.run loop that was closed Since commit cb0c09d8cecb, _get_running_loop can wrap loops from asyncio.run, so these loops need to be discarded if they've been closed. Fixes: cb0c09d8cecb ("Support coroutine exitfuncs for non-main loops") Bug: https://bugs.gentoo.org/938761 Bug: https://bugs.gentoo.org/761538 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/portage/util/futures/_asyncio/__init__.py | 7 +++++++ 1 file changed, 7 insertions(+)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=b30ddb1913e8aa2947d20e43f455d2060aa6257f commit b30ddb1913e8aa2947d20e43f455d2060aa6257f Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2024-08-31 20:08:35 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2024-09-01 06:59:44 +0000 asyncio: Avoid _wrap_loop prior to create_subprocess_exec Bug: https://bugs.gentoo.org/761538 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/portage/util/futures/_asyncio/__init__.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) https://gitweb.gentoo.org/proj/portage.git/commit/?id=e5457915f7929db3781ded384bdb089b0760221f commit e5457915f7929db3781ded384bdb089b0760221f Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2024-08-31 17:32:12 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2024-09-01 06:59:34 +0000 asyncio: Use default ensure_future implementation when possible When a loop argument is not given, use the default asyncio ensure_future implementation and avoid unnecessary _wrap_loop usage. Bug: https://bugs.gentoo.org/761538 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/portage/util/futures/_asyncio/__init__.py | 3 +++ 1 file changed, 3 insertions(+) https://gitweb.gentoo.org/proj/portage.git/commit/?id=7f6b2b04878209130d44fc06105f521bae2b2173 commit 7f6b2b04878209130d44fc06105f521bae2b2173 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2024-08-31 05:35:32 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2024-09-01 06:58:56 +0000 asyncio: Use default sleep implementation when possible When a loop argument is not given, use the default asyncio sleep implementation and avoid unnecessary _wrap_loop usage. Bug: https://bugs.gentoo.org/761538 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/portage/util/futures/_asyncio/__init__.py | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
