Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 685378 - sys-apps/portage: asynchronous operations for binary packages
Summary: sys-apps/portage: asynchronous operations for binary packages
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All All
: Normal normal (vote)
Assignee: Portage team
Depends on:
Blocks: 835380
  Show dependency tree
Reported: 2019-05-08 19:21 UTC by Zac Medico
Modified: 2024-01-05 10:42 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Note You need to log in before you can comment on or make changes to this bug.
Description Zac Medico gentoo-dev 2019-05-08 19:21:21 UTC
When don't want asynchronous code to block while waiting for a lock on a binary package repository, so there should be an asynchronous variant of any method that requires a lock. For example, Binpkg and BinpkgPrefetcher classes each call the binarytree.inject method within an asynchronous context, so an asynchronous variant of this method is needed.
Comment 1 Zac Medico gentoo-dev 2019-05-08 19:23:45 UTC
For the locks (also see bug 685236) we really should use a (python3.5+ only) asynchronous context manager:

Given that python2.7 is scheduled for retirement in 2020 (, maybe it's time to drop support for python3.4 and earlier.
Comment 2 Zac Medico gentoo-dev 2019-05-10 08:52:36 UTC
PEP 492 includes sample code that can be adapted to handle asynchronous context managers in older versions of python:

> mgr = (EXPR)
> aexit = type(mgr).__aexit__
> aenter = type(mgr).__aenter__(mgr)
> VAR = await aenter
> try:
>     BLOCK
> except:
>     if not await aexit(mgr, *sys.exc_info()):
>         raise
> else:
>     await aexit(mgr, None, None, None)

Here's a compat_coroutine version of it:

from portage.util.futures.compat_coroutine

def async_with(mgr, async_func):
  aexit = type(mgr).__aexit__
  aenter = type(mgr).__aenter__(mgr)

  aenter_result = (yield aenter)
    return_value = (yield async_func(aenter_result))
    if not (yield aexit(mgr, *sys.exc_info()):
    yield aexit(mgr, None, None, None)

Comment 3 Zac Medico gentoo-dev 2023-12-14 16:52:53 UTC
We might inject them in batches in order to reduce the number of package index updates (especially for parallel quickpkg as in bug 898648).