When don't want asynchronous code to block while waiting for a lock on a binary package repository, so there should be an asynchronous variant of any method that requires a lock. For example, Binpkg and BinpkgPrefetcher classes each call the binarytree.inject method within an asynchronous context, so an asynchronous variant of this method is needed.
For the locks (also see bug 685236) we really should use a (python3.5+ only) asynchronous context manager: https://www.python.org/dev/peps/pep-0492/#asynchronous-context-managers-and-async-with Given that python2.7 is scheduled for retirement in 2020 (https://pythonclock.org/), maybe it's time to drop support for python3.4 and earlier.
PEP 492 includes sample code that can be adapted to handle asynchronous context managers in older versions of python: > mgr = (EXPR) > aexit = type(mgr).__aexit__ > aenter = type(mgr).__aenter__(mgr) > > VAR = await aenter > try: > BLOCK > except: > if not await aexit(mgr, *sys.exc_info()): > raise > else: > await aexit(mgr, None, None, None) Here's a compat_coroutine version of it: from portage.util.futures.compat_coroutine @compat_coroutine.coroutine def async_with(mgr, async_func): aexit = type(mgr).__aexit__ aenter = type(mgr).__aenter__(mgr) aenter_result = (yield aenter) try: return_value = (yield async_func(aenter_result)) except: if not (yield aexit(mgr, *sys.exc_info()): raise else: yield aexit(mgr, None, None, None) compat_coroutine.coroutine_return(return_value)
We might inject them in batches in order to reduce the number of package index updates (especially for parallel quickpkg as in bug 898648).
