Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 591760

Summary: sys-apps/portage: [TRACKER] asyncio compatibility and interoperability
Product: Portage Development Reporter: Zac Medico <zmedico>
Component: CoreAssignee: Portage team <dev-portage>
Status: RESOLVED FIXED    
Severity: enhancement Keywords: Tracker
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: All   
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on: 613990, 614102, 614104, 614108, 614110, 614112, 614116, 649588, 653810, 653844, 653848, 653856, 653946, 654038, 654276, 654382, 654390, 655378    
Bug Blocks: 758719, 651804    

Description Zac Medico gentoo-dev 2016-08-20 19:27:33 UTC
Currently, portage's internal EventLoop exposes an iteration() method which is used as follows:

   while not condition:
       event_loop.iteration()

The asyncio.AbstractEventLoop.run_until_complete(future) interface is much nicer because it allows a loop to exit as soon as a future condition is met, reducing latency in situations where it is desirable for a loop to exit at the earliest opportunity. Therefore, it is desirable for portage's EventLoop to support this interface. Eliminating portage's EventLoop.iteration() method will make it possible for EventLoop to be implemented using an asyncio.AbstractEventLoop instance.
Comment 2 Zac Medico gentoo-dev 2016-08-23 16:54:04 UTC
The initial run_until_complete implementation is in the master branch:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=995f0f983386e2a82dbce65d4366ee7f58f59138

I'll keep this bug open to track progress of the work that is planned:

* Migrate all internal use of the EventLoop.iteration method to the new
run_until_complete(future) method, and remove the EventLoop.iteration
method (or make it private as long as it's needed to implement
run_until_complete for older python versions).

* Implement all EventLoop methods using asyncio.AbstractEventLoop
methods (but keep existing implementations for use with older python).
Comment 3 Zac Medico gentoo-dev 2017-03-21 05:40:57 UTC
Since asyncio.AbstractEventLoop has no equivalent to the EventLoop.idle_add method, we need to implement the AbstractEventLoop call_soon and call_soon_threadsafe methods, and use them to replace all idle_add usage.
Comment 6 Zac Medico gentoo-dev 2017-03-24 20:51:40 UTC
The next step is to implement add_done_callback and remove_done_callback methods for portage.util.futures.futures.Future, so that run_until_complete can use a done callback to trigger earlier return of the iteration method.
Comment 7 Zac Medico gentoo-dev 2017-03-26 10:22:44 UTC
I've posted a patch that implements add_done_callback:

https://archives.gentoo.org/gentoo-portage-dev/message/ca089280e6f3a06f8920d2c85c54fb92
https://github.com/gentoo/portage/pull/148
Comment 8 Zac Medico gentoo-dev 2017-03-26 21:40:59 UTC
add_done_callback is in the master branch:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=d90ff7c046b39de82558655375ea104cfa19b176

Since the asyncio event loop does not allow recursive calls to run_until_complete/run_forever methods (RuntimeError: This event loop is already running), any of the existing code that relies on this sort of recursive call will have to be migrated to use an asynchronous add_done_callback approach. Calls to EventLoop.iteration() and AsynchronousTask.wait() are the main things to look for. Both of these methods are going to have to raise RuntimeError if called while the event loop is already running.
Comment 9 Zac Medico gentoo-dev 2017-03-27 05:11:22 UTC
In testing, then only event loop recursion that I've triggered is in the SpawnProcess._pipe_logger_exit method where it calls self.wait() before the process exit status has become available, ultimately triggering recursion in the SubProcess._waitpid_loop method. The solution is to make SpawnProcess._pipe_logger_exit use child_watch_add trigger an asynchronous call to self.wait(). Here is the relevant part of the stack trace:

  File "pym/_emerge/SpawnProcess.py", line 174, in _pipe_logger_exit
    self.wait()
  File "pym/_emerge/AsynchronousTask.py", line 54, in wait
    self._wait()
  File "pym/_emerge/SubProcess.py", line 100, in _wait
    self._waitpid_loop()
  File "pym/_emerge/SpawnProcess.py", line 177, in _waitpid_loop
    SubProcess._waitpid_loop(self)
  File "pym/_emerge/SubProcess.py", line 109, in _waitpid_loop
    self.scheduler.iteration()
  File "pym/portage/util/_eventloop/EventLoop.py", line 240, in iteration
    raise RuntimeError("This event loop is already running")
RuntimeError: This event loop is already running
Comment 10 Zac Medico gentoo-dev 2017-03-27 06:32:19 UTC
I've got a patch that disables event loop recursion in this branch:

https://github.com/zmedico/portage/tree/event_loop_recursion

However, we need to be fairly confident that all cases like bug 613990 have been fixed before we entirely disable recursion.
Comment 11 Zac Medico gentoo-dev 2017-05-20 17:50:03 UTC
The asyncio event loop currently does not work in a forked process when the parent was running an event loop:

https://bugs.python.org/issue21998

Than needs to be fixed, or portage would have to stop running event loops in child processes (the global_event_loop function returns a separate EventLoop instance for each pid).
Comment 12 Larry the Git Cow gentoo-dev 2018-02-25 06:43:43 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=81baf80258393938152d6c8fc53d33d5f85de23c

commit 81baf80258393938152d6c8fc53d33d5f85de23c
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-02-25 06:17:40 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-02-25 06:19:18 +0000

    EventLoop: implement call_later for asyncio compat (bug 591760)
    
    Bug: https://bugs.gentoo.org/591760

 pym/portage/tests/ebuild/test_ipc_daemon.py | 12 ++++++------
 pym/portage/util/_eventloop/EventLoop.py    | 28 ++++++++++++++++++++++++++++
 2 files changed, 34 insertions(+), 6 deletions(-)}
Comment 13 Zac Medico gentoo-dev 2018-03-04 00:35:13 UTC
(In reply to Zac Medico from comment #11)
> The asyncio event loop currently does not work in a forked process when the
> parent was running an event loop:
> 
> https://bugs.python.org/issue21998
> 
> Than needs to be fixed, or portage would have to stop running event loops in
> child processes (the global_event_loop function returns a separate EventLoop
> instance for each pid).

I've found this workaround for asyncio that I've posted on the upstream issue:

> # https://bugs.python.org/issue21998#msg313196
> 
> _default_policy = asyncio.get_event_loop_policy()
> _pid_loop = {}
> 
> class MultiprocessingPolicy(asyncio.AbstractEventLoopPolicy):
>     def get_event_loop(self):
>         pid = os.getpid()
>         loop = _pid_loop.get(pid)
>         if loop is None:
>             loop = self.new_event_loop()
>             _pid_loop[pid] = loop
>         return loop
> 
>     def new_event_loop(self):
>         return _default_policy.new_event_loop()
> 
> asyncio.set_event_loop_policy(MultiprocessingPolicy())
Comment 14 Zac Medico gentoo-dev 2018-03-11 10:40:18 UTC
For fork support, this issue appears to be more relevant:

https://bugs.python.org/issue22087
Comment 15 Larry the Git Cow gentoo-dev 2018-03-16 17:18:26 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=a997f3e2d4318b8b9129d5cccd3c5f2c5677d11d

commit a997f3e2d4318b8b9129d5cccd3c5f2c5677d11d
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-03-16 08:40:42 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-03-16 17:17:53 +0000

    EventLoop: implement time method for asyncio compat (bug 591760)
    
    Use time.monotonic() which is available in Python 3.3 and later,
    and otherwise emulate it by using an offset to counteract any
    backward movements.
    
    Bug: https://bugs.gentoo.org/591760

 pym/portage/util/_eventloop/EventLoop.py | 19 +++++++++++++-----
 pym/portage/util/monotonic.py            | 34 ++++++++++++++++++++++++++++++++
 2 files changed, 48 insertions(+), 5 deletions(-)}
Comment 16 Larry the Git Cow gentoo-dev 2018-04-08 21:18:21 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=13621b62e32a7c21aa08247b33f1faa0a146d0d4

commit 13621b62e32a7c21aa08247b33f1faa0a146d0d4
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-08 21:15:39 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-08 21:15:39 +0000

    EventLoop: add get/set_debug methods for asyncio compat (bug 591760)
    
    Bug: https://bugs.gentoo.org/591760

 pym/portage/util/_eventloop/EventLoop.py | 14 ++++++++++++++
 1 file changed, 14 insertions(+)}
Comment 17 Larry the Git Cow gentoo-dev 2018-04-08 21:37:16 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=754010f346ec2455ea8c71a6af4796c10fd28d23

commit 754010f346ec2455ea8c71a6af4796c10fd28d23
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-08 21:36:07 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-08 21:36:07 +0000

    EventLoop: add is_closed method for asyncio compat (bug 591760)
    
    Bug: https://bugs.gentoo.org/591760

 pym/portage/util/_eventloop/EventLoop.py | 4 ++++
 1 file changed, 4 insertions(+)}
Comment 18 Larry the Git Cow gentoo-dev 2018-04-09 01:27:39 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=3b1c182750d82dc55b96ee111e356968ca9a9fb7

commit 3b1c182750d82dc55b96ee111e356968ca9a9fb7
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-09 01:25:25 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-09 01:25:52 +0000

    EventLoop: add call_at method for asyncio compat (bug 591760)
    
    Bug: https://bugs.gentoo.org/591760

 pym/portage/util/_eventloop/EventLoop.py | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)}
Comment 19 Larry the Git Cow gentoo-dev 2018-04-17 17:45:35 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=a9e8ebaa6979ccf0bb385e457d695bedc7b65bf5

commit a9e8ebaa6979ccf0bb385e457d695bedc7b65bf5
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-17 09:51:36 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-17 17:44:58 +0000

    Add async_iter_completed for asyncio migration (bug 591760)
    
    This serves as a wrapper around portage's internal TaskScheduler
    class, allowing TaskScheduler API consumers to be migrated to use
    asyncio interfaces.
    
    Bug: https://bugs.gentoo.org/591760

 .../tests/util/futures/test_iter_completed.py      | 37 ++++++++++++-
 pym/portage/util/futures/iter_completed.py         | 61 +++++++++++++++++++---
 2 files changed, 91 insertions(+), 7 deletions(-)}
Comment 20 Larry the Git Cow gentoo-dev 2018-04-21 20:17:08 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=3031af210e44af825096a7601fa368851428e661

commit 3031af210e44af825096a7601fa368851428e661
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-21 20:13:53 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-21 20:16:39 +0000

    CompositeTask._wait: recurse less (bug 591760)
    
    In order to reduce the amount of recursion, use a future to
    wait for task exit.
    
    Bug: https://bugs.gentoo.org/591760

 pym/_emerge/CompositeTask.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)}
Comment 21 Larry the Git Cow gentoo-dev 2018-04-22 17:50:02 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=d836b0d32797f1431ebeaa20e2a18ea5cb7c3f7c

commit d836b0d32797f1431ebeaa20e2a18ea5cb7c3f7c
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-22 15:56:29 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-22 17:48:52 +0000

    AsyncFunction: use _async_waitpid (bug 591760)
    
    When pid exit status is not yet available, use the Subprocess
    _async_waitpid() method to avoid event loop recursion.
    
    Bug: https://bugs.gentoo.org/591760

 pym/portage/util/_async/AsyncFunction.py | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)}
Comment 22 Larry the Git Cow gentoo-dev 2018-04-23 01:03:37 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=3eb46b05eb098507ea74108559bf77006e0f8142

commit 3eb46b05eb098507ea74108559bf77006e0f8142
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-23 00:49:15 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-23 00:49:15 +0000

    FileDigester: use _async_waitpid (bug 591760)
    
    When pid exit status is not yet available, use the Subprocess
    _async_waitpid() method to avoid event loop recursion.
    
    Bug: https://bugs.gentoo.org/591760

 pym/portage/util/_async/FileDigester.py | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)}
Comment 23 Larry the Git Cow gentoo-dev 2018-04-23 18:26:32 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=c23d093d0330a9318534a1c521fb00d6360fabc0

commit c23d093d0330a9318534a1c521fb00d6360fabc0
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-23 18:20:25 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-23 18:24:11 +0000

    tests: prove run_until_complete executes done callbacks
    
    Prove that done callbacks are executed before run_until_complete
    returns, which is how asyncio's default event loop behaves. This
    behavior was implemented in portage's internal event loop in commit
    25245d7eb86ed197b7d7cfead0dbe4ce8ad4bc5b.
    
    Bug: https://bugs.gentoo.org/591760

 .../futures/asyncio/test_run_until_complete.py     | 29 ++++++++++++++++++++++
 1 file changed, 29 insertions(+)}
Comment 24 Larry the Git Cow gentoo-dev 2018-04-24 06:57:17 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=7f1a362fea4ca0bf4cd0476ceb3a61afb8ef1a7e

commit 7f1a362fea4ca0bf4cd0476ceb3a61afb8ef1a7e
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-24 06:40:46 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-24 06:56:18 +0000

    EbuildMetadataPhase._output_handler: fix event loop recursion (bug 591760)
    
    When pid exit status is not yet available, use the Subprocess
    _async_waitpid() method to avoid event loop recursion.
    
    Bug: https://bugs.gentoo.org/591760

 pym/_emerge/EbuildMetadataPhase.py | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)}
Comment 25 Larry the Git Cow gentoo-dev 2018-04-25 07:30:22 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=a7c8ddd9968741fd51ba21500a9f6b9ef27bef15

commit a7c8ddd9968741fd51ba21500a9f6b9ef27bef15
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-25 07:28:44 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-25 07:28:44 +0000

    FetchTask._fetch_uri: use _async_wait() (bug 591760)
    
    Use _async_wait() to avoid event loop recursion in
    emirrordist --dry-run mode.
    
    Bug: https://bugs.gentoo.org/591760

 pym/portage/_emirrordist/FetchTask.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)}
Comment 26 Larry the Git Cow gentoo-dev 2018-04-26 05:49:00 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=384b61dd24a21aaccd7c643e58b07ea56c3cbdfc

commit 384b61dd24a21aaccd7c643e58b07ea56c3cbdfc
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-26 05:31:54 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-26 05:46:30 +0000

    PipeLogger._output_handler: use _async_wait() (bug 591760)
    
    Use _async_wait() to avoid event loop recursion.
    
    Bug: https://bugs.gentoo.org/591760

 pym/portage/util/_async/PipeLogger.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)}
Comment 27 Larry the Git Cow gentoo-dev 2018-04-26 06:20:15 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=f7b1b61f682f3d635aaded630d8e61f1756c8023

commit f7b1b61f682f3d635aaded630d8e61f1756c8023
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-26 06:12:19 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-26 06:13:57 +0000

    PipeReader._output_handler: use _async_wait() (bug 591760)
    
    Use _async_wait() to avoid event loop recursion.
    
    Bug: https://bugs.gentoo.org/591760

 pym/_emerge/PipeReader.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)}
Comment 28 Larry the Git Cow gentoo-dev 2018-04-26 07:12:23 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=94a37752ab21883065ed7150b902dfda2b45a5b3

commit 94a37752ab21883065ed7150b902dfda2b45a5b3
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-26 07:00:18 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-26 07:08:54 +0000

    AsynchronousLock: use _async_wait() (bug 591760)
    
    Use _async_wait() to avoid event loop recursion.
    
    Bug: https://bugs.gentoo.org/591760

 pym/_emerge/AsynchronousLock.py | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)}
Comment 29 Larry the Git Cow gentoo-dev 2018-04-26 08:06:04 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=d16206ccd315edbd4bd7a46c0b1aa9e59a46db43

commit d16206ccd315edbd4bd7a46c0b1aa9e59a46db43
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-26 07:45:32 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-26 08:02:28 +0000

    AsyncScheduler._poll(): fix event loop recursion (bug 591760)
    
    Call self._cleanup() and set the returncode, in order to avoid
    event loop recursion in self.wait().
    
    Bug: https://bugs.gentoo.org/591760

 pym/portage/util/_async/AsyncScheduler.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

https://gitweb.gentoo.org/proj/portage.git/commit/?id=03a635e3a95fbe4afc6fb2fae2e56c03ef946933

commit 03a635e3a95fbe4afc6fb2fae2e56c03ef946933
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-26 07:40:05 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-26 08:01:05 +0000

    AbstractPollTask._unregister_if_appropriate: use _async_wait() (bug 591760)
    
    Use _async_wait() to avoid event loop recursion. This change
    only affects PipeLogger, PipeReader, and FifoWriter, since other
    subclasses inherit the Subprocess implementation that was fixed
    in commit c5ab0696cec1f0f020d2d8ad416f9b1444a467de.
    
    Bug: https://bugs.gentoo.org/591760

 pym/_emerge/AbstractPollTask.py | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

https://gitweb.gentoo.org/proj/portage.git/commit/?id=f1fbe2dcf51984d5fd06a14541335618537aa006

commit f1fbe2dcf51984d5fd06a14541335618537aa006
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-26 07:38:28 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-26 07:58:29 +0000

    PipeReader._array_output_handler: use _async_wait() (bug 591760)
    
    Use _async_wait() to avoid event loop recursion.
    
    Bug: https://bugs.gentoo.org/591760

 pym/_emerge/PipeReader.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)}
Comment 30 Larry the Git Cow gentoo-dev 2018-04-26 08:46:06 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=391daef6fce981acd5a01e41a0f7238044c48877

commit 391daef6fce981acd5a01e41a0f7238044c48877
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-26 08:29:04 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-26 08:43:53 +0000

    MirrorDistTask._term_callback(): use _async_wait() (bug 591760)
    
    Use _async_wait() to avoid event loop recursion, but don't call
    it prematurely, since since that could trigger event loop recursion
    if the current (cancelled) task's exit callback does not set the
    returncode first.
    
    Bug: https://bugs.gentoo.org/591760

 pym/portage/_emirrordist/MirrorDistTask.py | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

https://gitweb.gentoo.org/proj/portage.git/commit/?id=9b93296e14e2e7f52cd2bf4d626eb7387f6786ab

commit 9b93296e14e2e7f52cd2bf4d626eb7387f6786ab
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-26 08:09:08 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-26 08:39:19 +0000

    CompositeTask._cancel(): use _async_wait() (bug 591760)
    
    After setting the returncode due to the current task being
    queued, use _async_wait() for consistency with the case
    where the current task needs to be cancelled, so that either
    case ultimately results in notification of exit listeners.
    
    Bug: https://bugs.gentoo.org/591760

 pym/_emerge/CompositeTask.py | 1 +
 1 file changed, 1 insertion(+)}
Comment 31 Larry the Git Cow gentoo-dev 2018-04-26 09:37:43 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=a65d93bdeb4d964603cd0ce9b0a75a571c9bdefa

commit a65d93bdeb4d964603cd0ce9b0a75a571c9bdefa
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-26 09:19:22 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-26 09:31:09 +0000

    SchedulerInterface: expose AbstractEventLoop methods (bug 591760)
    
    Bug: https://bugs.gentoo.org/591760

 pym/portage/util/_async/SchedulerInterface.py | 34 +++++++++++++++++++++++----
 1 file changed, 30 insertions(+), 4 deletions(-)}
Comment 32 Larry the Git Cow gentoo-dev 2018-04-29 06:09:03 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=be800bf0153a28ce034277d103a2021f93ac8b2e

commit be800bf0153a28ce034277d103a2021f93ac8b2e
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-29 05:10:27 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-29 05:58:58 +0000

    PopenProcess: add_child_handler asyncio compat (bug 591760)
    
    Migrate to asyncio's AbstractChildWatcher.add_child_handler
    interface.
    
    Bug: https://bugs.gentoo.org/591760

 pym/portage/util/_async/PopenProcess.py | 31 +++++++++++++++++++------------
 1 file changed, 19 insertions(+), 12 deletions(-)}
Comment 33 Larry the Git Cow gentoo-dev 2018-04-29 22:24:21 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=abf4b7abcdc53bbbd86eb6f79d45417b54ed64d2

commit abf4b7abcdc53bbbd86eb6f79d45417b54ed64d2
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-29 22:12:13 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-29 22:18:28 +0000

    AsyncScheduler: call_later asyncio compat (bug 591760)
    
    Use call_later for asyncio compatibility. Also remove the
    timeout_add method from SchedulerInterface, since there are
    no more consumers.
    
    Bug: https://bugs.gentoo.org/591760

 pym/portage/util/_async/AsyncScheduler.py     | 9 +++++++--
 pym/portage/util/_async/SchedulerInterface.py | 1 -
 2 files changed, 7 insertions(+), 3 deletions(-)

https://gitweb.gentoo.org/proj/portage.git/commit/?id=28fd93d4de0c1adcecb7ce10c7514d5718ccf43f

commit 28fd93d4de0c1adcecb7ce10c7514d5718ccf43f
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-29 21:58:38 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-29 22:18:28 +0000

    Scheduler: call_later asyncio compat (bug 591760)
    
    Use call_later for asyncio compatibility.
    
    Bug: https://bugs.gentoo.org/591760

 pym/_emerge/Scheduler.py | 39 +++++++++++++++++++--------------------
 1 file changed, 19 insertions(+), 20 deletions(-)

https://gitweb.gentoo.org/proj/portage.git/commit/?id=4da425966a82bfbbb68908010995941a44b45598

commit 4da425966a82bfbbb68908010995941a44b45598
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-04-29 21:39:28 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-04-29 21:41:07 +0000

    AbstractEbuildProcess: call_later asyncio compat (bug 591760)
    
    Use call_later for asyncio compatibility.
    
    Bug: https://bugs.gentoo.org/591760

 pym/_emerge/AbstractEbuildProcess.py | 11 ++++-------
 pym/_emerge/SubProcess.py            |  2 +-
 2 files changed, 5 insertions(+), 8 deletions(-)}
Comment 34 Zac Medico gentoo-dev 2018-05-08 17:43:30 UTC
Support is complete now, since bug 654390 was the last step.
Comment 35 Zac Medico gentoo-dev 2018-07-02 18:56:40 UTC
Fixed in portage-2.3.40-r1.