Summary: | sys-apps/portage: FetchIterator.__iter__ event loop recursion | ||
---|---|---|---|
Product: | Portage Development | Reporter: | Zac Medico <zmedico> |
Component: | Core | Assignee: | Portage team <dev-portage> |
Status: | RESOLVED FIXED | ||
Severity: | normal | Keywords: | InVCS |
Priority: | Normal | ||
Version: | unspecified | ||
Hardware: | All | ||
OS: | All | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Bug Depends on: | 648790 | ||
Bug Blocks: | 591760, 651804, 653856 |
I've posted a patch for review that will fix this bug: https://archives.gentoo.org/gentoo-portage-dev/message/7dc8c8f70e80bb5628541e245c2823db https://github.com/gentoo/portage/pull/315 The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=7c652d1b967f9c4c6a7fbd9fc5e46a0e57438a16 commit 7c652d1b967f9c4c6a7fbd9fc5e46a0e57438a16 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2018-04-25 06:42:10 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2018-04-28 13:56:42 +0000 FetchIterator: fix event loop recursion (bug 654038) Since construction of FetchTask instances requires results from aux_get calls that would trigger event loop recursion when executed synchronously, add an _async_fetch_tasks function to construct FetchTask instances asynchronously and return a Future. Use an _EbuildFetchTasks class to wait for the FetchTask instances to become available, and then execute them. Bug: https://bugs.gentoo.org/654038 pym/portage/_emirrordist/FetchIterator.py | 324 ++++++++++++++++++++---------- 1 file changed, 218 insertions(+), 106 deletions(-)} Fixed in portage-2.3.40-r1. |
The FetchIterator.__iter__ method needs to yield one asynchronous task per ebuild, in order to make this aux_get call asynchronous: > Traceback (most recent call last): > File "bin/emirrordist", line 21, in <module> > sys.exit(emirrordist_main(sys.argv[1:])) > File "pym/portage/_emirrordist/main.py", line 438, in emirrordist_main > signum = run_main_scheduler(MirrorDistTask(config)) > File "pym/portage/util/_async/run_main_scheduler.py", line 27, in run_main_scheduler > scheduler.wait() > File "pym/_emerge/AsynchronousTask.py", line 54, in wait > self._wait() > File "pym/portage/_emirrordist/MirrorDistTask.py", line 237, in _wait > CompositeTask._wait(self) > File "pym/_emerge/CompositeTask.py", line 90, in _wait > self.scheduler.iteration() > File "pym/portage/util/_eventloop/EventLoop.py", line 281, in iteration > return self._iteration(*args) > File "pym/portage/util/_eventloop/EventLoop.py", line 302, in _iteration > if self._run_timeouts(): > File "pym/portage/util/_eventloop/EventLoop.py", line 597, in _run_timeouts > if self._run_idle_callbacks(): > File "pym/portage/util/_eventloop/EventLoop.py", line 556, in _run_idle_callbacks > if x._callback(*x._args): > File "pym/_emerge/AsynchronousTask.py", line 74, in _async_wait_cb > self.wait() > File "pym/_emerge/AsynchronousTask.py", line 57, in wait > self._wait_hook() > File "pym/_emerge/AsynchronousTask.py", line 175, in _wait_hook > self._exit_listener_stack.pop()(self) > File "pym/portage/util/_async/AsyncScheduler.py", line 64, in _task_exit > self._schedule() > File "pym/_emerge/PollScheduler.py", line 154, in _schedule > self._schedule_tasks() > File "pym/portage/util/_async/AsyncScheduler.py", line 48, in _schedule_tasks > task = self._next_task() > File "pym/portage/util/_async/TaskScheduler.py", line 20, in _next_task > return next(self._task_iter) > File "pym/portage/_emirrordist/FetchIterator.py", line 72, in __iter__ > mytree=tree) > File "pym/portage/dbapi/porttree.py", line 587, in aux_get > myrepo=myrepo, loop=loop)) > File "pym/portage/util/_eventloop/EventLoop.py", line 827, in run_until_complete > self.iteration()