Bug 614112 – sys-apps/portage: EbuildBuildDir.lock triggers event loop recursion

• Product: Portage Development
• Status: RESOLVED FIXED

[Description Zac Medico 2017-03-28 04:25:22 UTC]

The EbuildBuildDir.lock method triggers event loop recursion in the catdir_lock.wait() and builddir_lock.wait() calls.

[Comment 1 Zac Medico 2018-04-21 09:09:12 UTC]

I've posted a patch series for review:

https://archives.gentoo.org/gentoo-portage-dev/message/ab45c90e7f0e9faab45614be43d64db6
https://github.com/gentoo/portage/pull/309

[Comment 2 Larry the Git Cow 2018-04-22 21:52:11 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=aa6ae16b606c6ed177279f6ef03651c006ff051e

commit aa6ae16b606c6ed177279f6ef03651c006ff051e
Author:     Zac Medico <[e-mail]>
AuthorDate: 2018-04-21 08:07:32 +0000
Commit:     Zac Medico <[e-mail]>
CommitDate: 2018-04-22 18:30:53 +0000

    EbuildBuildDir: remove synchronous lock method (bug 614112)
    
    The synchronous lock method can trigger event loop recursion if the
    event loop is already running, which is incompatible with asyncio's
    default event loop. Therefore, migrate the last consumers to use the
    async_lock method, and remove the synchronous lock method.
    
    Bug: https://bugs.gentoo.org/614112
    Bug: https://bugs.gentoo.org/649588

 pym/_emerge/EbuildBuildDir.py               | 47 -----------------------------
 pym/_emerge/Scheduler.py                    |  2 +-
 pym/portage/dbapi/vartree.py                |  2 +-
 pym/portage/package/ebuild/doebuild.py      |  9 ++++--
 pym/portage/tests/ebuild/test_ipc_daemon.py |  2 +-
 5 files changed, 9 insertions(+), 53 deletions(-)

https://gitweb.gentoo.org/proj/portage.git/commit/?id=c59348dc275b433d61735f8053e0db1a2ef6432b

commit c59348dc275b433d61735f8053e0db1a2ef6432b
Author:     Zac Medico <[e-mail]>
AuthorDate: 2018-04-21 07:59:00 +0000
Commit:     Zac Medico <[e-mail]>
CommitDate: 2018-04-22 18:30:53 +0000

    PackageUninstall: use async_lock (bug 614112)
    
    Asynchronously lock the build directory, and use AsyncTaskFuture
    to fit the resulting future into the CompositeTask framework that
    PackageUninstall uses.
    
    Bug: https://bugs.gentoo.org/614112

 pym/_emerge/PackageUninstall.py | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

https://gitweb.gentoo.org/proj/portage.git/commit/?id=0dfa50e1d85aea0d63497fb823139541bc989e2e

commit 0dfa50e1d85aea0d63497fb823139541bc989e2e
Author:     Zac Medico <[e-mail]>
AuthorDate: 2018-04-21 07:49:00 +0000
Commit:     Zac Medico <[e-mail]>
CommitDate: 2018-04-22 18:30:53 +0000

    EbuildBuild: use async_lock (bug 614112)
    
    Asynchronously lock the build directory, and use AsyncTaskFuture
    to fit the resulting future into the CompositeTask framework that
    EbuildBuild uses.
    
    Bug: https://bugs.gentoo.org/614112

 pym/_emerge/EbuildBuild.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

https://gitweb.gentoo.org/proj/portage.git/commit/?id=ee5305f26a0bdc6054307289c2487c241735bcbb

commit ee5305f26a0bdc6054307289c2487c241735bcbb
Author:     Zac Medico <[e-mail]>
AuthorDate: 2018-04-21 07:32:45 +0000
Commit:     Zac Medico <[e-mail]>
CommitDate: 2018-04-22 18:30:52 +0000

    Binpkg: use async_lock (bug 614112)
    
    Asynchronously lock the build directory, and use AsyncTaskFuture
    to fit the resulting future into the CompositeTask framework that
    Binpkg uses.
    
    Bug: https://bugs.gentoo.org/614112

 pym/_emerge/Binpkg.py | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

https://gitweb.gentoo.org/proj/portage.git/commit/?id=30c69adfc0ffa450ff3a4d4d176023db66171ae7

commit 30c69adfc0ffa450ff3a4d4d176023db66171ae7
Author:     Zac Medico <[e-mail]>
AuthorDate: 2018-04-21 06:36:29 +0000
Commit:     Zac Medico <[e-mail]>
CommitDate: 2018-04-22 18:30:52 +0000

    AbstractEbuildProcess: use async_lock (bug 614112)
    
    Asynchronously lock the build directory. The asynchronous lock delays
    creation of the pid, and it's possible for the _wait method to be called
    before the pid is available. Therefore, AbstractEbuildProcess._wait()
    must wait for the pid to become available before it can call the
    SpawnProcess._wait() method.
    
    Bug: https://bugs.gentoo.org/614112

 pym/_emerge/AbstractEbuildProcess.py | 30 ++++++++++++++++++++++++++----
 1 file changed, 26 insertions(+), 4 deletions(-)

https://gitweb.gentoo.org/proj/portage.git/commit/?id=9d7b43779d17ca9f698bb90e0ac7b1f43afa9575

commit 9d7b43779d17ca9f698bb90e0ac7b1f43afa9575
Author:     Zac Medico <[e-mail]>
AuthorDate: 2018-04-21 06:05:34 +0000
Commit:     Zac Medico <[e-mail]>
CommitDate: 2018-04-22 18:30:52 +0000

    EbuildBuildDir: add async_lock method (bug 614112)
    
    Asynchronoulsy handle locking of both the category directory and
    build directory.
    
    Bug: https://bugs.gentoo.org/614112

 pym/_emerge/EbuildBuildDir.py | 76 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 76 insertions(+)}

[Comment 3 Zac Medico 2018-07-02 18:40:41 UTC]

Fixed in portage-2.3.40-r1.