Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 924192 - sys-apps/portage: test start many redundant gpg keepalive daemon threads
Summary: sys-apps/portage: test start many redundant gpg keepalive daemon threads
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on: 921380
Blocks:
  Show dependency tree
 
Reported: 2024-02-10 03:45 UTC by Zac Medico
Modified: 2024-02-22 07:24 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Zac Medico gentoo-dev 2024-02-10 03:45:21 UTC
This issue was recently reported in a comment on the PR for bug 923368, and it's a symptom of the fact that the tests start many redundant gpg keepalive daemon threads:

https://github.com/gentoo/portage/pull/1246#issuecomment-1935537197

Exception in thread Thread-2 (gpg_keepalive):
Traceback (most recent call last):
  File "/usr/lib/pypy3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/pypy3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/home/sam/git/portage/lib/portage/gpg.py", line 106, in gpg_keepalive
    raise GPGException("GPG keepalive failed")
portage.exception.GPGException: GPG keepalive failed
Exception in thread Thread-4 (gpg_keepalive):
Traceback (most recent call last):
  File "/usr/lib/pypy3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/pypy3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/home/sam/git/portage/lib/portage/gpg.py", line 106, in gpg_keepalive
    raise GPGException("GPG keepalive failed")
portage.exception.GPGException: GPG keepalive failed
Exception in thread Thread-6 (gpg_keepalive):
Traceback (most recent call last):
  File "/usr/lib/pypy3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/pypy3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/home/sam/git/portage/lib/portage/gpg.py", line 106, in gpg_keepalive
    raise GPGException("GPG keepalive failed")
portage.exception.GPGException: GPG keepalive failed
Comment 1 Larry the Git Cow gentoo-dev 2024-02-10 06:09:50 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=03be12ec5f46629fa928e5fcd45d3fe6745d5d0a

commit 03be12ec5f46629fa928e5fcd45d3fe6745d5d0a
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2024-02-09 22:12:02 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2024-02-10 06:08:59 +0000

    GPG: Use threading.Event for thread safety
    
    Use threading.Event for thread safety during GPG stop, and use the
    wait method to improve responsiveness for stop requests.
    
    Bug: https://bugs.gentoo.org/924192
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 lib/portage/gpg.py | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

https://gitweb.gentoo.org/proj/portage.git/commit/?id=d7115d18dada572b6b10d6be30d0fa7fb325a2c8

commit d7115d18dada572b6b10d6be30d0fa7fb325a2c8
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2024-02-09 17:19:54 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2024-02-10 06:08:58 +0000

    GPG: Proactively stop to avoid "GPG keepalive failed" error in pypy ci jobs
    
    This seems to help mitigate pypy ci job hangs like those fba76a545f2
    triggered.
    
    Bug: https://bugs.gentoo.org/924192
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 bin/quickpkg                                     | 13 ++++++++-----
 lib/_emerge/actions.py                           | 10 +++++++---
 lib/portage/tests/gpkg/test_gpkg_gpg.py          | 23 ++++++++++++++++++++++-
 lib/portage/tests/gpkg/test_gpkg_metadata_url.py |  5 ++++-
 4 files changed, 41 insertions(+), 10 deletions(-)
Comment 2 Zac Medico gentoo-dev 2024-02-10 20:05:33 UTC
There's a similar case in test_gpkg_metadata_url_case where two test methods call self.start_http_server and both leave an http server daemon thread running in the background.
Comment 3 Larry the Git Cow gentoo-dev 2024-02-10 20:27:32 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=d320211266b95dc8cbea295cb234a607246d955f

commit d320211266b95dc8cbea295cb234a607246d955f
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2024-02-10 20:25:28 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2024-02-10 20:26:17 +0000

    test_gpkg_metadata_url_case: shutdown http server daemon threads
    
    Bug: https://bugs.gentoo.org/924192
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 lib/portage/tests/gpkg/test_gpkg_metadata_url.py | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
Comment 4 Larry the Git Cow gentoo-dev 2024-02-22 07:24:19 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=77c44c46194922509bc4f2b5cfc099412a560a69

commit 77c44c46194922509bc4f2b5cfc099412a560a69
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2024-02-22 07:23:40 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-02-22 07:23:50 +0000

    sys-apps/portage: add 3.0.62
    
    Closes: https://bugs.gentoo.org/663324
    Closes: https://bugs.gentoo.org/728046
    Closes: https://bugs.gentoo.org/891137
    Closes: https://bugs.gentoo.org/906368
    Closes: https://bugs.gentoo.org/916566
    Closes: https://bugs.gentoo.org/921170
    Closes: https://bugs.gentoo.org/921208
    Closes: https://bugs.gentoo.org/921400
    Closes: https://bugs.gentoo.org/922038
    Closes: https://bugs.gentoo.org/922142
    Closes: https://bugs.gentoo.org/923368
    Closes: https://bugs.gentoo.org/923750
    Closes: https://bugs.gentoo.org/923841
    Closes: https://bugs.gentoo.org/923852
    Closes: https://bugs.gentoo.org/923854
    Closes: https://bugs.gentoo.org/924192
    Closes: https://bugs.gentoo.org/924273
    Closes: https://bugs.gentoo.org/924585
    Closes: https://bugs.gentoo.org/921380
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-3.0.62.ebuild | 246 +++++++++++++++++++++++++++++++++
 2 files changed, 247 insertions(+)