Summary: | =sys-apps/portage-3.0.63 exits with python stack about async | ||
---|---|---|---|
Product: | Portage Development | Reporter: | Dan Goodliffe <gentoo> |
Component: | Core | Assignee: | Portage team <dev-portage> |
Status: | RESOLVED FIXED | ||
Severity: | normal | CC: | gentoo, zmedico |
Priority: | Normal | Keywords: | InVCS |
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
See Also: |
https://bugs.gentoo.org/show_bug.cgi?id=923750 https://github.com/gentoo/portage/pull/1295 |
||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Bug Depends on: | 925481 | ||
Bug Blocks: | |||
Attachments: | Screenshot of whole process |
Description
Dan Goodliffe
2024-03-01 10:26:01 UTC
Clarification on the exact command being run (it's in a script I'd forgotten about) emerge -uqDN @world @system --keep-going --with-bdeps=y --verbose-conflicts --backtrack=100 -k Can't be coincidence, 12 package update today, did 8 of 12, 8th was grub, email sent and then it errored out. Mail settings are in a make.defaults in a custom profile: 12 PORTAGE_ELOG_MAILSUBJECT="\${HOST} - package \${PACKAGE}" 13 PORTAGE_ELOG_MAILURI="dan@randomdan.homeip.net smtp.random.lan" 14 PORTAGE_ELOG_SYSTEM="mail" I don't *think* they're a problem because that's worked for years and the mails DO get sent... Mar 02 12:23:37 danielg exim[125590]: [1\2] 2024-03-02 12:23:37 Warning: purging the environment. Mar 02 12:23:37 danielg exim[125590]: [2/2] Suggested action: use keep_environment. Mar 02 12:23:37 danielg exim[125590]: 2024-03-02 12:23:37 1rgOOv-00000000Wfe-0pYv <= root@danielg.random.lan U=root P=local S=766 Mar 02 12:23:38 danielg exim[125592]: 2024-03-02 12:23:38 1rgOOv-00000000Wfe-0pYv => dan.goodliffe@randomdan.homeip.net R=smarthost T=smarthost_smtp H=virtualip1.random.lan [10.10.0.253] X=TLS1.3:TLS_AES_256_GCM_SHA384:256 CV=yes K C="250- 788 byte chunk, total 788\\n250 OK id=1rgOOv-0000000> Mar 02 12:23:38 danielg exim[125592]: 2024-03-02 12:23:38 1rgOOv-00000000Wfe-0pYv Completed Can't see anything amiss here. (In reply to Dan Goodliffe from comment #2) > Can't be coincidence, 12 package update today, did 8 of 12, 8th was grub, > email sent and then it errored out. > > Mail settings are in a make.defaults in a custom profile: > > 12 PORTAGE_ELOG_MAILSUBJECT="\${HOST} - package \${PACKAGE}" > 13 PORTAGE_ELOG_MAILURI="dan@randomdan.homeip.net smtp.random.lan" > 14 PORTAGE_ELOG_SYSTEM="mail" Did you see another traceback from portage? For the mail elog module there's not the same issue as the custom module shown in comment #0 because the mail elog module does not call the portage.process module (issues with the portage.process module arose from changes related to bug 916566). Created attachment 886559 [details]
Screenshot of whole process
Picked an arbitrary small package that I knew would email out, merged that on its own. Screenshot attached. (In reply to Dan Goodliffe from comment #4) > Created attachment 886559 [details] > Screenshot of whole process This is the same traceback as comment #0, and is for elog/mod_custom rather than elog/mod_mail. Are you sure you don't have "custom" in your PORTAGE_ELOG_SYSTEM? Damnit, apologies I do... I've overridden make.defaults in make.conf for some reason and I'd totally forgotten I'd done it or why... PORTAGE_ELOG_SYSTEM="custom" PORTAGE_ELOG_COMMAND="/usr/bin/mailx -s '${PACKAGE}' dan.goodliffe@randomdan.homeip.net < '${LOGFILE}'" (I can only presume this made sense to me at the time for a reason that maybe doesn't exist now) Removing those and using the default mail solution works fine, so it's an issue with the above or with custom, not with email specifically... that was coincidence. Okay, great, so there's nothing wrong with mod_mail and mod_custom is fixed in https://github.com/gentoo/portage/pull/1295. It would be nice to add async send to mod_mail still, but it's not critical for now because it doesn't use the event loop to send so it won't trigger the "RuntimeError: This event loop is already running" error. The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=c3ebdbb42e72335ca65335c855a82b99537c7606 commit c3ebdbb42e72335ca65335c855a82b99537c7606 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2024-03-03 06:30:50 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2024-03-03 06:30:50 +0000 elog/mod_custom: Spawn processes in background Since elog_process is typically called while the event loop is running, hold references to spawned processes and wait for them asynchronously, ultimately waiting for them if necessary when the AsyncioEventLoop _close_main method calls _async_finalize via portage.process.run_coroutine_exitfuncs(). ConfigProtectTestCase is useful for exercising this code, and this little make.globals patch can be used to test failure during finalize with this error message: !!! PORTAGE_ELOG_COMMAND failed with exitcode 1 --- a/cnf/make.globals +++ b/cnf/make.globals @@ -144 +144,2 @@ PORTAGE_ELOG_CLASSES="log warn error" -PORTAGE_ELOG_SYSTEM="save_summary:log,warn,error,qa echo" +PORTAGE_ELOG_SYSTEM="save_summary:log,warn,error,qa echo custom" +PORTAGE_ELOG_COMMAND="/bin/false" Bug: https://bugs.gentoo.org/925907 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/portage/elog/mod_custom.py | 73 +++++++++++++++++++++-- lib/portage/process.py | 29 +++++++++ lib/portage/util/_eventloop/asyncio_event_loop.py | 8 ++- 3 files changed, 105 insertions(+), 5 deletions(-) The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=f1e53b9368c215451e3af78b557d8b86a2db9677 commit f1e53b9368c215451e3af78b557d8b86a2db9677 Author: Sam James <sam@gentoo.org> AuthorDate: 2024-04-28 00:31:29 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2024-04-28 00:31:29 +0000 sys-apps/portage: add 3.0.64 Closes: https://bugs.gentoo.org/925481 Closes: https://bugs.gentoo.org/675748 Closes: https://bugs.gentoo.org/712222 Closes: https://bugs.gentoo.org/920330 Closes: https://bugs.gentoo.org/925213 Closes: https://bugs.gentoo.org/925863 Closes: https://bugs.gentoo.org/925888 Closes: https://bugs.gentoo.org/925907 Closes: https://bugs.gentoo.org/926048 Closes: https://bugs.gentoo.org/926221 Closes: https://bugs.gentoo.org/927059 Closes: https://bugs.gentoo.org/927632 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.64.ebuild | 242 +++++++++++++++++++++++++++++++++ 2 files changed, 243 insertions(+) |