Bug 936286 – Poor error message on failed fetch for binpkgs

• Product: Portage Development
• Status: CONFIRMED ---

[Description Sam James 2024-07-19 08:06:00 UTC]

meson reported this in their CI builder a few days ago at https://github.com/mesonbuild/meson/actions/runs/9976106143/job/27567470710?pr=13351.

The real failure is buried within a bunch of output, where we fail to fetch one binpkg:
```
#10 127.2 >>> Running pre-merge checks for sys-libs/compiler-rt-17.0.6
#10 127.2 --2024-07-17 14:30:27--  https://distfiles.gentoo.org/releases/amd64/binpackages/23.0/x86-64/sys-libs/compiler-rt/compiler-rt-17.0.6-1.gpkg.tar
#10 127.2 Resolving distfiles.gentoo.org... 89.187.177.17, 2a02:6ea0:c400::12
#10 127.4 Connecting to distfiles.gentoo.org|89.187.177.17|:443... failed: Connection timed out.
#10 187.4 Connecting to distfiles.gentoo.org|2a02:6ea0:c400::12|:443... failed: Network is unreachable.
```

Between that and the end, we get tonnes of output, and then:
```
#10 311.8 >>> Running pre-merge checks for media-libs/mesa-24.0.9
#10 312.5  * Messages for package sys-libs/libomp-17.0.6:
#10 312.5  * Unable to find kernel sources at /usr/src/linux
#10 312.5  * Unable to calculate Linux Kernel version for build, attempting to use running version
#10 312.5 
#10 312.5 
#10 312.6 [ERROR] Task was destroyed but it is pending!
#10 312.6 task: <Task cancelling name='Task-1945' coro=<BinpkgFetcher._main() done, defined at /usr/lib/python3.12/site-packages/_emerge/BinpkgFetcher.py:48> wait_for=<Future finished result=None> cb=[AsyncTaskFuture._done_callback()]>
#10 312.6 [ERROR] Task was destroyed but it is pending!
#10 312.6 task: <Task pending name='Task-1951' coro=<MultiprocessingProcess.wait() done, defined at /usr/lib/python3.12/site-packages/portage/process.py:406> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[SubProcess._async_waitpid_cb()]>
#10 312.6 [ERROR] Task was destroyed but it is pending!
#10 312.6 task: <Task pending name='Task-1952' coro=<MultiprocessingProcess._proc_join() done, defined at /usr/lib/python3.12/site-packages/portage/process.py:424> wait_for=<Future pending cb=[sleep.<locals>.cancel_callback() at /usr/lib/python3.12/site-packages/portage/util/futures/_asyncio/__init__.py:229, Task.task_wakeup()]> cb=[MultiprocessingProcess._proc_join_done()]>
#10 ERROR: process "/bin/sh -c /ci/install.sh" did not complete successfully: exit code: 1
------
312.5  * Unable to find kernel sources at /usr/src/linux
312.5  * Unable to calculate Linux Kernel version for build, attempting to use running version
312.5 
312.5 
Error: ERROR] Task was destroyed but it is pending!
312.6 task: <Task cancelling name='Task-1945' coro=<BinpkgFetcher._main() done, defined at /usr/lib/python3.12/site-packages/_emerge/BinpkgFetcher.py:48> wait_for=<Future finished result=None> cb=[AsyncTaskFuture._done_callback()]>
Error: ERROR] Task was destroyed but it is pending!
312.6 task: <Task pending name='Task-1951' coro=<MultiprocessingProcess.wait() done, defined at /usr/lib/python3.12/site-packages/portage/process.py:406> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[SubProcess._async_waitpid_cb()]>
Error: ERROR] Task was destroyed but it is pending!
312.6 task: <Task pending name='Task-1952' coro=<MultiprocessingProcess._proc_join() done, defined at /usr/lib/python3.12/site-packages/portage/process.py:424> wait_for=<Future pending cb=[sleep.<locals>.cancel_callback() at /usr/lib/python3.12/site-packages/portage/util/futures/_asyncio/__init__.py:229, Task.task_wakeup()]> cb=[MultiprocessingProcess._proc_join_done()]>
```

It ends up looking like Portage has crashed and it's not immediately obvious what failed, and you have to dig through the log to find it.

[Comment 1 Eli Schwartz 2024-07-19 08:11:27 UTC]

12:58 <tristan957> elibrokeit: any idea what's going on with gentoo? https://github.com/mesonbuild/meson/actions/runs/9976106143/job/27567470710?pr=13351
12:58 <tristan957> anything more than a sporadic failure
13:00 <tristan957> also is that PR good to go now?
13:00 <tristan957> not sure if your arch debug thing was satisfied
13:01 <sam_> huh, wtf
13:04 <@elibrokeit> #10 127.4 Connecting to distfiles.gentoo.org|89.187.177.17|:443... failed: Connection timed out.
13:04 <@elibrokeit> #10 187.4 Connecting to distfiles.gentoo.org|2a02:6ea0:c400::12|:443... failed: Network is unreachable.
13:04 <@elibrokeit> #10 187.5 >>> Failed to emerge sys-libs/compiler-rt-17.0.6
13:05 <@elibrokeit> I would say that is sporadic
13:05 <@elibrokeit> the portage cleanup code can sometimes be a bit hairy about failures, what with all the "[ERROR] Task was destroyed but it is pending!"
13:05 <@elibrokeit> however it only indicates something had previously failed anyway...
13:05 <sam_> ah, I missed that, it was buried
13:06 <sam_> (which is probably a bug in itself really)
13:06 <sam_> .. which is what you said :)



I see this "ERROR] Task was destroyed but it is pending" message in a bunch of odd places when cleaning up after other failures. I think probably if it ever does appear, it's a bug in cleanup code.