Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 780321 - sys-apps/portage: emerge --keep-going stops with "ebuild for XXX does not exist at:"
Summary: sys-apps/portage: emerge --keep-going stops with "ebuild for XXX does not exi...
Status: RESOLVED DUPLICATE of bug 639374
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core - Interface (emerge) (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-04-05 08:02 UTC by Kalin KOZHUHAROV
Modified: 2021-04-05 09:28 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Kalin KOZHUHAROV 2021-04-05 08:02:18 UTC
During a `emerge -vtq --keep-going y -e @world` emerge died due to a missing ebuild, most likely caused by an eix-sync cronjob like this:

[snip]
>>> Emerging (715 of 1610) sys-devel/lld-11.1.0::gentoo
>>> Installing (715 of 1610) sys-devel/lld-11.1.0::gentoo
>>> Emerging (716 of 1610) app-arch/p7zip-16.02-r6::gentoo
>>> Jobs: 715 of 1610 complete, 1 running           Load avg: 6.07, 3.96, 3.89!!! aux_get(): ebuild for 'app-arch/p7zip-16.02-r6' does not exist at:
!!!            /var/db/repos/gentoo/app-arch/p7zip/p7zip-16.02-r6.ebuild
Exception in callback AsynchronousTask._exit_listener_cb(<bound method...7f764bb9efd0>>)
handle: <Handle AsynchronousTask._exit_listener_cb(<bound method...7f764bb9efd0>>)>
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/portage/eclass_cache.py", line 32, in __getattr__
    self.mtime = obj = os.stat(self.location)[stat.ST_MTIME]
  File "/usr/lib/python3.9/site-packages/portage/__init__.py", line 231, in __call__
    rval = self._func(*wrapped_args, **wrapped_kwargs)
FileNotFoundError: [Errno 2] No such file or directory: b'/var/db/repos/gentoo/app-arch/p7zip/p7zip-16.02-r6.ebuild'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/portage/dbapi/porttree.py", line 550, in _pull_valid_cache
    ebuild_hash.mtime
  File "/usr/lib/python3.9/site-packages/portage/eclass_cache.py", line 35, in __getattr__
    raise FileNotFound(self.location)
portage.exception.FileNotFound: /var/db/repos/gentoo/app-arch/p7zip/p7zip-16.02-r6.ebuild

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/lib/python3.9/site-packages/_emerge/AsynchronousTask.py", line 201, in _exit_listener_cb
    listener(self)
  File "/usr/lib/python3.9/site-packages/_emerge/Scheduler.py", line 1297, in _merge_exit
    self._schedule()
  File "/usr/lib/python3.9/site-packages/_emerge/PollScheduler.py", line 152, in _schedule
    self._schedule_tasks()
  File "/usr/lib/python3.9/site-packages/_emerge/Scheduler.py", line 1638, in _schedule_tasks
    if self._schedule_tasks_imp():
  File "/usr/lib/python3.9/site-packages/_emerge/Scheduler.py", line 1790, in _schedule_tasks_imp
    self._task_queues.jobs.add(task)
  File "/usr/lib/python3.9/site-packages/_emerge/SequentialTaskQueue.py", line 24, in add
    self.schedule()
  File "/usr/lib/python3.9/site-packages/_emerge/SequentialTaskQueue.py", line 46, in schedule
    task.start()
  File "/usr/lib/python3.9/site-packages/_emerge/AsynchronousTask.py", line 30, in start
    self._start()
  File "/usr/lib/python3.9/site-packages/_emerge/MergeListItem.py", line 85, in _start
    self._start_task(build, self._default_final_exit)
  File "/usr/lib/python3.9/site-packages/_emerge/CompositeTask.py", line 113, in _start_task
    task.start()
  File "/usr/lib/python3.9/site-packages/_emerge/AsynchronousTask.py", line 30, in start
    self._start()
  File "/usr/lib/python3.9/site-packages/_emerge/EbuildBuild.py", line 48, in _start
    future=self.pkg.root_config.trees["porttree"].dbapi.\
  File "/usr/lib/python3.9/site-packages/portage/dbapi/porttree.py", line 675, in async_aux_get
    mydata, ebuild_hash = self._pull_valid_cache(mycpv, myebuild, mylocation)
  File "/usr/lib/python3.9/site-packages/portage/dbapi/porttree.py", line 555, in _pull_valid_cache
    raise PortageKeyError(cpv)
portage.exception.PortageKeyError: 'app-arch/p7zip-16.02-r6'


Reproducible: Didn't try

Steps to Reproduce:
1. start long emerge --keep-going y @world
2. delete an ebuild scheduled to be emerged
Actual Results:  
emerge stops and exits when the deleted ebuild is about to be merged

Expected Results:  
emerge stops and resumes

$ emerge -V
Portage 3.0.17 (python 3.9.2-final-0, default/linux/amd64/17.1, gcc-10.2.0, glibc-2.32-r7, 5.10.27-r15 x86_64)


I have seen this during the years, but now I think I know what causes it. I don't think it is (much) version/env dependent, but more of a design/spec what is fatal error and what `--keep-going y` can skip.

Feel free to edit the Subject, couldn't find very good way to describe it
Comment 1 Zac Medico gentoo-dev 2021-04-05 08:20:32 UTC
The sync-rcu setting from bug 662070 solves this problem. There are sync-rcu configuration instructions in the portage man page.

*** This bug has been marked as a duplicate of bug 639374 ***
Comment 2 Kalin KOZHUHAROV 2021-04-05 09:28:13 UTC
Thanks for the quick triage!

Yup bug #66270 seems like a good duplicate for this scenario, albeit it will only work when all users of the repo behave correctly (=RO) and rsync is used (according to the man page, i.e. git will break it?)...

I would expect that any issue with an ebuild (including missing ebuild) is skipped instead of stopping. Only system-wide issues (e.g. DISKFULL, OOM) should be treated as fatal.

As a workaround (sometime ago; not only for this), I came up with:

qlist -ICv |sort --random-sort |parallel --joblog emerge-all.log --line-buffer --tag -- "emerge --nodeps -q1 ={} 2>/dev/null"