Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 684940 - sys-apps/portage: emerge --sync / emaint sync needs locking in case of concurrent calls
Summary: sys-apps/portage: emerge --sync / emaint sync needs locking in case of concur...
Status: CONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 240187
  Show dependency tree
 
Reported: 2019-05-02 17:03 UTC by Zac Medico
Modified: 2024-02-19 01:45 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Zac Medico gentoo-dev 2019-05-02 17:03:45 UTC
Here's an example error from two concurrent emerge --sync operations interfering with each other:

> # running emerge --sync
> 
> rm: cannot remove '/var/portage/repos/gentoo/.tmp-unverified-download-quarantine': Directory not empty
> Traceback (most recent call last):
>   File "/usr/lib64/python3.6/site-packages/portage/util/_async/AsyncFunction.py", line 39, in _run
>     result = self.target(*(self.args or []), **(self.kwargs or {}))
>   File "/usr/lib64/python3.6/site-packages/portage/sync/controller.py", line 169, in sync
>     taskmaster.run_tasks(tasks, func, status, options=task_opts)
>   File "/usr/lib64/python3.6/site-packages/portage/sync/controller.py", line 68, in run_tasks
>     result = getattr(inst, func)(**kwargs)
>   File "/usr/lib64/python3.6/site-packages/portage/sync/syncbase.py", line 304, in sync
>     return self.update()
>   File "/usr/lib64/python3.6/site-packages/portage/sync/modules/rsync/rsync.py", line 333, in update
>     dosyncuri, timestamp, opts)
>   File "/usr/lib64/python3.6/site-packages/portage/sync/modules/rsync/rsync.py", line 705, in _do_rsync
>     command.append(self.download_dir)
>   File "/usr/lib64/python3.6/site-packages/portage/sync/syncbase.py", line 125, in download_dir
>     self._download_dir = self.repo_storage.init_update()
>   File "/usr/lib64/python3.6/site-packages/portage/util/futures/_sync_decorator.py", line 21, in wrapper
>     return loop.run_until_complete(func(*args, **kwargs))
>   File "/usr/lib64/python3.6/site-packages/portage/util/_eventloop/EventLoop.py", line 833, in run_until_complete
>     return future.result()
>   File "/usr/lib64/python3.6/site-packages/portage/util/futures/compat_coroutine.py", line 113, in _next
>     future = self._generator.throw(previous.exception())
>   File "/usr/lib64/python3.6/site-packages/portage/repository/storage/hardlink_quarantine.py", line 58, in init_update
>     yield self._check_call(['rm', '-rf', update_location])
>   File "/usr/lib64/python3.6/site-packages/portage/util/futures/compat_coroutine.py", line 111, in _next
>     future = self._generator.send(previous.result())
>   File "/usr/lib64/python3.6/site-packages/portage/repository/storage/hardlink_quarantine.py", line 53, in _check_call
>     format(p.returncode, ' '.join(cmd)))
> portage.repository.storage.interface.RepoStorageException: command exited with status 1: rm -rf /var/portage/repos/gentoo/.tmp-unverified-download-quarantine
Comment 1 Zac Medico gentoo-dev 2019-05-02 17:08:52 UTC
We could have a single global lock, or separate locks for each repository. If we have separate locks then we have to lock the master(s) of a given repository while it's syncing, and the order of acquiring locks has to be predictable in order to avoid deadlocks.
Comment 2 Zac Medico gentoo-dev 2024-02-19 01:45:21 UTC
Package moves acquire a global vardbapi lock since bug 587088.