Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 684940

Summary: sys-apps/portage: emerge --sync / emaint sync needs locking in case of concurrent calls
Product: Portage Development Reporter: Zac Medico <zmedico>
Component: CoreAssignee: Portage team <dev-portage>
Status: CONFIRMED ---    
Severity: normal CC: esigra, zmedico
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://bugs.gentoo.org/show_bug.cgi?id=587088
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 240187    

Description Zac Medico gentoo-dev 2019-05-02 17:03:45 UTC
Here's an example error from two concurrent emerge --sync operations interfering with each other:

> # running emerge --sync
> 
> rm: cannot remove '/var/portage/repos/gentoo/.tmp-unverified-download-quarantine': Directory not empty
> Traceback (most recent call last):
>   File "/usr/lib64/python3.6/site-packages/portage/util/_async/AsyncFunction.py", line 39, in _run
>     result = self.target(*(self.args or []), **(self.kwargs or {}))
>   File "/usr/lib64/python3.6/site-packages/portage/sync/controller.py", line 169, in sync
>     taskmaster.run_tasks(tasks, func, status, options=task_opts)
>   File "/usr/lib64/python3.6/site-packages/portage/sync/controller.py", line 68, in run_tasks
>     result = getattr(inst, func)(**kwargs)
>   File "/usr/lib64/python3.6/site-packages/portage/sync/syncbase.py", line 304, in sync
>     return self.update()
>   File "/usr/lib64/python3.6/site-packages/portage/sync/modules/rsync/rsync.py", line 333, in update
>     dosyncuri, timestamp, opts)
>   File "/usr/lib64/python3.6/site-packages/portage/sync/modules/rsync/rsync.py", line 705, in _do_rsync
>     command.append(self.download_dir)
>   File "/usr/lib64/python3.6/site-packages/portage/sync/syncbase.py", line 125, in download_dir
>     self._download_dir = self.repo_storage.init_update()
>   File "/usr/lib64/python3.6/site-packages/portage/util/futures/_sync_decorator.py", line 21, in wrapper
>     return loop.run_until_complete(func(*args, **kwargs))
>   File "/usr/lib64/python3.6/site-packages/portage/util/_eventloop/EventLoop.py", line 833, in run_until_complete
>     return future.result()
>   File "/usr/lib64/python3.6/site-packages/portage/util/futures/compat_coroutine.py", line 113, in _next
>     future = self._generator.throw(previous.exception())
>   File "/usr/lib64/python3.6/site-packages/portage/repository/storage/hardlink_quarantine.py", line 58, in init_update
>     yield self._check_call(['rm', '-rf', update_location])
>   File "/usr/lib64/python3.6/site-packages/portage/util/futures/compat_coroutine.py", line 111, in _next
>     future = self._generator.send(previous.result())
>   File "/usr/lib64/python3.6/site-packages/portage/repository/storage/hardlink_quarantine.py", line 53, in _check_call
>     format(p.returncode, ' '.join(cmd)))
> portage.repository.storage.interface.RepoStorageException: command exited with status 1: rm -rf /var/portage/repos/gentoo/.tmp-unverified-download-quarantine
Comment 1 Zac Medico gentoo-dev 2019-05-02 17:08:52 UTC
We could have a single global lock, or separate locks for each repository. If we have separate locks then we have to lock the master(s) of a given repository while it's syncing, and the order of acquiring locks has to be predictable in order to avoid deadlocks.
Comment 2 Zac Medico gentoo-dev 2024-02-19 01:45:21 UTC
Package moves acquire a global vardbapi lock since bug 587088.