Bug 321711 – Enhancement: Make emerge --sync respect a "lockfile concept" so a --sync won't run during an active merge

• Product: Portage Development
• Status: CONFIRMED ---

[Description Jeremy Olexa (darkside) (RETIRED) 2010-05-27 18:01:23 UTC]

It would be nice for emerge --sync to be ignored/delayed when an active merge is happening. The user facing problem is when you are compiling something and then an emerge --sync is ran (probably from cron) which invalidates your work at the next phase when the Manifest is re-checked.

A discussed feature/workaround for this would be to make emerge --sync respect a lockfile or so. In this fashion, a long compile job would *not* get halted in the middle. Think about someone making a whitespace change that invalidates the 3+ hour compilation job. This is annoying to the end user, admittedly slightly rare occurrence.

[Comment 1 Zac Medico 2010-05-28 00:09:16 UTC]

Maybe we can modify the portage.locks module to optionally use fcntl.LOCK_SH instead of LOCK_EX, or add a new function to do that if it doesn't fit the existing functions well.

[Comment 2 Brian Harring (RETIRED) 2010-05-28 04:02:52 UTC]

via flock, a LOCK_SH for readonly access, LOCK_EX for write access (sync'ing) applied to the repository directory (PORTDIR=/usr/portage, thus /usr/portage) would work for me in terms of pkgcore/portage interop.

Work for you?

[Comment 3 Zac Medico 2010-05-28 21:41:34 UTC]

If you have a lot of repos configured, and you have a build plan that uses lots of them, then you're going to have to hold individual shared locks for each repo that you're using until you're finished with it.

Alternatively we could go with a single global lock, like /var/lock/portage_repos or something like that, but maybe that's not granular enough for some use cases? If using a global lock, might also want to consider permissions issues. For example, with prefix-portage, the user may not have write access to the global /var/lock directory, but they still may want use this locking feature. In that case maybe it would make sense for them to use $EPREFIX/var/lock for such locks.

[Comment 4 Brian Harring (RETIRED) 2010-05-28 22:18:04 UTC]

(In reply to comment #3)
> If you have a lot of repos configured, and you have a build plan that uses lots
> of them, then you're going to have to hold individual shared locks for each
> repo that you're using until you're finished with it.
> 
> Alternatively we could go with a single global lock, like
> /var/lock/portage_repos or something like that, but maybe that's not granular
> enough for some use cases? If using a global lock, might also want to consider
> permissions issues. For example, with prefix-portage, the user may not have
> write access to the global /var/lock directory, but they still may want use
> this locking feature. In that case maybe it would make sense for them to use
> $EPREFIX/var/lock for such locks.

locking per repo suffices, although keep in mind that overlay repo's would need to lock what their 'overlaying' since eclasses and other bits can come from there.

While this can result in long held locks... that's valid.  You don't want a sync leveled during a 100 pkg build after all.