When working on an ebuild, I often would make tweaks in the repo directory. This used for work fine, but with portage-3.0.40 if I do an "emerge --sync" it fully resets the repo to upstream, so if there are any changes that weren't yet pushed they are gone. I could not find anything to disable this in the portage manpage. Please add a way to revert to the previous behavior. Reproducible: Always Steps to Reproduce: 1. Edit a file in a local repo 2. (optional) Commit the changes 3. Run emerge --sync Actual Results: All your changes are gone Expected Results: Changes should remain or sync should fail if there are conflicts as in older versions /etc/portage/repos.conf/local.conf [local] location = /var/db/repos/local sync-type = git sync-uri = https://<...>
I'm working on a toggle for it.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=92f8d27e034558edd749a08e5d61d579dc7172a8 commit 92f8d27e034558edd749a08e5d61d579dc7172a8 Author: Sam James <sam@gentoo.org> AuthorDate: 2022-12-17 05:51:02 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-21 01:28:03 +0000 sync: git: only clobber changes if repository is marked as non-volatile The fundamental issue with regard to Portage handling sync-type=git repositories is follows: whenever Portage is syncing something, our view was that we should prioritise successful completion of syncs, especially given it may be performed unattended. The rationale for this is that most people who use sync-type=git are doing so because they want quicker sync (to complete), a more reliable one (no Manifest race condition issues), and to get changes from upstream faster. Recently, Portage started doing two things: 1. Clobbering changes to repositories with sync-type=git If git is pulling from a CDN, Portage may on one sync receive state X, but on a subsequent sync receive state X-1. This can cause an odd state where git wants to resolve conflicts and manual intervention is required, hence needing git reset. This situation was often worse with sync-depth=1 and would lead to orphaned files hence needing git clean. 2. Defaulted to sync-depth=clone-depth=1. The motivation here was because of disk space growing unbounded otherwise. It makes sense when considered within the school of thought / motivation listed above (prioritising a successful sync and then making an optimisation based on how we achieve that). Further, there was additional motivation to avoid disk usage growing unbounded (which additionally could lead to a sync failing at some point). -- To allow users to opt-out of these destructive changes, we now have a repository setting called 'volatile'. * If volatile=yes, the repository is assumed to be user-modifiable and Portage will NOT prioritise successful sync over preserving repository state. * If volatile=no, the repository is assumed to only be modified by Portage, and Portage will try its best to ensure syncs continue to work. This is a followup to 7a336e85052530c62acf25b5df10e2d02a17e779 and 1b23d15e4c71d95c6c2520dcb85f8bf90ddf430c. Bug: https://bugs.gentoo.org/887025 See: 7a336e85052530c62acf25b5df10e2d02a17e779 See: 1b23d15e4c71d95c6c2520dcb85f8bf90ddf430c See: https://github.com/gentoo/portage/pull/801 See: https://github.com/gentoo/portage/pull/931 Closes: https://github.com/gentoo/portage/pull/939 Closes: https://github.com/gentoo/portage/pull/960 Signed-off-by: Sam James <sam@gentoo.org> NEWS | 53 +++++++++++++++++++++- lib/portage/sync/modules/git/git.py | 89 ++++++++++++++++++++++++++++++------- 2 files changed, 124 insertions(+), 18 deletions(-) https://gitweb.gentoo.org/proj/portage.git/commit/?id=cc20629da347393e6987fa58231e4b773ca0a5e7 commit cc20629da347393e6987fa58231e4b773ca0a5e7 Author: John Helmert III <ajak@gentoo.org> AuthorDate: 2022-11-20 04:47:13 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-21 01:28:03 +0000 sync: git: add GIT_CEILING_DIRECTORIES for update operations Bug: https://bugs.gentoo.org/887025 Signed-off-by: John Helmert III <ajak@gentoo.org> Closes: https://github.com/gentoo/portage/pull/939 Signed-off-by: Sam James <sam@gentoo.org> lib/portage/sync/modules/git/git.py | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) https://gitweb.gentoo.org/proj/portage.git/commit/?id=1696966564510e8359e3e29681dc1326c37c6e0b commit 1696966564510e8359e3e29681dc1326c37c6e0b Author: John Helmert III <ajak@gentoo.org> AuthorDate: 2022-11-03 02:48:59 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-21 01:28:03 +0000 sync: git: `git clean` the repo before updating With a shallow clone, git may leave orphaned files if the remote moves from state X -> X-1. If on pulling again, state X is returned, git will then abort to avoid clobbering untracked files. Bug: https://bugs.gentoo.org/887025 Signed-off-by: John Helmert III <ajak@gentoo.org> Signed-off-by: Sam James <sam@gentoo.org> lib/portage/sync/modules/git/git.py | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) https://gitweb.gentoo.org/proj/portage.git/commit/?id=ac405c994a0cce67e11c1bc7fb37fc02bba627ae commit ac405c994a0cce67e11c1bc7fb37fc02bba627ae Author: Sam James <sam@gentoo.org> AuthorDate: 2022-12-17 05:19:32 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-21 01:28:03 +0000 sync: git: allow truncating history if repository is marked volatile See 54a08c9f71dc37851fdb0364576581ca19df6580 for an explanation of the context here. If the repository is marked as volatile, it's assumed that Portage has complete control of it, and for Portage's purposes, there's no use in having extended history (in fact, if anything, it's deterimental, as it makes a sync more likely to fail due to the large disk space requirements over time). Bug: https://bugs.gentoo.org/824782 Bug: https://bugs.gentoo.org/887025 Signed-off-by: Sam James <sam@gentoo.org> lib/portage/sync/modules/git/git.py | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-) https://gitweb.gentoo.org/proj/portage.git/commit/?id=da8e7d193563a2b0c88a523083b2a3466ff4526e commit da8e7d193563a2b0c88a523083b2a3466ff4526e Author: Florian Schmaus <flow@gentoo.org> AuthorDate: 2022-12-14 16:09:50 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-21 01:28:02 +0000 sync: git: only perform shallow updates if the repository is shallow With the new default of shallow cloning/updating git repositories, added with f2207e41792d ("GitSync.update: default to "--depth 1" (bug 824782 comment 17)"), existing *non-shallow* repositories would become shallow after a "emerge --sync". This adds a check to see if the repository is already shallow, and only in this case performs a shallow clone. Thanks to Douglas Freed for pointing out that this should consider whether or not the user explicitly set sync-depth. Thanks-to: Douglas Freed <dwfreed@mtu.edu> Bug: https://bugs.gentoo.org/824782 Bug: https://bugs.gentoo.org/887025 Closes: https://github.com/gentoo/portage/pull/960 Signed-off-by: Sam James <sam@gentoo.org> NEWS | 2 ++ lib/portage/sync/modules/git/git.py | 28 +++++++++++++++++++++++----- 2 files changed, 25 insertions(+), 5 deletions(-) https://gitweb.gentoo.org/proj/portage.git/commit/?id=ef123a214708c85f9802f2a649b93137fd2ee3be commit ef123a214708c85f9802f2a649b93137fd2ee3be Author: Sam James <sam@gentoo.org> AuthorDate: 2022-12-17 05:12:46 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-21 01:28:02 +0000 config: Add 'volatile' configuration option This introduces a 'volatile' configuration option for repos.conf. * If a repository is marked as volatile (volatile=true), then Portage will assume the repository may be changed at any time, and that it may contain e.g. local changes. Portage will not perform optimizations based on assuming its integrity. It will also not perform destructive operations if it is a git repository (it will not prioritise being able to continue to sync). * If a repository is marked as non-volatile (volatile=false), then Portage will assume that only Portage changes the repository and that it will not be modified otherwise. Portage will perform optimizations based on assuming this, like not needing to verify the metadata cache for ebuilds & eclasses. Portage will prioritise the repository continuing to work and may e.g. 'git reset --hard' or 'git clean -fdx' in order to keep git repositories working for sync purposes. (If you do not like this, please set 'volatile=true'). Bug: https://bugs.gentoo.org/528394 Bug: https://bugs.gentoo.org/887025 See: https://github.com/gentoo/portage/pull/801 See: https://github.com/gentoo/portage/pull/931 See: https://github.com/gentoo/portage/pull/939 See: https://github.com/gentoo/portage/pull/960 Signed-off-by: Sam James <sam@gentoo.org> lib/portage/repository/config.py | 40 +++++++++++++++++++++++++++++++++++++++- man/portage.5 | 8 ++++++++ 2 files changed, 47 insertions(+), 1 deletion(-)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=84b50e99241623bf5557cea8e00a8178b9f01e14 commit 84b50e99241623bf5557cea8e00a8178b9f01e14 Author: Sam James <sam@gentoo.org> AuthorDate: 2022-12-26 07:16:56 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-26 07:17:16 +0000 sys-apps/portage: add 3.0.42 Bug: https://bugs.gentoo.org/881383 Bug: https://bugs.gentoo.org/882797 Closes: https://bugs.gentoo.org/884397 Closes: https://bugs.gentoo.org/884135 Closes: https://bugs.gentoo.org/884285 Closes: https://bugs.gentoo.org/887025 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.42.ebuild | 283 +++++++++++++++++++++++++++++++++ 2 files changed, 284 insertions(+)
