| Summary: | sys-apps/portage: support sync module option to 'git reset' before pulling | ||
|---|---|---|---|
| Product: | Portage Development | Reporter: | Michał Górny <mgorny> |
| Component: | Unclassified | Assignee: | Portage team <dev-portage> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | ajak, lssndrbarbieri |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | 883437, 887025 | ||
| Bug Blocks: | |||
|
Description
Michał Górny
2015-02-09 08:14:59 UTC
Does not option 2 will destroy distfiles content? I think it must be added to .gitignore. Why would I care about people crazy enough to put distfiles in a repository? (In reply to Michał Górny from comment #2) > Why would I care about people crazy enough to put distfiles in a repository? Yeah, since the git sync module has nothing to do with the default rsync sync module, within this context we can consider /usr/portage/disfiles as an unsupported legacy configuration. Ok then default value for DISTDIR must be changed to something outside of /usr/portage (In reply to nE0sIghT from comment #4) > Ok then default value for DISTDIR must be changed to something outside of > /usr/portage For bug 378603, the default DISTDIR will change along with a few others. I don't think we should consider that a blocker for this 'git reset' feature, as long as it's not enabled in the default configuration (and git is not currently the default configuration). This was done a little while ago. commit 7a336e85052530c62acf25b5df10e2d02a17e779 Author: John Helmert III <ajak@gentoo.org> Date: Sat Oct 29 21:33:44 2022 -0500 GitSync.update: clobber changes to git repo as rsync does Check `git diff --quiet` and if tree is dirty, do a hard reset to the upstream HEAD instead of simply merging. Thanks to a limitation in git, also `git reset --hard` if the merge fails. Closes: https://github.com/gentoo/portage/pull/931 Signed-off-by: John Helmert III <ajak@gentoo.org> Signed-off-by: Sam James <sam@gentoo.org> commit da8e7d193563a2b0c88a523083b2a3466ff4526e Author: Florian Schmaus <flow@gentoo.org> Date: Wed Dec 14 17:09:50 2022 +0100 sync: git: only perform shallow updates if the repository is shallow With the new default of shallow cloning/updating git repositories, added with f2207e41792d ("GitSync.update: default to "--depth 1" (bug 824782 comment 17)"), existing *non-shallow* repositories would become shallow after a "emerge --sync". This adds a check to see if the repository is already shallow, and only in this case performs a shallow clone. Thanks to Douglas Freed for pointing out that this should consider whether or not the user explicitly set sync-depth. Thanks-to: Douglas Freed <dwfreed@mtu.edu> Bug: https://bugs.gentoo.org/824782 Bug: https://bugs.gentoo.org/887025 Closes: https://github.com/gentoo/portage/pull/960 Signed-off-by: Sam James <sam@gentoo.org> commit ac405c994a0cce67e11c1bc7fb37fc02bba627ae Author: Sam James <sam@gentoo.org> Date: Sat Dec 17 05:19:32 2022 +0000 sync: git: allow truncating history if repository is marked volatile See 54a08c9f71dc37851fdb0364576581ca19df6580 for an explanation of the context here. If the repository is marked as volatile, it's assumed that Portage has complete control of it, and for Portage's purposes, there's no use in having extended history (in fact, if anything, it's deterimental, as it makes a sync more likely to fail due to the large disk space requirements over time). Bug: https://bugs.gentoo.org/824782 Bug: https://bugs.gentoo.org/887025 Signed-off-by: Sam James <sam@gentoo.org> commit 1696966564510e8359e3e29681dc1326c37c6e0b Author: John Helmert III <ajak@gentoo.org> Date: Wed Nov 2 21:48:59 2022 -0500 sync: git: `git clean` the repo before updating With a shallow clone, git may leave orphaned files if the remote moves from state X -> X-1. If on pulling again, state X is returned, git will then abort to avoid clobbering untracked files. Bug: https://bugs.gentoo.org/887025 Signed-off-by: John Helmert III <ajak@gentoo.org> Signed-off-by: Sam James <sam@gentoo.org> commit 92f8d27e034558edd749a08e5d61d579dc7172a8 Author: Sam James <sam@gentoo.org> Date: Sat Dec 17 05:51:02 2022 +0000 sync: git: only clobber changes if repository is marked as non-volatile The fundamental issue with regard to Portage handling sync-type=git repositories is follows: whenever Portage is syncing something, our view was that we should prioritise successful completion of syncs, especially given it may be performed unattended. The rationale for this is that most people who use sync-type=git are doing so because they want quicker sync (to complete), a more reliable one (no Manifest race condition issues), and to get changes from upstream faster. Recently, Portage started doing two things: 1. Clobbering changes to repositories with sync-type=git If git is pulling from a CDN, Portage may on one sync receive state X, but on a subsequent sync receive state X-1. This can cause an odd state where git wants to resolve conflicts and manual intervention is required, hence needing git reset. This situation was often worse with sync-depth=1 and would lead to orphaned files hence needing git clean. 2. Defaulted to sync-depth=clone-depth=1. The motivation here was because of disk space growing unbounded otherwise. It makes sense when considered within the school of thought / motivation listed above (prioritising a successful sync and then making an optimisation based on how we achieve that). Further, there was additional motivation to avoid disk usage growing unbounded (which additionally could lead to a sync failing at some point). -- To allow users to opt-out of these destructive changes, we now have a repository setting called 'volatile'. * If volatile=yes, the repository is assumed to be user-modifiable and Portage will NOT prioritise successful sync over preserving repository state. * If volatile=no, the repository is assumed to only be modified by Portage, and Portage will try its best to ensure syncs continue to work. This is a followup to 7a336e85052530c62acf25b5df10e2d02a17e779 and 1b23d15e4c71d95c6c2520dcb85f8bf90ddf430c. Bug: https://bugs.gentoo.org/887025 See: 7a336e85052530c62acf25b5df10e2d02a17e779 See: 1b23d15e4c71d95c6c2520dcb85f8bf90ddf430c See: https://github.com/gentoo/portage/pull/801 See: https://github.com/gentoo/portage/pull/931 Closes: https://github.com/gentoo/portage/pull/939 Closes: https://github.com/gentoo/portage/pull/960 Signed-off-by: Sam James <sam@gentoo.org> commit 16246ea32371ff70c8941756af75ddefb8c97f4e Author: Florian Schmaus <flow@gentoo.org> Date: Sat Dec 31 11:29:02 2022 +0100 git.py: fix "is-shallow-repository" check by stripping \n The output of git commands has a trailing newline character which we usually strip. Fixes: da8e7d193563a2b0c88a523083b2a3466ff4526e ("sync: git: only perform shallow updates if the repository is shallow") Signed-off-by: Florian Schmaus <flow@gentoo.org> Closes: https://github.com/gentoo/portage/pull/968 Signed-off-by: Sam James <sam@gentoo.org> commit f29cc2578b1c97164c1411b14a2dd8195b14ab1e Author: Florian Schmaus <flow@gentoo.org> Date: Wed Feb 22 10:07:31 2023 +0100 sync: git: fix code comment regarding clobbering the repository The logic the code comment stated was simply inverted, and so was the NEWS item. The actual implementation is correct. Signed-off-by: Florian Schmaus <flow@gentoo.org> Closes: https://github.com/gentoo/portage/pull/992 Signed-off-by: Sam James <sam@gentoo.org> commit 31a1cbea211633ddb162124ea42d008015f4ab5f Author: Florian Schmaus <flow@gentoo.org> Date: Wed Feb 22 15:11:17 2023 +0100 sync: git: report error if volatile repository Previously portage would not report a merge_cmd error if the repository was volatile. This could potentially be the cause of bug #895526, however if it unclear, at the time of writing this, if the repository in the bug is a volatile one. Bug: https://bugs.gentoo.org/895526 Signed-off-by: Florian Schmaus <flow@gentoo.org> Closes: https://github.com/gentoo/portage/pull/993 Signed-off-by: Sam James <sam@gentoo.org> |