| Summary: | Read only DISTDIR is not possible (FEATURES=skiprocheck) | ||
|---|---|---|---|
| Product: | Portage Development | Reporter: | orishi |
| Component: | Core | Assignee: | Portage team <dev-portage> |
| Status: | RESOLVED FIXED | ||
| Severity: | major | Keywords: | InVCS, REGRESSION |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| See Also: |
https://bugs.gentoo.org/show_bug.cgi?id=69763 https://bugs.gentoo.org/show_bug.cgi?id=175612 |
||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | |||
| Bug Blocks: | 377365, 697566, 697734 | ||
|
Description
orishi
2008-05-06 07:22:14 UTC
It's undocumented, but if you set FEATURES="skiprocheck -distlocks" in make.conf then it should work. We should document this in the make.conf.5 man page. This is fixed in 2.1.5. According to bug 175612, comment #12, this feature was broken by this commit: https://gitweb.gentoo.org/proj/portage.git/commit/?id=ebbde237d33e783c562cc6c70987969ac7228b96 commit ebbde237d33e783c562cc6c70987969ac7228b96 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2019-04-27 21:59:57 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2019-04-30 02:28:21 +0000 fetch: atomic downloads (bug 175612) Direct FETCHCOMMAND/RESUMECOMMAND output to a temporary file with a constant .__download__ suffix, and atomically rename the file to remove the suffix only after the download has completed successfully (includes digest verification when applicable). Also add unit tests to cover most fetch cases. Bug: https://bugs.gentoo.org/175612 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/_emerge/BinpkgVerifier.py | 4 +- lib/portage/package/ebuild/fetch.py | 105 ++++++++++----- lib/portage/tests/ebuild/test_fetch.py | 230 +++++++++++++++++++++++++++++++++ 3 files changed, 303 insertions(+), 36 deletions(-) I suppose we could make FEATURES="skiprocheck" disable the atomic download behavior that solves bug 175612. (In reply to Zac Medico from comment #4) > I suppose we could make FEATURES="skiprocheck" disable the atomic download > behavior that solves bug 175612. Or introduce a RENAMECOMMAND that defaults to "mv \"\${DISTDIR}/\${TMPFILE}\" \"\${DISTDIR}/\${FILE}\"" but that can be overridden in make.conf to allow performing the renames on a remote host. If you do overload the "skiprocheck" feature, you ought to rename it, perhaps to "rodistdir". Alternatively you could introduce a new feature, "atomicfetch", enabled by default, that could be disabled to suppress the atomic behavior. (In reply to Matt Whitlock from comment #5) > (In reply to Zac Medico from comment #4) > > I suppose we could make FEATURES="skiprocheck" disable the atomic download > > behavior that solves bug 175612. > > Or introduce a RENAMECOMMAND that defaults to "mv > \"\${DISTDIR}/\${TMPFILE}\" \"\${DISTDIR}/\${FILE}\"" but that can be > overridden in make.conf to allow performing the renames on a remote host. We'd also need a DELETECOMMAND since there are cases where we want to delete invalid files. Looking at the code, I see a couple of unlink calls where failures are silently allowed. Also, the _checksum_failure_temp_file function uses mkstemp, so it would be useful to have a command for creating temp files. Also, we currently ignore errors for chown/chmod calls, so maybe we could avoid that somehow. > If you do overload the "skiprocheck" feature, you ought to rename it, > perhaps to "rodistdir". Alternatively you could introduce a new feature, > "atomicfetch", enabled by default, that could be disabled to suppress the > atomic behavior. Is there any reason to disable the atomic fetch behavior, aside from the skiprocheck feature though? Here are some minimal fixes for the old skiprocheck behavior: https://archives.gentoo.org/gentoo-portage-dev/message/57f93a88dff0d2a179916f96f22fc0bd https://github.com/gentoo/portage/pull/467 (In reply to Zac Medico from comment #6) > Is there any reason to disable the atomic fetch behavior, aside from the > skiprocheck feature though? Not that I know of. I'd only want to disable it because it breaks the read-only DISTDIR use case. We can make it possible to pass digests and other information to FETCHCOMMAND, so that it can perform digest verification and atomic rename itself. The changes for bug 646898 are now an issue since mirror file layout data is cached in ${DISTDIR}/.mirror-cache.json. Also bug 697566 complicates things if we use FETCHCOMMAND to fetch mirror layout.conf which don't really want to save in DISTDIR. The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=6b5889afb1e80bc673ce782e65fc0f49ee7d0908 commit 6b5889afb1e80bc673ce782e65fc0f49ee7d0908 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2019-10-13 22:13:18 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2019-10-14 19:46:16 +0000 fetch: minimal skiprocheck fixes (bug 220533) Fix cases here fetch assumes that DISTDIR is writable when it's actually read-only. This preserves old behavior which allowed users to override FETCHCOMMAND to fetch files on a remote system, even though DISTDIR is locally mounted in read-only mode. Bug: https://bugs.gentoo.org/220533 Fixes: ebbde237d33e ("fetch: atomic downloads (bug 175612)") Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/portage/package/ebuild/fetch.py | 38 ++++++++++++++++++++-------------- lib/portage/tests/ebuild/test_fetch.py | 22 ++++++++++++++++++++ 2 files changed, 45 insertions(+), 15 deletions(-) The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=e18292c54af86bee8299b6a925d25eb6c55fd77b commit e18292c54af86bee8299b6a925d25eb6c55fd77b Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2019-10-14 22:53:35 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2019-10-14 23:01:06 +0000 sys-apps/portage: Bump to version 2.3.77 #220533 Fix FEATURES=skiprocheck read-only DISTDIR support #646898 Support GLEP 75 mirror structure #658648 Disable emerge --autounmask by default, except for package.use and package.license changes #695870 Improvements to ebuild(5) man page Bug: https://bugs.gentoo.org/697734 Bug: https://bugs.gentoo.org/220533 Bug: https://bugs.gentoo.org/646898 Bug: https://bugs.gentoo.org/658648 Bug: https://bugs.gentoo.org/695870 Package-Manager: Portage-2.3.77, Repoman-2.3.17 Signed-off-by: Zac Medico <zmedico@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-2.3.77.ebuild | 261 +++++++++++++++++++++++++++++++++ 2 files changed, 262 insertions(+) |
