| Summary: | sys-apps/portage: partial downloads trigger generation of incorrect digests in manifests | ||
|---|---|---|---|
| Product: | Portage Development | Reporter: | Pacho Ramos <pacho> |
| Component: | Core | Assignee: | Portage team <dev-portage> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | phajdan.jr |
| Priority: | Normal | Keywords: | InVCS |
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | All | ||
| See Also: |
https://bugs.gentoo.org/show_bug.cgi?id=442526 https://bugs.gentoo.org/show_bug.cgi?id=588442 https://bugs.gentoo.org/show_bug.cgi?id=688124 https://bugs.gentoo.org/show_bug.cgi?id=220533 https://bugs.gentoo.org/show_bug.cgi?id=821571 |
||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | |||
| Bug Blocks: | 377365, 683434 | ||
|
Description
Pacho Ramos
2007-04-22 19:38:27 UTC
I want say "Manifest" metadata -> manifest *** Bug 175609 has been marked as a duplicate of this bug. *** (In reply to comment #2) > *** Bug 175609 has been marked as a duplicate of this bug. *** > Sorry, my network is awful :-S Maybe fetchcommand should make a temporary file until the fetch completes and then move it into place when it's done? That way if your download was incomplete the digest would complain about missing distfiles. -Alec It is great idea :-D Thanks This is much more feasible nowadays since FETCHCOMMAND has been updated to -O ${DISTDIR}/${FILE} instead of -P ${DISTDIR}:
http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=466cfb920412a47c1bf8e47628dd59f90a7220b2
*** Bug 477942 has been marked as a duplicate of this bug. *** *** Bug 485458 has been marked as a duplicate of this bug. *** Patch posted for review: https://archives.gentoo.org/gentoo-portage-dev/message/fa73b65f75541be31d5f329a8d99ea7c https://github.com/gentoo/portage/pull/421 The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=ebbde237d33e783c562cc6c70987969ac7228b96 commit ebbde237d33e783c562cc6c70987969ac7228b96 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2019-04-27 21:59:57 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2019-04-30 02:28:21 +0000 fetch: atomic downloads (bug 175612) Direct FETCHCOMMAND/RESUMECOMMAND output to a temporary file with a constant .__download__ suffix, and atomically rename the file to remove the suffix only after the download has completed successfully (includes digest verification when applicable). Also add unit tests to cover most fetch cases. Bug: https://bugs.gentoo.org/175612 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/_emerge/BinpkgVerifier.py | 4 +- lib/portage/package/ebuild/fetch.py | 105 ++++++++++----- lib/portage/tests/ebuild/test_fetch.py | 230 +++++++++++++++++++++++++++++++++ 3 files changed, 303 insertions(+), 36 deletions(-) The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=227c1e8bcb5a6fc377be46dbac33ef9a3dcaa27d commit 227c1e8bcb5a6fc377be46dbac33ef9a3dcaa27d Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2019-04-30 02:53:01 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2019-04-30 02:55:00 +0000 sys-apps/portage: version bump to 2.3.66 #175612 download distfiles to temp file and rename atomically #651678 enable FEATURES=strict-keepdir behavior for new EAPIs Bug: https://bugs.gentoo.org/683434 Bug: https://bugs.gentoo.org/175612 Bug: https://bugs.gentoo.org/651678 Package-Manager: Portage-2.3.66, Repoman-2.3.12 Signed-off-by: Zac Medico <zmedico@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-2.3.66.ebuild | 259 +++++++++++++++++++++++++++++++++ 2 files changed, 260 insertions(+) @Zac Medico:
How is this supposed to work when DISTDIR is a read-only network mount and FETCHCOMMAND is configured to fetch files onto the remote system? Such a setup was working for me until this change. Now what I see is that the file downloads successfully onto the DISTDIR host, but Portage then fails to rename it.
Read-only DISTDIR is supposedly a supported configuration, judging by the make.conf(5) man page, which reads:
skiprocheck
Skip write access checks on DISTDIR when fetching files. This is
useful when FETCHCOMMAND and RESUMECOMMAND are used to forward fetch
requests to a server that exposes DISTDIR as a read-only NFS share.
A read-only DISTDIR is not compatible with the distlocks, so it is
recommended to also add "-distlocks" to FEATURES in order to avoid
warning messages that are triggered by this incompatibility.
This use case no longer seems possible since Portage tries to rename downloaded files itself rather than exposing any way to send those rename requests to the DISTDIR host.
(In reply to Matt Whitlock from comment #12) > @Zac Medico: > How is this supposed to work when DISTDIR is a read-only network mount and > FETCHCOMMAND is configured to fetch files onto the remote system? Such a > setup was working for me until this change. Now what I see is that the file > downloads successfully onto the DISTDIR host, but Portage then fails to > rename it. > > Read-only DISTDIR is supposedly a supported configuration, judging by the > make.conf(5) man page, which reads: > > skiprocheck > Skip write access checks on DISTDIR when fetching files. This is > useful when FETCHCOMMAND and RESUMECOMMAND are used to forward fetch > requests to a server that exposes DISTDIR as a read-only NFS share. > A read-only DISTDIR is not compatible with the distlocks, so it is > recommended to also add "-distlocks" to FEATURES in order to avoid > warning messages that are triggered by this incompatibility. > > This use case no longer seems possible since Portage tries to rename > downloaded files itself rather than exposing any way to send those rename > requests to the DISTDIR host. Thanks for the report, I've re-opened bug 220533. |
