For example, I run: ebuild example-1.0.ebuild digest Download starts, if network goes down, a wrong digest is done and later "emerge example" fails because it doesn't try to re-download source file again. For being able to redigest it, I have to remove digest-example-1.0, Metadata and downloaded part sources from distfiles dir. Reproducible: Always Steps to Reproduce: 1. ebuild example-1.0.ebuild digest 2. Switch down network 3. Actual Results: A wrong metadata and digest files are generated Expected Results: When network is down and full source tarball couldn't be downloaded, ebuild should stop making the digest and doesn't generate any file My emerge --info: Portage 2.1.2.2 (default-linux/amd64/2006.1, gcc-4.1.1, glibc-2.5-r0, 2.6.20-ck1 x86_64) ================================================================= System uname: 2.6.20-ck1 x86_64 AMD Athlon(tm) 64 Processor 3200+ Gentoo Base System release 1.12.9 Timestamp of tree: Sun, 22 Apr 2007 11:20:01 +0000 ccache version 2.4 [enabled] dev-java/java-config: 1.3.7, 2.0.31-r5 dev-lang/python: 2.4.3-r4 dev-python/pycrypto: 2.0.1-r5 dev-util/ccache: 2.4-r7 sys-apps/sandbox: 1.2.17 sys-devel/autoconf: 2.13, 2.61 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10 sys-devel/binutils: 2.16.1-r3 sys-devel/gcc-config: 1.3.15-r1 sys-devel/libtool: 1.5.22 virtual/os-headers: 2.6.19.2-r2 ACCEPT_KEYWORDS="amd64" AUTOCLEAN="yes" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=k8 -O2 -pipe -msse3" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config" CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/gconf /etc/java-config/vms/ /etc/revdep-rebuild /etc/splash /etc/terminfo /etc/texmf/web2c" CXXFLAGS="-march=k8 -O2 -pipe -msse3" DISTDIR="/usr/distfiles" FEATURES="autoaddcvs ccache collision-protect cvs distlocks metadata-transfer multilib-strict parallel-fetch sandbox sfperms strict" GENTOO_MIRRORS="http://ftp.belnet.be/mirror/rsync.gentoo.org/gentoo/" LANG="es_ES.UTF-8" LC_ALL="es_ES.UTF-8" LINGUAS="es en_US" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_EXTRA_OPTS="--exclude-from=/etc/portage/rsync_excludes" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --filter=H_**/files/digest-*" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/portage/local/layman/sunrise /usr/portage/local/layman/musicbrainz /usr/local/portage" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="X a52 aac acpi alsa amd64 arts asf audiofile bash-completion beagle bitmap-fonts bzip2 bzlib cairo caps cdb cdda cddb cdparanoia cdr cli cpdflib cpudetection cracklib cross crypt cups curl dbus dga divx4linux dlloader dri dts dvb dvd dvdr dvdread encode erandom escreen esd evo evolution exif fam fbcon flac foomaticdb fortran ftp galago gb gcj ggi gif gimp gimpprint glitz glut glx gnome gnome-print gphoto2 gpm gstreamer gtk gtk2 gtkhtml hal iconv idn imagemagick imlib isdnlog ithreads jabber java jpeg kde kdeenablefinal kdehiddenvisibility latex lcms libg++ libnotify logrotate mad madwifi midi mikmod mime mng mono motif mp3 mpeg mpi mplayer musepack musicbrainz nas nautilus ncurses nethack network nls nocardbus nptl nptlonly nvidia ogg oggvorbis openal opengl pam pcre pdf perl pic plotutils png posix ppds pppd python qt3 qt4 quicktime readline reflection rtc ruby scanner sdl seamonkey session slang slp sockets spell spl sse3 ssl startup-notification svg sysvipc tcltk tcpd tetex theora threads tiff timidity tk totem trayicon truetype truetype-fonts type1-fonts unicode usb v4l v4l2 vcd videos vim vorbis wma wmf xcomposite xine xml xml2 xorg xpm xv xvid zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" DVB_CARDS="usb-dib0700" ELIBC="glibc" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="es en_US" USERLAND="GNU" VIDEO_CARDS="nvidia nv vesa fbdev vga" Unset: CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LDFLAGS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS
I want say "Manifest" metadata -> manifest
*** Bug 175609 has been marked as a duplicate of this bug. ***
(In reply to comment #2) > *** Bug 175609 has been marked as a duplicate of this bug. *** > Sorry, my network is awful :-S
Maybe fetchcommand should make a temporary file until the fetch completes and then move it into place when it's done? That way if your download was incomplete the digest would complain about missing distfiles. -Alec
It is great idea :-D Thanks
This is much more feasible nowadays since FETCHCOMMAND has been updated to -O ${DISTDIR}/${FILE} instead of -P ${DISTDIR}: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=466cfb920412a47c1bf8e47628dd59f90a7220b2
*** Bug 477942 has been marked as a duplicate of this bug. ***
*** Bug 485458 has been marked as a duplicate of this bug. ***
Patch posted for review: https://archives.gentoo.org/gentoo-portage-dev/message/fa73b65f75541be31d5f329a8d99ea7c https://github.com/gentoo/portage/pull/421
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=ebbde237d33e783c562cc6c70987969ac7228b96 commit ebbde237d33e783c562cc6c70987969ac7228b96 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2019-04-27 21:59:57 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2019-04-30 02:28:21 +0000 fetch: atomic downloads (bug 175612) Direct FETCHCOMMAND/RESUMECOMMAND output to a temporary file with a constant .__download__ suffix, and atomically rename the file to remove the suffix only after the download has completed successfully (includes digest verification when applicable). Also add unit tests to cover most fetch cases. Bug: https://bugs.gentoo.org/175612 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/_emerge/BinpkgVerifier.py | 4 +- lib/portage/package/ebuild/fetch.py | 105 ++++++++++----- lib/portage/tests/ebuild/test_fetch.py | 230 +++++++++++++++++++++++++++++++++ 3 files changed, 303 insertions(+), 36 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=227c1e8bcb5a6fc377be46dbac33ef9a3dcaa27d commit 227c1e8bcb5a6fc377be46dbac33ef9a3dcaa27d Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2019-04-30 02:53:01 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2019-04-30 02:55:00 +0000 sys-apps/portage: version bump to 2.3.66 #175612 download distfiles to temp file and rename atomically #651678 enable FEATURES=strict-keepdir behavior for new EAPIs Bug: https://bugs.gentoo.org/683434 Bug: https://bugs.gentoo.org/175612 Bug: https://bugs.gentoo.org/651678 Package-Manager: Portage-2.3.66, Repoman-2.3.12 Signed-off-by: Zac Medico <zmedico@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-2.3.66.ebuild | 259 +++++++++++++++++++++++++++++++++ 2 files changed, 260 insertions(+)
@Zac Medico: How is this supposed to work when DISTDIR is a read-only network mount and FETCHCOMMAND is configured to fetch files onto the remote system? Such a setup was working for me until this change. Now what I see is that the file downloads successfully onto the DISTDIR host, but Portage then fails to rename it. Read-only DISTDIR is supposedly a supported configuration, judging by the make.conf(5) man page, which reads: skiprocheck Skip write access checks on DISTDIR when fetching files. This is useful when FETCHCOMMAND and RESUMECOMMAND are used to forward fetch requests to a server that exposes DISTDIR as a read-only NFS share. A read-only DISTDIR is not compatible with the distlocks, so it is recommended to also add "-distlocks" to FEATURES in order to avoid warning messages that are triggered by this incompatibility. This use case no longer seems possible since Portage tries to rename downloaded files itself rather than exposing any way to send those rename requests to the DISTDIR host.
(In reply to Matt Whitlock from comment #12) > @Zac Medico: > How is this supposed to work when DISTDIR is a read-only network mount and > FETCHCOMMAND is configured to fetch files onto the remote system? Such a > setup was working for me until this change. Now what I see is that the file > downloads successfully onto the DISTDIR host, but Portage then fails to > rename it. > > Read-only DISTDIR is supposedly a supported configuration, judging by the > make.conf(5) man page, which reads: > > skiprocheck > Skip write access checks on DISTDIR when fetching files. This is > useful when FETCHCOMMAND and RESUMECOMMAND are used to forward fetch > requests to a server that exposes DISTDIR as a read-only NFS share. > A read-only DISTDIR is not compatible with the distlocks, so it is > recommended to also add "-distlocks" to FEATURES in order to avoid > warning messages that are triggered by this incompatibility. > > This use case no longer seems possible since Portage tries to rename > downloaded files itself rather than exposing any way to send those rename > requests to the DISTDIR host. Thanks for the report, I've re-opened bug 220533.