Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 145601 - emerge loops on single file download after previous download errors
Summary: emerge loops on single file download after previous download errors
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: x86 Linux
: High normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
: 150487 153040 (view as bug list)
Depends on:
Blocks: 147007 149110
  Show dependency tree
 
Reported: 2006-08-30 07:28 UTC by Christoph Probst
Modified: 2006-10-27 14:04 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
properly resume after wget fails and leaves a partial download (resume.patch,941 bytes, patch)
2006-09-18 19:40 UTC, Zac Medico
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Christoph Probst 2006-08-30 07:28:26 UTC
Today I was having an issue when updateing my system as usual (emerge -uD world). In the beginning the download of one of the source files failed multiple times due to connection problems. Suddenly wget decided to rename the files tar.bz2.1, tar.bz2.2, tar.bz2.3, ... and emerge kept on restarting the download. I stopped it manually at .5 and I guess it wouldn't had stopped by itself.

Have a look at the following output and you'll understand what I mean. This can certainly be fix manually, though it shouldn't happen at all. The output of emerge --info is below aswell.

-----------------------------8<--------------------------------------------
>>> Emerging (4 of 36) kde-base/kdepim-3.5.4 to /
>>> Downloading 'http://gentoo.osuosl.org/distfiles/kdepim-3.5.4.tar.bz2'
--15:02:24--  http://gentoo.osuosl.org/distfiles/kdepim-3.5.4.tar.bz2
           => `/usr/portage/distfiles/kdepim-3.5.4.tar.bz2'
Resolving gentoo.osuosl.org... 64.50.236.52, 64.50.238.52
Connecting to gentoo.osuosl.org|64.50.236.52|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12,925,075 (12M) [application/x-tar]

66% [========================================================================================>                                              ] 8,556,191     --.--K/s    ETA 03:53

15:10:02 (18.27 KB/s) - Read error at byte 8556191/12925075 (Connection timed out). Retrying.

[...]

--15:14:23--  http://gentoo.osuosl.org/distfiles/kdepim-3.5.4.tar.bz2
  (try: 5) => `/usr/portage/distfiles/kdepim-3.5.4.tar.bz2'
Connecting to gentoo.osuosl.org|64.50.236.52|:80... connected.
HTTP request sent, awaiting response... Read error (Connection timed out) in headers.
Giving up.

>>> Downloading 'http://www.ibiblio.org/pub/Linux/distributions/gentoo/distfiles/kdepim-3.5.4.tar.bz2'
--15:15:33--  http://www.ibiblio.org/pub/Linux/distributions/gentoo/distfiles/kdepim-3.5.4.tar.bz2
           => `/usr/portage/distfiles/kdepim-3.5.4.tar.bz2.1'
Resolving www.ibiblio.org... 152.2.210.80
Connecting to www.ibiblio.org|152.2.210.80|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://distro.ibiblio.org/pub/linux/distributions/gentoo/distfiles/kdepim-3.5.4.tar.bz2 [following]
--15:15:53--  http://distro.ibiblio.org/pub/linux/distributions/gentoo/distfiles/kdepim-3.5.4.tar.bz2
           => `/usr/portage/distfiles/kdepim-3.5.4.tar.bz2.1'
Resolving distro.ibiblio.org... 152.46.7.109
Connecting to distro.ibiblio.org|152.46.7.109|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12,925,075 (12M) [application/x-bzip2]

 0% [                                                                                                                                       ] 32,837        --.--K/s  ETA 8:57:11

15:17:16 (399.93 B/s) - Read error at byte 32837/12925075 (Connection timed out). Retrying.

[...]

--15:18:55--  http://distro.ibiblio.org/pub/linux/distributions/gentoo/distfiles/kdepim-3.5.4.tar.bz2
  (try: 3) => `/usr/portage/distfiles/kdepim-3.5.4.tar.bz2.1'
Connecting to distro.ibiblio.org|152.46.7.109|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 12,925,075 (12M), 12,885,354 (12M) remaining [application/x-bzip2]

100%[======================================================================================================================================>] 12,925,075     9.50K/s    ETA 00:00

15:35:06 (13.00 KB/s) - `/usr/portage/distfiles/kdepim-3.5.4.tar.bz2.1' saved [12925075/12925075]

>>> Downloading 'http://ibiblio.org/pub/mirrors/kde/stable/3.5.4/src/kdepim-3.5.4.tar.bz2'
--15:35:06--  http://ibiblio.org/pub/mirrors/kde/stable/3.5.4/src/kdepim-3.5.4.tar.bz2
           => `/usr/portage/distfiles/kdepim-3.5.4.tar.bz2.2'
Resolving ibiblio.org... 152.2.210.80
Connecting to ibiblio.org|152.2.210.80|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://mirrors.ibiblio.org/pub/mirrors/kde/stable/3.5.4/src/kdepim-3.5.4.tar.bz2 [following]
--15:35:12--  http://mirrors.ibiblio.org/pub/mirrors/kde/stable/3.5.4/src/kdepim-3.5.4.tar.bz2
           => `/usr/portage/distfiles/kdepim-3.5.4.tar.bz2.2'
Resolving mirrors.ibiblio.org... 152.2.210.65
Connecting to mirrors.ibiblio.org|152.2.210.65|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12,925,075 (12M) [application/x-bzip2]

100%[======================================================================================================================================>] 12,925,075    14.58K/s    ETA 00:00

15:47:08 (17.64 KB/s) - `/usr/portage/distfiles/kdepim-3.5.4.tar.bz2.2' saved [12925075/12925075]
-----------------------------8<--------------------------------------------


-----------------------------8<--------------------------------------------
# emerge --info
Portage 2.1.1_rc1-r1 (default-linux/x86/2006.0, gcc-3.4.6/vanilla, glibc-2.4-r3, 2.6.14-suspend2-r5 i686)
=================================================================
System uname: 2.6.14-suspend2-r5 i686 AMD-K7(tm) Processor
Gentoo Base System version 1.12.4
Last Sync: Mon, 28 Aug 2006 23:30:01 +0000
distcc 2.18.3 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
ccache version 2.4 [disabled]
app-admin/eselect-compiler: 2.0.0_rc2-r1
dev-lang/python:     2.3.5, 2.4.3-r3
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     2.4-r2
dev-util/confcache:  [Not Present]
sys-apps/sandbox:    1.2.18.1
sys-devel/autoconf:  2.13, 2.60
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2
sys-devel/binutils:  2.17
sys-devel/gcc-config: 1.3.13-r3
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.17
ACCEPT_KEYWORDS="x86 ~x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=athlon -Os -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/eselect/compiler /etc/gconf /etc/java-config/vms/ /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c"
CXXFLAGS="-O2 -mcpu=i686 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks metadata-transfer sandbox sfperms strict"
GENTOO_MIRRORS=" http://gentoo.osuosl.org http://www.ibiblio.org/pub/Linux/distributions/gentoo"
LINGUAS="de"
MAKEOPTS="-s -j1"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="x86 3dnow 3dnowext X a52 aac aalib acl acpi alsa apache2 arts avi berkdb bitmap-fonts browserplugin bzlib calendar chipcard cpdflib crypt cups dba dga divx4linux dri dv dvd dvdread elibc_glibc encode exif extras fame foomaticdb ftp gd gd-external gdbm gif gphoto2 gtk gtk2 hbci iconv ijs imagemagick imap imlib input_devices_keyboard input_devices_mouse ipv6 java jpeg kde kdeenablefinal kernel_linux lcms libg++ linguas_de maildir mbox mikmod mmx mp3 mpeg mysql nas ncurses nls nptl nptlonly ogg oggvorbis opengl pam pcre pdflib perl png pnp python qt qtmt quicktime readline real samba sdl session sftplogging simplexml smartcard sockets spell sqlite ssl subtitles tetex tiff truetype truetype-fonts type1-fonts usb userland_GNU vcd video_cards_r128 video_cards_rage128 vorbis win32codecs wmf xml xml2 xmlrpc xmms xorg xv xvid zlib"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, PORTAGE_RSYNC_EXTRA_OPTS
-----------------------------8<--------------------------------------------
Comment 1 Doug Goldstein (RETIRED) gentoo-dev 2006-09-17 17:00:23 UTC
Same issue here..

$ emerge --info
Portage 2.1.2_pre1 (default-linux/x86/2006.1/desktop, gcc-4.1.1, glibc-2.4-r3, 2.6.17-gentoo-r8 i686)
=================================================================
System uname: 2.6.17-gentoo-r8 i686 Intel(R) Pentium(R) M processor 1.80GHz
Gentoo Base System version 1.12.5
Last Sync: Sun, 17 Sep 2006 21:30:07 +0000
ccache version 2.4 [enabled]
app-admin/eselect-compiler: [Not Present]
dev-java/java-config: 1.3.3-r1, 2.0.28-r1
dev-lang/python:     2.4.3-r3
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     2.4-r5
dev-util/confcache:  [Not Present]
sys-apps/sandbox:    1.2.18.1
sys-devel/autoconf:  2.13, 2.60
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2
sys-devel/binutils:  2.17
sys-devel/gcc-config: 1.3.13-r3
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.17-r1
ACCEPT_KEYWORDS="x86 ~x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-Os -march=pentium-m -fomit-frame-pointer -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/X11/xkb"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/gconf /etc/java-config/vms/ /etc/revdep-rebuild /etc/splash /etc/terminfo"
CXXFLAGS="-Os -march=pentium-m -fomit-frame-pointer -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs autoconfig ccache cvs distlocks metadata-transfer sandbox sfperms sign strict"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"
LDFLAGS="-Wl,--as-needed"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/home/cardoe/workspace/gnome /home/cardoe/workspace/gentopia/overlay /home/cardoe/workspace/ferringb"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="x86 X a52 aac acpi alsa avahi bash-completion beagle berkdb bitmap-fonts bzip2 cairo cdr cli crypt dbus dlloader dri dts dvd dvdr elibc_glibc encode exif fam firefox gdbm gif glitz gnome gstreamer gtk hal howl input_devices_keyboard input_devices_mouse ipv6 isdnlog jpeg kernel_linux lcms libg++ libnotify lirc_devices_mceusb2 live mad mjpeg mmx mono mp3 mpeg mythtv ncurses nptl nptlonly oav ogg opengl pam pcre pdf perl png ppds pppd python qt3 quicktime readline reflection samba sdl session spell sse sse2 ssl svg tcpd tiff truetype truetype-fonts type1-fonts udev unicode userland_GNU video_cards_ati video_cards_radeon vorbis win32codecs xml xorg xv xvid zlib"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, PORTAGE_RSYNC_EXTRA_OPTS

Comment 2 Zac Medico gentoo-dev 2006-09-18 19:40:58 UTC
Created attachment 97380 [details, diff]
properly resume after wget fails and leaves a partial download

This patch should fix it (in svn r4478).  It will case RESUMECOMMAND to be called when the previous fetch failed and left a partial download.
Comment 3 Zac Medico gentoo-dev 2006-09-22 19:24:24 UTC
This has been released in 2.1.2_pre1-r1.
Comment 4 Jakub Moc (RETIRED) gentoo-dev 2006-10-08 10:59:29 UTC
*** Bug 150487 has been marked as a duplicate of this bug. ***
Comment 5 Zac Medico gentoo-dev 2006-10-27 14:04:57 UTC
*** Bug 153040 has been marked as a duplicate of this bug. ***