When attempting to download a file of over two gigabytes, wget dies, and I see: zsh: file size limit exceeded wget --ignore-length http://not.the.real.url/bigfile.tar. Also, wget without --ignore-length thinks that a 2.5 gig file has a negative length, and dies at zero bytes. This seems to be a known bug in wget; http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=137989 mentions it and has a patch (which I've not tried). Reproducible: Always Steps to Reproduce: 1. Try to download a file over 2 gigabytes in size with wget. 2. Try again with --ignore-length as it will have downloaded 0 bytes, claiming a negative length. 3. At 2 gigabytes, it will stop downloading and you will recieve a "file size limit exceeded" message. Actual Results: Wget refuses to download more than zero bytes of the file without --ignore-length, as it thinks that the Content-Length header is a negative number. With --ignore-length, it downloads two gigabytes and stops. Expected Results: Wget should correctly deal with large files and long Content-Length headers. I can create files larger than this using dd, which do not appear to cause any problems; there are no limits/ulimits/quotas/etc which prevent me from having files > 2 gigabytes large. Portage 2.0.49-r10 (default-x86-1.4, gcc-3.2.3, glibc-2.3.2-r1, 2.4.22) ================================================================= System uname: 2.4.22 i686 AMD Athlon(TM) XP 2500+ Gentoo Base System version 1.4.3.10p1 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CFLAGS="-mcpu=athlon-xp -O2 -pipe" CHOST="i686-pc-linux-gnu" COMPILER="gcc3" CONFIG_PROTECT="/etc /var/qmail/control /usr/kde/2/share/config /usr/kde/3/share/config /usr/X11R6/lib/X11/xkb /usr/kde/3.1/share/config /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/config" CONFIG_PROTECT_MASK="/etc/gconf /etc/env.d" CXXFLAGS="-mcpu=athlon-xp -O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="sandbox ccache autoaddcvs" GENTOO_MIRRORS="http://www.mirror.ac.uk/sites/www.ibiblio.org/gentoo/ ftp://gentoo.inode.at/source/ http://ftp.uni-erlangen.de/pub/mirrors/gentoo" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="x86 apm avi gif jpeg libg++ mad mikmod mpeg ncurses pdflib png xml2 xmms zlib gdbm berkdb slang readline tetex bonobo X sdl gpm tcpd pam libwww ssl perl imlib oggvorbis gtk qt mozilla 3dnow acl alsa -arts canna cdr cjk crypt csope cups dvd encode -esd foomaticdb freewnn gatos -gnome guile ipv6 java jikes junit -kde -motif nls opengl -oss python quicktime ruby scanner -svga spell tcltk truetype usb videos xinerama xv video_cards_radeon"
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=181634 We need a hacker :/
wget-1.8.2-r3 in portage should solve this.
Still have this problem with wget 1.9.1. Download died when it got to 2GB, with error: "File size limit exceeded"
reopening for new wget
We need someone to wiggle the patch in, doesn't apply clean.
Created attachment 20969 [details, diff] Patch to fix 2G limit Patch for the new wget (1.9.1) that resolves the 2G download limit.
Created attachment 20970 [details, diff] patch to fix 2GB download limit Cleaned up patch.
seemant / katerina can you please test this new patch? Thanks.
Created attachment 31870 [details, diff] wget-cvs_LFS.diff (for wget-cvs version) Alvaro Lopez <alvaro@gnu.org> hacked wget-cvs to support LFS (Large File Support) Patch attached, and you can find it at http://www.alobbs.com/cherokee/wget-cvs_LFS.diff
Why isn't the 1.9.1 patch in portage yet ? Doesn't it work ?
I've got some problems downloading >2GB files. After googeling a bit, I've tried a CVS version of wget from here: http://software.lpetrov.net/wget-LFS/ and it works for mit. But it's no solution at all. So I noticed the patch there and I tried it, but it doesn't work. A friend of me and I modified it do work with the 1.9.1 in portage. The patch can be find here: http://polynomial-c.homelinux.net/pub/gentoo/portage/net-misc/wget/files/wget-LFS-20040909-gentoo.patch.bz2 But I haven't testet it yet. It might work or not.
Hi, there's also an (at least for x86 working) ebuild in http://polynomial-c.homelinux.net/pub/gentoo/portage/net-misc/wget/ The patch Rene mentioned applys against unpatched wget-1.9.1 Testing would be highly appreciated... Poly
*** Bug 75596 has been marked as a duplicate of this bug. ***
Hmm, that doesn't work for me and it seems to work with these additional cflags: -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE but I think -D_FILE_OFFSET_BITS=64 should be enough.
*** Bug 86569 has been marked as a duplicate of this bug. ***
Created attachment 54647 [details, diff] LFS patch against wget-1.9.1 from Fedora Core 2 The folks at Redhat are using this, and I've had no problems with it.
The patch from #16 works fine.
ouch, I haven't been monitoring my bugs. I added wget-1.9.1-r4 into portage yesterday with a set of Mandrake patches, the last one being largefilesupport. Can you all please test and report?
closing due to lack of response
*** Bug 92525 has been marked as a duplicate of this bug. ***
Sorry for the lack of of response, it works great. I just finished downloading the SUSE 9.2 DVD iso, no problems.
Bug is still present, please reopen. # wget -c ftp://ftp.freenet.de/pub/ftp.suse.com/pub/suse/i386/10.0/iso/SUSE-10.0-EvalDVD-i386-GM. iso --10:17:13-- ftp://ftp.freenet.de/pub/ftp.suse.com/pub/suse/i386/10.0/iso/SUSE-10.0-EvalDVD-i386- GM.iso => `SUSE-10.0-EvalDVD-i386-GM.iso' Resolving ftp.freenet.de... 194.97.2.68, 194.97.2.67, 194.97.2.70, ... Connecting to ftp.freenet.de|194.97.2.68|:21... connected. Logging in as anonymous ... Logged in! ==> SYST ... done. ==> PWD ... done. ==> TYPE I ... done. ==> CWD /pub/ftp.suse.com/pub/suse/i386/10.0/iso ... done. ==> SIZE SUSE-10.0-EvalDVD-i386-GM.iso ... done. ==> PASV ... done. ==> REST 2147483647 ... done. ==> RETR SUSE-10.0-EvalDVD-i386-GM.iso ... done. Length: 3,621,957,632 (3.4G), 1,474,473,985 (1.4G) remaining 59% [++++++++++++++++++++++++++++++++++++++++++++++ ] 2,147,483,647 --.--K/s nothing happens. # emerge info Portage 2.0.51.22-r3 (default-linux/x86/2005.0, gcc-3.3.6, glibc-2.3.5-r2, 2.6.11-hardened-r15 i686) ================================================================= System uname: 2.6.11-hardened-r15 i686 Intel(R) Xeon(TM) CPU 3.00GHz Gentoo Base System version 1.6.13 ccache version 2.3 [disabled] dev-lang/python: 2.3.5-r2, 2.4.2 sys-apps/sandbox: 1.2.12 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1 sys-devel/binutils: 2.15.92.0.2-r10 sys-devel/libtool: 1.5.20 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -march=pentium4 -fomit-frame-pointer" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/ 3.4/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-O2 -march=pentium4 -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig distlocks sandbox sfperms strict" GENTOO_MIRRORS="ftp://gentoo-mirror/gentoo ftp://ftp.uni-erlangen.de/pub/mirrors/gentoo" LANG="C" LINGUAS="de" MAKEOPTS="-j1" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://gentoo-mirror/gentoo-portage" USE="x86 X acpi acpi4linux apache2 avi bash-completion berkdb bigger-fonts bitmap-fonts browserplugin bzlib crypt cups curldevfs26 dnd dvdread eds emboss fam font-server foomaticdb fortran freetype ftp gd gdbm gif gimp gimpprint gstreamer gtk gtk2 hardened hardenedphp iconv imagemagick imap imlib innodb ithreads java jpeg jpeg2k junit kde libg++ libgd libwww mailbox maildir mbox md5sum mime mmx mmx2 mozdevelop mozilla mozsvg mozxmlterm mp3 ncurses network nls nocd nsplugin ogg oss pam pam_console pdf pdflib perl png posix python qt quicktime readline rtc samba scanner silverxp sse sse2 ssl svg tcpd tga tiff truetype truetype-fonts type1 type1-fonts uudeview v4l v4l2 vorbis xml2 xprint xvid zlib linguas_de userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LC_ALL, LDFLAGS, PORTDIR_OVERLAY # emerge -pv wget These are the packages that I would merge, in order: Calculating dependencies ...done! [ebuild R ] net-misc/wget-1.10.2 -build -debug -ipv6 +nls -socks5 +ssl -static 4 kB
Your problem is not wget's fault, rather it's the mirror you are using. Try a different mirror, ftp://suse.mirrors.tds.net/pub/suse/i386/10.0/iso/SUSE-10.0-EvalDVD-i386-GM.iso works well.
