Trying to emerge a package that for some reason is not available at a mirror causes some http-servers to send "403 Forbidden" messages. emerge does not correctly interpret these messages. emerge instead considers this message to be the first part of the file to download and only asks for the remaining bytes when contacting the next mirror. This results in a corrupted download and digest verification. Usually, this can be fixed by just reemerging the package, but if a majority of mirrors do not have the package available - as e.g. the dialog-1.0.20050206 package right now, see bug 91994 - it is virtually impossible to fetch the package using emerge. Reproducible: Always Steps to Reproduce: 1. Set up a dummy webserver / look for a package that has disappeared on mirrors. 2. Try to emerge the package, making emerge contact the dummy webserver. 3. Make the webserver send the following document attached. Actual Results: emerge saves this error message as the first part of the package to be downloaded. It continues to query other mirrors for the remaining bytes and failes the digest verification after the download. Expected Results: Discarded the error message the same way it does already some others error messages (e.g. 404). One possibility would be to generally discard partial downloads of less than 10k, say. Portage 2.0.51.19 (default-darwin/macos/10.3, gcc-3.3, libsystem-7.1-r0, 7.9.0 Power Macintosh) ================================================================= System uname: 7.9.0 Power Macintosh powerpc macos-20041118 Python: [2.3 (#1, Sep 13 2003, 00:49:11)] distcc 2.0.1-zeroconf powerpc-apple-darwin7.0 (protocol 1) (default port 3632) [disabled] dev-lang/python: [Not Present] sys-apps/sandbox: [Not Present] sys-devel/autoconf: [Not Present] sys-devel/automake: [Not Present] sys-devel/binutils: [Not Present] sys-devel/libtool: [Not Present] virtual/os-headers: 7.1 ACCEPT_KEYWORDS="ppc-macos" AUTOCLEAN="yes" CFLAGS="-O2 -pipe" CHOST="powerpc-apple-darwin" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/env.d" CXXFLAGS="-O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs autoconfig ccache collision-protect distlocks sfperms strict userpriv" GENTOO_MIRRORS="" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="ppc X aqua emacs emboss gdbm libwww ncurses offensive png ppc-macos readline tetex unicode userland_Darwin kernel_Darwin elibc_Darwin" Unset: ASFLAGS, CBUILD, CTARGET, LANG, LC_ALL, LDFLAGS, LINGUAS
Created attachment 61225 [details] error message that confuses emerge Send this file using your dummy mirror to observe the bug.
Could you please post HTTP headers too?
Created attachment 61227 [details] http headers returned by server I saved these headers using curl -D. If emerge uses any special curl flags, please tell me.
As portage never sees the headers there isn't a sane way for us to solve this. Reassigning as this should be done by the fetcher, and this is osx/curl specific.
from man curl(1) : -f/--fail (HTTP) Fail silently (no output at all) on server errors. This is mostly done like this to better enable scripts etc to better deal with failed attempts. In normal cases when a HTTP server fails to deliver a document, it returns a HTML document stating so (which often also describes why and more). This flag will prevent curl from outputting that and fail silently instead. If this option is used twice, the second will again disable silent failure. So, I'll add the -f flag to default-darwin/make.defaults after some testing, which should fix the problem.
I'd like to try this fix, too. Can you tell me where to set the -f flag?
Fixed in CVS. Thanks Kito for the '-f' insight. I use wget, but I was able to test this fix. Tobias Hahn: Just do an rsync in about 30 minutes and you'll have the fix. :) Just in case you can't wait, throw '-f' as an argument to curl in $PORTDIR/profiles/default-darwin/ make.defaults
