Platform is SUSE Enterprise 10 SP2. The `emerge -e system' step fails. The reason seems to be that download of dev-lang/python-2.6.5-r1 fails: >>> Emerging (79 of 103) dev-lang/python-2.6.5-r1 * Fetching files in the background. To view fetch progress, run * `tail -f /local/tmp/k/var/log/emerge-fetch.log` in another * terminal. >>> Downloading 'http://distfiles.gentoo.org/distfiles/Python-2.6.5.tar.bz2' --2010-04-15 04:22:26-- http://distfiles.gentoo.org/distfiles/Python-2.6.5.tar.bz2 Resolving distfiles.gentoo.org... 156.56.247.195, 199.6.1.174, 204.152.191.39, ... Connecting to distfiles.gentoo.org|156.56.247.195|:80... failed: Connection timed out. Connecting to distfiles.gentoo.org|199.6.1.174|:80... failed: Connection timed out. [ lots of similar failure indications cut ] --2010-04-15 04:56:46-- (try: 5) http://www.python.org/ftp/python/2.6.5/Python-2.6.5.tar.bz2 Connecting to www.python.org|82.94.164.162|:80... failed: Connection timed out. Giving up. !!! Couldn't download 'Python-2.6.5.tar.bz2'. Aborting.
The timeouts indicate a local error at your side to me. If your host restricted by a firewall or something?
HTTP to the outer world is through a proxy, yes. But the HTTP access works in general, there are lots of downloads that succeed before the Python download. A bit of a mystery right now. I can download the http://distfiles.gentoo.org/distfiles/Python-2.6.5.tar.bz2 manually with wget (through the proxy) with no problems, so it is not a case of adult-sites blocking (which has in fact blocked some minor Prefix omponent before). Just a thought, could it have to do with the "Fetching files in the background" feature? If I grep the console log for that string I get just one hit, and that is when trying to download Python.
I'd say unlikely as it should be using the same code, but in the background. Or it would be that the proxy settings aren't well propagated in that case. Does the problem go away if you set FEATURES="-parallel-fetch"?
I'll try it out. I see now that there were somewhat similar problems back in 2008, bug 233103.
Doing FEATURES="-parallel-fetch" does not help. Downloading is now done in the foreground, but it fails nevertheless: >>> Emerging (79 of 103) dev-lang/python-2.6.5-r1 >>> Downloading 'http://distfiles.gentoo.org/distfiles/Python-2.6.5.tar.bz2' --2010-04-15 14:55:16-- http://distfiles.gentoo.org/distfiles/Python-2.6.5.tar.bz2 Resolving distfiles.gentoo.org... 149.20.20.135, 156.56.247.195, 199.6.1.174, ... Connecting to distfiles.gentoo.org|149.20.20.135|:80... failed: Connection timed out. Connecting to distfiles.gentoo.org|156.56.247.195|:80... failed: Connection timed out.
I looked in older logs. It seems that the problem appeared for the first time yesterday (2010-04-14).
there has hardly been any activity over the last few days
Same result this morning, download of python-2.6.5-r1 fails. It is as if the download behaviour changes in the middle of the `emerge -e system'. For package 77, and many before, it looks like >>> Emerging (77 of 103) sys-libs/gdbm-1.8.3-r4 >>> Downloading 'http://distfiles.gentoo.org/distfiles/gdbm-1.8.3.tar.gz' --2010-04-16 03:53:29-- http://distfiles.gentoo.org/distfiles/gdbm-1.8.3.tar.gz Resolving www-proxy.ericsson.se... 153.88.253.150 Connecting to www-proxy.ericsson.se|153.88.253.150|:8080... connected. Proxy request sent, awaiting response... 200 OK Length: 228695 (223K) [application/x-gzip] Saving to: `/local/scratch/nightly/2010-04-16/usr/portage/distfiles/gdbm-1.8.3.tar.gz' Package 78 is perl-5.10.1, which is probably already in usr/portage/distfiles. Then for package 79 there is the failure; note that www-proxy.ericsson.se is no longer referenced: >>> Emerging (79 of 103) dev-lang/python-2.6.5-r1 >>> Downloading 'http://distfiles.gentoo.org/distfiles/Python-2.6.5.tar.bz2' --2010-04-16 03:58:24-- http://distfiles.gentoo.org/distfiles/Python-2.6.5.tar.bz2 Resolving distfiles.gentoo.org... 216.165.129.135, 140.211.166.134, 149.20.20.135, ... Connecting to distfiles.gentoo.org|216.165.129.135|:80... failed: Connection timed out. The symptoms are a little like those in bug 233103. In comment #18 of that bug I found that the environment in which downloading occurred had suddenly lost the http_proxy environment variable.
I didn't notice that before. I'd say the case here is the proxy settings getting lost somehow, causing the trouble following. @zmedico: any idea on where this could get lost? Can e.g an eclass change cause this behaviour somehow?
I will try as a workaround to set up a etc/wgetrc file that specifies http_proxy and ftp_proxy. Outcome to be reported later today.
Oops, the previous comment got stuck because of the Bugzilla password prompt. Anyway, good news: Bootstrapping succeeds with the workaround. Almost to good to be true; I insert the etc/wgetrc file before the `emerge --oneshot sed'. Somewhere along the way it gets overwritten by the etc/wgetrc file of wget-1.12-r1, which is nothing but comments. So either it has its beneficial effect sometime before it is overwritten, or bootstrapping would have worked anyway ... I should make another test without the workaround really. BTW, I checked, I don't have a personal ~/.wgetrc file around that would explain the success.
Still an issue, Rabbe?
Now I do have a very similar problem in stable main portage-2.1.8.3: repoman fails to download files, because the necessary *_proxy variables (found both in current env as well as /etc/make.conf) are not passed to subsequent wget call. Interesting enough, this did work a few minutes before... Have digged into this: In portage/package/ebuild/doebuild.py:doebuild_environment(), mysettings.envion() does not return *_proxy any more after assigning "T" in line 208, which is: mysettings["T"] = os.path.join(mysettings["PORTAGE_BUILDDIR"], "temp") Reason is in portage/package/ebuild/config.py:config.environ(), where filter_calling_env becomes True whenever $T is set and does exist: 2524 filter_calling_env = False 2525 if phase not in ('clean', 'cleanrm', 'depend'): 2526 temp_dir = self.get('T') 2527 if temp_dir is not None and \ 2528 os.path.exists(os.path.join(temp_dir, 'environment')): 2529 filter_calling_env = True As I did try to merge this new ebuild before, its $T already exists, which is the trigger for this problem at all... When I delete $PORTAGE_BUILDDIR for this package, repoman's download does work again.
So - is it intended behavior that distfiles cannot be refetched once $PORTAGE_BUILDDIR/temp does exist?
This problem, whatever it was, seems to have healed over time. I have removed my workarounds base on editing etc/wgetrc. All I do now is to have http_proxy=http://x.y.z:nnnn ftp_proxy=http://x.y.z:nnnn https_proxy=http://x.y.z:nnnn in the environment, and bootstrapping succeeds all the way.
(In reply to comment #13) > Reason is in portage/package/ebuild/config.py:config.environ(), where > filter_calling_env becomes True whenever $T is set and does exist: This should fix that: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=154aa71455be8564134d14167f839eb9fdc8160f There's another proxy fix for ROOT support here: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=1530b3ae365005c40afe0a08860641b1d4b53c92
This is fixed in 2.1.9.25.
