Created attachment 325630 [details] emerge --info I would like to use an ftps/https binhost for portage. From discussion on the forum, I was told the problem is that FETCHCOMMAND is not used for the Packages file? (http://forums.gentoo.org/viewtopic-t-935402.html) I believe I have the FETCHCOMMAND set correctly, as using it from the command line I can fetch files from my host. Further info here: http://forums.gentoo.org/viewtopic-t-935402.html From make.conf ====================== FETCHCOMMAND_FTP= "/usr/bin/curl -R \${URI} -o ${DISTDIR}/\${FILE} --ssl" RESUMECOMMAND_FTP="/usr/bin/curl -R \${URI} -o ${DISTDIR}/\${FILE} --ssl -C -" FETCHCOMMAND_HTTPS= "/usr/bin/curl -R \${URI} -o ${DISTDIR}/\${FILE} --digest --fail" RESUMECOMMAND_HTTPS="/usr/bin/curl -R \${URI} -o ${DISTDIR}/\${FILE} --digest --fail -C -" PORTAGE_BINHOST="ftp://user:password@domain.tld" (or https://) output of "sudo emerge links" ===================== kai@kai-laptop ~ % sudo emerge -a links Password: % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1136k 100 1136k 0 0 187k 0 0:00:06 0:00:06 --:--:-- 250k !!! Binhost package index has no TIMESTAMP field. Fetching bininfo from ftp://user:*password*@domain.tld !!! 421 Sorry, cleartext sessions are not accepted on this server. These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild R ] www-client/links-2.7 Would you like to merge these packages? [Yes/No] no ================== It seems that FETCHCOMMAND is being used, but then portage complains about the TIMESTAMP field, so I am not entirely sure what is happening. The error looks very similar for https when authentication is enabled, but the emerge works when it is disabled.
What ftps/https server applications are you using? From personal testing, I know that it works with thttpd (with or without authentication, and also with or without encryption via stunnel).
Created attachment 325634 [details] Apache configuration
I am using pure-ftpd-1.0.36 with -Y 3 set, and apache-2.2.22-r1 (config attached). The problem with HTTPS is that I need to use SNI, as I have several websites hosted on the computer for testing purposes.
We've had issues reported with pure-ftpd before (see bug #350139), but none with apache.
(In reply to comment #4) > We've had issues reported with pure-ftpd before (see bug #350139), but none > with apache. I think this may be a different issue, it doesn't hang as described in that report. According to the progress printout shown the file transfers completely. It is only after this that portage complains about lack of a TIMESTAMP field (I wasn't able to dig out exactly where this comes from in the source). The output of "sudo emerge links" in the original report demonstrates this a bit. Let me know if I should attach anything else (I can do an apache version, but it is very similar. The binhost is the second of several HTTPS vhosts).
I think there may be a larger problem with either my setup or the FETCHCOMMAND - If I use HTTPS but turn off authentication it fetches the package index file fine, but then fails (after fetching is complete) as follows on the first binary: >>> Emerging binary (1 of 43) dev-libs/expat-2.1.0-r2 * Fetching '/usr/portage/packages/dev-libs/expat-2.1.0-r2.tbz2' in * the background. To view fetch progress, run `tail -f /var/log * /emerge-fetch.log` in another terminal. % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 209k 100 209k 0 0 43272 0 0:00:04 0:00:04 --:--:-- 50894 !!! Fetching Binary failed for 'dev-libs/expat-2.1.0-r2' >>> Failed to emerge dev-libs/expat-2.1.0-r2, Log file: >>> '/var/tmp/portage/dev-libs/expat-2.1.0-r2/temp/build.log' I am not sure whether this was also the case before, or whether I had simply not tried further once seeing it did not fail on fetching Packages. The only thing that appears to be working is a plain HTTP server without authentication. With normal FTP it hangs as in the linked bug (bug #350139) - whatever is being used to fetch files does not respect pure-ftpd's port flag (-p port1:port2) for passive FTP (I have only opened a few through my firewall, and they are not being used). I think the built-in python function is used rather than cURL, as the progress bar never appears, and I do not have this issue with cURL. I'm not sure exactly what the scope of this is anymore, please let me know what you want me to attach or test.
The ftp hang is supposed to be fixed in Python 3.3: http://bugs.python.org/issue11199 http://hg.python.org/cpython/file/bd8afb90ebf2/Misc/NEWS That fix is not included in Python 2.7.3 or 3.2.3, since their NEWS files do not mention it. You can test it with Python 3.3 if you install sys-apps/portage with USE=python3 enabled.
Snapshots of Python 2.7.4_pre* and 3.2.4_pre* are available for testing in Progress Overlay.
