Hi, sometimes, when an invalid distfile gets distributed to our mirrors but corrected later, people using net-proxy/http-replicator will have to manually fix the distfile due to the used "--flat" parameter: 1. net-misc/openssh-6.6.1_p1-r4 was released. There was a problem with "openssh-6.6.1p1-hpnssh14v5.diff.xz". But the wrong file was already distributed on the mirrors. 2. The problem got fixed, the source (http://dev.gentoo.org/~polynomial-c/openssh-6.6.1p1-hpnssh14v5.diff.xz) now serves the right file. 3. When using emerge, emerge will notice that the file downloaded is invalid and will try another mirror and finally will go to the source. But due to the "--flat" parameter, this doesn't work with http-replicator, see: [ INIT ] Replicator started at srv1:8080 [ IDLE ] Wed Oct 1 16:10:10 2014 [ BUSY ] Wed Oct 1 16:10:49 2014 [ 0.00 ] Wed Oct 1 16:10:49 2014 0.00 Accepted request from 10.82.0.1:59403 0.00 Client sends GET http://ftp.uni-erlangen.de/pub/mirrors/gentoo/distfiles/openssh-6.6.1p1-hpnssh14v5.diff.xz HTTP/1.1 0.00 Switching to HttpProtocol 0.00 Cache position: openssh-6.6.1p1-hpnssh14v5.diff.xz 0.00 Requesting address info for ftp.uni-erlangen.de:80 0.00 Connecting to 131.188.12.211:80 0.03 Server responds HTTP/1.1 200 OK 0.03 Preparing new file in cache 0.03 Switching to DataResponse 0.03 Replicator responds HTTP/1.1 200 OK 0.04 Transaction successfully completed 0.04 Finalized /srv/distfile-cache/openssh-6.6.1p1-hpnssh14v5.diff.xz [ IDLE ] Wed Oct 1 16:10:49 2014 [ BUSY ] Wed Oct 1 16:10:49 2014 [ 0.00 ] Wed Oct 1 16:10:49 2014 0.00 Accepted request from 10.82.0.1:59405 0.00 Client sends GET http://mirror.netcologne.de/gentoo/distfiles/openssh-6.6.1p1-hpnssh14v5.diff.xz HTTP/1.1 0.00 Switching to HttpProtocol 0.00 Cache position: openssh-6.6.1p1-hpnssh14v5.diff.xz 0.00 Static mode; serving file directly from cache 0.00 Reading complete file from cache 0.00 Switching to DataResponse 0.00 Replicator responds HTTP/1.1 200 OK 0.00 Transaction successfully completed [ IDLE ] Wed Oct 1 16:10:49 2014 [ BUSY ] Wed Oct 1 16:10:49 2014 [ 0.00 ] Wed Oct 1 16:10:49 2014 0.00 Accepted request from 10.82.0.1:59406 0.00 Client sends GET http://dev.gentoo.org/~polynomial-c/openssh-6.6.1p1-hpnssh14v5.diff.xz HTTP/1.1 0.00 Switching to HttpProtocol 0.00 Cache position: openssh-6.6.1p1-hpnssh14v5.diff.xz 0.00 Static mode; serving file directly from cache 0.00 Reading complete file from cache 0.00 Switching to DataResponse 0.00 Replicator responds HTTP/1.1 200 OK 0.00 Transaction successfully completed [...] As you can see, we first tried to fetch "openssh-6.6.1p1-hpnssh14v5.diff.xz" from ftp.uni-erlangen.de. This file got cached. But because ftp.uni-erlangen.de contains the invalid file, emerge tried to download the same file from another mirror (mirror.netcologne.de in this case). But due to the set "--flat" parameter, http-replicator doesn't understand that we are requesting a different file. It just sees "Ah, I already have the requested file 'openssh-6.6.1p1-hpnssh14v5.diff.xz', let's serve it from cache!". So actually we never tried another mirror. ftp.uni-erlangen.de was the only mirror we tried... There are multiple bugs where people faced this problem. See bug 524028, bug 509874 or bug 523986 for example. I am not sure how to fix that problem. Because "repcacheman" (shipped with net-proxy/http-replicator) heavily relies on a flat hierarchy, similar to "/usr/portage/distfiles". 1) Move "--flat" from the runscript to "/etc/conf.d/http-replicator". I.e something like # repcacheman relies on a cache structure similar to "/usr/portage/distfiles". # NOTICE: If one mirror serves an invalid file, you have to clean up your http-replicator # cache by hand because emerge's retry attemps will always result in a cache hit (=serving # the invalid file from cache instead of actually re-downloading the file from another URI) DAEMON_OPTS="$DAEMON_OPTS --flat" ...this will at least inform people reading configurations ;) 2) Finally, find a way to get repocacheman working in a non-flat environment. Reproducible: Always
A subset of this problem is fixed in version 3.0-r5 [same file name but different URL], as long as the client side uses the correct FETCHCOMMAND (see bug 442874). However, the specific issue of a source file being replaced in-place is still broken and requires manual intervention to recover. If someone wants to work on this, perhaps portage could be tweaked to add additional headers (similar to bug 442874) when retrying the download after a hash/checksum failure, and then http-replicator watches for the extra headers. Perhaps a "ignore cached" or a "expecting this sha256" extra HTTP header.
*** Bug 503226 has been marked as a duplicate of this bug. ***
I would not advise blindly removing the --flat option from http-replicator-4.0_alpha2. In addition to the repcacheman concern mentioned above, from my memory of a quick look at the code a few years ago, unless --flat is specified there does not appear to be anything to protect a 4.0 server from someone requesting "GET /../../../../any/file/on/your/server". (3.0 looked OK, though). Perhaps this should be considered a security issue, although I think the default configuration isn't vulnerable. See also: - bug 442874, comment 18. - If you follow some links from the ebuild-indicated homepage (moved..., issues...), you can find an upstream bug report I filed about this a few years ago, but it hasn't had any activity: https://github.com/gertjanvanzwieten/replicator/issues/4 - I may add a more detailed comment to bug 676758 (stabilize 4.0) about other things 3.0 still does better than 4.0.
