# env SYNC="rsync://rsync-gentoo.planetmirror.com/gentoo-portage" emerge sync >>> starting rsync with rsync://203.16.234.26/gentoo-portage... >>> checking server timestamp ... Welcome to PlanetMirror's rsync service for Gentoo Linux ... .receiving file list ... 0 files to consider client: nothing to do: perhaps you need to specify some filenames or the --recursive option? rsync error: some files could not be transferred (code 23) at main.c(653) (error occured 3 times)
Planetmirror Admin, Your machine's timestamp is off by about 12 hours. Please make sure you are syncing to rsync1.us.gentoo.org per the rsync server document. As for not being in the rsync.au.gentoo.org rotation, were you ever in it? I do not see your IP in said DNS records. For Infrastructure, Jeffrey
We are definitely syncing from that server, and we're definitely doing it every 30 minutes. I believe this might be related to something I raised on the mirror mailing list a couple of times and went un-resolved. Almost every time we run a mirror sync, we get the following error: rsync error: timeout in data send/receive (code 30) at io.c(109) rsync: connection unexpectedly closed (2591764 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(342) So far, no one has been able to identify where the problem might lie. I've tried the rsync mailing list, and numerous mirror lists, google searches and have yet to turn up a definitive cause. Perhaps the maintainer for rsync1.us.gentoo.org could look at the server logs for the rsync processes at 04 and 34 past the hour (that's when we sync), for client IP 203.16.234.17.
Daniel, The rsync issue is something we have bugged in our bugzilla database, but no one has been able to tell us anything more, as you can tell. I see you syncing to rsync1.us since this is a snippet of the logs: gentoo1 log # grep 203.16 daemon.log Jun 11 04:04:03 gentoo1 rsyncd[4460]: rsync on gentoo-portage/ from tiktok.planetmirror.com (203.16.234.17) Jun 11 05:04:02 gentoo1 rsyncd[4625]: rsync on gentoo-portage/ from tiktok.planetmirror.com (203.16.234.17) Jun 11 07:04:01 gentoo1 rsyncd[4965]: rsync on gentoo-portage/ from tiktok.planetmirror.com (203.16.234.17) Jun 11 07:34:03 gentoo1 rsyncd[5056]: rsync on gentoo-portage/ from tiktok.planetmirror.com (203.16.234.17) Even though your timestamp is 2 days old: Wed, 09 Jun 2004 02:50:00 +0000 I'm going to pass this up the infrastructure ranks and see if they know what could be going on. -Jeffrey
Daniel, Try downgrading to rsync 2.6.0. We have had issues with 2.6.2 giving that 'connection unexpectedly closed' error, and downgrading usually helps. Please try that. Thanks! -Jeffrey
I was using 2.6.0 rsync --version rsync version 2.6.0 protocol version 27 Copyright (C) 1996-2004 by Andrew Tridgell and others <http://rsync.samba.org/> Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles, IPv6, 64-bit system inums, 64-bit internal inums env SYNC="rsync://rsync-gentoo.planetmirror.com/gentoo-portage" emerge sync >>> starting rsync with rsync://203.16.234.25/gentoo-portage... >>> checking server timestamp ... Welcome to PlanetMirror's rsync service for Gentoo Linux rsync1.au.gentoo.org Contact PlanetMirror Support support@planetmirror.com You can find a web front end to this archive at: http://planetmirror.com You can also access this service via FTP at ftp://ftp.planetmirror.com !!! --- !!! If you are a regular Gentoo or PM user, please consider supporting your distribution and PlanetMirror by subscribing to a Premium account via the web front end. !!! --- !!! receiving file list ... 1 file to consider Number of files: 1 Number of files transferred: 0 Total file size: 32 bytes Total transferred file size: 0 bytes Literal data: 0 bytes Matched data: 0 bytes File list size: 32 Total bytes written: 207 Total bytes read: 615 wrote 207 bytes read 615 bytes 328.80 bytes/sec total size is 32 speedup is 0.04 >>> >>> SERVER OUT OF DATE: rsync://203.16.234.25/gentoo-portage >>> >>> retry ... spider portage # date -u Mon Jun 21 14:01:40 UTC 2004
Are you attempting to sync with yourself? Try and sync with rsync1.us.gentoo.org. Others are not reporting issues with rsync-2.6.0 and rsync1.us. If you try and sync with yourself, it will be out date. I just checked the timestamp this afternoon: IP: 203.16.234.25 BUG: Checked (UTC) Timestamp ================================================ 18:16:28 Sun, 13 Jun 2004 05:20:00 +0000 -Jeffrey
As I posted earlier, our sync script is definitely syncing from rsync1.us.gentoo.org. Can I ask what prompted the suggestion we might be syncing from ourselves? If we were, wouldn't our timestamp stay the same, rather than being 12 hours behind? Just wondering :) As a thought, could we sync from a single IP from the rsync1.us.gentoo.org cluster? Is there anything else I can give you that will be of assistance (such as rsync logs etc)?
Could you check to see if you have any files with an mtime/ctime newer than June 13th? It's possible your metadata/timestamp.chk file isn't updating for some reason, while other files are updating fine. (though I don't think that's the problem) If that fails, could you try deleting your current tree and fetching a new one from rsync1.us? --kurt
I ran a find over our tree and found no files newer than June 13th. I believe this is closely related (in the sense that, we're lagging behind because this other bug exists) to another bug I've raised here, in the gentoo-mirror list, and in various other places, in that when we rsync, we start to see a transfer, but get: rsync error: timeout in data send/receive (code 30) at io.c(109) rsync: connection unexpectedly closed (2623094 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(342) rsync: connection unexpectedly closed (2623094 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(342) To date, no-one's been able to help out with this one. Any idea of some other testing I can do to see what speeds we get from rsync1.us? Basically I don't think we can do much w.r.t. the portage tree until the rsync/timeout issue is resolved.
OK, can you try to perform a complete sync w/ kite.gentoo.org? rsync -av kite.gentoo.org::gentoo-portage/ /tmp/ obviously adjust /tmp/ to taste :) We're not seeing this problem on any other mirror, so I'm puzzled as to what might be the cause.
Just an update since this has been open for a while. # date -u; rsync --version; env SYNC="rsync://rsync-gentoo.planetmirror.com/gentoo-portage" emerge sync Sun Oct 3 08:11:29 UTC 2004 rsync version 2.6.0 protocol version 27 Copyright (C) 1996-2004 by Andrew Tridgell and others <http://rsync.samba.org/> Capabilities: 64-bit files, socketpairs, hard links, acls, symlinks, batchfiles, IPv6, 64-bit system inums, 64-bit internal inums rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it under certain conditions. See the GNU General Public Licence for details. >>> starting rsync with rsync://203.16.234.26/gentoo-portage... >>> checking server timestamp ... Welcome to PlanetMirror's rsync service for Gentoo Linux rsync1.au.gentoo.org Contact PlanetMirror Support support@planetmirror.com You can find a web front end to this archive at: http://planetmirror.com You can also access this service via FTP at ftp://ftp.planetmirror.com !!! --- !!! If you are a regular Gentoo or PM user, please consider supporting your distribution and PlanetMirror by subscribing to a Premium account via the web front end. !!! --- !!! receiving file list ... 1 file to consider Number of files: 1 Number of files transferred: 0 Total file size: 32 bytes Total transferred file size: 0 bytes Literal data: 0 bytes Matched data: 0 bytes File list size: 32 Total bytes written: 207 Total bytes read: 615 wrote 207 bytes read 615 bytes 53.03 bytes/sec total size is 32 speedup is 0.04 >>> >>> SERVER OUT OF DATE: rsync://203.16.234.26/gentoo-portage >>> Immediately afterwards I tried: # date -u; rsync --version; env SYNC="rsync://rsync-gentoo.planetmirror.com/gentoo-portage" emerge --debug sync Sun Oct 3 08:14:34 UTC 2004 rsync version 2.6.0 protocol version 27 Copyright (C) 1996-2004 by Andrew Tridgell and others <http://rsync.samba.org/> Capabilities: 64-bit files, socketpairs, hard links, acls, symlinks, batchfiles, IPv6, 64-bit system inums, 64-bit internal inums rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it under certain conditions. See the GNU General Public Licence for details. >>> starting rsync with rsync://203.16.234.25/gentoo-portage... >>> checking server timestamp ... Welcome to PlanetMirror's rsync service for Gentoo Linux .... rsync: connection unexpectedly closed (571 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(189) >>> retry ...
Okay I checked our logs. At the time below that the Rsync failed, I see a number of connection attempts, all from an Internode IP address. The errors in our log at those times are: rsync: This server does not support --checksum (-c) BUT - both our servers have "refuse options = checksum" in the config. So why one would work (albeit out of date) and the other failed is beyond me. As for why it's out of date - well, every time I try to set up a sync with most of the official servers, I get errors (as outlined elsewhere in this bug).
Just checked the mirror, its last sync was back in September. I'm removing its dns record. Please reopen bug if you wish to become a gentoo rsync mirror again. Thanks! -Jeffrey
