Hi Gentoo folk, By request of some local gentoo users I have set up portage mirror at rsync://rsync.megamirror.com/gentoo-portage (and gentoo-x86-portage). So the data is available to more people it would be great if you could include it as an offical mirror. It is currently configured to rsync to rsync2.us.gentoo.org every half hour at :00 and :30. A little info about the server: Name: rsync.megamirror.com IP: 203.220.0.2 Contact Name: Rob Wise Contact Email: rob@megamirror.com Location: Sydney, Australia Bandwidth: 100Mbps local LAN, GigE uplink to Comindico (AS9942). Additionally, I have mirrored the entire Gentoo archive which can be seen at (http|ftp)://ftp.megamirror.com/pub/gentoo/ I could use a little guidance on an appropriate mirroring frequency for the archive. If you need any more info please let me know. Thanks, Rob Reproducible: Always Steps to Reproduce:
We'll start out getting the rsync mirror going. I'm just starting to catch up with older source mirror requests, so I'll get back to you on that. For the rsync part, I've added 203.220.0.2 to the access list for rsync1.us.gentoo.org. Please update your rsync cron jobs to point to rsync1.us.gentoo.org and post back here when you've done so. We'll monitor the server for a couple days to make sure things are working fine. Thanks.
Hi Peter, I have updated our mirror script to use rsync1.us.gentoo.org and given it a test run. It seems fine, so I'll let cron do its thing for a while. Cheers, Rob
Just FYI the mirror update script has been failing frequently since my previous comment with a read timeout message. This seems to have been related to a bug in the version of rsync we were using, so I have upgraded to the latest release. It seems to be working better now, but I'll keep an eye on it. My apologies if this has disrupted your monitoring. Cheers, Rob
Actually, its still not working right. I'm seeing the following output from rsync: receiving file list ... done rsync: connection unexpectedly closed (1341935 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(165) rsync: connection unexpectedly closed (1341935 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(165) Can you check and see whats in the logs on your end? I had no problems syncing with rsync2.us so I'm curious to find out what the problem is. Cheers, Rob
Unfortunately, there's a good chance your running into a problem with rsync1.us running out of RAM when you hit it. For now, there's not much we can do about it. We are working on removing some non rsync related duties from that machine. Hopefully, that will be happening soon. We're also working on replacing it entirely, but that isn't for another 3 months or so. In the meantime, we can move onto the source mirror. I noticed the last update for that was on 25 Aug. Are you still interested in providing a source mirror?
No worries.. Would you prefer we moved back to rsync2.us for the time being to reduce load on rsync1? We're happy to provide a source mirror. We've currently mirroring 'www.ibiblio.org::gentoo' on a non-scheduled basis roughly twice per week. If you let me know your preferred master site and recommended miorring frequency I'm happy to make the required changes. Cheers, Rob
rsync1.us is our only master mirror. Syncing with rsync2.us will just put unecessary load onto that machine. I'm going to monitor the rsync mirror again for a while and see if anything has improved. If not, then we'll not be able to add it until we can get it resolved. For the source mirror, sync with ibiblio every 4 hours beginning at midnight local time. Once I've verified that syncing is working OK, I'll send some instructions for accessing the private distfiles mirror. More info is on the Source Mirror "how to" page (http://www.gentoo.org/doc/en/source_mirrors.xml).
The syncs are not occuring for hours at a time in many cases. If you manually sync at :15 or :45 past the hour (instead of :00 or :30), what is the contents of metadata/timestamp.chk ? Before running the sync manually, run date -uR to get a time to compare it with (the date in timestamp.chk is in date -uR format). I'm wondering if the distance isn't also a factor here.
Hi Peter, I've been hunting around for additional information on the rsync error but I havne't been able to find much. Someone has suggested turning on the -W flag so I'll give that a try for a while. Re the dates, none of the unix boxes here support the -R flag, but it looks like the date is the timestamp file is UTC with offset shown. I'll keep an eye on it. I'll have to automated sources mirroring going in the next couple of days. I will advise when its done. Cheers, Rob
Unfortunately it doesn't look like the various rsync options I've tried have made any difference to the success rate of the mirror runs. If you guys have no objection I'd like to go back to mirroring from rsync2.us until after your master server has been upgraded. Will that be ok?
Generally we prefer that people don't sync with secondary mirrors for long periods of time. Alot of load can be put on the servers with little benefit to the community (since it won't be an official mirror). I suppose it would primarily be up to the owner/provider of rsync2.us.
We've resolved the problem with rsync1.us. Let me know when you're syncing with rsync1.us, and I'll monitor it for a while. Thanks.
Removed access to rsync1.us Are you still interested in offering this portage mirror?
Closing. If this mirror becomes available again, just reopen the bug. Thanks for offering to support Gentoo.
