I like to get added as a new mirror for the gentoo dist files. Our server is reachable over IPv6 and IPv4 with the following names: ftp.join.uni-muenster.de (IPv4 & IPv6) (recommended) ftp.ipv6.uni-muenster.de (IPv6-only) ftp6.uni-muenster.de (IPv4) (usually useless and unneccessary) IPv4 address: 128.176.191.21 IPv6 address: 2001:638:500:101:201:2ff:fedd:5056 As contact you can use my email address schild@uni-muenster.de or the address of our team (join@uni-muenster.de). I add my signature for further contact details or you may take a look at our projects homepage (http://www.join.uni-muenster.de/?lang=en). -- JOIN - IP Version 6 in the WiN Christian Schild A DFN project Westfaelische Wilhelms-Universitaet Muenster http://www.join.uni-muenster.de Zentrum fuer Informationsverarbeitung Team: join@uni-muenster.de Roentgenstrasse 9-13 Priv: schild@uni-muenster.de D-48149 Muenster / Germany GPG-/PGP-Key-ID: 6EBFA081 Fon: +49 251 83 31638, fax: +49 251 83 31653 Reproducible: Always Steps to Reproduce: 1. 2. 3.
We're a bit behind on setting up new mirrors right now, but give us a week or so and we'll get things caught up.
Sorry for the delay. Can you confirm the CPU & RAM of the server and if it has at least 5Mbps full duplex bandwidth? Thanks.
Thanks for the info. No problem with any "DUPs". The information looks familiar, so I may have accidentally deleted a message. In the future, if you could reply directly to the bug report, that would be helpful. I've added 128.176.191.21 to the access list for rsync1.us.gentoo.org. Please update your rsync cron jobs to point to rsync1.us.gentoo.org and post back here when you've done so. We'll monitor the server for a couple days to make sure things are working fine. Thanks. Christian Schild <join@uni-muenster.de> wrote: >Hi, > >I am not sure, if this msg reached you, so I resend it. Sorry for DUPs. > >So long, > Christian > >-----Weitergeleitete Nachricht----- > >From: Christian Schild <schild@uni-muenster.de> >To: bugzilla-daemon@wwwjr.cws.oregonstate.edu >Cc: join@uni-muenster.de >Subject: Re: [Bug 23159] I want to become an new gentoo mirror >Date: 11 Aug 2003 07:58:40 +0200 > >Am Fre, 2003-07-25 um 19.38 schrieb >bugzilla-daemon@wwwjr.cws.oregonstate.edu: >> http://bugs.gentoo.org/show_bug.cgi?id=23159 >> >> >> >> >> >> ------- Additional Comments From pjp@gentoo.org 2003-25-07 10:38 EST ------- >> Sorry for the delay. Can you confirm the CPU & RAM of the server and if >> it has at least 5Mbps full duplex bandwidth? > >Hi, > >The CPU is a 700 MHz P-III and memory is 128 MB. 128MB might be a little >bit low, but the system is doing ftp services exclusively. I try to add >a few MB anyway. > >The system is connected with 100MBit full duplex to our backbone, the >universities upstream is currently 155MBit. > >So long, > Christian >-- >JOIN - IP Version 6 in the WiN Christian Schild >A DFN project Westfaelische Wilhelms-Universitaet Muenster >http://www.join.uni-muenster.de Zentrum fuer Informationsverarbeitung >Team: join@uni-muenster.de Roentgenstrasse 9-13 >Priv: schild@uni-muenster.de D-48149 Muenster / Germany >GPG-/PGP-Key-ID: 6EBFA081 Fon: +49 251 83 31638, fax: +49 251 83 31653 > >
I tried to change to server to rsync1.us.gentoo.org but I can't connect to it with rsync. ----- $ rsync -an rsync://rsync1.us.gentoo.org rsync: failed to connect to rsync1.us.gentoo.org: Connection timed out rsync error: error in socket IO (code 10) at clientserver.c(83) ----- The source address is correct: ----- [snip] 231.177715 128.176.191.21 -> 128.193.0.34 TCP 4084 > rsync [SYN, ECN, CWR] Seq=2999329989 Ack=0 Win=5840 Len=0 234.173319 128.176.191.21 -> 128.193.0.34 TCP 4084 > rsync [SYN, ECN, CWR] Seq=2999329989 Ack=0 Win=5840 Len=0 240.173322 128.176.191.21 -> 128.193.0.34 TCP 4084 > rsync [SYN, ECN, CWR] Seq=2999329989 Ack=0 Win=5840 Len=0 [snip] ----- Our rsync is version 2.5.6-26.
I skipped a step, but that should be fixed now. Let me know if you still have any problems connecting.
I changed the server to rsync1.us.gentoo.org now, and retrieving the four distfiles directories twice a day. Is this sufficient? One can get them with ftp at ftp://ftp.join.uni-muenster.de/pub/linux/distributions/gentoo or with rsync at ftp.join.uni-muenster.de::gentoo The name ftp.join.uni-muenster.de returns both, an IPv4 and IPv6 address. If you want to make sure, you use IPv6-only, you can use the name ftp.ipv6.uni-muenster.de for both above methods. So retrieving the distfiles works fine. I am not sure, if I made myself clear in my first message, we want to do both, mirroring the distfiles and also offering the rsync portage stuff. But I have difficulties to retrieve the portage tree from the server, the rsync connections closes down after some time. Do you have to open up your server a little bit more for that?
The rsync mirror document (http://www.gentoo.org/doc/en/rsync.xml) should cover everything that needs to be done. Syncing needs to occur twice every hour (at :00 and :30). As for the 4 directories, I'm not sure which ones you are referring to. I've not actually set up a mirror, so I could be wrong, but I think "Code listing 5.2: rsync-gentoo-portage.sh" (in the link above) configures what is necessary. If that still isn't clear, let me know and I'll get someone to explain it more clearly. Thanks.
I clicked submit too soon. Regarding the source mirror files, those are handled differently. The instructions I mentioned above are only for the rsync side. For source mirror instructions, someone will need to send those to you seperately. Regarding the IPv6 address, we can add that after the mirror has been tested and determined to be working successfully. I don't have IPv6 setup, so I'm only able to test via IPv4.
Ok sorry for the confusion, in my last msg I was primarily reffering to mirroring the source distfiles (the four dirs experimental, releases, distfiles and snapshots). That works fine. Concerning the portage rsync, I still observe problems retrieving the portage tree. I sometimes (sometimes!) get the error: ----- rsync: connection unexpectedly closed (1285686 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(165) ----- The error doesn't appear, if I skip the --delete option.
Unfortunately, I'm not too familiar with troubleshooting rsync problems. There's something definately wrong. In addition to some syncs being missed, there are some that take an unusually long time to complete. For example, 07:20:09 Fri, 15 Aug 2003 06:55:00 +0000 (25 minutes to complete) 08:50:27 Fri, 15 Aug 2003 08:47:00 +0000 (17 minutes) For reference, I'm including some data. I'll see if I can get someone to look at it. In the meantime, can you post your rsyncd.conf and cron job information for syncing? Thanks. Checked (UTC) Timestamp ================================================ 23:50:13 Wed, 13 Aug 2003 23:31:03 +0000 ... 07:20:13 Thu, 14 Aug 2003 06:31:03 +0000 x 07:50:14 Thu, 14 Aug 2003 06:31:03 +0000 x 08:20:13 Thu, 14 Aug 2003 06:31:03 +0000 x 08:50:13 Thu, 14 Aug 2003 06:31:03 +0000 x 09:20:13 Thu, 14 Aug 2003 08:31:03 +0000 x 09:50:13 Thu, 14 Aug 2003 08:31:03 +0000 x 10:20:12 Thu, 14 Aug 2003 08:31:03 +0000 x 14:20:12 Thu, 14 Aug 2003 13:31:04 +0000 x 18:50:13 Thu, 14 Aug 2003 18:01:01 +0000 x ... 06:50:09 Fri, 15 Aug 2003 06:48:00 +0000 x 07:20:09 Fri, 15 Aug 2003 06:55:00 +0000 x 07:50:09 Fri, 15 Aug 2003 06:55:00 +0000 x 08:20:09 Fri, 15 Aug 2003 08:05:00 +0000 x 08:50:27 Fri, 15 Aug 2003 08:47:00 +0000 x 09:20:09 Fri, 15 Aug 2003 08:47:00 +0000 x 09:50:09 Fri, 15 Aug 2003 08:47:00 +0000 x 10:20:09 Fri, 15 Aug 2003 08:47:00 +0000 x 10:50:09 Fri, 15 Aug 2003 08:47:00 +0000 x 11:20:08 Fri, 15 Aug 2003 08:47:00 +0000 x 11:50:09 Fri, 15 Aug 2003 11:22:00 +0000 x 12:20:09 Fri, 15 Aug 2003 11:22:00 +0000 x 12:50:09 Fri, 15 Aug 2003 11:22:00 +0000 x 13:20:10 Fri, 15 Aug 2003 11:22:00 +0000 x 13:50:09 Fri, 15 Aug 2003 11:22:00 +0000 x 14:20:10 Fri, 15 Aug 2003 11:22:00 +0000 x 14:50:09 Fri, 15 Aug 2003 11:22:00 +0000 x
Right now the crontab says: ----- 0,30 * * * * rsync --stats --progress -vrlptD --timeout=600 --delete rsync://rsync1.us.gentoo.org/gentoo-portage/ /raid/ftp/pub/mirrors/gentoo.org/gentoo-portage/ >> /tmp/gentoo-rsync-watch 2>&1 ----- stats, progress and the pipe are only present for debugging. I think the rsyncd.conf is of no relevance here, cause it only hold the config for our own rsync server. The problem occurs, when running rsync as a client. Please note, that everything works fine, when I use other portage mirror servers. Namely I checked it with trumpetti.atm.tut.fi (over IPv6) and ftp.snt.utwente.nl (over IPv4). Everything works fine there. I assume the download takes so long, because for the file deletion process rsync has to check the local filesystem. Our server is a very large raid with quite a lot of files and indeed, it takes some time to check all portage files (but usually less than, lets say 5 mins (haven't measured it)). This might be a reason, why the connection to your server breaks, maybe there is a server side timeout? Again, I don't have this problem when connecting to other servers. I recall from other mirroring lists, that there are such kind of problems, when on client and server different versions of rsync are used. I'm running version 2.5.6-26, what version is your server?
I changed two things. First I added more memory (1G now). This means most of the files are cached and the "to delete"-check is much faster most of the times and we probably don't have this timeout-like error anymore. Second, I browsed the rsync archives, it seems that others had exactly the same problem (see http://www.mail-archive.com/rsync@lists.samba.org/msg07305.html). Their "solution" was to add --compress, so I did that too. So you might want to start your monitoring again.
*sigh* It doesn't work (always). I even tried a CVS version of rsync. I still assume this is a server sided (your) timeout. It may happen, that I don't access you server for more than 3 minutes (3:15 last time) while checking my local filesystem and then I get the error message. This seems to be an rsync bug, but I suggest you should try to ease up your timeout settings. Another solution might be, I rsync with the server twice every time. In the second run, the local files are cached, and the download will complete. But this also sounds sick to me.
I've been watching this on the server side, and things seem to be fine up until before 0700 UTC today: Aug 27 05:00:23 eagle rsyncd[707]: rsync on gentoo-portage/ from V6SERV01.UNI-MUENSTER.DE (128.176.191.21) Aug 27 05:30:13 eagle rsyncd[1341]: rsync on gentoo-portage/ from V6SERV01.UNI-MUENSTER.DE (128.176.191.21) Aug 27 06:00:29 eagle rsyncd[2130]: rsync on gentoo-portage/ from V6SERV01.UNI-MUENSTER.DE (128.176.191.21) Aug 27 06:30:19 eagle rsyncd[5104]: rsync on gentoo-portage/ from V6SERV01.UNI-MUENSTER.DE (128.176.191.21) Aug 27 06:59:03 eagle rsyncd[5792]: rsync on gentoo-portage/ from V6SERV01.UNI-MUENSTER.DE (128.176.191.21) Aug 27 07:00:29 eagle rsyncd[5875]: rsync on gentoo-portage/ from V6SERV01.UNI-MUENSTER.DE (128.176.191.21) Aug 27 07:29:02 eagle rsyncd[6323]: rsync on gentoo-portage/ from V6SERV01.UNI-MUENSTER.DE (128.176.191.21) Aug 27 07:32:52 eagle rsyncd[6430]: rsync on gentoo-portage/ from V6SERV01.UNI-MUENSTER.DE (128.176.191.21) Aug 27 07:59:03 eagle rsyncd[7090]: rsync on gentoo-portage/ from V6SERV01.UNI-MUENSTER.DE (128.176.191.21) Aug 27 08:02:06 eagle rsyncd[7200]: rsync on gentoo-portage/ from V6SERV01.UNI-MUENSTER.DE (128.176.191.21) Aug 27 08:29:03 eagle rsyncd[7664]: rsync on gentoo-portage/ from V6SERV01.UNI-MUENSTER.DE (128.176.191.21) Aug 27 08:31:14 eagle rsyncd[7765]: rsync on gentoo-portage/ from V6SERV01.UNI-MUENSTER.DE (128.176.191.21) Aug 27 08:59:02 eagle rsyncd[8608]: rsync on gentoo-portage/ from V6SERV01.UNI-MUENSTER.DE (128.176.191.21) Aug 27 09:01:33 eagle rsyncd[8705]: rsync on gentoo-portage/ from V6SERV01.UNI-MUENSTER.DE (128.176.191.21) This off-time pattern continues throughout the day. Can you fill me in on what this shift might be? As far as the occasional connection timeout issue, that seems to be a problem that we are dealing with on rsync1.us due to a little overload. We are working on this problem and should have it fixed in a few months. Running --compress shouldn't do anything, because we set no compression on the server side. Cheers!
I changed rsync behaviour this morning. I still observed timeout-like errors about every second time. My dirty hack is to run rsync twice. In the second run, all local files are cached in memory, and the 'delete'-check is fast enough to not get caught by the timeout. That way syncing works 100% at all times. The second run takes about 1 minute, so I shifted the start to :29/:59 to get synced with all other servers.
One more thing.. We noticed that you were syncing your distfiles module against rsync1.us.gentoo.org We didn't realize that that module was publicly accessible, but you need to sync that module off of gentoo.oregonstate.edu Please make that change as soon as possible, and that should free up some resources on rsync1.us.gentoo.org and maybe clear that timeout issue. Cheers
Excellent. I stopped syncing the distfiles from rsync1.us. Do I have to ask on the gentoo-mirrors list to get access to gentoo.oregonstate.edu?
> Do I have to ask on the gentoo-mirrors list to get access to > gentoo.oregonstate.edu? No, I'll get some instructions to you tomorrow on what to do for the source mirroring. I'm catching up on old source mirror requests. Thanks for your patience.
I've sent you some instructions on accessing our private distfiles mirror. Please update your scripts to point to this mirror. Once your syncing, I'll verify a couple and then update our mirrors page with your server info. Thanks.
I'm syncing the distfiles twice a day from ftp.oregonstate.edu now.
> I'm syncing the distfiles twice a day from ftp.oregonstate.edu now. Mirrors need to be syncing every 4 hours, beginning with midnight local time. Will the server be able to handle that?
Sure, as long as the download wouldn't take longer than four hours :) (download rate is really slow, below 100kb/s all the time)
I tried ftp'ing ftp.join.uni-muenster.de/pub/linux/distributions/gentoo/distfiles/timestamp.chk but received this error: Resolving ftp.join.uni-muenster.de... done. Connecting to ftp.join.uni-muenster.de[128.176.191.21]:21... connected. Logging in as anonymous ... Logged in! ==> SYST ... done. ==> PWD ... done. ==> TYPE I ... done. ==> CWD /pub/linux/distributions/gentoo/distfiles ... done. ==> PORT ... done. ==> RETR timestamp.chk ... Error in server response, closing control connection. Retrying. I had no problems accessing the file with a web browser (ftp://ftp....)
The mirror seems to be syncing OK, so we're going to add it. I must have a client that doesn't like wuftpd-server. Hopefully, there won't be many others with the same problem. I've subscribed <schild@uni-muenster.de> to our gentoo-mirrors mailing list (low volume). This list is used to discuss mirror-related issues. On the rsync side, the mirror is still not syncing consistantly enough. I have learned that it is likely a problem on our end. rsync1.us is running out of RAM when under heavy load. We're working on removing some non-syncing duties from the box, but that might take a while. If you're willing to wait until we can resolve the issue, I'd be glad to post back here once we're able to start testing again. In the meantime, I'm going to temporarily turn off rsync1.us.gentoo.org access for this box. Then, when we're ready, I'll turn it back on and we can begin testing. Thanks for supporting Gentoo. Christian Schild <schild@uni-muenster.de> wrote: >Hiho, > >From experience I know that some distributions or ftp clients don't work >well together with wuftpd-server. >But I tried the same (wget) on several distributions (gentoo, debian, >mandrake, freebsd) and hosts (even a remote host) and it worked all the >time: >----- >$ wget >ftp://ftp.join.uni-muenster.de/pub/linux/distributions/gentoo/distfiles/timestamp.chk >--08:02:47-- >ftp://ftp.join.uni-muenster.de/pub/linux/distributions/gentoo/distfiles/timestamp.chk > => `timestamp.chk' >Aufl
The mirror seems to be syncing OK, so we're going to add it. I must have a client that doesn't like wuftpd-server. Hopefully, there won't be many others with the same problem. I've subscribed <schild@uni-muenster.de> to our gentoo-mirrors mailing list (low volume). This list is used to discuss mirror-related issues. On the rsync side, the mirror is still not syncing consistantly enough. I have learned that it is likely a problem on our end. rsync1.us is running out of RAM when under heavy load. We're working on removing some non-syncing duties from the box, but that might take a while. If you're willing to wait until we can resolve the issue, I'd be glad to post back here once we're able to start testing again. In the meantime, I'm going to temporarily turn off rsync1.us.gentoo.org access for this box. Then, when we're ready, I'll turn it back on and we can begin testing. Thanks for supporting Gentoo. Christian Schild <schild@uni-muenster.de> wrote: >Hiho, > >From experience I know that some distributions or ftp clients don't work >well together with wuftpd-server. >But I tried the same (wget) on several distributions (gentoo, debian, >mandrake, freebsd) and hosts (even a remote host) and it worked all the >time: >----- >$ wget >ftp://ftp.join.uni-muenster.de/pub/linux/distributions/gentoo/distfiles/timestamp.chk >--08:02:47-- >ftp://ftp.join.uni-muenster.de/pub/linux/distributions/gentoo/distfiles/timestamp.chk > => `timestamp.chk' >Auflösen des Hostnamen »ftp.join.uni-muenster.de«.... fertig. >Verbindungsaufbau zu ftp.join.uni-muenster.de[128.176.191.21]:21... >verbunden. >Anmelden als anonymous ... Angemeldet! >==> SYST ... fertig. ==> PWD ... fertig. >==> TYPE I ... fertig. ==> CWD >/pub/linux/distributions/gentoo/distfiles ... fertig. >==> PASV ... fertig. ==> RETR timestamp.chk ... fertig. >Länge: 32 (unmaßgeblich) > >100%[========================================================================>] 32 31.25K/s ETA 00:00 > >08:02:47 (31.25 KB/s) - »timestamp.chk« gespeichert [32] > >----- > >Maybe a temporary problem? Some non-default wget settings? > >So long, > Christian
I stopped syncing from rsync1.us for now and changed to one of the other secondary rsync-servers for the time beeing. You added our site to the mirror list webpage, but the links are broken. The name is very long, you might want to change it to "Universitiy of Muenster/JOIN" or "Uni Muenster/JOIN" to be consistent with other (german university) names. I don't know if you need to list us three times. Maybe it is enough to add the asterisk for IPv6 support (like for tut.fi, again to make your website consistent :-) You could also add rsync-support for the mirror (rsync://ftp.join.uni-muenster.de/gentoo/ , also IPv6). The same host names for IPv4 and/or IPv6 apply here for rsync access.
If you hadn't checked, the Mirrors page is updated. Thanks for pointing out the * (denoting IPv6)
We've resolved some load problems that rsync1.us was having. I've also enabled added 128.176.191.21 back on the access list (should be able to sync again in about 30 minutes). If you're still interested, repoint to rsync1.us and we'll see if the problems continue. Thanks.
A few minutes ago I pointed our gentoo-portage mirror back to rsync1.us.
This server is now rsync14.de.gentoo.org. Please update the motd file to include that information. Both IPv4 & IPv6 addresses should work. I've subscribed <schild@uni-muenster.de> to our gentoo-mirrors mailing list (low volume). This list is used to discuss mirror-related issues. Thanks for supporting Gentoo.
