Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 30681 - new rsync mirror
Summary: new rsync mirror
Status: RESOLVED FIXED
Alias: None
Product: Mirrors
Classification: Unclassified
Component: New Server (show other bugs)
Hardware: All Linux
: High normal
Assignee: Mirror Admins
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2003-10-08 09:59 UTC by Dwayne J. Hart
Modified: 2003-12-30 17:13 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dwayne J. Hart 2003-10-08 09:59:19 UTC
Server information: mirror.cs.mun.ca,134.153.48.2
Contact information: Dwayne J. Hart dwayne@cs.mun.ca
Comment 1 Peter Penkala gentoo-dev 2003-10-13 20:34:09 UTC
Can you update the MOTD file with the information requested in the
rsync mirror guide? (http://www.gentoo.org/doc/en/rsync.xml)
Comment 2 Peter Penkala gentoo-dev 2003-10-17 21:01:29 UTC
I've added 134.153.48.2 to the access list for rsync1.us.gentoo.org.
You should be able to sync with rsync1.us in about 30 minutes.  
Please update your rsync cron jobs to point to rsync1.us.gentoo.org
and post back here when you've done so.  We'll monitor the server 
for a couple days to make sure things are working fine.

Thanks.
Comment 3 Dwayne J. Hart 2003-10-20 11:07:25 UTC
I have updated the cron jobs to point to rsync1.us.gentoo.org server as per
your request. 
Comment 4 Peter Penkala gentoo-dev 2003-10-20 21:59:02 UTC
Monitoring the server now.
Comment 5 Peter Penkala gentoo-dev 2003-10-22 06:18:29 UTC
Are you seeing any problems on your end?  I've noticed several 
occurances where the sync didn't finish (no timestamp) and when
the timestamp was old.  It appears that the server is taking a
very long time to finish syncing with rsync1.us.

Checked (UTC)   Timestamp                         Expected
==========================================================
     15:20:14   Tue, 21 Oct 2003 14:20:01 +0000   14:50
     20:50:14   Tue, 21 Oct 2003 20:20:01 +0000   OK
     21:23:20   timestamp missing                 20:50
     21:53:21   timestamp missing                 21:20
     22:23:21   timestamp missing                 21:50
     22:53:21   timestamp missing                 22:20
     23:20:17   Tue, 21 Oct 2003 22:24:00 +0000   22:50
     02:56:26   Wed, 22 Oct 2003 02:20:00 +0000   OK
     03:23:19   timestamp missing                 02:50
     03:53:20   timestamp missing                 03:20
     04:50:14   Wed, 22 Oct 2003 03:50:01 +0000   04:20
     05:20:11   Wed, 22 Oct 2003 03:50:01 +0000   04:50
     06:20:11   Wed, 22 Oct 2003 05:20:01 +0000   05:50
Comment 6 Peter Penkala gentoo-dev 2003-10-23 06:16:53 UTC
Another chunk of syncs that didn't seem to finish. Time in ()
was expected.  Let me know if you see anything in logs, etc.

Checked (UTC)   Timestamp
===============================================
     05:20:05   Thu, 23 Oct 2003 04:20:00 +0000 (04:50)
     05:50:05   Thu, 23 Oct 2003 04:20:00 +0000 (05:20)
     06:20:05   Thu, 23 Oct 2003 04:20:00 +0000 (05:50)
     06:50:06   Thu, 23 Oct 2003 04:20:00 +0000 (06:20)
     07:20:05   Thu, 23 Oct 2003 04:20:00 +0000 (06:50)
Comment 7 Peter Penkala gentoo-dev 2003-10-29 18:19:20 UTC
This server is still having alot of problems with syncing.
Are you noticing any errors on your end?
Comment 8 Dwayne J. Hart 2003-10-30 06:02:12 UTC
The times which you are checking for the time stamp. My server is being pounded
by anonymous ftp requests. In order to combat this problem I have modified
the proftp.conf file to allow one session per user. Hopefully this will allow
my 01:30 and 02:00 (NST) sync to succeed.
Comment 9 Peter Penkala gentoo-dev 2003-11-01 23:55:17 UTC
Still seeing alot of misses.

Checked (UTC)   Timestamp
============================================================
     05:50:07   Sat, 01 Nov 2003 04:50:01 +0000 (05:20)
     05:20:07   Sun, 02 Nov 2003 04:20:00 +0000 (04:50)
     05:50:06   Sun, 02 Nov 2003 04:20:00 +0000 (05:20)
     06:50:06   Sun, 02 Nov 2003 05:51:00 +0000 (06:20)
     07:50:08   Sun, 02 Nov 2003 06:50:01 +0000 (07:20)
Comment 10 Peter Penkala gentoo-dev 2003-11-02 22:24:52 UTC
Still getting hammered?
Comment 11 Peter Penkala gentoo-dev 2003-11-23 18:56:32 UTC
Thanks for your interest in supporting Gentoo.
Comment 12 Dwayne J. Hart 2003-11-24 07:06:08 UTC
------- Additional Comment #10 From Peter Penkala 2003-11-02 22:24 PST ------- 
Still getting hammered?

I am looking at implementing two network cards in the server. The first card will listen to outside connections and the other will listen to the internal network. Hopefully this will help alleviate some of the network problems that we were encountering.

I have a side question... is it better for the server to be a low powered dual processor or a high powered single processor?
Comment 13 Peter Penkala gentoo-dev 2003-11-24 17:10:36 UTC
> I am looking at implementing two network cards in the server.
OK, I'll wait until you have more info.  Please keep us updated (any
long delays, etc.)

> is it better for the server to be a low powered dual processor or a 
> high powered single processor?
I don't know of any benchmark test, but both should do OK (depending
on how "low powered" the dual is).  Most of the load comes from disk
activity and network bandwidth.  If the minimum is met (PIII 500 / 256MB
RAM), it shouldn't be much of a problem.  If the server is running other
tasks, then that will also have an impact.
Comment 14 Dwayne J. Hart 2003-11-25 14:54:58 UTC
Hi, the two network card solution has been implemented. The xinetd configuration 
file for proftp has been configured so that it will listen to ftp requets on the new 
network card. Hopefully, this will help to alleviate some of the network traffic bottle
neck.

Also, as of this morning I took the opportunity to increase the amount of physical 
ram in the server from 256Mb to 1Gb.

I look forward to hearing from you again.
Comment 15 Dwayne J. Hart 2003-11-25 16:55:54 UTC
Hi could you please add access for mirro.cs.mun.ca (134.153.48.27). It seems as
though my rsync traffic goes through both nics when it is trying to download from your
server. Thanks.
Comment 16 Dwayne J. Hart 2003-11-26 12:05:22 UTC
Started update at Wed Nov 26 14:00:00 NST 2003
@ERROR: access denied to gentoo-portage from mirro.cs.mun.ca (134.153.48.27)
rsync: connection unexpectedly closed (202 bytes read so far)
rsync error: error in rsync protocol data stream (code 12) at io.c(165)
End: Wed Nov 26 14:00:00 NST 2003
Started update at Wed Nov 26 14:30:00 NST 2003
@ERROR: access denied to gentoo-portage from mirror.cs.mun.ca (134.153.48.2)
rsync: connection unexpectedly closed (202 bytes read so far)
rsync error: error in rsync protocol data stream (code 12) at io.c(165)
End: Wed Nov 26 14:30:00 NST 2003


Can you please tell me why my server (mirror.cs.mun.ca/mirro.cs.mun.ca) is being
denied access.
Comment 17 Peter Penkala gentoo-dev 2003-12-02 07:06:54 UTC
Sorry, looks like I forgot to add it back in.  Should be able to sync
again within 30 minutes.
Comment 18 Dwayne J. Hart 2003-12-02 11:01:17 UTC
mirrror.cs.mun.ca is now syncing. Could you add mirro.cs.mun.ca (134.153.48.27) to the access list. It appears as though when I sync from your server the program makes us of one or the other network card. Thanks.
Comment 19 Peter Penkala gentoo-dev 2003-12-05 12:22:23 UTC
> It appears as though when I sync from your server the program 
> makes us of one or the other network card.
Interesting.  One card can't be made as the default?
Comment 20 Dwayne J. Hart 2003-12-12 06:38:45 UTC
hi, we have made some modifications to the scripts which keep the software on the
mirror server up to date. could you please start to audit our gentoo tree. thanks.
Comment 21 Peter Penkala gentoo-dev 2003-12-12 09:48:40 UTC
Watching again.  Will let you know how it goes.
Comment 22 Peter Penkala gentoo-dev 2003-12-15 08:51:33 UTC
The first timestamp is when syncing appeared to be working OK.
It went for a while without misses, but they start up again on
Sunday.  Do the logs indicate any connection problems around 
the misses?

Checked (UTC)   Timestamp
============================================================
     16:20:09   Sat, 13 Dec 2003 15:50:00 +0000 OK
     ...
     04:50:09   Sun, 14 Dec 2003 03:50:01 +0000 (04:20)
     16:50:09   Sun, 14 Dec 2003 15:50:01 +0000 (16:20)
     05:20:09   Mon, 15 Dec 2003 04:20:01 +0000 (04:50)
     07:50:08   Mon, 15 Dec 2003 06:50:01 +0000 (07:20)
     09:20:08   Mon, 15 Dec 2003 08:21:01 +0000 (08:50)
     14:50:09   Mon, 15 Dec 2003 13:50:00 +0000 (14:20)
     15:20:09   Mon, 15 Dec 2003 13:50:00 +0000 (14:50)
     15:50:08   Mon, 15 Dec 2003 13:50:00 +0000 (15:20)
Comment 23 Dwayne J. Hart 2003-12-15 10:32:29 UTC
Hi, the logs reveal a time out problem.

Started update at Sun Dec 14 13:00:00 NST 2003
io timeout after 600 seconds - exiting
rsync error: timeout in data send/receive (code 30) at io.c(103)
End: Sun Dec 14 13:12:56 NST 2003

Started update at Mon Dec 15 01:30:00 NST 2003
io timeout after 600 seconds - exiting
rsync error: timeout in data send/receive (code 30) at io.c(103)
End: Mon Dec 15 01:44:58 NST 2003

Started update at Mon Dec 15 04:00:00 NST 2003
io timeout after 600 seconds - exiting
rsync error: timeout in data send/receive (code 30) at io.c(103)
End: Mon Dec 15 04:12:55 NST 2003

Started update at Mon Dec 15 11:00:00 NST 2003
io timeout after 600 seconds - exiting
rsync error: timeout in data send/receive (code 30) at io.c(103)
End: Mon Dec 15 11:12:53 NST 2003
Started update at Mon Dec 15 11:30:00 NST 2003
io timeout after 600 seconds - exiting
rsync error: timeout in data send/receive (code 30) at io.c(103)
rsync: connection unexpectedly closed (1624249 bytes read so far)
rsync error: error in rsync protocol data stream (code 12) at io.c(165)
End: Mon Dec 15 11:53:56 NST 2003
Started update at Mon Dec 15 12:00:00 NST 2003
io timeout after 600 seconds - exiting
rsync error: timeout in data send/receive (code 30) at io.c(103)
End: Mon Dec 15 12:11:52 NST 2003
Comment 24 Dwayne J. Hart 2003-12-16 07:47:15 UTC
Hi, I just wanted to let you know that everything has been syncing fine since for the past 12 hours.
Comment 25 Peter Penkala gentoo-dev 2003-12-30 10:58:19 UTC
Looks like there have been some errors, but I can't be certain, so I'll
add this mirror and see if there are any out of sync reports.

This server is now rsync3.ca.gentoo.org.  Please update the motd file
to include that information.

Also, I've subscribed <dwayne@cs.mun.ca> to our gentoo-mirrors mailing 
list (low volume).  This list is used to discuss mirror-related issues.

Thanks for supporting Gentoo.
Comment 26 Dwayne J. Hart 2003-12-30 17:13:40 UTC
Hi,

Just to let you know that I did find the following information in the rsync log files when mirror.cs.mun.ca was trying to sync with the gentoo master server.

Started update at Sat Dec 27 02:00:00 NST 2003
rsync: getaddrinfo: rsync1.us.gentoo.org 873: Temporary failure in name resoluti
on
rsync error: error in socket IO (code 10) at clientserver.c(83)
End: Sat Dec 27 02:00:56 NST 2003
Started update at Sat Dec 27 02:30:00 NST 2003
rsync: getaddrinfo: rsync1.us.gentoo.org 873: Temporary failure in name resoluti
on
rsync error: error in socket IO (code 10) at clientserver.c(83)
End: Sat Dec 27 02:30:56 NST 2003
Started update at Sat Dec 27 03:00:00 NST 2003
rsync: getaddrinfo: rsync1.us.gentoo.org 873: Temporary failure in name resoluti
on
rsync error: error in socket IO (code 10) at clientserver.c(83)
End: Sat Dec 27 03:00:57 NST 2003
Started update at Sat Dec 27 03:30:00 NST 2003
rsync: getaddrinfo: rsync1.us.gentoo.org 873: Temporary failure in name resoluti
on
rsync error: error in socket IO (code 10) at clientserver.c(83)
End: Sat Dec 27 03:30:56 NST 2003

Then everything cleared up after this incident and then I experienced the normal timeout on the rsync this afternoon.

Started update at Tue Dec 30 17:00:00 NST 2003
io timeout after 600 seconds - exiting
rsync error: timeout in data send/receive (code 30) at io.c(103)
End: Tue Dec 30 17:13:48 NST 2003

The motd on mirror has been updated as per your request.