Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 234083 - [rsync;distfiles|all] New mirror Cambrium BV (The Netherlands)
Summary: [rsync;distfiles|all] New mirror Cambrium BV (The Netherlands)
Status: RESOLVED FIXED
Alias: None
Product: Mirrors
Classification: Unclassified
Component: New Server (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Mirror Admins
URL: mirror.cambrium.nl
Whiteboard: due:2008/09/01;dist:http://mirror.cam...
Keywords:
Depends on:
Blocks:
 
Reported: 2008-08-06 13:01 UTC by Mark van Herpen
Modified: 2008-09-01 03:49 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mark van Herpen 2008-08-06 13:01:32 UTC
Hi,

I'm a sysadmin at Cambrium BV in the Netherlands. We are a company that provides DSL access and hosting to customers of different ISP's. 

I've set up a full gentoo-source and gentoo-portage mirror for ftp, http and rsync. Can it be added to the list of official mirrors? The server can be found at 

http://mirror.cambrium.nl/pub/os/linux/gentoo/
http://mirror.cambrium.nl/pub/os/linux/gentoo-portage/

ftp://mirror.cambrium.nl/pub/os/linux/gentoo/
ftp://mirror.cambrium.nl/pub/os/linux/gentoo-portage/

rsync://mirror.cambrium.nl/gentoo
rsync://mirror.cambrium.nl/gentoo-portage

The mirror can be accessed via ipv4 (217.19.16.188) and ipv6 (2a02:58:3:2:80::1)

Grtz,

Mark van Herpen
Cambrium BV
mirrors at cambrium.nl


Reproducible: Always
Comment 1 Shyam Mani (RETIRED) gentoo-dev 2008-08-06 16:46:08 UTC
Mark,

Thanks for wanting to help out gentoo with a mirror.

I've added your mirror to our test system and we'll keep you posted on the results.

In the meanwhile, can you please let us know the hardware specs of the machine, bandwidth and how many concurrent users you plan to support?

Thanks!
Comment 2 Mark van Herpen 2008-08-07 09:10:16 UTC
Shyam,

Cambrium is connected directly on the Amsterdam Internet Exchange (AMS-IX) with 1 20Gig link and 1 10G link. We have a total of 10 Gig transit connected to our transit partners.
The mirror server is connected with a 1G link to our core routers.

The server has got a 3.0 Gig Xeon processor with 8 gig of ram.

The configuration for http,ftp and rsync is that there can be max 250 connections to each service. There has never been a 'max connection limit reached' on the services, but if it's necessary we can change this. 

Grtz,

Mark van Herpen
Cambrium BV
mirrors at cambrium.nl
Comment 3 Shyam Mani (RETIRED) gentoo-dev 2008-08-18 03:53:30 UTC
Hi Mark,

The http distfiles mirror seems okay :

mirror.cambrium.nl/pub/os/linux/gentoo/ is in sync, current delta : 5.18 hours
Timestamp on Mirror   : Sat, 16 Aug 2008 22:00:02 +0000
Current time in UTC   : Sun, 17 Aug 2008 03:11:05 +0000

The ftp mirror has an issue, we can't even connect to it :

*ERROR* : return value of wget during mirrorcheck was 1
*ERROR* : return value of wget for release timestamp check was 1

rsync is okay as well (for distfiles) but gentoo-portage has a lag :

*ERROR* : gentoo-portage mirror is lagging by over 40 mins, lag is : 56.17 mins
Timestamp on Mirror   : Sun, 17 Aug 2008 02:15:01 +0000
Current time in UTC   : Sun, 17 Aug 2008 03:11:11 +0000

Can you please look into ftp and sync gentoo-portage from rsync1.de.gentoo.org? 

Thanks
Comment 4 Mark van Herpen 2008-08-18 10:59:36 UTC
Hi Shyam,

I've fixed the FTP issue and added this service to our monitoring system.
I've also changed the rsync host, this was rsync.nl.gentoo.org and is now rsync1.de.gentoo.org. 

Can you see if these changes have fixed the problem?

Grtz,

Mark
Comment 5 Shyam Mani (RETIRED) gentoo-dev 2008-08-25 07:04:29 UTC
(In reply to comment #4)
> Hi Shyam,
> 
> I've fixed the FTP issue and added this service to our monitoring system.
> I've also changed the rsync host, this was rsync.nl.gentoo.org and is now
> rsync1.de.gentoo.org. 
> 
> Can you see if these changes have fixed the problem?

Distfiles seems to have an issue, you'll need to sync once every 4 hours as mentioned at http://www.gentoo.org/doc/en/source_mirrors.xml

Right now, there's a lag :

*ERROR* : distfiles is over 5.5 hours behind, lag is : 6.01 hours
Timestamp on Mirror   : Sun, 24 Aug 2008 21:10:06 +0000
Current time in UTC   : Mon, 25 Aug 2008 03:10:44 +0000

rsync (gentoo-portage) seems to be fine now :

mirror.cambrium.nl is in sync, current delta : 24.6 mins
Timestamp on Mirror   : Mon, 25 Aug 2008 02:46:02 +0000
Current time in UTC   : Mon, 25 Aug 2008 03:10:38 +0000
Comment 6 Mark van Herpen 2008-08-25 14:18:59 UTC
Hi Shyam,

Rsyncing the distfiles from rsync1.de.gentoo.org/gentoo doesn't work, only the the gentoo-portage rsync works from this mirror. 

I've scheduled the cronjob 5 minutes later every 4 hour, and sync from a Dutch mirror. Now it should work. 

But this is just a temporary fix I guess, because I see in the rsync-gentoo.sh script that we can sync from the main server after the mirror is added and we've recieved a username and password?

Grtz,

Mark
Comment 7 Shyam Mani (RETIRED) gentoo-dev 2008-08-26 11:25:03 UTC
(In reply to comment #6)
> Hi Shyam,
 
> Rsyncing the distfiles from rsync1.de.gentoo.org/gentoo doesn't work, only the
> the gentoo-portage rsync works from this mirror. 

As mentioned in the doc in comment #5, you should be syncing from ftp.ussg.iu.edu::gentoo-distfiles

> I've scheduled the cronjob 5 minutes later every 4 hour, and sync from a Dutch
> mirror. Now it should work. 

It looks okay now.

mirror.cambrium.nl/pub/os/linux/gentoo/ is in sync, current delta : 3.17 hours
Timestamp on Mirror   : Tue, 26 Aug 2008 00:00:03 +0000
Current time in UTC   : Tue, 26 Aug 2008 03:10:13 +0000
 
> But this is just a temporary fix I guess, because I see in the rsync-gentoo.sh
> script that we can sync from the main server after the mirror is added and
> we've recieved a username and password?

Yes, that is correct. I'll let the checks run for this week and add you in.
Comment 8 Shyam Mani (RETIRED) gentoo-dev 2008-09-01 03:49:34 UTC
All good, adding you in!

Thanks for helping out Gentoo. I have added your mirror to our list at
http://www.gentoo.org/main/en/mirrors.xml. Please start syncing from
masterdistfiles.gentoo.org as the email I have sent to you instructs. We would
like you to subscribe to the gentoo-mirrors Mailing List by sending an email to
gentoo-mirrors-subscribe@lists.gentoo.org, as we send out notifications of
updates, releases, and changes in policy to the mirrors.  
Please also monitor http://mirrorstats.gentoo.org as this application is used
to keep track of the up-to-date nature of our mirrors.

I have added the IP you specified in the bug (217.19.16.188) to
our access database, please start syncing to rsync1.us.gentoo.org per our
Rsync document at http://www.gentoo.org/doc/en/rsync.xml.

Please also monitor http://mirrorstats.gentoo.org/rsync as this application is
used to keep track of the up-to-date nature of our mirrors

For Infrastructure,

Shyam Mani

You are mirror rsync1.nl.gentoo.org in the rsync.nl.gentoo.org rotation. Additionally, you've also been added to the rsync.europe.gentoo.org rotation.