Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 139005 - IPv6 (only) rsync rotation
Summary: IPv6 (only) rsync rotation
Status: RESOLVED FIXED
Alias: None
Product: Mirrors
Classification: Unclassified
Component: Feature Request (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Mirror Admins
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 150196 212095 215924 236556
  Show dependency tree
 
Reported: 2006-07-03 07:40 UTC by Martin
Modified: 2010-07-28 15:38 UTC (History)
11 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Martin 2006-07-03 07:40:12 UTC
I think it would be a good idea to add an additional rsync rotation, which only consists of IPv6 capable rsync mirrors.

At the moment the only way to definately get an IPv6-connection seems to be the manual choice of a specific mirror, which goes against the concept of using the rotation.
Comment 1 Bernhard Schmidt 2006-07-19 05:16:55 UTC
How are you currently syncing against an IPv6 mirror? The code to allow IPv6 name resolution has just been added to portage (Bug #37124). This patch tries IPv6 first on IPv6-enabled boxes until manually overridden.

I would therefor propose adding AAAA records to the normal rotations. Especially to allow people that "just have" IPv6 to sync with it without manual intervention.

There are at least two mirrors for the Europe rotation (ftp.join.uni-muenster.de and Bug #138763), prolly some more if just asked.
Comment 2 Lorenz Kiefner 2006-07-26 03:51:34 UTC
Keep in mind that some users have an ill-configured network with ipv6 enabled and configured, but without ipv6-connectivity. They would connect to a ipv6-rsync in first place, wait for a timeout and move on to a ipv4-host. It is current best practise to have three rotations: one with *only ipv6*, one with *only ipv4* and one with both. These rotations should be named rsync.region.ipv4.gentoo.org, rsync.region.ipv6.gentoo.org and rsync.region.gentoo.org - so everyone can choose his favorite.
Comment 3 Bernhard Schmidt 2006-07-26 04:38:03 UTC
I think the possibility to force IPv4-only or IPv6-only operation with the use of  PORTAGE_RSYNC_OPTS and PORTAGE_RSYNC_EXTRA_OPTS should be enough. After all, if the user has accidentally configured IPv6 but no connectivity (I don't think that happens too often) he will observe those issues with other applications as well.
Comment 4 Lorenz Kiefner 2006-07-26 04:58:02 UTC
Yes, you are right - there are enough options to force ipv4 or ipv6 on application level.

But to see how often users are upset about "slow connections" just ask google about "firefox ipv6" - you get about 1,5 million results and most of them advice you to turn off ipv6 in firefox to speed up their connection and avoid ipv6 timeouts. Really not funny.

I would be happy to see some AAAA records in rsync.region.gentoo.org, but I think there should be an easy way to choose between ipv4 and ipv6 (or mixed mode).

The user-friendliest way is surely this rsync.region{.ipv[46]}.gentoo.org-thing, but if it is too much work I would be happy with only one rotation and the application level switch.
Comment 5 Bernhard Schmidt 2006-07-30 09:35:57 UTC
Maybe this is exactly the situation where we do want IPv6 enabled by default, but print an info message when we sync to an IPv6 mirror? Something like

*** You are trying to sync to an IPv6 rsync mirror now. If you experience problems please have a look at <website>

or maybe set the timeout pretty low and print a similar message after an IPv6 mirror has failed?

emerge is typically ran manually and this way people with broken installations would get a better pointer to the problem than the usual invisible Firefox problems.
Comment 6 Shyam Mani (RETIRED) gentoo-dev 2006-09-05 12:24:34 UTC
Guys,

We now have an rsync.ipv6.gentoo.org to which I'll be adding the IPv6 mirrors shortly (in a day or two or over the weekend). Since we don't have many ipv6 mirrors, I think just rsync.ipv6.gentoo.org seems enough in addition to the mirror being listed as an AAAA under the respective country rotation.

Thoughts?
Comment 7 Bernhard Schmidt 2006-09-10 12:40:33 UTC
Hi,

> We now have an rsync.ipv6.gentoo.org to which I'll be adding the IPv6 mirrors
> shortly (in a day or two or over the weekend). Since we don't have many ipv6
> mirrors, I think just rsync.ipv6.gentoo.org seems enough in addition to the
> mirror being listed as an AAAA under the respective country rotation.

I'd rather see one rotation per continent. At the moment rsync.ipv6.gentoo.org only lists one mirror, which is HE.net (world's worst tunnelbroker) in .us. Most IPv6 users are probably in Asia and Europe.

Based on replies on gentoo-mirrors and others we have:

Europe: ftp.snt.utwente.nl (rsync2.nl), ftp.join.uni-muenster.de (rsync??.de)
Asia: ftp.twaren.net (rsync6.tw)
America: the existing one and someone from bigfiber.net

I guess there will come more once we start, but it does not make sense for European users to use American or even Asian mirrors.

I'd suggest something like rsync.ipv6.europe.gentoo.org or ipv6.rsync.europe.gentoo.org. 

Regards,
Bernhard
Comment 8 Bernhard Schmidt 2006-09-26 16:27:45 UTC
Any news on that issue?

Another naming idea would be rsync6.((continent|cctld).)?gentoo.org, that is an approach often used in existing IPv6 deployments.

I've meanwhile had a look and found a few more possibly IPv6-enabled boxes in the rotations (getting reverse name to a listed IP and try to get an AAAA record for it)

130.230.54.100    trumpetti.atm.tut.fi     2001:708:310:54::2 (works)
193.190.198.20    niue.belnet.be           2001:6a8:3c80:0:203:baff:fe39:f931 (no rsync on IPv6, should be an easy fix)
203.178.137.175   ftp.nara.wide.ad.jp      2001:200:0:1::800:21 (works)


The first one is already listed in the official rsync.fi.gentoo.org rotation!

berni@obelix:~$ host rsync.fi.gentoo.org
rsync.fi.gentoo.org has address 130.230.54.100
rsync.fi.gentoo.org has IPv6 address 2001:708:310:54::101

The same issue unfortunately applies for the rsync.us rotation

rsync.us.gentoo.org has IPv6 address 2001:470:1f00:264::1003

this one is down for a long time now. It is most probably meant to be gentoo.llarian.net (209.221.142.124, 2001:470:1f01:164::4), but since this one is connected to HE.net (again: worlds worst tunnelbroker) it will cause problems. I'd advise pulling this AAAA record from DNS ASAP.
Comment 9 Bernhard Schmidt 2006-10-23 16:47:07 UTC
ping?

There is a new mirror listed in Bug #150196 as well

Regards,
Bernhard
Comment 10 Dick Marinus 2007-06-07 16:42:13 UTC
Please also add these mirrors to rsync.ipv6.gentoo.org

rsync://mercant.join.uni-muenster.de/gentoo-portage/
rsync://vlaai.snt.utwente.nl/gentoo-portage/

It seems like 2001:470:1f00:264::1003 (rsync.ipv6.gentoo.org) is down at this moment...
Comment 11 Alex Smith 2007-11-27 14:00:58 UTC
Ok, well I got fed up of waiting for Gentoo ppl to do this so I have an "Unoffical" one.

I don't like having just one IPv6 host in my SYNC hence the reason for doing this.

SYNC="rsync://gentoo-ipv6.alex-smith.org/gentoo-portage"

Will work, although there is only 4 mirrors listed at the moment it's an improvement on what Gentoo's mirror team's done...

gentoo-ipv6.alex-smith.org has IPv6 address 2001:708:310:54::101
gentoo-ipv6.alex-smith.org has IPv6 address 2001:200:0:1::800:21
gentoo-ipv6.alex-smith.org has IPv6 address 2001:638:500:101::21
gentoo-ipv6.alex-smith.org has IPv6 address 2001:708:310:54::2

If there's any breakages with the ones in that list (Out of dates, connection refused) or you have another that works that you want added feel free to drop me an email:

asmith AT asmhosting DOT com

(p.s. gentoo mirror team, im not trying to tread on your feet here, just it's been a rather long time!)
Comment 12 Stephan Kupfer 2008-04-04 00:21:50 UTC
Whats up with this here?

Are there any informations about a ipv6 only (rsync) rotation jet?
Comment 13 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2008-04-04 00:47:00 UTC
Don't get your knickers in a twist, we haven't forgotten about you.
The new rsync1.us (which is actually rsync0.uk) finally has (native) IPv6 support [albatross.ipv6.gentoo.org], one of the first Gentoo infrastructure boxes to do so. So the mirrormon is getting IPv6 foo, then we can start populating rsync.ipv6.gentoo.org with more mirrors than only the broken one it has now.
Comment 14 Stephan Kupfer 2008-06-21 19:55:52 UTC
Alex, whats with your Ipv6 rotation?

If someone is interested i can build a dns round robin like yours... 
Comment 15 Mihai Moldovan 2009-04-28 13:59:26 UTC
Hi,

Seems like this has gone to sleep once again...

I'm bumping this thread again.

Currently, Alex's rotation is gone and rsync.ipv6.gentoo.org still contains only one faulty server.

Best regards,


Mihai Moldovan
Comment 16 Mihai Moldovan 2009-05-09 16:08:13 UTC
Well OK, albatross.ipv6.gentoo.org seems to respond now, but it is denying my connection attempts, seems like this isn't too public yet?
Comment 17 Bernhard Schmidt 2009-05-09 18:21:06 UTC
According to http://archives.gentoo.org/gentoo-mirrors/msg_7a4887f7f5dca5adccb1846a076c18e4.xml rsync.ipv6.gentoo.org is not supposed to be a public IPv6 rotation, but an IPv6-enabled masterserver for the other mirrors. 

The naming is less than perfect, but still.
Comment 18 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2009-05-09 22:00:06 UTC
I'm NOT offering rsync.ipv6.gentoo.org in beta yet.

Just rsync1.ipv6.gentoo.org is in beta first of all, I'm extremely short on other IPv6 resources for monitoring of IPv6 mirrors.

The problem is that I need some more Gentoo infrastructure boxes with IPv6 so I can offer rsync.ipv6.gentoo.org from them, and thereafter rsync$N.$CC.ipv6.gentoo.org in the same fashion as the IPv4 rotation.
Comment 19 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2009-06-09 08:55:25 UTC
rsync.ipv6.gentoo.org lives! Bugs in here please if you run into any (I had it broken earlier, so take this as the time after which bugs should be reported, nothing prior).
Comment 20 Steven Davies 2009-06-26 21:44:46 UTC
Works well, but it'd be nice to have its v6 address on the Server Address line! :-)

Is the notice(-2) anything to worry about? It's definitely syncing over v6.

root@arthur:~> emerge --sync
Notice: (-2, 'Name or service not known')
>>> Starting rsync with rsync://rsync.ipv6.gentoo.org/gentoo-portage...
>>> Checking server timestamp ...
Welcome to flycatcher.gentoo.org
 
Server Address : 81.93.240.111
Contact Name   : mirror-admin@gentoo.org
Hardware       : 8 x Intel(R) Xeon(R) CPU           E5405  @ 2.00GHz, 16084MB RAM
Comment 21 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2009-06-27 02:22:13 UTC
MOTD updated for rsync{,.ipv6}.gentoo.org boxes updated to include IPv6 address and useful aliases.

Could you trace the notice warning please? I haven't seen it before.
Comment 22 Steven Davies 2009-06-27 13:06:39 UTC
Here's an strace of it (it's about half way down)

socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.0.1")}, 28) = 0
fcntl64(3, F_GETFL)                     = 0x2 (flags O_RDWR)
fcntl64(3, F_SETFL, O_RDWR|O_NONBLOCK)  = 0
gettimeofday({1246106985, 750860}, NULL) = 0
poll([{fd=3, events=POLLOUT}], 1, 0)    = 1 ([{fd=3, revents=POLLOUT}])
send(3, "\361A\1\0\0\1\0\0\0\0\0\0\5rsync\4ipv6\6gentoo\3or"..., 39, MSG_NOSIGNAL) = 39
poll([{fd=3, events=POLLIN}], 1, 5000)  = 1 ([{fd=3, revents=POLLIN}])
ioctl(3, FIONREAD, [132])               = 0
recvfrom(3, "\361A\201\200\0\1\0\1\0\1\0\0\5rsync\4ipv6\6gentoo\3or"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("1
27.0.0.1")}, [16]) = 132
close(3)                                = 0
fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 5), ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb804f000
write(1, "Notice: (-2, 'Name or service not"..., 42) = 42
open("/var/log/emerge.log", O_WRONLY|O_CREAT|O_APPEND|O_LARGEFILE, 0666) = 3
fstat64(3, {st_mode=S_IFREG|0660, st_size=5109838, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb804e000
fstat64(3, {st_mode=S_IFREG|0660, st_size=5109838, ...}) = 0
_llseek(3, 5109838, [5109838], SEEK_SET) = 0
fstat64(3, {st_mode=S_IFREG|0660, st_size=5109838, ...}) = 0
stat64("/var/log/emerge.log", {st_mode=S_IFREG|0660, st_size=5109838, ...}) = 0
fcntl64(3, F_SETLK64, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}, 0xbf87fab4) = 0
fstat64(3, {st_mode=S_IFREG|0660, st_size=5109838, ...}) = 0
_llseek(3, 5109838, [5109838], SEEK_SET) = 0
gettimeofday({1246106985, 754685}, NULL) = 0
write(3, "1246106985: >>> Starting rsync wi"..., 81) = 81
fcntl64(3, F_SETLKW64, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}, 0xbf87fb14) = 0
close(3)                                = 0
munmap(0xb804e000, 4096)                = 0
write(1, ">>> Starting rsync with rsync://r"..., 72) = 72

This seems to correspond to line 12850 in __init__.py (in Portage 2.1.6.13)
Comment 23 Steven Davies 2009-06-27 13:20:44 UTC
Sorry for the bugspam, this seems to have been caused by my /etc/hosts file not containing the address ::1. This was fixed in bug 25859 nearly 6 years ago so I'm not sure why it was missing from my copy.
Comment 24 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2009-06-27 16:45:17 UTC
Thanks for tracing the notice anyway.
Comment 25 Petr Pisar 2009-06-28 20:18:07 UTC
rsync.ipv6.gentoo.org contains only one address. Any will to add the other ones <https://bugs.gentoo.org/buglist.cgi?quicksearch=ipv6+mirror>? I have these in my /etc/hosts too:

2001:708:310:54::2 rsync6.europe.gentoo.org
2001:708:310:54::101 rsync6.europe.gentoo.org
2001:6f8:93c::1 rsync6.europe.gentoo.org
Comment 26 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2009-06-28 23:09:40 UTC
rsync.gentoo.org and rsync.ipv6.gentoo.org are deliberately following the same principle - they contain only servers run by Gentoo infrastructure. There aren't many of those with IPv6 at all.

For community-run IPv6, I think we do the same country subcoding as IPv4, in addition to loading AAAA records into the same base.
Eg:
; For the individual mirrors
rsync3.$CC.gentoo.org IN A 123.123.123.123
rsync3.$CC.gentoo.org IN AAAA 2001::...:A1
; For the country rotations:
rsync.$CC.gentoo.org IN A 123.123.123.123
ipv6.rsync.$CC.gentoo.org IN AAAA 2001::...:A1

mirrormon still isn't IPv6 capable, mostly as that sponsor does not have IPv6 transit presently.
Comment 27 Steven Davies 2009-07-02 16:57:50 UTC
I can't seem to connect to the server any more, with the following error:

rsync: failed to connect to rsync.ipv6.gentoo.org: No route to host (113)
rsync error: error in socket IO (code 10) at clientserver.c(124) [receiver=3.0.5]

Here's a traceroute6 showing some kind of issue, I'm guessing at Gentoo's ISP..

root@bedevere:~> traceroute6 rsync.ipv6.gentoo.org
traceroute to flycatcher-rsync.ipv6.gentoo.org (2001:758:f00:4732::a1) from 2001:470:9095:1:201:4aff:fe05:96ac, 30 hops max, 16 byte packets
 1  2001:470:9095:1::1 (2001:470:9095:1::1)  0.243 ms  0.149 ms  0.131 ms
 2  daviessm.tunnel.tserv5.lon1.ipv6.he.net (2001:470:1f08:243::1)  28.394 ms  26.404 ms  28.84 ms
 3  gige-g4-6.core1.lon1.he.net (2001:470:0:67::1)  25.028 ms  32.446 ms  25.525 ms
 4  10gigabitethernet1-1.core1.par1.he.net (2001:470:0:42::2)  35.398 ms  40.935 ms  28.649 ms
 5  equinox.ipv6.panap.fr (2001:860:0:6:0:3:5393:1)  30.365 ms  29.517 ms  29.81 ms
 6  2001:758:2309:3932:49:1:0:1 (2001:758:2309:3932:49:1:0:1)  90.955 ms  150.307 ms  199.895 ms
 7  2001:758:2309:3932:49:3:0:2 (2001:758:2309:3932:49:3:0:2)  29.815 ms  32.388 ms  31.473 ms
 8  2001:758:2309:3932:49:3:0:2 (2001:758:2309:3932:49:3:0:2)  3026.54 ms !H  3027.5 ms !H  3027.52 ms !H
Comment 28 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2009-07-02 21:43:57 UTC
The server had an unscheduled downtime, and I just picked up a bug in baselayout1 that happened on the reboot.

It is back. If you do get another downtime or other issue, please open a new bug, do NOT continue to comment on this one.
Comment 29 Marcel Pennewiß 2009-12-23 07:55:51 UTC
(In reply to comment #26)
> For community-run IPv6, I think we do the same country subcoding as IPv4, in
> addition to loading AAAA records into the same base.

ACK. Until now it seems not realized. Why?

> mirrormon still isn't IPv6 capable, mostly as that sponsor does not have IPv6
> transit presently.

What is mirrormon? I didn't find any information.
It sounds like a tool to monitor if the mirrors being up-to-date.

Maybe we could find a second sponsor with native IPv6 connectivity if this is the whole problem.
Comment 30 Jeremy Olexa (darkside) (RETIRED) archtester Gentoo Infrastructure gentoo-dev Security 2009-12-23 18:27:09 UTC
The state of ipv6 rotation in Gentoo is poor. It is on the todo list for the upcoming months. mirmon is this: http://mirrorstats.gentoo.org/ but that was mostly killed while I have been setting up the replacement (which has ipv6 capabilities)
Comment 31 Marcel Pennewiß 2009-12-23 20:15:53 UTC
(In reply to comment #30)
> It is on the todo list for the
> upcoming months.

if someone could provide assistance (however) to you so let us know.

> mirmon is this: http://mirrorstats.gentoo.org/ but that was
> mostly killed while I have been setting up the replacement (which has ipv6
> capabilities)

which replacement do you use?

ipv6 will be more and more significant to all users and gentoo should be on the cutting-edge ;)
Comment 32 Antonio Quartulli 2010-07-07 15:39:11 UTC
Any news about this bug? Is there any ipv6 plan for rsync rotation?
Comment 33 Sascha Hlusiak 2010-07-07 17:43:57 UTC
So far at least the official german rsync pool (rsync.de.gentoo.org) does anready contain an ipv6 entry (which is mirror.opteamax.de):

rsync.de.gentoo.org has IPv6 address 2001:67c:1420::1:50

Nothing more needed than putting PORTAGE_RSYNC_EXTRA_OPTS="-6" in your make.conf.

The name could contain more servers though, that are v6 capable, like (taken from mirrorstats.gentoo.org, untested): mirror.netcologne.de, ftp.join.uni-muenster.de
Comment 34 Petr Pisar 2010-07-07 17:52:50 UTC
Read this thread <http://thread.gmane.org/gmane.linux.gentoo.mirrors/35>. You could see there is "a lot" of IPv6 capable mirrors.
Comment 35 Jeremy Olexa (darkside) (RETIRED) archtester Gentoo Infrastructure gentoo-dev Security 2010-07-07 17:58:24 UTC
Yup, at this time there are no longer any plans to create ipv6 *only* rotations.
Comment 36 Jeremy Olexa (darkside) (RETIRED) archtester Gentoo Infrastructure gentoo-dev Security 2010-07-28 14:42:25 UTC
All, rsync.$CC.gentoo.org rotations have AAAA records where appropriate. Moving forward, we are adding A & AAAA records for new mirrors. I'm going to close this bug as fixed.
Comment 37 Petr Pisar 2010-07-28 14:56:30 UTC
Please do not forget on region-wide mirrors like rsync.europe.gentoo.org. All of them are A only right now.
Comment 38 Jeremy Olexa (darkside) (RETIRED) archtester Gentoo Infrastructure gentoo-dev Security 2010-07-28 15:03:59 UTC
(In reply to comment #37)
> Please do not forget on region-wide mirrors like rsync.europe.gentoo.org. All
> of them are A only right now.
> 

Yup, we can't add anymore to rsync.europe or it will go over the 512byte UDP packet size. (and cause issues like bug 324121)

We either need to implement GeoIP or some trickery to "load balance" the rotation (probably via a unbound DNS resolver plugin/extension)
Comment 39 Petr Pisar 2010-07-28 15:27:28 UTC
(In reply to comment #38)
> 
> Yup, we can't add anymore to rsync.europe or it will go over the 512byte UDP
> packet size. (and cause issues like bug 324121)
> 
I know that infrastructure guys are conservative. But this is ultraconservative. Have you heard about DNSSEC that requires long UDP packets and that has been deployed to root server already? I guess the 512B limit is over.
Comment 40 Jeremy Olexa (darkside) (RETIRED) archtester Gentoo Infrastructure gentoo-dev Security 2010-07-28 15:38:09 UTC
(In reply to comment #39)
> (In reply to comment #38)
> > 
> > Yup, we can't add anymore to rsync.europe or it will go over the 512byte UDP
> > packet size. (and cause issues like bug 324121)
> > 
> I know that infrastructure guys are conservative. But this is
> ultraconservative. Have you heard about DNSSEC that requires long UDP packets
> and that has been deployed to root server already? I guess the 512B limit is
> over.
> 

Yes, but it affects our users still. That is a bad thing so I guess the limit is still in the wild. Anyway, you don't need to convince me or the other Gentoo infra team members, we get it. Let's stop commenting on this bug now as the case is closed for the sake of being less noisy. :)