| Summary: | IPv6 (only) rsync rotation | ||
|---|---|---|---|
| Product: | Mirrors | Reporter: | Martin <smallone> |
| Component: | Feature Request | Assignee: | Mirror Admins <mirror-admin> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | a, asmith, berni, bertrand, bugs, candrews, gentoo-bugzilla2, gentoo.org, gentoo, ionic, petr.pisar |
| Priority: | High | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | |||
| Bug Blocks: | 150196, 212095, 215924, 236556 | ||
|
Description
Martin
2006-07-03 07:40:12 UTC
How are you currently syncing against an IPv6 mirror? The code to allow IPv6 name resolution has just been added to portage (Bug #37124). This patch tries IPv6 first on IPv6-enabled boxes until manually overridden. I would therefor propose adding AAAA records to the normal rotations. Especially to allow people that "just have" IPv6 to sync with it without manual intervention. There are at least two mirrors for the Europe rotation (ftp.join.uni-muenster.de and Bug #138763), prolly some more if just asked. Keep in mind that some users have an ill-configured network with ipv6 enabled and configured, but without ipv6-connectivity. They would connect to a ipv6-rsync in first place, wait for a timeout and move on to a ipv4-host. It is current best practise to have three rotations: one with *only ipv6*, one with *only ipv4* and one with both. These rotations should be named rsync.region.ipv4.gentoo.org, rsync.region.ipv6.gentoo.org and rsync.region.gentoo.org - so everyone can choose his favorite. I think the possibility to force IPv4-only or IPv6-only operation with the use of PORTAGE_RSYNC_OPTS and PORTAGE_RSYNC_EXTRA_OPTS should be enough. After all, if the user has accidentally configured IPv6 but no connectivity (I don't think that happens too often) he will observe those issues with other applications as well. Yes, you are right - there are enough options to force ipv4 or ipv6 on application level.
But to see how often users are upset about "slow connections" just ask google about "firefox ipv6" - you get about 1,5 million results and most of them advice you to turn off ipv6 in firefox to speed up their connection and avoid ipv6 timeouts. Really not funny.
I would be happy to see some AAAA records in rsync.region.gentoo.org, but I think there should be an easy way to choose between ipv4 and ipv6 (or mixed mode).
The user-friendliest way is surely this rsync.region{.ipv[46]}.gentoo.org-thing, but if it is too much work I would be happy with only one rotation and the application level switch.
Maybe this is exactly the situation where we do want IPv6 enabled by default, but print an info message when we sync to an IPv6 mirror? Something like *** You are trying to sync to an IPv6 rsync mirror now. If you experience problems please have a look at <website> or maybe set the timeout pretty low and print a similar message after an IPv6 mirror has failed? emerge is typically ran manually and this way people with broken installations would get a better pointer to the problem than the usual invisible Firefox problems. Guys, We now have an rsync.ipv6.gentoo.org to which I'll be adding the IPv6 mirrors shortly (in a day or two or over the weekend). Since we don't have many ipv6 mirrors, I think just rsync.ipv6.gentoo.org seems enough in addition to the mirror being listed as an AAAA under the respective country rotation. Thoughts? Hi,
> We now have an rsync.ipv6.gentoo.org to which I'll be adding the IPv6 mirrors
> shortly (in a day or two or over the weekend). Since we don't have many ipv6
> mirrors, I think just rsync.ipv6.gentoo.org seems enough in addition to the
> mirror being listed as an AAAA under the respective country rotation.
I'd rather see one rotation per continent. At the moment rsync.ipv6.gentoo.org only lists one mirror, which is HE.net (world's worst tunnelbroker) in .us. Most IPv6 users are probably in Asia and Europe.
Based on replies on gentoo-mirrors and others we have:
Europe: ftp.snt.utwente.nl (rsync2.nl), ftp.join.uni-muenster.de (rsync??.de)
Asia: ftp.twaren.net (rsync6.tw)
America: the existing one and someone from bigfiber.net
I guess there will come more once we start, but it does not make sense for European users to use American or even Asian mirrors.
I'd suggest something like rsync.ipv6.europe.gentoo.org or ipv6.rsync.europe.gentoo.org.
Regards,
Bernhard
Any news on that issue? Another naming idea would be rsync6.((continent|cctld).)?gentoo.org, that is an approach often used in existing IPv6 deployments. I've meanwhile had a look and found a few more possibly IPv6-enabled boxes in the rotations (getting reverse name to a listed IP and try to get an AAAA record for it) 130.230.54.100 trumpetti.atm.tut.fi 2001:708:310:54::2 (works) 193.190.198.20 niue.belnet.be 2001:6a8:3c80:0:203:baff:fe39:f931 (no rsync on IPv6, should be an easy fix) 203.178.137.175 ftp.nara.wide.ad.jp 2001:200:0:1::800:21 (works) The first one is already listed in the official rsync.fi.gentoo.org rotation! berni@obelix:~$ host rsync.fi.gentoo.org rsync.fi.gentoo.org has address 130.230.54.100 rsync.fi.gentoo.org has IPv6 address 2001:708:310:54::101 The same issue unfortunately applies for the rsync.us rotation rsync.us.gentoo.org has IPv6 address 2001:470:1f00:264::1003 this one is down for a long time now. It is most probably meant to be gentoo.llarian.net (209.221.142.124, 2001:470:1f01:164::4), but since this one is connected to HE.net (again: worlds worst tunnelbroker) it will cause problems. I'd advise pulling this AAAA record from DNS ASAP. ping? There is a new mirror listed in Bug #150196 as well Regards, Bernhard Please also add these mirrors to rsync.ipv6.gentoo.org rsync://mercant.join.uni-muenster.de/gentoo-portage/ rsync://vlaai.snt.utwente.nl/gentoo-portage/ It seems like 2001:470:1f00:264::1003 (rsync.ipv6.gentoo.org) is down at this moment... Ok, well I got fed up of waiting for Gentoo ppl to do this so I have an "Unoffical" one. I don't like having just one IPv6 host in my SYNC hence the reason for doing this. SYNC="rsync://gentoo-ipv6.alex-smith.org/gentoo-portage" Will work, although there is only 4 mirrors listed at the moment it's an improvement on what Gentoo's mirror team's done... gentoo-ipv6.alex-smith.org has IPv6 address 2001:708:310:54::101 gentoo-ipv6.alex-smith.org has IPv6 address 2001:200:0:1::800:21 gentoo-ipv6.alex-smith.org has IPv6 address 2001:638:500:101::21 gentoo-ipv6.alex-smith.org has IPv6 address 2001:708:310:54::2 If there's any breakages with the ones in that list (Out of dates, connection refused) or you have another that works that you want added feel free to drop me an email: asmith AT asmhosting DOT com (p.s. gentoo mirror team, im not trying to tread on your feet here, just it's been a rather long time!) Whats up with this here? Are there any informations about a ipv6 only (rsync) rotation jet? Don't get your knickers in a twist, we haven't forgotten about you. The new rsync1.us (which is actually rsync0.uk) finally has (native) IPv6 support [albatross.ipv6.gentoo.org], one of the first Gentoo infrastructure boxes to do so. So the mirrormon is getting IPv6 foo, then we can start populating rsync.ipv6.gentoo.org with more mirrors than only the broken one it has now. Alex, whats with your Ipv6 rotation? If someone is interested i can build a dns round robin like yours... Hi, Seems like this has gone to sleep once again... I'm bumping this thread again. Currently, Alex's rotation is gone and rsync.ipv6.gentoo.org still contains only one faulty server. Best regards, Mihai Moldovan Well OK, albatross.ipv6.gentoo.org seems to respond now, but it is denying my connection attempts, seems like this isn't too public yet? According to http://archives.gentoo.org/gentoo-mirrors/msg_7a4887f7f5dca5adccb1846a076c18e4.xml rsync.ipv6.gentoo.org is not supposed to be a public IPv6 rotation, but an IPv6-enabled masterserver for the other mirrors. The naming is less than perfect, but still. I'm NOT offering rsync.ipv6.gentoo.org in beta yet. Just rsync1.ipv6.gentoo.org is in beta first of all, I'm extremely short on other IPv6 resources for monitoring of IPv6 mirrors. The problem is that I need some more Gentoo infrastructure boxes with IPv6 so I can offer rsync.ipv6.gentoo.org from them, and thereafter rsync$N.$CC.ipv6.gentoo.org in the same fashion as the IPv4 rotation. rsync.ipv6.gentoo.org lives! Bugs in here please if you run into any (I had it broken earlier, so take this as the time after which bugs should be reported, nothing prior). Works well, but it'd be nice to have its v6 address on the Server Address line! :-) Is the notice(-2) anything to worry about? It's definitely syncing over v6. root@arthur:~> emerge --sync Notice: (-2, 'Name or service not known') >>> Starting rsync with rsync://rsync.ipv6.gentoo.org/gentoo-portage... >>> Checking server timestamp ... Welcome to flycatcher.gentoo.org Server Address : 81.93.240.111 Contact Name : mirror-admin@gentoo.org Hardware : 8 x Intel(R) Xeon(R) CPU E5405 @ 2.00GHz, 16084MB RAM MOTD updated for rsync{,.ipv6}.gentoo.org boxes updated to include IPv6 address and useful aliases.
Could you trace the notice warning please? I haven't seen it before.
Here's an strace of it (it's about half way down)
socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.0.1")}, 28) = 0
fcntl64(3, F_GETFL) = 0x2 (flags O_RDWR)
fcntl64(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
gettimeofday({1246106985, 750860}, NULL) = 0
poll([{fd=3, events=POLLOUT}], 1, 0) = 1 ([{fd=3, revents=POLLOUT}])
send(3, "\361A\1\0\0\1\0\0\0\0\0\0\5rsync\4ipv6\6gentoo\3or"..., 39, MSG_NOSIGNAL) = 39
poll([{fd=3, events=POLLIN}], 1, 5000) = 1 ([{fd=3, revents=POLLIN}])
ioctl(3, FIONREAD, [132]) = 0
recvfrom(3, "\361A\201\200\0\1\0\1\0\1\0\0\5rsync\4ipv6\6gentoo\3or"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("1
27.0.0.1")}, [16]) = 132
close(3) = 0
fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 5), ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb804f000
write(1, "Notice: (-2, 'Name or service not"..., 42) = 42
open("/var/log/emerge.log", O_WRONLY|O_CREAT|O_APPEND|O_LARGEFILE, 0666) = 3
fstat64(3, {st_mode=S_IFREG|0660, st_size=5109838, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb804e000
fstat64(3, {st_mode=S_IFREG|0660, st_size=5109838, ...}) = 0
_llseek(3, 5109838, [5109838], SEEK_SET) = 0
fstat64(3, {st_mode=S_IFREG|0660, st_size=5109838, ...}) = 0
stat64("/var/log/emerge.log", {st_mode=S_IFREG|0660, st_size=5109838, ...}) = 0
fcntl64(3, F_SETLK64, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}, 0xbf87fab4) = 0
fstat64(3, {st_mode=S_IFREG|0660, st_size=5109838, ...}) = 0
_llseek(3, 5109838, [5109838], SEEK_SET) = 0
gettimeofday({1246106985, 754685}, NULL) = 0
write(3, "1246106985: >>> Starting rsync wi"..., 81) = 81
fcntl64(3, F_SETLKW64, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}, 0xbf87fb14) = 0
close(3) = 0
munmap(0xb804e000, 4096) = 0
write(1, ">>> Starting rsync with rsync://r"..., 72) = 72
This seems to correspond to line 12850 in __init__.py (in Portage 2.1.6.13)
Sorry for the bugspam, this seems to have been caused by my /etc/hosts file not containing the address ::1. This was fixed in bug 25859 nearly 6 years ago so I'm not sure why it was missing from my copy. Thanks for tracing the notice anyway. rsync.ipv6.gentoo.org contains only one address. Any will to add the other ones <https://bugs.gentoo.org/buglist.cgi?quicksearch=ipv6+mirror>? I have these in my /etc/hosts too: 2001:708:310:54::2 rsync6.europe.gentoo.org 2001:708:310:54::101 rsync6.europe.gentoo.org 2001:6f8:93c::1 rsync6.europe.gentoo.org rsync.gentoo.org and rsync.ipv6.gentoo.org are deliberately following the same principle - they contain only servers run by Gentoo infrastructure. There aren't many of those with IPv6 at all. For community-run IPv6, I think we do the same country subcoding as IPv4, in addition to loading AAAA records into the same base. Eg: ; For the individual mirrors rsync3.$CC.gentoo.org IN A 123.123.123.123 rsync3.$CC.gentoo.org IN AAAA 2001::...:A1 ; For the country rotations: rsync.$CC.gentoo.org IN A 123.123.123.123 ipv6.rsync.$CC.gentoo.org IN AAAA 2001::...:A1 mirrormon still isn't IPv6 capable, mostly as that sponsor does not have IPv6 transit presently. I can't seem to connect to the server any more, with the following error: rsync: failed to connect to rsync.ipv6.gentoo.org: No route to host (113) rsync error: error in socket IO (code 10) at clientserver.c(124) [receiver=3.0.5] Here's a traceroute6 showing some kind of issue, I'm guessing at Gentoo's ISP.. root@bedevere:~> traceroute6 rsync.ipv6.gentoo.org traceroute to flycatcher-rsync.ipv6.gentoo.org (2001:758:f00:4732::a1) from 2001:470:9095:1:201:4aff:fe05:96ac, 30 hops max, 16 byte packets 1 2001:470:9095:1::1 (2001:470:9095:1::1) 0.243 ms 0.149 ms 0.131 ms 2 daviessm.tunnel.tserv5.lon1.ipv6.he.net (2001:470:1f08:243::1) 28.394 ms 26.404 ms 28.84 ms 3 gige-g4-6.core1.lon1.he.net (2001:470:0:67::1) 25.028 ms 32.446 ms 25.525 ms 4 10gigabitethernet1-1.core1.par1.he.net (2001:470:0:42::2) 35.398 ms 40.935 ms 28.649 ms 5 equinox.ipv6.panap.fr (2001:860:0:6:0:3:5393:1) 30.365 ms 29.517 ms 29.81 ms 6 2001:758:2309:3932:49:1:0:1 (2001:758:2309:3932:49:1:0:1) 90.955 ms 150.307 ms 199.895 ms 7 2001:758:2309:3932:49:3:0:2 (2001:758:2309:3932:49:3:0:2) 29.815 ms 32.388 ms 31.473 ms 8 2001:758:2309:3932:49:3:0:2 (2001:758:2309:3932:49:3:0:2) 3026.54 ms !H 3027.5 ms !H 3027.52 ms !H The server had an unscheduled downtime, and I just picked up a bug in baselayout1 that happened on the reboot. It is back. If you do get another downtime or other issue, please open a new bug, do NOT continue to comment on this one. (In reply to comment #26) > For community-run IPv6, I think we do the same country subcoding as IPv4, in > addition to loading AAAA records into the same base. ACK. Until now it seems not realized. Why? > mirrormon still isn't IPv6 capable, mostly as that sponsor does not have IPv6 > transit presently. What is mirrormon? I didn't find any information. It sounds like a tool to monitor if the mirrors being up-to-date. Maybe we could find a second sponsor with native IPv6 connectivity if this is the whole problem. The state of ipv6 rotation in Gentoo is poor. It is on the todo list for the upcoming months. mirmon is this: http://mirrorstats.gentoo.org/ but that was mostly killed while I have been setting up the replacement (which has ipv6 capabilities) (In reply to comment #30) > It is on the todo list for the > upcoming months. if someone could provide assistance (however) to you so let us know. > mirmon is this: http://mirrorstats.gentoo.org/ but that was > mostly killed while I have been setting up the replacement (which has ipv6 > capabilities) which replacement do you use? ipv6 will be more and more significant to all users and gentoo should be on the cutting-edge ;) Any news about this bug? Is there any ipv6 plan for rsync rotation? So far at least the official german rsync pool (rsync.de.gentoo.org) does anready contain an ipv6 entry (which is mirror.opteamax.de): rsync.de.gentoo.org has IPv6 address 2001:67c:1420::1:50 Nothing more needed than putting PORTAGE_RSYNC_EXTRA_OPTS="-6" in your make.conf. The name could contain more servers though, that are v6 capable, like (taken from mirrorstats.gentoo.org, untested): mirror.netcologne.de, ftp.join.uni-muenster.de Read this thread <http://thread.gmane.org/gmane.linux.gentoo.mirrors/35>. You could see there is "a lot" of IPv6 capable mirrors. Yup, at this time there are no longer any plans to create ipv6 *only* rotations. All, rsync.$CC.gentoo.org rotations have AAAA records where appropriate. Moving forward, we are adding A & AAAA records for new mirrors. I'm going to close this bug as fixed. Please do not forget on region-wide mirrors like rsync.europe.gentoo.org. All of them are A only right now. (In reply to comment #37) > Please do not forget on region-wide mirrors like rsync.europe.gentoo.org. All > of them are A only right now. > Yup, we can't add anymore to rsync.europe or it will go over the 512byte UDP packet size. (and cause issues like bug 324121) We either need to implement GeoIP or some trickery to "load balance" the rotation (probably via a unbound DNS resolver plugin/extension) (In reply to comment #38) > > Yup, we can't add anymore to rsync.europe or it will go over the 512byte UDP > packet size. (and cause issues like bug 324121) > I know that infrastructure guys are conservative. But this is ultraconservative. Have you heard about DNSSEC that requires long UDP packets and that has been deployed to root server already? I guess the 512B limit is over. (In reply to comment #39) > (In reply to comment #38) > > > > Yup, we can't add anymore to rsync.europe or it will go over the 512byte UDP > > packet size. (and cause issues like bug 324121) > > > I know that infrastructure guys are conservative. But this is > ultraconservative. Have you heard about DNSSEC that requires long UDP packets > and that has been deployed to root server already? I guess the 512B limit is > over. > Yes, but it affects our users still. That is a bad thing so I guess the limit is still in the wild. Anyway, you don't need to convince me or the other Gentoo infra team members, we get it. Let's stop commenting on this bug now as the case is closed for the sake of being less noisy. :) |
