Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 280764 - 193.1.193.64 (ftp.heanet.ie): broken for several months now
Summary: 193.1.193.64 (ftp.heanet.ie): broken for several months now
Status: RESOLVED FIXED
Alias: None
Product: Mirrors
Classification: Unclassified
Component: Server Problem (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Mirror Admins
URL:
Whiteboard: due:2009/10/07;dist:||;portage:rsync1...
Keywords:
: 290060 (view as bug list)
Depends on:
Blocks:
 
Reported: 2009-08-08 14:50 UTC by Nikos Chantziaras
Modified: 2010-02-18 21:04 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Nikos Chantziaras 2009-08-08 14:50:57 UTC
For several months now, ftp.heanet.ie (rotated in rsync://rsync.europe.gentoo.org) isn't syncing.  It just stays on the "receiving file list ..." step forever, with almost no traffic going to it or from it.

Is there a way to block this specific mirror on my machine?
Comment 1 Arun Raghavan (RETIRED) gentoo-dev 2009-08-11 17:21:09 UTC
Seems to be fine for now as per mirrorstats (http://mirrorstats.gentoo.org/rsync/). Leaving this open for now - will add this to our automatic monitoring system after clearing the backlog.
Comment 2 Martin Gebert 2009-08-21 08:14:24 UTC
Experienced the same problem yesterday and this morning a couple of times; had to switch to rsync.gentoo.org to be able to sync. Please remove this mirror; if it isn't borken completely, it's at least unreliable.
Comment 3 Jeremy Olexa (darkside) (RETIRED) archtester gentoo-dev Security 2009-10-02 02:22:09 UTC
(In reply to comment #2)
> Experienced the same problem yesterday and this morning a couple of times; had
> to switch to rsync.gentoo.org to be able to sync. Please remove this mirror; if
> it isn't borken completely, it's at least unreliable.
> 

Sorry, it looks good here. I'll add it to the testing system for one week.
Comment 4 Nikos Chantziaras 2009-10-02 16:04:59 UTC
(In reply to comment #3)
> Sorry, it looks good here. I'll add it to the testing system for one week.

It's broken right now.  It wasn't yesterday, or last week, or 20 minutes ago, but right now.

It's random when that server breaks.
Comment 5 Jeremy Olexa (darkside) (RETIRED) archtester gentoo-dev Security 2009-10-13 04:42:12 UTC
Hard to debug. I'll see if I can get some assistance on this issue.
Comment 6 Nikos Chantziaras 2009-10-23 13:57:51 UTC
For a few days now, it seems to work but is extremely slow.
Comment 7 Jeremy Olexa (darkside) (RETIRED) archtester gentoo-dev Security 2009-10-23 21:49:08 UTC
Alright, can we get a traceroute from you in an attempt to possibly debug this some more?
Comment 8 Nikos Chantziaras 2009-10-23 21:54:00 UTC
Tracing the path to 193.1.193.64 on TCP port 80 (http), 30 hops max
 1  10.0.0.138  1.279 ms  0.561 ms  0.552 ms
 2  * 80.106.108.20 11.569 ms *
 3  80.106.229.161  11.240 ms  11.564 ms  12.227 ms
 4  athe-crsb-athe7609a-1.backbone.otenet.net (79.128.227.129)  11.259 ms
    athe-crsb-athe7609a-2.backbone.otenet.net (79.128.227.133)  11.722 ms
    athe-crsb-athe7609a-1.backbone.otenet.net (79.128.227.129)  11.586 ms
 5  62.75.3.1  11.240 ms
    ten0-0-5-0-crs02-otenet.ath.oteglobe.net (62.75.3.9)  12.429 ms  11.826 ms
 6  pos7-0-0-cr02.lon.oteglobe.net (62.75.4.66)  64.257 ms
    ten0-7-5-0-crs01-tbn.fra.oteglobe.net (62.75.4.214)  65.687 ms
    pos7-0-0-cr02.lon.oteglobe.net (62.75.4.66)  64.006 ms
 7  194.25.208.37  70.458 ms  71.159 ms
    TenGigabitEthernet4-3.ar3.FRA3.gblx.net (67.17.197.205)  65.956 ms
 8  xe-9-2-0-0.fra20.ip4.tinet.net (213.200.65.149)  65.659 ms  125.165 ms
    194.25.6.189  71.404 ms
 9  62.156.138.54  71.256 ms
    so-1-1-0.dub20.ip4.tinet.net (89.149.187.9)  92.802 ms
    62.156.138.54  72.908 ms
10  heanet-limited-gw.ip4.tinet.net (77.67.64.206)  96.958 ms  93.334 ms
    ldn-bb1-link.telia.net (80.91.251.18)  71.653 ms
11  te5-1-blanch-sr1.services.hea.net (193.1.236.2)  93.307 ms  93.231 ms
    dln-b3-link.telia.net (80.91.249.134)  91.445 ms
12  heanet-ic-126792-dln-b3.c.telia.net (213.248.88.10)  83.059 ms  83.489 ms
    ftp.heanet.ie (193.1.193.64) [open]  94.056 ms
Comment 9 Jeremy Olexa (darkside) (RETIRED) archtester gentoo-dev Security 2009-10-27 19:48:16 UTC
HEAnet mirror admins, We would appreciate it if you could help us debug this issue. As suggested on this bug, there appears to be some connectivity issues from Greece and Germany to your mirror. My personal computers in the USA cannot reproduce this issue. It is suggested that it happens at random times. Any ideas?
Comment 10 HEAnet mirror admin 2009-10-28 09:41:48 UTC
Hi,

We've been encountering this bug on our mirror cluster every so often. It's likely the user was hitting
one of the nodes with a hung rsyncd:

http://www.mail-archive.com/rsync@lists.samba.org/msg23199.html

I'm planning to upgrade to rsync 3.x in the next couple of days, which should resolve it. 

For comparison I ran the following from a machine in Germany:

rsync -4Pavn rsync://ftp.heanet.ie/gentoo-portage runs fine.

Traceroute to ftp.heanet.ie (rsync port):

tcptraceroute -p 873 ftp.heanet.ie
traceroute to ftp.heanet.ie (193.1.193.64), 30 hops max, 40 byte packets
 1  static.88-198-67-225.clients.your-server.de (88.198.67.225)  0.251 ms  0.227 ms  0.227 ms
 2  hos-tr4.juniper2.rz6.hetzner.de (213.239.252.129)  2.033 ms  1.034 ms  1.291 ms
 3  hos-bb1.juniper1.ffm.hetzner.de (213.239.240.224)  4.735 ms  5.361 ms  5.242 ms
 4  ffm-b2-link.telia.net (213.248.92.81)  4.576 ms  3.409 ms  3.437 ms
 5  ffm-bb1-link.telia.net (80.91.252.173)  8.829 ms  3.748 ms  4.099 ms
 6  prs-bb1-link.telia.net (80.91.252.229)  13.537 ms  13.643 ms  13.516 ms
 7  ldn-bb1-link.telia.net (80.91.247.7)  20.799 ms  20.747 ms  20.736 ms
 8  dln-b3-link.telia.net (80.91.249.134)  30.988 ms  30.999 ms  30.980 ms
 9  heanet-ic-126792-dln-b3.c.telia.net (213.248.88.10)  31.884 ms  31.597 ms  31.961 ms
10  te5-1-blanch-sr1.services.hea.net (193.1.236.2)  32.077 ms  31.977 ms  32.093 ms
11  ftp.heanet.ie (193.1.193.64)  31.937 ms  31.913 ms  32.347 ms

Also, I noticed our contact details aren't listed on http://mirrorstats.gentoo.org/rsync/  
Could they be updated, please?:

Admin contact: HEAnet Mirror Admin
Email: mirrors@heanet.ie

rg
Comment 11 Jeremy Olexa (darkside) (RETIRED) archtester gentoo-dev Security 2009-11-16 01:05:55 UTC
(In reply to comment #10)

(thanks for the prompt response)

> I'm planning to upgrade to rsync 3.x in the next couple of days, which should
> resolve it. 

Is it completed now?

> 
> For comparison I ran the following from a machine in Germany:
> 
> rsync -4Pavn rsync://ftp.heanet.ie/gentoo-portage runs fine.
> 
> Traceroute to ftp.heanet.ie (rsync port):
> 
> tcptraceroute -p 873 ftp.heanet.ie
> traceroute to ftp.heanet.ie (193.1.193.64), 30 hops max, 40 byte packets
>  1  static.88-198-67-225.clients.your-server.de (88.198.67.225)  0.251 ms 
> 0.227 ms  0.227 ms
>  2  hos-tr4.juniper2.rz6.hetzner.de (213.239.252.129)  2.033 ms  1.034 ms 
> 1.291 ms
>  3  hos-bb1.juniper1.ffm.hetzner.de (213.239.240.224)  4.735 ms  5.361 ms 
> 5.242 ms
>  4  ffm-b2-link.telia.net (213.248.92.81)  4.576 ms  3.409 ms  3.437 ms
>  5  ffm-bb1-link.telia.net (80.91.252.173)  8.829 ms  3.748 ms  4.099 ms
>  6  prs-bb1-link.telia.net (80.91.252.229)  13.537 ms  13.643 ms  13.516 ms
>  7  ldn-bb1-link.telia.net (80.91.247.7)  20.799 ms  20.747 ms  20.736 ms
>  8  dln-b3-link.telia.net (80.91.249.134)  30.988 ms  30.999 ms  30.980 ms
>  9  heanet-ic-126792-dln-b3.c.telia.net (213.248.88.10)  31.884 ms  31.597 ms 
> 31.961 ms
> 10  te5-1-blanch-sr1.services.hea.net (193.1.236.2)  32.077 ms  31.977 ms 
> 32.093 ms
> 11  ftp.heanet.ie (193.1.193.64)  31.937 ms  31.913 ms  32.347 ms
> 
> Also, I noticed our contact details aren't listed on
> http://mirrorstats.gentoo.org/rsync/  
> Could they be updated, please?:

We are replacing that page soon. So, no work will be done there. Sorry.

> 
> Admin contact: HEAnet Mirror Admin
> Email: mirrors@heanet.ie
> 
> rg
> 

Comment 12 Jeremy Olexa (darkside) (RETIRED) archtester gentoo-dev Security 2009-11-16 01:33:46 UTC
*** Bug 290060 has been marked as a duplicate of this bug. ***
Comment 13 HEAnet mirror admin 2009-11-16 09:55:58 UTC
Yep, we upgraded to rsync 3.0.6 and it has remained stable for about 10 days now. 
Comment 14 Jeremy Olexa (darkside) (RETIRED) archtester gentoo-dev Security 2009-11-16 21:27:38 UTC
(In reply to comment #13)
> Yep, we upgraded to rsync 3.0.6 and it has remained stable for about 10 days
> now. 
> 

Thanks for the update.

Resolving this bug, please reopen or file a new one if there are future problems. Thanks.
Comment 15 Nikos Chantziaras 2010-01-22 17:14:47 UTC
Damn. For over a month now, the HEANet mirror is so slow again. And I mean *slow*; as it would be running on dial-up. :P An emerge --sync takes long; longest I left it sync was about 10 minutes where it only reached the games-* hierarchy, then I aborted. Reopening.

HEAnet mirror admin: Could you check what's wrong?

tcptraceroute:

 1  10.0.0.138  1.273 ms  0.605 ms  0.713 ms
 2  80.106.108.20  164.700 ms * *
 3  80.106.229.177  11.456 ms  11.400 ms  11.792 ms
 4  athe-crsb-athe7609b-2.backbone.otenet.net (79.128.227.141)  11.799 ms
    athe-crsa-athe7609b-2.backbone.otenet.net (79.128.227.13)  12.099 ms
    athe-crsb-athe7609b-1.backbone.otenet.net (79.128.227.137)  11.996 ms
 5  62.75.3.1  12.370 ms
    ten0-0-0-0-crs02-otenet.ath.OTEGlobe.net (62.75.3.13)  13.043 ms  11.562 ms
 6  62.75.4.66  61.557 ms
    62.75.4.150  59.649 ms  59.495 ms
 7  ldn-b3-link.telia.net (195.12.255.141)  66.439 ms  80.626 ms
    ffm-b10-link.telia.net (195.12.255.165)  57.595 ms
 8  ffm-bb1-link.telia.net (80.91.251.123)  53.578 ms
    ffm-bb1-link.telia.net (80.91.251.249)  54.276 ms
    ldn-bb1-link.telia.net (80.91.251.166)  66.962 ms
 9  dln-b3-link.telia.net (80.91.249.134)  74.837 ms
    prs-bb1-link.telia.net (80.91.254.205)  69.706 ms
    dln-b3-link.telia.net (80.91.249.134)  74.356 ms
10  heanet-ic-126792-dln-b3.c.telia.net (213.248.88.10)  78.063 ms
    ldn-bb1-link.telia.net (80.91.247.35)  72.554 ms
    heanet-ic-126792-dln-b3.c.telia.net (213.248.88.10)  78.330 ms
11  dln-b3-link.telia.net (80.91.249.134)  82.347 ms  83.217 ms
    te5-1-blanch-sr1.services.hea.net (193.1.236.2)  75.689 ms
12  ftp.heanet.ie (193.1.193.64) [open]  77.358 ms  76.241 ms  76.914 ms
Comment 16 HEAnet mirror admin 2010-01-26 11:12:29 UTC
Hi,

We'll have a look into this, there was a large traffic spike last week which seemed to coincide with the poor performance you were seeing.

Just for comparison, here are the speeds from two hosts in Germany and the Netherlands - whilst downloading a 700Mb ISO:

Germany (Hetzner):

rsync --stats -v -P ftp.heanet.ie::gentoo/releases/x86/2008.0/livecd/livecd-i686-installer-2008.0-r1.iso .

...

Number of files: 1
Number of files transferred: 1
Total file size: 721944576 bytes
Total transferred file size: 721944576 bytes
Literal data: 721944576 bytes
Matched data: 0 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 47
Total bytes received: 722033331

sent 47 bytes  received 722033331 bytes  7330288.10 bytes/sec
total size is 721944576  speedup is 1.00

(TCP) Traceroute from this host:

tcptraceroute ftp.heanet.ie 80
Selected device eth0, address 188.40.137.138, port 51140 for outgoing packets
Tracing the path to ftp.heanet.ie (193.1.193.64) on TCP port 80 (www), 30 hops max
 1  static.129.137.40.188.clients.your-server.de (188.40.137.129)  0.735 ms  0.540 ms  0.373 ms
 2  hos-tr4.juniper2.rz10.hetzner.de (213.239.227.225)  0.155 ms  0.212 ms  0.141 ms
 3  hos-bb1.juniper1.ffm.hetzner.de (213.239.240.224)  4.826 ms  4.781 ms  4.773 ms
 4  xe-2-3-0.271.fra21.ip4.tinet.net (77.67.64.17)  4.825 ms  4.867 ms  4.825 ms
 5  so-1-1-0.dub20.ip4.tinet.net (89.149.187.9)  28.619 ms  28.590 ms  28.599 ms
 6  heanet-limited-gw.ip4.tinet.net (77.67.64.206)  35.603 ms  62.761 ms  45.003 ms
 7  te5-1-blanch-sr1.services.hea.net (193.1.236.2)  31.787 ms  31.871 ms  31.802 ms
 8  ftp.heanet.ie (193.1.193.64) [open]  31.694 ms  31.706 ms  31.702 ms

Netherlands (Leaseweb):

rsync --stats -v -P ftp.heanet.ie::gentoo/releases/x86/2008.0/livecd/livecd-i686-installer-2008.0-r1.iso .

...

Number of files: 1
Number of files transferred: 1
Total file size: 721944576 bytes
Total transferred file size: 721944576 bytes
Literal data: 721944576 bytes
Matched data: 0 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 47
Total bytes received: 722033331

sent 47 bytes  received 722033331 bytes  10241608.20 bytes/sec
total size is 721944576  speedup is 1.00

(ICMP) Traceroute from this host:

traceroute -I ftp.heanet.ie
traceroute to ftp.heanet.ie (193.1.193.64), 64 hops max, 60 byte packets
 1  hosted.by.leaseweb.com (85.17.139.252)  1.636 ms  0.448 ms  0.485 ms
 2  be11.crs.evo.leaseweb.net (85.17.100.225)  0.979 ms  0.956 ms  0.578 ms
 3  adm-evo-link.telia.net (213.248.88.197)  0.391 ms  0.453 ms  0.487 ms
 4  adm-b2-link.telia.net (80.91.248.245)  0.981 ms  0.962 ms  0.985 ms
 5  adm-bb1-link.telia.net (80.91.253.181)  0.985 ms  0.954 ms  0.983 ms
 6  ldn-bb1-link.telia.net (80.91.253.145)  9.486 ms  9.458 ms  9.479 ms
 7  dln-b3-link.telia.net (80.91.249.134)  19.970 ms  19.951 ms  19.473 ms
 8  heanet-ic-126792-dln-b3.c.telia.net (213.248.88.10)  20.975 ms  20.473 ms  22.451 ms
 9  te5-1-blanch-sr1.services.hea.net (193.1.236.2)  20.481 ms  20.459 ms  20.473 ms
10  ftp.heanet.ie (193.1.193.64)  20.471 ms  20.455 ms  20.479 ms
Comment 17 Jeremy Olexa (darkside) (RETIRED) archtester gentoo-dev Security 2010-02-18 21:04:45 UTC
(In reply to comment #16)
> Hi,
> 
> We'll have a look into this, there was a large traffic spike last week which
> seemed to coincide with the poor performance you were seeing.

Your prompt response is always appreciated, thx