Summary: | Connection to dev.gentoo.org is very slow | ||
---|---|---|---|
Product: | Gentoo Infrastructure | Reporter: | Thomas Deutschmann (RETIRED) <whissi> |
Component: | Dev box issues | Assignee: | Gentoo Infrastructure <infra-bugs> |
Status: | IN_PROGRESS --- | ||
Severity: | normal | CC: | sam |
Priority: | Normal | ||
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Attachments: | wget.log |
Description
Thomas Deutschmann (RETIRED)
2020-09-21 13:58:49 UTC
For comparison, it maxes out my download bandwidth from my home in Detroit, MI, USA.
> floppym@naomi tmp % time wget -4 -S https://dev.gentoo.org/~whissi/dist/libvpx/libvpx-testdata-1.9.0.tar.xz
> --2020-09-21 10:24:55-- https://dev.gentoo.org/~whissi/dist/libvpx/libvpx-testdata-1.9.0.tar.xz
> Resolving dev.gentoo.org (dev.gentoo.org)... 140.211.166.183
> Connecting to dev.gentoo.org (dev.gentoo.org)|140.211.166.183|:443... connected.
> HTTP request sent, awaiting response...
> HTTP/1.1 200 OK
> Date: Mon, 21 Sep 2020 14:24:56 GMT
> Server: Apache
> Last-Modified: Tue, 11 Aug 2020 22:09:57 GMT
> ETag: "1581c6d-1adfb67c-5aca154dc7740"
> Accept-Ranges: bytes
> Content-Length: 450868860
> Keep-Alive: timeout=15, max=100
> Connection: Keep-Alive
> Content-Type: application/x-xz
> Length: 450868860 (430M) [application/x-xz]
> Saving to: ‘libvpx-testdata-1.9.0.tar.xz’
>
> libvpx-testdata-1.9.0.ta 100%[==================================>] 429.98M 11.8MB/s in 35s
>
> 2020-09-21 10:25:32 (12.1 MB/s) - ‘libvpx-testdata-1.9.0.tar.xz’ saved [450868860/450868860]
>
> wget -4 -S 11.04s user 2.98s system 38% cpu 36.643 total
>
> floppym@naomi tmp % traceroute -4 dev.gentoo.org
> traceroute to dev.gentoo.org (140.211.166.183), 30 hops max, 60 byte packets
> 1 192.168.0.1 (192.168.0.1) 1.743 ms 2.051 ms 2.352 ms
> 2 d149-67-1-56.try.wideopenwest.com (67.149.56.1) 144.304 ms 144.255 ms 144.863 ms
> 3 d47-69-97-57.col.wideopenwest.com (69.47.57.97) 18.719 ms 18.772 ms 18.861 ms
> 4 76-73-165-180.knology.net (76.73.165.180) 18.949 ms 21.080 ms 19.031 ms
> 5 ip65-46-186-97.z186-46-65.customer.algx.net (65.46.186.97) 26.053 ms 25.849 ms 25.930 ms
> 6 207.88.13.13.ptr.us.xo.net (207.88.13.13) 74.089 ms 72.215 ms 72.135 ms
> 7 198.32.251.209 (198.32.251.209) 85.457 ms 79.763 ms 83.628 ms
> 8 dc-svl-agg8--svl-agg4-100ge-1.cenic.net (137.164.11.29) 84.404 ms 84.339 ms dc-svl-agg8--svl-agg4-100ge-2.cenic.net (137.164.11.31) 80.624 ms
> 9 dc-svl-agg10--svl-agg8-300g.cenic.net (137.164.11.80) 81.499 ms * 76.697 ms
> 10 lon--cenic-100ge.cenic.net (137.164.3.103) 71.977 ms 70.990 ms 70.944 ms
> 11 eugn-p1-gw.nero.net (207.98.64.196) 80.457 ms 79.744 ms 77.812 ms
> 12 corv-p2-gw.nero.net (207.98.64.13) 83.734 ms 82.461 ms 79.748 ms
> 13 corv-car1-gw.nero.net (207.98.64.17) 70.789 ms 70.932 ms 75.366 ms
> 14 * * *
> 15 * * *
> 16 * * *
> 17 * * *
> 18 * * *
> 19 * * *
> 20 * * *
> 21 * * *
> 22 * * *
> 23 * * *
> 24 * * *
> 25 * * *
> 26 * * *
> 27 * * *
> 28 * * *
> 29 * * *
> 30 * * *
Created attachment 661789 [details]
wget.log
(In reply to Thomas Deutschmann from comment #2) > Created attachment 661789 [details] > wget.log Can you do a tcp capture? I'm curious about the tcp window params for your slow link (receive and congestion window.) In my test: TCP segment sizes ranged from 1448 to 7420. My TCP window was 61 segments. My RTT is 54ms If we round: 1000ms in a second / 54ms ~= 20 complete RTTs in a second. 61 inflights per RTT * 20 = 1220 segments per second. When segments are 1448: 1220 * 1448 == 1766560 bytes / second (70mbps) When segments are 7420: 1220 * 7420 == 9052400 bytes / second (13mbps) Now these are all theoretical speeds (we don't actually sustain 61 inflghts consistently, rtt is not quite 54ms all the time, the connection does not sustain a particular segment size, etc.) Find me on IRC and we can do a test. But in theory: throughput = RTT * window_size * segment_size I'm curious to know what the bottleneck is here; is the window size just small? is the kernel picking small segments (e.g. because big segments are lost on the link somewhere, causing a stall and impacting window fill?) (In reply to Alec Warner from comment #3) > (In reply to Thomas Deutschmann from comment #2) > > Created attachment 661789 [details] > > wget.log > > Can you do a tcp capture? I'm curious about the tcp window params for your > slow link (receive and congestion window.) > > In my test: > > TCP segment sizes ranged from 1448 to 7420. > My TCP window was 61 segments. > My RTT is 54ms > > If we round: > 1000ms in a second / 54ms ~= 20 complete RTTs in a second. > 61 inflights per RTT * 20 = 1220 segments per second. > > When segments are 1448: 1220 * 1448 == 1766560 bytes / second (70mbps) > When segments are 7420: 1220 * 7420 == 9052400 bytes / second (13mbps) > > Now these are all theoretical speeds (we don't actually sustain 61 inflghts > consistently, rtt is not quite 54ms all the time, the connection does not > sustain a particular segment size, etc.) > > Find me on IRC and we can do a test. But in theory: > > throughput = RTT * window_size * segment_size > > I'm curious to know what the bottleneck is here; is the window size just > small? is the kernel picking small segments (e.g. because big segments are > lost on the link somewhere, causing a stall and impacting window fill?) Testing today with whissi and others: - TCP window: 61 - segment size: 1440 - These two things result in the 'slow' path (< 1MB/s) due to unfill windows and longer RTTs for europeans. The cause appears to "the internet is bad" and various links are congested. This in turn causes loss, which causes tcp stack to reduce segment sizes (to 1440, the nominal smallest segment size; I haven't seen segments smaller than this.) Loss at some links was 20-80%; which..I mean..you can't get a good speed over that kind of link using standard control algorithms. FWIW I reported this upstream (to the OSL) but I don't expect a short term solution. -A |