Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 904968 - net-analyzer/speedtest-cli doesn't work anymore
Summary: net-analyzer/speedtest-cli doesn't work anymore
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Conrad Kostecki
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-04-24 16:29 UTC by Thomas Capricelli
Modified: 2023-05-07 12:34 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
rebased patch (7906c4bdc36b969212526d71e83a2ecea5739704.patch,3.40 KB, text/plain)
2023-05-03 14:34 UTC, Ben Kohler
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Capricelli 2023-04-24 16:29:56 UTC
vivaldi ~ # speedtest
Retrieving speedtest.net configuration...
Testing from Renater (134.157.15.237)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
ERROR: Unable to connect to servers to test latency.


It used to work on all my servers, and now it fails on all servers. I eliminated 'local' network problems (firewall in or near the servers).
There are some (but few) people reporting this error since ~oct 2022. Could it be that the speedtest service is not available anymore ?

I notice the gentoo package hasn't been updated for long. Could that be the problem ? 

Reproducible: Always
Comment 1 Ben Kohler gentoo-dev 2023-04-24 17:09:01 UTC
Related? https://github.com/sivel/speedtest-cli/pull/796
Comment 2 Thomas Capricelli 2023-04-24 18:27:33 UTC
there's no --verbose option, so i couldn't find the url for 'Retrieving speedtest.net configuration'.

In your linked ticket, there's curl and i have the same answer:


curl -iL https://www.speedtest.net/speedtest-servers.php
-> 429 answer, "Too many requests"

Which i dont understand, i haven't tried for months, and almost all servers are standalone (not NAT'ing other computers).

I find it hard to believe that all my servers are blacklisted at the same time, they are in different physical location, their ip are completely unrelated.. ?

On the ticket there's a link to a patch
https://github.com/sivel/speedtest-cli/pull/796/commits/7906c4bdc36b969212526d71e83a2ecea5739704

where they replace the url by : https://www.speedtest.net/api/js/servers

I can confirm this one works. I can't understand from the ticket if this commit was merged, if so when / which release.

It doesn't seem to be in 'mainline' file: https://github.com/sivel/speedtest-cli/blob/master/speedtest.py
Comment 3 Ben Kohler gentoo-dev 2023-04-24 18:32:19 UTC
No it's not merged-- but we can add it in ::gentoo if needed. I'm assigning this to the gentoo maintainer, let's see what he says.
Comment 4 Thomas Capricelli 2023-04-24 20:33:21 UTC
For that matters, net-analyzer/speedtest++ doesn't work either. The error seems similar:

Finding fastest server... Unable to download server list. Try again later

Am I the only one using speedtest on servers .. ?? Or the only where it fails ?
Comment 5 Conrad Kostecki gentoo-dev 2023-05-02 22:08:38 UTC
This does not seem a general problem, as it works for me just fine. I am not able to reproduce in any way.

$ speedtest
Retrieving speedtest.net configuration...
Testing from Hetzner Online GmbH (178.63.55.74)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Der TechNik (Falkenstein) [324.57 km]: 0.574 ms
Testing download speed................................................................................
Download: 702.04 Mbit/s
Testing upload speed......................................................................................................
Upload: 643.07 Mbit/s

I am not sure about the patch. It does not apply against 2.1.3. Did you had modified the patch?
Comment 6 Thomas Capricelli 2023-05-03 13:08:25 UTC
I did not try the patch, just noticed the part that changes the url being requested for the list of servers.

As said, i have the problem from a whole bunch of different servers, i highly doubt all of them have been "banned" at the same time. I barely use speedtest (no cron or whatever)

What about the url i gave that reports "429 / too many requests". Does this work for you ?

  curl -iL https://www.speedtest.net/speedtest-servers.php
Comment 7 Ben Kohler gentoo-dev 2023-05-03 14:11:10 UTC
(In reply to Conrad Kostecki from comment #5)
> This does not seem a general problem, as it works for me just fine. I am not
> able to reproduce in any way.
> 
You're not able to reproduce because your IP isn't blocked/banned/throttled-- see the details in the upstream PR:

curl -iL https://www.speedtest.net/speedtest-servers.php
HTTP/2 429 
date: Thu, 02 Mar 2023 18:57:11 GMT
content-type: text/html
content-length: 591
retry-after: 0
accept-ranges: bytes
cf-cache-status: DYNAMIC
set-cookie: __cf_bm=lsVV04VP4EnXe_XACbglTNypxfwSwVdCSBZCsKmb5_A-1677783431-0-AfzPaNeYOsnL5DGl66fR1OEsrx7cZqQIPsosKJdrzyfOmli02FaYmmpSXco13lETXCmgU0VgfrY5Mk2mj/t5r+U=; path=/; expires=Thu, 02-Mar-23 19:27:11 GMT; domain=.www.speedtest.net; HttpOnly; Secure; SameSite=None
server: cloudflare
cf-ray: 7a1bdd2f09e40d73-MRS

<html><head><title>Too Many Requests</title></head><body><h1>429 Error - Too Many Requests</h1><p>Your IP address is making too many requests. We ask that you limit requests to no more than 1 per second.</p><p>We make an attempt to be flexible, allowing small bursts in traffic. You have exceeded this threshold however and have been blocked.</p><p>Please reduce the speed at which your requests are made and try again after a while. If you feel you have reached this message in error, please contact us at noc @ speedtest.net and reference this error code.</p><p>Thank you</p></body></html>
Comment 8 Ben Kohler gentoo-dev 2023-05-03 14:34:10 UTC
Created attachment 861086 [details]
rebased patch

This rebased patch works for me but we may be better off taking a snapshot and using the existing PR patch as-is.
Comment 9 Conrad Kostecki gentoo-dev 2023-05-07 12:32:51 UTC
(In reply to Thomas Capricelli from comment #6)
> What about the url i gave that reports "429 / too many requests". Does this
> work for you ?
> 
>   curl -iL https://www.speedtest.net/speedtest-servers.php

Yes, works absolutly fine for me.

---------

Anyway, I tested the patch now on multiple machines, seems fine for me. I will add this one. But I would still recommend, that you talk to Ookla. This still looks for me, like they have some outdated? rules and block some ip ranges they should not.
Comment 10 Larry the Git Cow gentoo-dev 2023-05-07 12:34:20 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=2ca0c72fa042dc510f74b128fe513db0cb9a934c

commit 2ca0c72fa042dc510f74b128fe513db0cb9a934c
Author:     Conrad Kostecki <conikost@gentoo.org>
AuthorDate: 2023-05-07 12:33:35 +0000
Commit:     Conrad Kostecki <conikost@gentoo.org>
CommitDate: 2023-05-07 12:34:10 +0000

    net-analyzer/speedtest-cli: switch to json serverlist
    
    Closes: https://bugs.gentoo.org/904968
    Signed-off-by: Conrad Kostecki <conikost@gentoo.org>

 .../speedtest-cli-2.1.3-json-serverlist.patch      | 88 ++++++++++++++++++++++
 ...1.3-r1.ebuild => speedtest-cli-2.1.3-r2.ebuild} |  2 +
 2 files changed, 90 insertions(+)