Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 503638 - app-portage/mirrorselect: mirrorselect-test (for --deep) does not seem to exist on most mirrors
Summary: app-portage/mirrorselect: mirrorselect-test (for --deep) does not seem to exi...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage Tools Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-03-06 16:23 UTC by Michał Górny
Modified: 2017-08-17 16:22 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-03-06 16:23:14 UTC
The 'mirrorselect-test' file used by 'mirrorselect --deep' seems not to exist on most of our mirrors. As a result, it usually adds the single mirror that has the file for some reason rather than the best mirror possible :).

* Downloading mirrorselect-test files from each mirror... [1 of 91]
_deeptime(): maxtime is 10
deeptime(): ip's for host gentoo.bloodhost.ru: ['37.139.54.5']
deeptime(): testing url: http://37.139.54.5/distfiles/mirrorselect-test
deeptime(): connection to host gentoo.bloodhost.ru
            returned HTTPError: HTTP Error 404: Not Found for ip 37.139.54.5
            Switching back to original url
deeptime(): connection to host gentoo.bloodhost.ru
            returned HTTPError: HTTP Error 404: Not Found for ip 37.139.54.5
            Switching back to original url
deeptime(): unable to connect to host gentoo.bloodhost.ru
* Downloading mirrorselect-test files from each mirror... [2 of 91]
_deeptime(): maxtime is 10
deeptime(): ip's for host mirror.mcs.anl.gov: ['146.137.96.7']
deeptime(): testing url: http://146.137.96.7/pub/gentoo/distfiles/mirrorselect-test
deeptime(): connection to host mirror.mcs.anl.gov
            returned HTTPError: HTTP Error 404: Not Found for ip 146.137.96.7
            Switching back to original url
deeptime(): connection to host mirror.mcs.anl.gov
            returned HTTPError: HTTP Error 404: Not Found for ip 146.137.96.7
            Switching back to original url
deeptime(): unable to connect to host mirror.mcs.anl.gov
Comment 1 Brian Dolbec (RETIRED) gentoo-dev 2014-03-06 17:05:08 UTC
running:

mirrorselect -DH4 -s 5 -d 3

...

deeptest(): got 91 hosts, and returned ['http://tux.rainside.sk/gentoo/', 'http://ftp.daum.net/gentoo/', 'http://gentoo.gg3.net/']


Those are the only 3 servers mirroring that file, at least for http protocol.
Comment 2 Jeremy Olexa (darkside) (RETIRED) archtester gentoo-dev Security 2014-03-08 03:27:40 UTC
I've explained to Brian on how to fix this issue.
Comment 3 Brian Dolbec (RETIRED) gentoo-dev 2014-03-08 04:21:21 UTC
Temporaily fixed by adding the file to distfiles-local.
I will add that file to the SRC_URI in the next release (soon).  That will fix it for all installed versions.
Comment 4 jms 2015-11-25 00:10:15 UTC
is Bug 523312 a duplicate?

mirrorselect -s5 -b10 -o -D
* Using url: https://api.gentoo.org/mirrors/distfiles.xml
* Downloading a list of mirrors...
 Got 162 mirrors.
* Downloading mirrorselect-test files from each mirror... [58 of 162]Traceback (most recent call last):
  File "/usr/lib64/python3.4/site-packages/mirrorselect/selectors.py", line 305, in deeptime
    addr_family, socket.SOCK_STREAM, 0, socket.AI_ADDRCONFIG):
  File "/usr/lib64/python3.4/socket.py", line 533, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.4/mirrorselect", line 61, in <module>
    MirrorSelect().main(sys.argv)
  File "/usr/lib64/python3.4/site-packages/mirrorselect/main.py", line 375, in main
    urls = self.select_urls(hosts, options)
  File "/usr/lib64/python3.4/site-packages/mirrorselect/main.py", line 324, in select_urls
    selector = Deep(hosts, options, self.output)
  File "/usr/lib64/python3.4/site-packages/mirrorselect/selectors.py", line 226, in __init__
    self.deeptest()
  File "/usr/lib64/python3.4/site-packages/mirrorselect/selectors.py", line 253, in deeptest
    mytime, ignore = self.deeptime(host, maxtime)
  File "/usr/lib64/python3.4/site-packages/mirrorselect/selectors.py", line 305, in deeptime
    addr_family, socket.SOCK_STREAM, 0, socket.AI_ADDRCONFIG):
  File "/usr/lib64/python3.4/site-packages/mirrorselect/selectors.py", line 199, in timeout_handler
    raise TimeoutException()
mirrorselect.selectors.TimeoutException
Comment 5 Brian Dolbec (RETIRED) gentoo-dev 2015-11-25 03:26:03 UTC
No, the timeout exception you got was a result of the name resolution error.

Your error was not due to the file being missing, it was due to the name resolution (DNS) failure.  Maybe it picked a bad mirror that was offline or failed.  The list may not have been updated yet.  It is a manually edited list.

If you find a mirror that continues to fail, please report it in our bugzilla so the infrastructure team can take care of it.


Side note: 

You can also specify any file known to exist on the mirrors.  See the options.  That way you can test larger or smaller files.


I forgot about this bug...

I should be able to close it as resolved since the ebuild has the test file listed in the SRC_URI for a long time.
Comment 6 jms 2015-11-25 09:18:57 UTC
(In reply to Brian Dolbec from comment #5)
> No, the timeout exception you got was a result of the name resolution error.
> 
> Your error was not due to the file being missing, it was due to the name
> resolution (DNS) failure.  Maybe it picked a bad mirror that was offline or
> failed.  The list may not have been updated yet.  It is a manually edited
> list.
> 
> If you find a mirror that continues to fail, please report it in our
> bugzilla so the infrastructure team can take care of it.
> 
> 
> Side note: 
> 
> You can also specify any file known to exist on the mirrors.  See the
> options.  That way you can test larger or smaller files.
> 
> 
> I forgot about this bug...
> 
> I should be able to close it as resolved since the ebuild has the test file
> listed in the SRC_URI for a long time.

Thanks for you reply
yes increasing to -t20 solve the issue
mirrorselect -s5 -b10 -o -D -t20

I wonder if -t20 should be set as default to avoid Bug 523312 mirrorselect.selectors.TimeoutException

> I forgot about this bug...
> 
> I should be able to close it as resolved since the ebuild has the test file
> listed in the SRC_URI for a long time.
please do so to compensate me opening new ones Bug 566778
;)
Thanks