Bug 914457 – app-portage/mirrorselect: enhancement requests

• Product: Gentoo Linux
• Status: CONFIRMED ---

[Description Toralf Förster 2023-09-20 08:44:01 UTC]

IMO mirrorselect should avoid using 8 servers from the same /24 subnet, e.g.:

< GENTOO_MIRRORS="https://mirrors.aliyun.com/gentoo/ \
<     https://47.246.44.230/gentoo/ \
<     https://47.246.44.231/gentoo/ \
<     https://47.246.44.228/gentoo/ \
<     https://47.246.44.225/gentoo/ \
<     https://47.246.44.226/gentoo/ \
<     https://47.246.44.227/gentoo/ \
<     https://47.246.44.224/gentoo/"


Furthermore it should check if an ip address and a hostname are identical, e.g. for fau:

GENTOO_MIRRORS="https://ftp-stud.hs-esslingen.de/pub/Mirrors/gentoo/ \
    https://155.4.110.241/gentoo/gentoo-distfiles/ \
    https://137.226.34.46/gentoo/ \
    https://ftp.fau.de/gentoo \
    https://129.143.116.10/pub/Mirrors/gentoo/ \
    https://mirror.netcologne.de/gentoo/ \
    https://131.188.12.211/gentoo \
    https://ftp.lysator.liu.se/gentoo/"

$> host ftp.fau.de
ftp.fau.de has address 131.188.12.211
ftp.fau.de has IPv6 address 2001:638:a000:1021:21::1

[Comment 1 Toralf Förster 2023-09-20 09:53:30 UTC]

I run tehse 2 commands in the row :

mirrorselect -H -s 8 -a
mirrorselect -H -s 8

and got now:

$> mirrorselect -H -s 8
* Using url: https://api.gentoo.org/mirrors/distfiles.xml
* Limiting test to http hosts. 
* Downloading a list of mirrors...
 Got 251 mirrors.
* Using netselect to choose the top 8 mirrors...Done.
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.11/mirrorselect", line 55, in <module>
    MirrorSelect().main(sys.argv)
  File "/usr/lib/python3.11/site-packages/mirrorselect/main.py", line 469, in main
    self.change_config(
  File "/usr/lib/python3.11/site-packages/mirrorselect/main.py", line 107, in change_config
    hosts[i] = hosts[i].decode("utf-8")
               ^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xcf in position 37: unexpected end of data

[Comment 2 Toralf Förster 2023-09-20 11:15:58 UTC]

Created attachment 871005 [details]
reproducer for the utf-8 issue

[Comment 3 Toralf Förster 2023-09-20 11:18:00 UTC]

Comment on attachment 871005 [details]
reproducer for the utf-8 issue

wrong file

[Comment 4 Toralf Förster 2023-09-20 11:33:26 UTC]

(In reply to Toralf Förster from comment #3)
> Comment on attachment 871005 [details]
> reproducer for the utf-8 issue
> 
> wrong file

My fault, I interrupted mirrorselect before so garbage landed into make.conf. Should mirrorselect be hardened to not be fooled by mess produced by the user?

[Comment 5 Toralf Förster 2023-09-20 12:51:08 UTC]

And mirrorselect returns server which itself have issues like:

$ curl https://47.246.44.231/gentoo
curl: (60) SSL: no alternative certificate subject name matches target host name '47.246.44.231'
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.