Gentoo prefix bootstrap script fails at downloading portage on M2 mac. * Fetching portage-20230709.tar.bz2 wget --no-check-certificate http://distfiles.gentoo.org/distfiles/portage-20230709.tar.bz2 --2023-09-01 17:38:54-- http://distfiles.gentoo.org/distfiles/portage-20230709.tar.bz2 Resolving distfiles.gentoo.org... 89.187.177.16, 156.146.36.23 Connecting to distfiles.gentoo.org|89.187.177.16|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2023-09-01 17:38:54 ERROR 404: Not Found. wget --no-check-certificate http://distfiles.prefix.bitzolder.nl/distfiles/portage-20230709.tar.bz2 URL transformed to HTTPS due to an HSTS policy --2023-09-01 17:38:54-- https://distfiles.prefix.bitzolder.nl/distfiles/portage-20230709.tar.bz2 Resolving distfiles.prefix.bitzolder.nl... 45.137.90.8 Connecting to distfiles.prefix.bitzolder.nl|45.137.90.8|:443... connected. HTTP request sent, awaiting response... 302 Look Elsewhere Location: http://distfiles.gentoo.org/distfiles/d0/portage-20230709.tar.bz2 [following] --2023-09-01 17:38:55-- http://distfiles.gentoo.org/distfiles/d0/portage-20230709.tar.bz2 Resolving distfiles.gentoo.org... 156.146.36.23, 89.187.177.16 Connecting to distfiles.gentoo.org|156.146.36.23|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2023-09-01 17:38:55 ERROR 404: Not Found. wget --no-check-certificate http://distfiles.prefix.bitzolder.nl/prefix/distfiles/portage-20230709.tar.bz2 URL transformed to HTTPS due to an HSTS policy --2023-09-01 17:38:55-- https://distfiles.prefix.bitzolder.nl/prefix/distfiles/portage-20230709.tar.bz2 Resolving distfiles.prefix.bitzolder.nl... 45.137.90.8 Connecting to distfiles.prefix.bitzolder.nl|45.137.90.8|:443... connected. HTTP request sent, awaiting response... 302 Look Elsewhere Location: https://distfiles.prefix.bitzolder.nl/prefix/distfiles/d0/portage-20230709.tar.bz2 [following] --2023-09-01 17:38:55-- https://distfiles.prefix.bitzolder.nl/prefix/distfiles/d0/portage-20230709.tar.bz2 Reusing existing connection to distfiles.prefix.bitzolder.nl:443. HTTP request sent, awaiting response... 404 Not Found 2023-09-01 17:38:55 ERROR 404: Not Found. wget --no-check-certificate http://rsync.prefix.bitzolder.nl/snapshots/portage-20230709.tar.bz2 --2023-09-01 17:38:56-- http://rsync.prefix.bitzolder.nl/snapshots/portage-20230709.tar.bz2 Resolving rsync.prefix.bitzolder.nl... 77.172.37.201, 45.137.90.8 Connecting to rsync.prefix.bitzolder.nl|77.172.37.201|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2023-09-01 17:38:56 ERROR 404: Not Found. !!! downloading http://rsync.prefix.bitzolder.nl/snapshots/portage-20230709.tar.bz2 failed! Reproducible: Always Actual Results: stage1 fails
Yeah, it only seems to have snapshots from August there...
I looked into: https://rsync.prefix.bitzolder.nl/snapshots/ But SSL cert is not valid and even if I go insecure way I see 404 for it. DNS resolution from my end: $ dig +short A rsync.prefix.bitzolder.nl 45.137.90.8 77.172.37.201
77.172.37.201 fails for me, but 45.137.90.8 is OK
this is 2-way stupid I changed the retrieval to use the rsync slave, but never thought of that the snapshots are gone of course Second is the failure of rsync2, will look into it. I'm affraid it got blocked :(
rsync2 seems fine now grobian@woodpecker ~ $ rsync rsync://77.172.37.201 ___| | _ \ _|_) | _ \ __ \ __| _ \ _ \ | | __| _ \ | |\ \ / | | __/ | | | ( | ( | ___/ | __/ __| | ` < \____|\___|_| _|\__|\___/ \___/ _| _| \___|_| _| _/\_\ Welcome to rsync2.prefix.bitzolder.nl. This server generates its own tree, refreshed at 26,56 of every hour. In case of trouble, please contact: grobian@gentoo.org gentoo-portage-prefix Gentoo Prefix Portage tree mirror
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/proj/prefix.git/commit/?id=e392ba57a6cbcd5401f53c5a8a3fc9bd304f2040 commit e392ba57a6cbcd5401f53c5a8a3fc9bd304f2040 Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2023-09-01 15:25:29 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2023-09-01 15:25:29 +0000 scripts/bootstrap-prefix: only use Prefix mirrors for Prefix tree Bug: https://bugs.gentoo.org/913415 Signed-off-by: Fabian Groffen <grobian@gentoo.org> scripts/bootstrap-prefix.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
ok, the file should now be downloadable grobian@woodpecker ~ $ curl -I "https://distfiles.prefix.bitzolder.nl/prefix/distfiles/d0/portage-20230709.tar.bz2" HTTP/2 200 server: nginx/1.25.2 date: Fri, 01 Sep 2023 15:30:32 GMT content-type: application/octet-stream content-length: 52221658 last-modified: Fri, 01 Sep 2023 15:29:32 GMT etag: "64f2035c-31cd6da" strict-transport-security: max-age=31536000 accept-ranges: bytes thanks
Seems like DNS issue still not fixed: $ curl -k --connect-to "rsync.prefix.bitzolder.nl:443:77.172.37.201:443" https://rsync.prefix.bitzolder.nl/snapshots/portage-20230709.tar.bz2 <html> <head><title>404 Not Found</title></head> <body> <center><h1>404 Not Found</h1></center> <hr><center>nginx/1.25.2</center> </body> </html>
the file is not on the rsync slave, it's on the mirror https://distfiles.prefix.bitzolder.nl/prefix/distfiles/portage-20230709.tar.bz2 cert is Let's Encrypt signed
@Fabian, DNS for me resolves to 2 ips still: dig +short rsync.prefix.bitzolder.nl 45.137.90.8 77.172.37.201 One of them works just fine, another fails: cono@cmac ~ $ curl --head --connect-to "rsync.prefix.bitzolder.nl:443:45.137.90.8:443" https://rsync.prefix.bitzolder.nl/snapshots/ HTTP/2 200 server: nginx/1.25.2 date: Sat, 02 Sep 2023 14:59:19 GMT content-type: text/html strict-transport-security: max-age=31536000 cono@cmac ~ $ curl --head --connect-to "rsync.prefix.bitzolder.nl:443:77.172.37.201:443" https://rsync.prefix.bitzolder.nl/snapshots/ curl: (60) SSL: no alternative certificate subject name matches target host name 'rsync.prefix.bitzolder.nl' More details here: https://curl.se/docs/sslcerts.html curl failed to verify the legitimacy of the server and therefore could not establish a secure connection to it. To learn more about this situation and how to fix it, please visit the web page mentioned above. Certificate which returned from this IP issues for another CN: $ openssl s_client -connect 77.172.37.201:443 -servername rsync.prefix.bitzolder.nl < /dev/null | openssl x509 -noout -text | grep bitzolder depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1 verify return:1 depth=1 C = US, O = Let's Encrypt, CN = R3 verify return:1 depth=0 CN = hera.orakel.bitzolder.nl verify return:1 DONE Subject: CN = hera.orakel.bitzolder.nl DNS:hera.orakel.bitzolder.nl
grobian is saying the new script shouldn't even be trying to fetch from there. are you using the latest copy?
https://gitweb.gentoo.org/repo/proj/prefix.git/tree/scripts/bootstrap-prefix.sh#n2252 For me it looks like it still uses rsync.prefix.bitzolder.nl
(In reply to cono from comment #12) > https://gitweb.gentoo.org/repo/proj/prefix.git/tree/scripts/bootstrap-prefix. > sh#n2252 > > For me it looks like it still uses rsync.prefix.bitzolder.nl It does, when the mirror fails. We need this for testing newer and latest snapshots. I'll look into the cert failure for rsync2. Thanks for pointing it out!
There was a typo that caused the nginx config not to be generated correctly, in that server_name did not contain rsync.prefix.b.n for rsync2. This is fixed now, thanks!
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/proj/prefix.git/commit/?id=69b7e2d62be6532978740e3e18b72b8fa8d0788d commit 69b7e2d62be6532978740e3e18b72b8fa8d0788d Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2023-09-03 07:31:57 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2023-09-03 07:31:57 +0000 scripts: make bootstrap snapshot available on rsync slaves In bug #913415 a bunch of confusing and wrong things came out, and in an attempt to fix this, the historical "prefix-overlay" fetched from dev was replaced with ordinary "portage" from rsync snapshots dir. However, this relies on the prefix mirror to provide the file, which can easily be overridden by the user. So retrieve the snapshot in use and retain it on the rsync slaves to make this in the very least less confusing for those following the locations, and ensure it works for people with custom mirror setups. Bug: https://bugs.gentoo.org/913415 Signed-off-by: Fabian Groffen <grobian@gentoo.org> scripts/bootstrap-prefix.sh | 3 +++ scripts/rsync-generation/mksnapshot.sh | 36 ++++++++++++++++++++++------------ 2 files changed, 27 insertions(+), 12 deletions(-)
