I'm seeing a lot of 404 responses when I try to fetch distfiles for mail-client/aerc. Fetching the files from the upstream location works fine. % ebuild aerc-0.3.0-r1.ebuild clean fetch >>> Downloading 'http://distfiles.gentoo.org/distfiles/ca/aerc-0.3.0.tar.gz' * aerc-0.3.0.tar.gz BLAKE2B SHA512 size ;-) ... [ ok ] >>> Downloading 'http://distfiles.gentoo.org/distfiles/26/cloud.google.com%2Fgo%2F@v%2Fv0.34.0.mod' curl: (22) The requested URL returned error: 404 Not Found >>> Downloading 'https://proxy.golang.org//cloud.google.com/go/@v/v0.34.0.mod' * cloud.google.com%2Fgo%2F@v%2Fv0.34.0.mod BLAKE2B SHA512 size ;-) ... [ ok ] >>> Downloading 'http://distfiles.gentoo.org/distfiles/e3/git.sr.ht%2F~sircmpwn%2Fgetopt%2F@v%2Fv0.0.0-20190621174457-292febf82fd0.zip' curl: (22) The requested URL returned error: 404 Not Found >>> Downloading 'https://proxy.golang.org//git.sr.ht/~sircmpwn/getopt/@v/v0.0.0-20190621174457-292febf82fd0.zip' * git.sr.ht%2F~sircmpwn%2Fgetopt%2F@v%2Fv0.0.0-20190621174457-292febf82fd0.zip BLAKE2B SHA512 size ;-) ... [ ok ] >>> Downloading 'http://distfiles.gentoo.org/distfiles/3a/git.sr.ht%2F~sircmpwn%2Fgetopt%2F@v%2Fv0.0.0-20190621174457-292febf82fd0.mod' curl: (22) The requested URL returned error: 404 Not Found >>> Downloading 'https://goproxy.io//git.sr.ht/~sircmpwn/getopt/@v/v0.0.0-20190621174457-292febf82fd0.mod' * git.sr.ht%2F~sircmpwn%2Fgetopt%2F@v%2Fv0.0.0-20190621174457-292febf82fd0.mod BLAKE2B SHA512 size ;-) ... [ ok ] >>> Downloading 'http://distfiles.gentoo.org/distfiles/a9/git.sr.ht%2F~sircmpwn%2Fpty%2F@v%2Fv0.0.0-20190330154901-3a43678975a9.zip' curl: (22) The requested URL returned error: 404 Not Found >>> Downloading 'https://goproxy.io//git.sr.ht/~sircmpwn/pty/@v/v0.0.0-20190330154901-3a43678975a9.zip' * git.sr.ht%2F~sircmpwn%2Fpty%2F@v%2Fv0.0.0-20190330154901-3a43678975a9.zip BLAKE2B SHA512 size ;-) ... [ ok ]
This is portage's fault; it needs to percent-encode percent signs in the mirrored distfile name before passing it to FETCHCOMMAND. For example, instead of this: http://distfiles.gentoo.org/distfiles/26/cloud.google.com%2Fgo%2F@v%2Fv0.34.0.mod We need to fetch this: http://distfiles.gentoo.org/distfiles/26/cloud.google.com%252Fgo%252F@v%252Fv0.34.0.mod
Note that this double percent-encoding should probably only happen when fetching from mirrors. SRC_URI may contain percent-encoded URIs that should not be double-encoded by portage.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=4c18f523bb86a8be4c148f365dabee06fca2e4fa commit 4c18f523bb86a8be4c148f365dabee06fca2e4fa Author: Mike Gilbert <floppym@gentoo.org> AuthorDate: 2020-05-31 13:11:41 +0000 Commit: Mike Gilbert <floppym@gentoo.org> CommitDate: 2020-05-31 20:32:46 +0000 Escape percent-signs in filename when fetching from mirrors Bug: https://bugs.gentoo.org/719810 Signed-off-by: Mike Gilbert <floppym@gentoo.org> lib/portage/package/ebuild/fetch.py | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=54d572cbc3ef2b646283b826813a47e5268bf734 commit 54d572cbc3ef2b646283b826813a47e5268bf734 Author: Mike Gilbert <floppym@gentoo.org> AuthorDate: 2020-05-31 21:15:29 +0000 Commit: Mike Gilbert <floppym@gentoo.org> CommitDate: 2020-05-31 21:15:29 +0000 Revert "Escape percent-signs in filename when fetching from mirrors" This probably breaks emirrordist; we will need separate methods for layout.get_path() when it is used client-side. This reverts commit 4c18f523bb86a8be4c148f365dabee06fca2e4fa. Bug: https://bugs.gentoo.org/719810 Signed-off-by: Mike Gilbert <floppym@gentoo.org> lib/portage/package/ebuild/fetch.py | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=c238d5f7ed264179c263f5a2da983c4ee50b4f00 commit c238d5f7ed264179c263f5a2da983c4ee50b4f00 Author: Mike Gilbert <floppym@gentoo.org> AuthorDate: 2020-05-31 21:24:25 +0000 Commit: Mike Gilbert <floppym@gentoo.org> CommitDate: 2020-05-31 21:30:14 +0000 Escape percent-signs in portage.package.ebuild.fetch.get_mirror_url() This avoids double-escaping in emirrordist. We only want to escape the path when fetching the file from the mirror, not when mirroring the file. Bug: https://bugs.gentoo.org/719810 Fixes: 4c18f523bb86a8be4c148f365dabee06fca2e4fa Signed-off-by: Mike Gilbert <floppym@gentoo.org> lib/portage/package/ebuild/fetch.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=be77cd7fe8703bb94f2cc289c395d4dd6798928b commit be77cd7fe8703bb94f2cc289c395d4dd6798928b Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2020-06-03 07:45:18 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2020-06-03 07:55:40 +0000 sys-apps/portage: 2.3.100-r1 revbump for bug 719810 #719810 fix fetch for file names with percent-encoded characters Bug: https://bugs.gentoo.org/721152 Bug: https://bugs.gentoo.org/719810 Package-Manager: Portage-2.3.100, Repoman-2.3.22 Signed-off-by: Zac Medico <zmedico@gentoo.org> sys-apps/portage/Manifest | 1 + .../portage/{portage-2.3.100.ebuild => portage-2.3.100-r1.ebuild} | 5 ++++- 2 files changed, 5 insertions(+), 1 deletion(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=5330fc62703c0e2c535eb2ac8247d43faf501c8e commit 5330fc62703c0e2c535eb2ac8247d43faf501c8e Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2020-06-14 23:33:47 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2020-06-14 23:47:09 +0000 sys-apps/portage: Bump to version 2.3.101 #661518 repos.conf: Add bool sync-openpgp-key-refresh option #709746 New PORTAGE_LOG_FILTER_FILE_CMD variable specifies a command that filters build log output to a log file #719810 Escape percent-signs in mirror url #725934 _better_cache._scan_cat: Avoid stat calls #728046 ecompress: Prefix eqawarn messages with QA Notice Bug: https://bugs.gentoo.org/721152 Bug: https://bugs.gentoo.org/661518 Bug: https://bugs.gentoo.org/709746 Bug: https://bugs.gentoo.org/719810 Bug: https://bugs.gentoo.org/725934 Bug: https://bugs.gentoo.org/728046 Package-Manager: Portage-2.3.101, Repoman-2.3.22 Signed-off-by: Zac Medico <zmedico@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-2.3.101.ebuild | 263 ++++++++++++++++++++++++++++++++ 2 files changed, 264 insertions(+)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=20b6a36fbf791bcc7d42bd429cf8116399a014a7 commit 20b6a36fbf791bcc7d42bd429cf8116399a014a7 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2020-09-12 21:23:28 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2020-09-12 22:08:45 +0000 get_mirror_url: urlquote only for ftp, http, and https (bug 741474) It's necessary to use urlquote for correct behavior with ftp, http, and https, since it's possible for file names to contain percent encoded characters that need to be protected by an additional layer of percent encoding. For other protocols such as sftp and rsync, all characters are interpreted literally, so urlquote must not be used. Fixes: c238d5f7ed264179c263f5a2da983c4ee50b4f00 Bug: https://bugs.gentoo.org/719810 Bug: https://bugs.gentoo.org/741474 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/portage/package/ebuild/fetch.py | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
