gemato 14.x and prior used to attempt WKD via https://gentoo.org/, and then fail over to hkps://keys.gentoo.org/ if that failed. But the current versions in the tree - 15.2 (stable) and 16.1 (unstable) both abort immediately if they cannot connect to gentoo.org:443 - at least when behind a restrictive web proxy. Discovered this in an environment where CONNECT gentoo.org:443 was not permitted, but CONNECT keys.gentoo.org:443 was - which worked fine previously. The behavior with older gemato's: # equery l portage gemato * Searching for portage ... [IP-] [ ] sys-apps/portage-3.0.8:0 * Searching for gemato ... [I--] [??] app-portage/gemato-14.5:0 # . /etc/portage/make.conf && export http_proxy && /usr/bin/emerge --sync >>> Syncing repository 'gentoo' into '/usr/portage'... * Using keys from /usr/share/openpgp-keys/gentoo-release.asc * Refreshing keys via WKD ... [ !! ] * Refreshing keys from keyserver hkps://keys.gentoo.org ... [ ok ] [SNIP] [ The make.conf dance was required because old Gemato did not know how to find proxy settings from make.conf itself; it got support for that recently, see https://github.com/gentoo/portage/pull/607 ] But versions 15.2 (stable) and 16.1 (unstable) both abort after the first attempt fails: # equery l portage gemato * Searching for portage ... [IP-] [ ] sys-apps/portage-3.0.8:0 * Searching for gemato ... [IP-] [ ] app-portage/gemato-16.1:0 # /usr/bin/emerge --sync >>> Syncing repository 'gentoo' into '/usr/portage'... * Using keys from /usr/share/openpgp-keys/gentoo-release.asc * Refreshing keys via WKD ...Traceback (most recent call last): File "/usr/lib/python3.7/site-packages/urllib3/connectionpool.py", line 667, in urlopen self._prepare_proxy(conn) File "/usr/lib/python3.7/site-packages/urllib3/connectionpool.py", line 932, in _prepare_proxy conn.connect() File "/usr/lib/python3.7/site-packages/urllib3/connection.py", line 317, in connect self._tunnel() File "/usr/lib/python3.7/http/client.py", line 927, in _tunnel message.strip())) OSError: Tunnel connection failed: 403 Forbidden During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.7/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/usr/lib/python3.7/site-packages/urllib3/connectionpool.py", line 727, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "/usr/lib/python3.7/site-packages/urllib3/util/retry.py", line 439, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='gentoo.org', port=443): Max retries exceeded with url: /.well-known/openpgpkey/hu/9tik1tjkx1fe3wke63tpd7iikyogfbtq?l=repomirrorci (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 403 Forbidden'))) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.7/site-packages/portage/util/_async/AsyncFunction.py", line 39, in _run result = self.target(*(self.args or []), **(self.kwargs or {})) File "/usr/lib/python3.7/site-packages/portage/sync/controller.py", line 165, in sync taskmaster.run_tasks(tasks, func, status, options=task_opts) File "/usr/lib/python3.7/site-packages/portage/sync/controller.py", line 65, in run_tasks result = getattr(inst, func)(**kwargs) File "/usr/lib/python3.7/site-packages/portage/sync/modules/webrsync/webrsync.py", line 89, in sync self._refresh_keys(openpgp_env) File "/usr/lib/python3.7/site-packages/portage/sync/syncbase.py", line 268, in _refresh_keys if openpgp_env.refresh_keys_wkd(): File "/usr/lib/python3.7/site-packages/gemato/openpgp.py", line 419, in refresh_keys_wkd resp = requests.get(url, proxies=proxies) File "/usr/lib/python3.7/site-packages/requests/api.py", line 76, in get return request('get', url, params=params, **kwargs) File "/usr/lib/python3.7/site-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/usr/lib/python3.7/site-packages/requests/sessions.py", line 530, in request resp = self.send(prep, **send_kwargs) File "/usr/lib/python3.7/site-packages/requests/sessions.py", line 643, in send r = adapter.send(request, **kwargs) File "/usr/lib/python3.7/site-packages/requests/adapters.py", line 510, in send raise ProxyError(e, request=request) requests.exceptions.ProxyError: HTTPSConnectionPool(host='gentoo.org', port=443): Max retries exceeded with url: /.well-known/openpgpkey/hu/9tik1tjkx1fe3wke63tpd7iikyogfbtq?l=repomirrorci (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 403 Forbidden'))) Action: sync for repo: gentoo, returned code = 1 # gemato has an internal knob allow_wkd, but I can't figure out the right way to turn that when gemato is used by emerge, or to otherwise tell portage not to try WKD. If I hack up /usr/lib/python3.7/site-packages/portage/sync/syncbase.py to comment out the 'Refreshing keys via WKD' section entirely, it proceeds to refreshing from keyserver and works fine.
My bad. The code only accounts for non-200 status but I've completely forgotten to handle connect failures etc.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=eeda16253769493db3bf50be19d7392302a5e89b commit eeda16253769493db3bf50be19d7392302a5e89b Author: Michał Górny <mgorny@gentoo.org> AuthorDate: 2020-10-01 12:17:09 +0000 Commit: Michał Górny <mgorny@gentoo.org> CommitDate: 2020-10-01 12:22:00 +0000 app-portage/gemato: Bump to 16.2 Bug: https://bugs.gentoo.org/745771 Signed-off-by: Michał Górny <mgorny@gentoo.org> app-portage/gemato/Manifest | 1 + app-portage/gemato/gemato-16.2.ebuild | 43 +++++++++++++++++++++++++++++++++++ 2 files changed, 44 insertions(+)
Could you confirm that 16.2 solves your issue? If it does, I'll fast-stabilize it.
Yes! Thanks @mgorny, with gemato-16.2 I once again get the expected behavior: # equery l gemato portage * Searching for gemato ... [I-O] [ ] app-portage/gemato-16.2:0 * Searching for portage ... [IP-] [ ] sys-apps/portage-3.0.8:0 # emerge --sync >>> Syncing repository 'gentoo' into '/usr/portage'... * Using keys from /usr/share/openpgp-keys/gentoo-release.asc * Refreshing keys via WKD ... [ !! ] * Refreshing keys from keyserver hkps://keys.gentoo.org ... [ ok ] Fetching most recent snapshot ... [snip]
Arch teams, please fast-stabilize this version.
amd64 arm arm64 hppa ppc ppc64 sparc x86 (ALLARCHES) done all arches done
