This might be a case of YMMV, but currently I'm unable to emerge --sync because kps://hkps.pool.sks-keyservers.net seems to be down: -------------------------------8<-------------------------------- * Using keys from /usr/share/openpgp-keys/gentoo-release.asc * Refreshing keys from keyserver ...OpenPGP keyring refresh failed: gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net gpg: keyserver refresh failed: Connection refused OpenPGP keyring refresh failed: gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net gpg: keyserver refresh failed: Connection refused OpenPGP keyring refresh failed: gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net gpg: keyserver refresh failed: Connection refused OpenPGP keyring refresh failed: gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net gpg: keyserver refresh failed: Connection refused OpenPGP keyring refresh failed: gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net gpg: keyserver refresh failed: Connection refused OpenPGP keyring refresh failed: gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net gpg: keyserver refresh failed: Connection refused OpenPGP keyring refresh failed: gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net gpg: keyserver refresh failed: Connection refused OpenPGP keyring refresh failed: gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net gpg: keyserver refresh failed: Connection refused <wait forever> ------------------->8--------------------------------------------- In my opinion emerge shouldn't wait forever/stall in this rather rare case - after a couple of tries it should give up trying to refresh the keys and continue syncing - using the previously fetched keys, because a key change/update isn't that common, so there's usually no problem when occasionally a key refresh doesn't work.
Oh, I forgot to mention portage's version: 2.3.40
While your key refresh issue appears to be persistent, there are other key refresh issues that are intermittent and can benefit from a large number of retries. The retries can be configured in /etc/portage/repos.conf, see bug 649276, comment 7.
Actually you've probably got some problem other than hkps.pool.sks-keyservers.net, since it actually refers to a number of hosts: > $ host hkps.pool.sks-keyservers.net > hkps.pool.sks-keyservers.net has address 193.164.133.100 > hkps.pool.sks-keyservers.net has address 193.224.163.43 > hkps.pool.sks-keyservers.net has address 176.9.147.41 > hkps.pool.sks-keyservers.net has address 68.187.0.77 > hkps.pool.sks-keyservers.net has address 51.15.53.138 > hkps.pool.sks-keyservers.net has address 18.191.65.131 > hkps.pool.sks-keyservers.net has address 37.191.226.104 > hkps.pool.sks-keyservers.net has IPv6 address 2600:1f16:41e:bd0a::73:6b73 > hkps.pool.sks-keyservers.net has IPv6 address 2001:bc8:4700:2300::10:f15 > hkps.pool.sks-keyservers.net has IPv6 address 2a02:c205:3001:3626::1 > hkps.pool.sks-keyservers.net has IPv6 address 2001:738:0:600:216:3eff:fe02:42
(In reply to Zac Medico from comment #2) > While your key refresh issue appears to be persistent, there are other key > refresh issues that are intermittent and can benefit from a large number of > retries. The retries can be configured in /etc/portage/repos.conf, see bug > 649276, comment 7. Thanks, I was unaware of this option. Manually syncing (without adjusting the options) isn't really an issue, just an inconvenience - but non-interactive one (e.g. cronjobs/systemd timers) might be if the process waits forever and won't continue/terminate. I'm wondering if something like a deadlock timer would be a good idea - if the emerge --sync process doesn't finish in (for example) 10 minutes it should be killed/timed out. But this might be more of a feature request.
(In reply to Zac Medico from comment #3) > Actually you've probably got some problem other than > hkps.pool.sks-keyservers.net, since it actually refers to a number of hosts: > > > $ host hkps.pool.sks-keyservers.net > > hkps.pool.sks-keyservers.net has address 193.164.133.100 > > hkps.pool.sks-keyservers.net has address 193.224.163.43 > > hkps.pool.sks-keyservers.net has address 176.9.147.41 > > hkps.pool.sks-keyservers.net has address 68.187.0.77 > > hkps.pool.sks-keyservers.net has address 51.15.53.138 > > hkps.pool.sks-keyservers.net has address 18.191.65.131 > > hkps.pool.sks-keyservers.net has address 37.191.226.104 > > hkps.pool.sks-keyservers.net has IPv6 address 2600:1f16:41e:bd0a::73:6b73 > > hkps.pool.sks-keyservers.net has IPv6 address 2001:bc8:4700:2300::10:f15 > > hkps.pool.sks-keyservers.net has IPv6 address 2a02:c205:3001:3626::1 > > hkps.pool.sks-keyservers.net has IPv6 address 2001:738:0:600:216:3eff:fe02:42 Could very well be - on another machine (outside of the company's network) emerge --sync (including fetching the keys) works, so I guess there's some screw up in the net/firewall setup here. However, it's a nice edge case test :-)
(In reply to Mark Nowiasz from comment #4) > (In reply to Zac Medico from comment #2) > > While your key refresh issue appears to be persistent, there are other key > > refresh issues that are intermittent and can benefit from a large number of > > retries. The retries can be configured in /etc/portage/repos.conf, see bug > > 649276, comment 7. > > Thanks, I was unaware of this option. Manually syncing (without adjusting > the options) isn't really an issue, just an inconvenience - but > non-interactive one (e.g. cronjobs/systemd timers) might be if the process > waits forever and won't continue/terminate. I'm wondering if something like > a deadlock timer would be a good idea - if the emerge --sync process doesn't > finish in (for example) 10 minutes it should be killed/timed out. But this > might be more of a feature request. Yeah sure we can add a deadlock timer, but we already have this setting for the key refresh step: sync-openpgp-key-refresh-retry-overall-timeout = 1200 Combined time limit for all refresh attempts, in units of seconds.
There are are 2 more timeouts, there's the rsync --timeout=180 setting that can be overridden via the sync-rsync-extra-opts setting in repos.conf, and there's also a make.conf PORTAGE_RSYNC_INITIAL_TIMEOUT=15 seconds setting that limits the time to create a connection. But yeah, some kind of overall timeout makes sense. However, the key refresh operates a pool of sks servers, and rsync operates on a separate pool of servers, so it also makes sense to have separate timeouts.
Given the last comments, this is slightly diverging from the original problem, so let me add my 2 cents. I have been poking the portage code for similar case and the relevant code is here: https://github.com/gentoo/portage/blob/master/pym/portage/sync/modules/rsync/rsync.py#L148 From a security standpoint (with performance in mind), isn't refreshing the keys every sync a bit too much? While retries and timeouts can be specified, I guess we need another setting based on the age of the cashed keys; I guess a default value "sync-openpgp-key-max-age = 2,629,746 # 1 month" should be fine for keys expected to be updated once per year. So the logic is, never try to refresh keys unless they are older than 1 month; when refreshing, use all the sync-openpgp-key-refresh-retry-* options as now. Fail if keys are older and cannot be refreshed.
The key refresh should be relatively fast and inexpensive compared to the rsync call, so trying to avoid refreshing the keys should not be worthwhile. I suppose an optimal configuration would be to use a local sks service that synchronizes data from the sks-keyservers.net pool in the background. In order to optimize for this sort of configuration, Gentoo should provide its own sks pool, in order to minimize the number of keys that users have to synchronize. (In reply to Kalin KOZHUHAROV from comment #8) > So the logic is, never try to refresh keys unless they are older than 1 > month; when refreshing, use all the sync-openpgp-key-refresh-retry-* options > as now. Fail if keys are older and cannot be refreshed. The thing is, if the keys are suddenly revoked due to a breach on the 7th day of the month, you'll be vulnerable for the remaining 21 days of the month.
Tn the original post in this bug the keyserver unavailability was described as a "rather rare case". On the contrary: I run into it every single day. I was going to report a bug about the thing being down all the time when I found this bug.
In reply to Carlos Konstanski from comment #10) > ... On the contrary: I run into it every single day ... +1 Cross-Reference: please c.f. Bug 647696 - portage: gpg: keyserver refresh failed: No data and its comments https://bugs.gentoo.org/647696#c10 ff. (In reply to Zac Medico from comment #9) > The key refresh should be relatively fast and inexpensive ... ..................^^^^^^ Zac, I deeply agree. But in fact, consulting via dns01.mnet-online.de , # host hkps.pool.sks-keyservers.net delivers like an infantile first-try random number generator :-( Any hints about different behaviour depending upon "source" of request?
In app-portage/gemato-14.0, keys are fetched via WKD by default, and it only falls back to hkps if one or more keys in the keychain (provided by app-crypt/openpgp-keys-gentoo-release) fails to import from WKD: https://github.com/mgorny/gemato/commit/909390c25a0ab589a4ae10d20cb9e321a51163b2
WORKSFORME: Upgrading from app-portage/gemato-13.0-r1:0 to (~) app-portage/gemato-14.0:0 seems to verify (Zac Medico's comment #9) > The key refresh should be relatively fast and inexpensive compared to the > rsync call, so trying to avoid refreshing the keys should not be worthwhile. Thanks a lot!
The original issue was related to hkps://hkps.pool.sks-keyservers.net, but defaults have since changed to use WKD with hkps://keys.gentoo.org fallback.
