Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 873133 - emerge --sync stucks on getting key via WKD
Summary: emerge --sync stucks on getting key via WKD
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Infrastructure
Classification: Unclassified
Component: Other (show other bugs)
Hardware: All Linux
: Normal major (vote)
Assignee: Gentoo Infrastructure
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-09-27 06:29 UTC by Rafal Kupiec
Modified: 2023-08-17 06:52 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Rafal Kupiec 2022-09-27 06:29:03 UTC
# emerge --sync
>>> Syncing repository 'gentoo' into '/usr/repos/gentoo-portage'...
 * Using keys from /usr/share/openpgp-keys/gentoo-release.asc
 * Refreshing keys via WKD ...

and stuck already for an hour.

Tried to `gpg --debug all --auto-key-locate wkd -vvvvv --locate-keys developer@gentoo.org` and got:

gpg: error retrieving 'developer@gentoo.org' via WKD: No data
gpg: error reading key: No data

Reproducible: Always

Steps to Reproduce:
1. emerge --sync
2.
3.
Actual Results:  
# gpg --debug all --auto-key-locate wkd -vvvvv --locate-keys developer@gentoo.org
gpg: reading options from '[cmdline]'
gpg: using character set 'utf-8'
gpg: enabled debug flags: packet mpi crypto filter iobuf memory cache memstat trust hashing ipc clock lookup extprog
gpg: DBG: [no clock] start
gpg: /root/.gnupg/trustdb.gpg: trustdb created
gpg: using pgp trust model
gpg: DBG: [no clock] keydb_new
gpg: DBG: [no clock] keydb_search enter
gpg: DBG: keydb_search: 1 search descriptions:
gpg: DBG: keydb_search   0: SUBSTR: 'developer@gentoo.org'
gpg: DBG: internal_keydb_search: searching keybox (resource 0 of 1)
gpg: DBG: internal_keydb_search: searched keybox (resource 0 of 1) => EOF
gpg: DBG: [no clock] keydb_search leave (not found)
gpg: no running dirmngr - starting '/usr/bin/dirmngr'
gpg: waiting for the dirmngr to come up ... (5s)
gpg: DBG: chan_5 <- # Home: /root/.gnupg
gpg: DBG: chan_5 <- # Config: /root/.gnupg/dirmngr.conf
gpg: DBG: chan_5 <- OK Dirmngr 2.3.7 at your service
gpg: connection to the dirmngr established
gpg: DBG: chan_5 -> GETINFO version
gpg: DBG: chan_5 <- D 2.3.7
gpg: DBG: chan_5 <- OK
gpg: DBG: chan_5 -> WKD_GET -- developer@gentoo.org
gpg: DBG: chan_5 <- S SOURCE https://gentoo.org

gpg: DBG: chan_5 <- S WARNING http_redirect_cleanup 0 changed from 'https://gentoo.org/.well-known/openpgpkey/hu/8ssm33j13uke6j94cmw3gbu58o49bf8z?l=developer' to 'https://www.gentoo.org/.well-known/openpgpkey/hu/8ssm33j13uke6j94cmw3gbu58o49bf8z?l=developer'
gpg: WARNING: unacceptable HTTP redirect from server was cleaned up
gpg: (further info: changed from 'https://gentoo.org/.well-known/openpgpkey/hu/8ssm33j13uke6j94cmw3gbu58o49bf8z?l=developer' to 'https://www.gentoo.org/.well-known/openpgpkey/hu/8ssm33j13uke6j94cmw3gbu58o49bf8z?l=developer')
gpg: DBG: chan_5 <- S PROGRESS tick ? 0 0
gpg: DBG: chan_5 <- ERR 167772218 No data <Dirmngr>
gpg: error retrieving 'developer@gentoo.org' via WKD: No data
gpg: error reading key: No data
gpg: DBG: chan_5 -> BYE
gpg: DBG: [no clock] keydb_release
gpg: DBG: [no clock] stop
gpg: keydb: handles=1 locks=0 parse=0 get=0
gpg:        build=0 update=0 insert=0 delete=0
gpg:        reset=0 found=0 not=1 cache=0 not=0
gpg: kid_not_found_cache: count=0 peak=0 flushes=0
gpg: sig_cache: total=0 cached=0 good=0 bad=0
gpg: objcache: keys=0/0/0 chains=0,0..0 buckets=0/0 attic=0
gpg: objcache: uids=0/0/0 chains=0,0..0 buckets=0/0
gpg: random usage: poolsize=600 mixed=0 polls=0/0 added=0/0
              outmix=0 getlvl1=0/0 getlvl2=0/0
gpg: rndjent stat: collector=0x0000000000000000 calls=0 bytes=0
gpg: secmem usage: 0/65536 bytes in 0 blocks
Comment 1 Alexey Shvetsov archtester gentoo-dev 2022-10-17 10:02:56 UTC
I have same issue.
Comment 2 David Sardari 2022-10-26 08:29:40 UTC
app-portage/gemato falls back to hkp/hkps [1] in case of errors while fetching over WKD and uses hkps://keys.gentoo.org:

❯ grep sync-openpgp-keyserver /usr/share/portage/config/repos.conf
sync-openpgp-keyserver = hkps://keys.gentoo.org

The fallback will always occur for "developer@gentoo.org" due to it not being a UID for a valid public key. The hkps keyserver currently just resolves this "UID" to 47 public keys:

❯ export GNUPGHOME="$(mktemp -d)"
❯ gpg --auto-key-locate hkps://keys.gentoo.org --locate-external-keys developer@gentoo.org 2>&1 | tail -n 5
gpg: key 350AAD7C2B859DE3: public key "Christian Faulhammer <christian@faulhammer.org>" imported
gpg: key 979CAF40D0455535: public key "Anthony G. Basile <basile@virtual.dyc.edu>" imported
gpg: Total number processed: 47
gpg:               imported: 47
gpg: error retrieving 'developer@gentoo.org' via hkps://keys.gentoo.org: No fingerprint

Therefore, above output of "--locate-keys" should be fine.

[1] https://github.com/projg2/gemato/blob/805ca36a222c5649b16134e818f8c8b23415c7a2/gemato/openpgp.py#L468-L471
Comment 3 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-10-26 08:33:14 UTC
Right, the problem is that it then hangs
Comment 4 David Sardari 2022-10-26 09:11:40 UTC
In the end, portage/gemato tries to refresh the keys at:

❯ grep sync-openpgp-key-path /usr/share/portage/config/repos.conf
sync-openpgp-key-path = /usr/share/openpgp-keys/gentoo-release.asc
❯ equery belongs /usr/share/openpgp-keys/gentoo-release.asc
 * Searching for /usr/share/openpgp-keys/gentoo-release.asc ...
sec-keys/openpgp-keys-gentoo-release-20220101 (/usr/share/openpgp-keys/gentoo-release.asc)

Only 4 public keys should be covered:

❯ sed -n '/^# Keys included:/,/^$/p' /var/db/repos/gentoo/sec-keys/openpgp-keys-gentoo-release/openpgp-keys-gentoo-release-20220101.ebuild
# Keys included:
# DCD05B71EAB94199527F44ACDB6B8C1F96D8BF6D
# D99EAC7379A850BCE47DA5F29E6438C817072058
# 13EBBDBEDE7A12775DFDB1BABB572E0E2D182910
# EF9538C9E8E64311A52CDEDFA13D0EF1914E7A72

I don't know why "emerge --sync" tries to fetch keys belonging to "developer@gentoo.org" on your machine. You should check the file the "sync-openpgp-key-path" setting points to on your machine.
Comment 5 David Sardari 2022-10-26 09:13:01 UTC
Ah, "developer@gentoo.org" was just an example. The .asc file should be checked anyway.
Comment 6 David Sardari 2022-10-26 09:36:02 UTC
And, should "emerge --sync" hang again, you should check the output of:

gpg --debug all -vvvvv --auto-key-locate wkd --locate-external-keys infrastructure@gentoo.org
gpg --debug all -vvvvv --auto-key-locate wkd --locate-external-keys releng@gentoo.org
gpg --debug all -vvvvv --auto-key-locate wkd --locate-external-keys repomirrorci@gentoo.org
Comment 7 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2022-10-27 21:08:27 UTC
WKD advanced is online again.

$ T=$(mktemp -d) ; gpg --homedir $T  --auto-key-locate wkd --locate-external-keys  infrastructure@gentoo.org releng@gentoo.org repomirrorci@gentoo.org  ; rm -rf "$T"
gpg: keybox '/tmp/tmp.KcIpfNLMh3/pubring.kbx' created
gpg: /tmp/tmp.KcIpfNLMh3/trustdb.gpg: trustdb created
gpg: key A13D0EF1914E7A72: public key "Gentoo repository mirrors (automated git signing key) <repomirrorci@gentoo.org>" imported
gpg: Total number processed: 1
gpg:               imported: 1
gpg: no ultimately trusted keys found
gpg: key 9E6438C817072058: public key "Gentoo Linux Release Engineering (Gentoo Linux Release Signing Key) <releng@gentoo.org>" imported
gpg: key BB572E0E2D182910: public key "Gentoo Linux Release Engineering (Automated Weekly Release Key) <releng@gentoo.org>" imported
gpg: Total number processed: 2
gpg:               imported: 2
gpg: no ultimately trusted keys found
gpg: key DB6B8C1F96D8BF6D: public key "Gentoo ebuild repository signing key (Automated Signing Key) <infrastructure@gentoo.org>" imported
gpg: Total number processed: 1
gpg:               imported: 1
gpg: no ultimately trusted keys found
pub   rsa4096 2018-05-28 [C] [expires: 2024-07-01]
      EF9538C9E8E64311A52CDEDFA13D0EF1914E7A72
uid           [ unknown] Gentoo repository mirrors (automated git signing key) <repomirrorci@gentoo.org>
sub   rsa2048 2018-05-28 [S] [expires: 2024-07-01]

pub   dsa1024 2004-07-20 [SC] [expires: 2024-01-01]
      D99EAC7379A850BCE47DA5F29E6438C817072058
uid           [ unknown] Gentoo Linux Release Engineering (Gentoo Linux Release Signing Key) <releng@gentoo.org>
sub   elg2048 2004-07-20 [E] [expires: 2024-01-01]

pub   rsa4096 2011-11-25 [C] [expires: 2024-07-01]
      DCD05B71EAB94199527F44ACDB6B8C1F96D8BF6D
uid           [ unknown] Gentoo ebuild repository signing key (Automated Signing Key) <infrastructure@gentoo.org>
sub   rsa4096 2011-11-25 [S] [expires: 2024-07-01]
Comment 8 Larry the Git Cow gentoo-dev 2023-08-17 06:52:58 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=9268a92b9666eaaf263999b18220c0d56d8c476c

commit 9268a92b9666eaaf263999b18220c0d56d8c476c
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2023-08-13 04:36:04 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2023-08-17 06:52:55 +0000

    sync: rsync, git: respect --debug for gemato
    
    Respect --debug and pass it down to gemato so we get nice debugging output
    when e.g. 'refreshing keys' is stuck.
    
    Bug: https://bugs.gentoo.org/646194
    Bug: https://bugs.gentoo.org/647696
    Bug: https://bugs.gentoo.org/691666
    Bug: https://bugs.gentoo.org/779766
    Bug: https://bugs.gentoo.org/873133
    Bug: https://bugs.gentoo.org/906875
    Bug: https://github.com/projg2/gemato/issues/7
    Bug: https://github.com/projg2/gemato/issues/25
    Signed-off-by: Sam James <sam@gentoo.org>

 lib/portage/sync/modules/git/git.py     | 15 +++++++++++++--
 lib/portage/sync/modules/rsync/rsync.py | 11 +++++++++--
 lib/portage/sync/syncbase.py            | 12 ++++++++----
 3 files changed, 30 insertions(+), 8 deletions(-)