Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 736756 - WKD: Advanced: key missing
Summary: WKD: Advanced: key missing
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Infrastructure
Classification: Unclassified
Component: Other (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Infrastructure
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-08-11 15:45 UTC by Jonas Stein
Modified: 2022-10-27 21:07 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jonas Stein gentoo-dev 2020-08-11 15:45:37 UTC
https://metacode.biz/openpgp/web-key-directory

Direct: key: https://gentoo.org/.well-known/openpgpkey/hu/6f7g7knuat978eqwsw6cdfa8n1u8w3na?l=jstein

works fine, but

Advanced: key: https://openpgpkey.gentoo.org/.well-known/openpgpkey/gentoo.org/hu/6f7g7knuat978eqwsw6cdfa8n1u8w3na?l=jstein

should work too. In order to support the usage of openpgp, we should support WKD as good as possible. 

The fix seems to be simply to copy the files.

Reproducible: Always
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-08-14 14:30:46 UTC
Keys work now.  What we still need to do is provide some explanatory page on openpgpkey.gentoo.org.
Comment 2 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-08-14 14:30:56 UTC
(any suggestions?)
Comment 3 Martin 2020-08-14 22:37:02 UTC
Is this bug then causing the following confusing errors for an "emerge --sync" when WKD falls back to next try hkps:


>>> Syncing repository 'gentoo' into '/usr/portage'...
 * Using keys from /usr/share/openpgp-keys/gentoo-release.asc
 * Refreshing keys via WKD ...                                                                                                                                                                                                                                   [ !! ]
 * Refreshing keys from keyserver hkps://keys.gentoo.org ...OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://keys.gentoo.org
gpg: keyserver refresh failed: No keyserver available


Regards,
Martin
Comment 4 Martin 2020-08-14 23:26:00 UTC
Further note:

Trying:

# gpg --locate-key openpgp-auth+{l1,l2-dev,l2-srv}@gentoo.org
gpg: error retrieving 'openpgp-auth+l2-srv@gentoo.org' via WKD: Address family not supported by protocol
gpg: error reading key: Address family not supported by protocol
gpg: error retrieving 'openpgp-auth+l2-dev@gentoo.org' via WKD: Address family not supported by protocol
gpg: error reading key: Address family not supported by protocol
gpg: error retrieving 'openpgp-auth+l1@gentoo.org' via WKD: Address family not supported by protocol
gpg: error reading key: Address family not supported by protocol


Yet https://keys.gentoo.org/pks/lookup?search=auth&op=vindex gives the expected successful results...


Hopefully that is of help?

Thanks,
Martin
Comment 5 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-08-15 06:03:46 UTC
So... it seems that enabling advanced WKD finds yet another bug in GnuPG?  Awesome.

For the record, I'm really seriously considering replacing GnuPG with something sane-ish in gemato.  However, so far experiences with the alternatives prove totally vulnerable to the first attack I can think of.
Comment 6 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-08-15 06:17:22 UTC
Ok, disabled the domain for now.  To be reenabled once we have the GnuPG fixed and in stable.
Comment 7 Michael Hofmann 2020-08-15 13:36:13 UTC
In the last two years, we've had many, many, many, many "OpenPGP keyring refresh failed:" threads in the Gentoo forums. Something is wrong with the tree verification implementation.

Those frequent errors scare away new users and irritate experienced users. "emerge --sync" should just work out of the box. No excuses. 

So please either make tree verification work or turn it off. 

The current situation is unsatisfactory!
Comment 8 Martin 2020-08-15 16:07:46 UTC
(In reply to Michael Hofmann from comment #7)
> In the last two years, we've had many, many, many, many "OpenPGP keyring
> refresh failed:" threads in the Gentoo forums. Something is wrong with the
> tree verification implementation.
How many?

And how frequent?

I'm genuinely interested in that I'm always impressed for how well emerge works!


> Those frequent errors scare away new users and irritate experienced users.
> "emerge --sync" should just work out of the box. No excuses. 
Please post a FAQ for any/all "emerge --sync"?

There is an awful lot in the chain that all has to be in place for that little command to complete successfully... (Including client networking, the internet, Gentoo volunteers' servers, and the Gentoo infrastructure itself.)


> So please either make tree verification work or turn it off. 
The tree verification is a Very Good Idea... This is why this is called Gentoo and not some proprietary name...

 
> The current situation is unsatisfactory!
Please volunteer to help!


Another aside/suggestion is for whatever error messages are generated for them to include a link for where to look for help or a fix.


Thanks,
Martin

(From just another user who knows not enough but still enough to appreciate the effort that goes into making things work ;-) )
Comment 9 Martin 2020-08-15 16:13:58 UTC
(In reply to Michał Górny from comment #6)
> Ok, disabled the domain for now.  To be reenabled once we have the GnuPG
> fixed and in stable.

Thanks. Good try.


Running "emerge --sync" just now works fine oncemore.


That sync included updates to:

sys-apps/portage-2.3.103-r1
app-crypt/gnupg-2.2.20-r1
app-portage/gemato-14.4-r1


And "emerge --sync" works after updating to those.

Regards,
Martin
Comment 10 Thomas Deutschmann (RETIRED) gentoo-dev 2020-08-15 17:32:42 UTC
A new gnupg version containing a patch which should address the problem

> Address family not supported by protocol

was added to repository. To allow user to receive that version, the infrastructure change which triggered the bug in gnupg was temporarily reverted.
Comment 11 Michael Hofmann 2020-08-15 18:20:48 UTC
> How many?
>
> And how frequent?

Go to Google and search for: 

  site:forums.gentoo.org "OpenPGP keyring refresh failed"

I get at least 50 entries. Here are some:

- https://forums.gentoo.org/viewtopic-t-1100202.html
- https://forums.gentoo.org/viewtopic-t-1100116.html
- https://forums.gentoo.org/viewtopic-t-1100352.html
- https://forums.gentoo.org/viewtopic-t-1082488.html
- https://forums.gentoo.org/viewtopic-p-8280724.html
- https://forums.gentoo.org/viewtopic-t-1103156.html
- https://forums.gentoo.org/viewtopic-p-8361568.html
- https://forums.gentoo.org/viewtopic-t-1112206.html
- https://forums.gentoo.org/viewtopic-t-1108628.html
- https://forums.gentoo.org/viewtopic-t-1083474.html
- https://forums.gentoo.org/viewtopic-t-1097244.html
...

Do you want more?

And every time a see a user struggling with tree verification, I ask myself: why do our developers torture our users so much? It's not only that tree verification doesn't work reliably. The stupid error messages are even worse - because they don't tell users anything about the cause of the error or what they can do to make their systems work again. 

I'm sorry to be blunt. But this really has to be improved!
Comment 12 Thomas Deutschmann (RETIRED) gentoo-dev 2020-08-15 18:51:55 UTC
Don't get me wrong, I agree with your comment.

But do you think it is still that worse? Notice that most of the shown examples are showing SKS keyserver usage which is really a problem. However, we have switched to WKD usage for some time now. Aside the current problem with non-IPv6-enabled hosts I am curious if situation is still that bad from your POV.
Comment 13 Martin 2020-08-15 23:55:39 UTC
(In reply to Thomas Deutschmann from comment #12)

> ... Aside the current problem
> with non-IPv6-enabled hosts...

?... My systems are all non-IPv6 (indeed, IPv6 is disabled,) due to my ISP not supporting IPv6.

No problem noticed due to that lack of IPv6.

Or what should I test?
Comment 14 Martin 2020-08-15 23:58:27 UTC
(In reply to Michael Hofmann from comment #11)
> > How many?
> >
> > And how frequent?
> 
> Go to Google and search for: 
> 
>   site:forums.gentoo.org "OpenPGP keyring refresh failed"
> 
> I get at least 50 entries. Here are some:


Do they not indicate that a FAQ is needed to explain the often confusing/misleading error messages?

The error messages themselves are correct. However, they are not actually helpful for most users...!
Comment 15 Michael Hofmann 2020-08-18 14:09:51 UTC
> Thomas Deutschmann wrote: 
> But do you think it is still that worse?

I agree. It definitely got better after switching to WKD. 

But still...

It would be nice to have a list of common error messages and an explanation how 
to solve those issues. Maybe we can add a new section "error messages and what to do" to https://wiki.gentoo.org/wiki/Project:Portage/Repository_verification? It should be easy: we can copy and paste text blocks from the forum threads.

Error messages of portage/emerge should point to that explanation. Maybe something like

   try {
       do the verification stuff
   } catch {
       print "Something went wrong during tree verification"
       print "For further information, please look at ...."
   }
Comment 16 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-09-28 12:04:14 UTC
Is time to try the experiment again?
Comment 17 Thomas Deutschmann (RETIRED) gentoo-dev 2020-09-28 12:51:39 UTC
Let's schedule this for 2020-10-05 so we have a clear cut.
Comment 18 Scott Howard 2021-03-22 17:59:06 UTC
the dirmgr fix that was described in comment 10 seems to no longer be needed

https://gitweb.gentoo.org/repo/gentoo.git/commit/app-crypt/gnupg/files?id=810410a8c6b411bd8b1ac60ceb28d37af27256b1

can the infrastructure team try to re-enable the domain again so we can make progress on getting WKD working smoothly?
Comment 19 Scott Howard 2021-03-29 22:39:13 UTC
actually, with dirmngr in gnupg 2.2.27 (https://bugs.gentoo.org/show_bug.cgi?id=777876), my client is successfully contacting the WKD server at gentoo.org instead of openpgpkey.gentoo.org. So I think this is resolved.


repos.conf:

[gentoo]
location = /var/db/repos/gentoo
sync-type = git
sync-uri = https://anongit.gentoo.org/git/repo/sync/gentoo.git
sync-git-verify-commit-signature = true
sync-depth = 1

emerge --sync:

 * Using keys from /usr/share/openpgp-keys/gentoo-release.asc
 * Refreshing keys via WKD ...                                 [ ok ]
 * Trusted signature found on top commit
=== Sync completed for gentoo

I still don't understand why it needs to perform the WKD lookup after each sync though
Comment 20 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2022-10-27 21:07:13 UTC
WKD advanced is online again.