Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 647696 - portage: gpg: keyserver refresh failed: No data
Summary: portage: gpg: keyserver refresh failed: No data
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Unclassified (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on: 649276
Blocks: 686768
  Show dependency tree
 
Reported: 2018-02-15 08:08 UTC by Anton Kochkov
Modified: 2020-08-15 18:35 UTC (History)
16 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
strace -a 256 -v -f emerge --sync (strace.txt.bz2,160.90 KB, application/octet-stream)
2020-01-01 10:55 UTC, Martin Mokrejš
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Anton Kochkov 2018-02-15 08:08:02 UTC
sent 42.24K bytes  received 11.08M bytes  71.56K bytes/sec
total size is 224.37M  speedup is 20.16
INFO:root:Refreshing keys from keyserver...
ERROR:root:OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net
gpg: keyserver refresh failed: No data

q: Updating ebuild cache in /usr/portage ... 
q: Finished 37403 entries in 0.255664 seconds

emerge --info output:

Portage 2.3.24 (python 2.7.14-final-0, default/linux/amd64/17.0/no-multilib/hardened, gcc-6.4.0, glibc-2.26-r5, 3.3.2-hardened x86_64)
=================================================================
System uname: Linux-3.3.2-hardened-x86_64-Intel-R-_Core-TM-_i7_CPU_930_@_2.80GHz-with-gentoo-2.4.1
KiB Mem:     8167824 total,    901244 free
KiB Swap:   16016792 total,  15928776 free
Timestamp of repository gentoo: Thu, 15 Feb 2018 07:15:01 +0000
Head commit of repository gentoo: 95ca72fdc863d8e43a0160462312b28ba94c46da
sh bash 4.4_p19
ld GNU ld (Gentoo 2.29.1 p2) 2.29.1
app-shells/bash:          4.4_p19::gentoo
dev-java/java-config:     2.2.0-r3::gentoo
dev-lang/perl:            5.26.1-r1::gentoo
dev-lang/python:          2.7.14-r1::gentoo, 3.6.4::gentoo
dev-util/cmake:           3.10.2::gentoo
dev-util/pkgconfig:       0.29.2::gentoo
sys-apps/baselayout:      2.4.1-r2::gentoo
sys-apps/openrc:          0.34.11::gentoo
sys-apps/sandbox:         2.12::gentoo
sys-devel/autoconf:       2.69-r4::gentoo
sys-devel/automake:       1.13.4-r1::gentoo, 1.15.1-r1::gentoo
sys-devel/binutils:       2.29.1-r1::gentoo, 2.30::gentoo
sys-devel/gcc:            5.4.0-r4::gentoo, 6.4.0-r1::gentoo, 7.1.0-r1::gentoo, 7.3.0::gentoo
sys-devel/gcc-config:     1.9.1::gentoo
sys-devel/libtool:        2.4.6-r4::gentoo
sys-devel/make:           4.2.1-r1::gentoo
sys-kernel/linux-headers: 4.15::gentoo (virtual/os-headers)
sys-libs/glibc:           2.26-r5::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://rsync.us.gentoo.org/gentoo-portage
    priority: -1000
    sync-rsync-verify-metamanifest: yes
    sync-rsync-extra-opts: 

x-portage
    location: /usr/local/portage
    masters: gentoo
    priority: 0

gentoo-xvilka
    location: /var/lib/layman/gentoo-xvilka
    masters: gentoo
    priority: 1

godin
    location: /var/lib/layman/godin
    masters: gentoo
    priority: 50

ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* -@EULA dlj-1.1"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=core2 -mtune=generic -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /var/bind /var/qmail/alias /var/qmail/control /var/spool/munin-async/.ssh /var/vpopmail/etc"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.6/ext-active/ /etc/php/cgi-php5.6/ext-active/ /etc/php/cli-php5.6/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-march=core2 -mtune=generic -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync multilib-strict news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j9"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="acl amd64 bzip2 crypt cvs cxx git gnutls hardened iconv ipv6 lighttpd mercurial mmx ncurses nls nptl openmp pam pcre perl php pie postgresql python readline sbcl seccomp sse sse2 sse4 ssl ssp ssse3 subversion unicode xattr xml xmlrpc xsl xtpax zlib" ABI_X86="64" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="glibc" INPUT_DEVICES="libinput keyboard mouse" KERNEL="linux" LCD_DEVICES="ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6 php7-0" POSTGRES_TARGETS="postgres9_5" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" RUBY_TARGETS="ruby24" USERLAND="GNU" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2018-02-16 18:09:04 UTC
Are you able reproduce it consistently? It might be an interim network problem.
Comment 2 Zac Medico gentoo-dev 2018-02-18 21:35:24 UTC
There's a report here that “emaint sync -r gentoo” consistently fails like this when executed via a cron job:

https://forums.gentoo.org/viewtopic-p-8185252.html?sid=19689b0028e30d86d5c57aa377437376#8185252
Comment 3 Zac Medico gentoo-dev 2018-02-18 23:13:22 UTC
(In reply to Zac Medico from comment #2)
> There's a report here that “emaint sync -r gentoo” consistently fails like
> this when executed via a cron job:
> 
> https://forums.gentoo.org/viewtopic-p-8185252.
> html?sid=19689b0028e30d86d5c57aa377437376#8185252

I've tested `emaint sync -r gentoo` in a cron job, and it works for me:

INFO:root:Refreshing keys from keyserver...
INFO:root:Keys refreshed.
INFO:root:Manifest timestamp: 2018-02-18 22:38:17 UTC
INFO:root:Valid OpenPGP signature found:
INFO:root:- primary key: DCD05B71EAB94199527F44ACDB6B8C1F96D8BF6D
INFO:root:- subkey: E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250
INFO:root:- timestamp: 2018-02-18 22:38:17 UTC
Comment 4 Zac Medico gentoo-dev 2018-02-23 17:08:51 UTC
Please make sure you have >=app-crypt/gnupg-2.2.4-r2, since the dependency has been updated:

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=9cdccbb4f67a6e3586df619a689ee9a7306ffe7d
Comment 5 email200202 2018-03-01 06:14:04 UTC
It happened today. I tried three times and it was repeatable.

>>> Syncing repository 'gentoo' into '/usr/portage'... 
>>> Starting rsync with rsync://176.28.50.119/gentoo-portage... 
Welcome to quetzal.gentoo.org / rsync.gentoo.org 

Server Address : 176.28.50.119, 2a01:488:67:1000:b01c:3277:0:1 
Contact Name   : mirror-admin@gentoo.org 
Hardware       : 4 x Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz, 16040MB RAM 
Sponsor        : Host Europe, Cologne, Germany, EU 

      ........... 
      ........... 

sent 37.10K bytes  received 4.11M bytes  34.15K bytes/sec 
total size is 225.65M  speedup is 54.39 
INFO:root:Refreshing keys from keyserver... 
ERROR:root:OpenPGP keyring refresh failed: 
gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net 
gpg: keyserver refresh failed: No data 

q: Updating ebuild cache in /usr/portage ... 
q: Finished 37493 entries in 0.560954 seconds 

Action: sync for repo: gentoo, returned code = 1
Comment 6 Kristian Fiskerstrand (RETIRED) gentoo-dev 2018-03-03 16:47:28 UTC
(In reply to email200202 from comment #5)
> It happened today. I tried three times and it was repeatable.


Could you try increasing your dirmngr logging verbosity to see what happens here? see debug and log-file for dirmngr.conf in man dirmngr, it requires a restart of the daemon -- gpgconf --kill dirmngr .
Comment 7 Zac Medico gentoo-dev 2018-03-26 08:24:35 UTC
I experienced the "No data" error today, in a cron job that runs every 30 minutes. The cron job calls gemato's OpenPGPEnvironment refresh_keys method, with a retry decorator created with dev-python/tenacity as follows:

tenacity.retry(
  wait=tenacity.wait_random_exponential(multiplier=1, max=128, exp_base=2),
  stop=(tenacity.stop_after_attempt(20) | tenacity.stop_after_delay(300)))

So, it failed after 20 attempts of 300 seconds, whichever came first.
Comment 8 Zac Medico gentoo-dev 2018-04-16 20:10:55 UTC
This issue tends to express itself on Mondays, today my cron job failed at 13:55:07 UTC, following 20 minutes of retry:

tenacity.retry(
  wait=tenacity.wait_random_exponential(multiplier=1, max=128, exp_base=2),
  stop=(tenacity.stop_after_attempt(40) | tenacity.stop_after_delay(1200)))
Comment 9 Dirkjan Ochtman gentoo-dev 2018-07-22 08:42:36 UTC
I also saw this in my weekly emerge sync cron job. I have gnupg-2.2.8, gemato-13.0-r1 and portage-2.3.40-r1.

OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net
gpg: keyserver refresh failed: No data
Comment 10 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2018-07-22 08:57:04 UTC
For the record, we're now testing WKD [1] deployment on gentoo.org, so we might switch to refetching keys over WKD instead (which should be much more stable).  However, I still need more opinions before deploying that (and it's going to be custom-made since GPG doesn't seem to have an option for that).

[1]:https://wiki.gnupg.org/WKD
Comment 11 Kerin Millar 2018-07-22 09:44:47 UTC
I welcome the endeavour to find a key distribution solution that scales more effectively. In practice, the public keyserver pool is slow and unreliable. This, along with portage's requirement to retrieve keys upon each sync, drove me back to using git.

As concerns WKD, wouldn't the --auto-key-locate and --locate-keys options be doing most of the work?
Comment 12 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2018-07-22 09:51:28 UTC
(In reply to Kerin Millar from comment #11)
> As concerns WKD, wouldn't the --auto-key-locate and --locate-keys options be
> doing most of the work?

Sadly, they do anything only if the key is not present in the local keyring.  They lack a variant refreshing the existing key.
Comment 13 Kerin Millar 2018-07-22 09:52:46 UTC
(In reply to Michał Górny from comment #12)
> (In reply to Kerin Millar from comment #11)
> > As concerns WKD, wouldn't the --auto-key-locate and --locate-keys options be
> > doing most of the work?
> 
> Sadly, they do anything only if the key is not present in the local keyring.
> They lack a variant refreshing the existing key.

Ah, that's too bad. Thank you for clarifying.
Comment 14 Manfred Knick 2018-07-25 11:49:52 UTC
Today,

   portage "emerge --sync" / "emaint sync -r gentoo"

fail / stall with multiple

   OpenPGP keyring refresh failed:
   gpg: 4 Schlüssel werden per hkps://hkps.pool.sks-keyservers.net aktualisiert
   gpg: Refresh vom Schlüsselserver fehlgeschlagen: Allgemeiner Fehler

!
   # host hkps.pool.sks-keyservers.net
   hkps.pool.sks-keyservers.net has address 37.191.226.104     <-----

! Only _one_ address in list ?

   # ping 37.191.226.104
   PING 37.191.226.104 (37.191.226.104) 56(84) bytes of data.

fails with "100% packet loss".

Entering [http://hkps.pool.sks-keyservers.net/] in Browser,
I get

   OpenPGP Public Key Server Commands

   Welcome to keys2.kfwebs.net, a keyserver for OpenPGP hosted by KF Webs,
   part of the pool.sks-keyservers.net round-robin.

Entering "gentoo" as search string fails:

   Error handling request
   Error handling request. Exception raised. 


Any additional Info I could provide?
Comment 15 Manfred Knick 2018-07-25 11:55:49 UTC
(In reply to Manfred Knick from comment #14)
Seems to be a temporary problem with the pool again:
Now:

#  host hkps.pool.sks-keyservers.net
hkps.pool.sks-keyservers.net has address 37.191.226.104
hkps.pool.sks-keyservers.net has IPv6 address 2a02:c205:3001:3626::1
hkps.pool.sks-keyservers.net has IPv6 address 2a01:4a0:59:1000:223:9eff:fe00:100f
hkps.pool.sks-keyservers.net has IPv6 address 2001:470:1:116::6

sync succeeds again.

Note: Still no additional different members in list.
Comment 16 Manfred Knick 2018-08-01 15:53:05 UTC
( QUESTION to Michał Górny from comment #12)

> Sadly, they do anything only if the key is not present in the local keyring.
> They lack a variant refreshing the existing key.

[ https://wiki.gnupg.org/WKD#Implementations ] proudly presents:

    WKD lookup is implemented in GnuPG since v2.1.12.
        ^^^^^^                                                      < ---
            It is enabled by default since 2.1.23..

    WKS server and client tools are part of GnuPG since v2.1.14


@ Michał: Do I understand you correctly stating "lookup" =/= "refresh" ?

          Thanks in advance!


# equery list -p app-crypt/gnupg
     [-P-] [  ] app-crypt/gnupg-1.4.21:0
     [IP-] [  ] app-crypt/gnupg-2.2.8:0
     [-P-] [ ~] app-crypt/gnupg-2.2.9:0

# gpg --help | grep key | grep refresh 
     --refresh-keys          ...

# man gpg

      --refresh-keys

              Request updates from a keyserver for keys that already exist on the local keyring. This is useful for updating a key with the latest signatures, user IDs, etc. Calling this with no arguments will refresh the entire keyring. Option --keyserver must be used to give the name of the keyserver for all keys that do not have preferred keyservers set (see --keyserver-options honor-keyserver-url).
Comment 17 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2018-08-01 16:33:20 UTC
Yes, WKD is used only by --locate-key.

Nevertheless, gemato-14.0 implements WKD-based key refresh.
Comment 18 James McMechan 2018-08-19 00:33:51 UTC
Well I seem to have a version of the error not depending on network issues...
I thought I would try again with sys-apps/portage-2.3.40-r1 since it is supposed to fix the network problem.

On my main box at the end of 29:40 of repeating failures I got:

------------------------
# date
Sat Aug 18 15:12:20 PDT 2018

# emaint sync -a
>>> Syncing repository 'gentoo' into '/usr/portage'...
 * Using keys from /usr/share/openpgp-keys/gentoo-release.asc
 * Refreshing keys from keyserver ...OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net
gpg: keyserver refresh failed: End of file

<...>
!!! Manifest verification impossible due to keyring problem:
OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net
gpg: keyserver refresh failed: End of file

q: Updating ebuild cache in /usr/portage ...
q: Finished 35499 entries in 75.900674 seconds

Action: sync for repo: gentoo, returned code = 1

# date
Sat Aug 18 15:41:00 PDT 2018
------------------------
Note that it took 28:40 while failing but only reported 75.900674 seconds


Meanwhile my other boxes are succeeding on the first try...
AMD64 - failed repeatedly for ~30 minutes
SPARC - takes a few seconds, no retries
MIPS64 - takes a few seconds, no retries
ARM - takes a few seconds, no retries

The problem being that my AMD64 box is the one I usually use as a local gentoo portage mirror, and main workstation.

Note I have both IPV4 & IPV6 configured and working, I can from any of the boxes do ping -6 -c10 ipv6.google.com with 0% packet loss.

How exactly should I investigate this?
The error message is nearly useless are there any logs to look at or will I have stick a strace wrapper on something.

Next I think I will try forcing hkps.pool.sks-keyservers.net to 216.66.15.2 rather than 37.191.226.104 since the second does not appear to be working though why they would have a down system as the preferred ipv4 address is not clear.

Still no joy ping -4 & -6 hkps.pool.sks-keyservers.net gets 0% packet lost yet still fails the key refresh
Comment 19 Zac Medico gentoo-dev 2018-08-19 01:47:27 UTC
(In reply to James McMechan from comment #18)
> Well I seem to have a version of the error not depending on network issues...
> I thought I would try again with sys-apps/portage-2.3.40-r1 since it is
> supposed to fix the network problem.

With app-portage/gemato-14.0, keys are fetched via WKD by default, and it only falls back to hkps if one or more keys in the keychain (provided by app-crypt/openpgp-keys-gentoo-release) fails to import from WKD:

https://github.com/mgorny/gemato/commit/909390c25a0ab589a4ae10d20cb9e321a51163b2

Also make sure you have the latest app-crypt/openpgp-keys-gentoo-release.
Comment 20 James McMechan 2018-08-19 16:30:21 UTC
(In reply to Zac Medico from comment #19)
> (In reply to James McMechan from comment #18)
> > Well I seem to have a version of the error not depending on network issues...
> > I thought I would try again with sys-apps/portage-2.3.40-r1 since it is
> > supposed to fix the network problem.
> 
> With app-portage/gemato-14.0, keys are fetched via WKD by default, and it
> only falls back to hkps if one or more keys in the keychain (provided by
> app-crypt/openpgp-keys-gentoo-release) fails to import from WKD:
> 

I had a fully up to date system with gemato-13.0-r1
Both my arm & sparc boxes had 13.0-r1 and worked...
My MIPS64 box already had 14.0 and worked...

I have done accept_keywords app-portage/gemato-14.0 ~amd64 same results
My amd64 box does not work :(

------------------------
# date;emaint sync -a;date
Sat Aug 18 21:23:58 PDT 2018
>>> Syncing repository 'gentoo' into '/usr/portage'...
 * Using keys from /usr/share/openpgp-keys/gentoo-release.asc
 * Refreshing keys from keyserver ...OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net
gpg: keyserver refresh failed: End of file

<...>

!!! Manifest verification impossible due to keyring problem:
OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net
gpg: keyserver refresh failed: End of file

q: Updating ebuild cache in /usr/portage ...
q: Finished 35499 entries in 0.284056 seconds

Action: sync for repo: gentoo, returned code = 1


Sat Aug 18 22:02:14 PDT 2018
------------------------
I still seems to be trying hkps not WKD though I can't be sure what it is doing.
How does one get more info, mysterious failure does not help much...
There does not appear to be either a config file nor a log file, anywhere I can find it.

> https://github.com/mgorny/gemato/commit/
> 909390c25a0ab589a4ae10d20cb9e321a51163b2
> 
> Also make sure you have the latest app-crypt/openpgp-keys-gentoo-release.

My portage tree only seems to have version app-crypt/gentoo-keys-201807020151 which was not installed.
I installed it, still same failures.
Note it still absent on the arm & mips64 boxes I just assumed I needed /usr/share/openpgp-keys/gentoo-release.asc which has present.
I may have tried emerge -1 gentoo-keys back when rsync-verify first came out.

If this is required should it not be a dependency?

------------------------
Sun Aug 19 07:51:30 PDT 2018

>>> Syncing repository 'gentoo' into '/usr/portage'...
 * Using keys from /usr/share/openpgp-keys/gentoo-release.asc
 * Refreshing keys from keyserver ...OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net
gpg: keyserver refresh failed: End of file

<...>

!!! Manifest verification impossible due to keyring problem:
OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net
gpg: keyserver refresh failed: End of file

q: Updating ebuild cache in /usr/portage ...
q: Finished 35499 entries in 104.552178 seconds

Action: sync for repo: gentoo, returned code = 1


Scanning Configuration files...
Exiting: Nothing left to do; exiting. :)
Sun Aug 19 08:31:59 PDT 2018
------------------------
[ebuild   R    ] app-crypt/gentoo-keys-201807020151::gentoo  0 KiB
[ebuild   R    ] sys-apps/portage-2.3.40-r1::gentoo  USE="(ipc) native-extensions rsync-verify xattr -build -doc -epydoc -gentoo-dev (-selinux)" PYTHON_TARGETS="python2_7 python3_6 (-pypy) -python3_4 -python3_5" 991 KiB
Comment 21 Zac Medico gentoo-dev 2018-08-19 19:30:16 UTC
(In reply to James McMechan from comment #20)
> I still seems to be trying hkps not WKD though I can't be sure what it is
> doing.

What version of openpgp-keys-gentoo-release do you have? The gentoo-keys package is no longer used by portage. You need openpgp-keys-gentoo-release-20180706.
Comment 22 James McMechan 2018-08-22 19:51:41 UTC
app-crypt/openpgp-keys-gentoo-release-20180706 was already installed
re-emerged it.
No noticeable change, how can I get status of what is happening with
all these failures?
Comment 23 Zac Medico gentoo-dev 2018-08-22 21:13:47 UTC
(In reply to James McMechan from comment #22)
> app-crypt/openpgp-keys-gentoo-release-20180706 was already installed
> re-emerged it.
> No noticeable change, how can I get status of what is happening with
> all these failures?

You can enable dirmngr debug and log-file in /etc/gnupg/dirmngr.conf, as suggested in comment #6. Instead of doing a full sync, just use this command to refresh keys and verify the manifest:

> gemato openpgp-verify --openpgp-key /usr/share/openpgp-keys/gentoo-release.asc /usr/portage/Manifest
Comment 24 James McMechan 2018-08-23 03:31:35 UTC
(In reply to Zac Medico from comment #23)
> (In reply to James McMechan from comment #22)
> > app-crypt/openpgp-keys-gentoo-release-20180706 was already installed
> > re-emerged it.
> > No noticeable change, how can I get status of what is happening with
> > all these failures?
> 
> You can enable dirmngr debug and log-file in /etc/gnupg/dirmngr.conf, as
> suggested in comment #6. Instead of doing a full sync, just use this command
> to refresh keys and verify the manifest:
> 
> > gemato openpgp-verify --openpgp-key /usr/share/openpgp-keys/gentoo-release.asc /usr/portage/Manifest

Actually it seemed I needed to set logging in /root/.gnupg/dirmngr.conf

#gnuconf -kill dirmngr

kills dirmngr but apparently it does not restart with the gemato commandline above.

The logfile ended with
handler for fd 6 started
connection from process 16093 (0:0)
socket file has been removed - shutting down

if I restart it with
#gnuconf --launch dirmngr

I get more log data mostly just setup ending with
2018-08-22 20:18:58 dirmngr[16195.6] handler for fd 6 started
2018-08-22 20:18:58 dirmngr[16195.6] connection from process 16192 (0:0)
2018-08-22 20:18:58 dirmngr[16195.6] handler for fd 6 terminated

and no more data from running gemato...
my config file is now
"
verbose
verbose
verbose
debug-level guru
debug-all
log-file /tmp/dirmngr.log
"
#gnuconf --reload dirmngr

But I get no further status in the log from the gemato command line
the logfile ends with
2018-08-22 20:26:09 dirmngr[16195.6]            trusted certificates: 134 (133,0,0,1)
2018-08-22 20:26:09 dirmngr[16195.6] DBG: chan_6 -> OK
2018-08-22 20:26:09 dirmngr[16195.6] DBG: chan_6 <- [eof]
2018-08-22 20:26:09 dirmngr[16195.6] handler for fd 6 terminated

and I get no later timestamps after running the gemato commandline.
Comment 25 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2018-08-23 05:24:44 UTC
Do you have GnuPG built with USE=ssl?
Comment 26 James McMechan 2018-08-23 06:43:05 UTC
(In reply to Michał Górny from comment #25)
> Do you have GnuPG built with USE=ssl?

Yes, emerge -pv gnupg reports

[ebuild   R    ] app-crypt/gnupg-2.2.8::gentoo  USE="bzip2 readline smartcard ssl usb -doc -ldap -nls (-selinux) -tofu -tools -wks-server"
Comment 27 Anton Kochkov 2019-11-29 16:25:04 UTC
By the way, never met this error since, maybe it was fixed already?
Comment 28 Zac Medico gentoo-dev 2019-11-29 17:13:38 UTC
We can consider this fixed by WKD support in gemato, and the sync-openpgp-keyserver = hkps://keys.gentoo.org setting:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=ed2a826f8d2fc5b74a714e0e37561cec25abc79b
Comment 29 Martin Mokrejš 2020-01-01 10:28:29 UTC
I don't think it is fixed:

host2 # emerge --sync; layman -S; emerge qgis
>>> Syncing repository 'gentoo' into '/scratch/usr/portage'...
 * Using keys from /usr/share/openpgp-keys/gentoo-release.asc
 * Refreshing keys via WKD ...                                                                                                                                                                                                                                                                                        [ !! ]
 * Refreshing keys from keyserver hkps://keys.gentoo.org ...OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://keys.gentoo.org
gpg: keyserver refresh failed: No keyserver available

OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://keys.gentoo.org
gpg: keyserver refresh failed: No keyserver available

OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://keys.gentoo.org
gpg: keyserver refresh failed: No keyserver available

OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://keys.gentoo.org
gpg: keyserver refresh failed: No keyserver available

OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://keys.gentoo.org
gpg: keyserver refresh failed: No keyserver available

OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://keys.gentoo.org
gpg: keyserver refresh failed: No keyserver available

OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://keys.gentoo.org
gpg: keyserver refresh failed: No keyserver available

OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://keys.gentoo.org
gpg: keyserver refresh failed: No keyserver available

!!! Manifest verification impossible due to keyring problem:
OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://keys.gentoo.org
gpg: keyserver refresh failed: No keyserver available


 * IMPORTANT: 5 news items need reading for repository 'gentoo'.
 * Use eselect news read to view new items.


Action: sync for repo: gentoo, returned code = 1

 * An update to portage is available. It is _highly_ recommended
 * that you update portage now, before any other packages are updated.

 * To update portage, run 'emerge --oneshot portage' now.




 * Fetching remote list...
 * Fetch Ok

 * Syncing selected overlay(s)...
 * Running Git... # ( cd /var/lib/layman/haskell  && /usr/bin/git pull )
remote: Enumerating objects: 185, done.
remote: Counting objects: 100% (185/185), done.
remote: Compressing objects: 100% (95/95), done.
remote: Total 185 (delta 97), reused 173 (delta 90), pack-reused 0
...



I had same issue host1 where I worked it around it by enabling IPV6 in kernel .config and on the home router. I have this problem also on another host behing the same router, say host2, which kernel I will need to recompile first.

host1 (fixed few days ago, I am little bit guessing it was just the below .config change, or some additonal associated with it thanks to "make oldconfig"):

# diff -u -w linux-5.3.7/.config linux-5.4.6/.config
--- linux-5.3.7/.config	2019-11-12 01:02:28.247849190 +0100
+++ linux-5.4.6/.config	2019-12-30 13:20:18.691687193 +0100
...
-# CONFIG_IPV6 is not set
+CONFIG_IPV6=y

I think I also had to enable IPv6 support in my home router.

At the moment the host1 is at:

host1# emerge -pv gemato gnupg openpgp-keys-gentoo-release portage
[ebuild   R    ] app-crypt/openpgp-keys-gentoo-release-20191030::gentoo  USE="-test" 0 KiB
[ebuild   R    ] dev-python/pyxattr-0.6.1-r1::gentoo  USE="-doc -test" PYTHON_TARGETS="python2_7* python3_6 python3_7 -pypy3 -python3_8 (-pypy%) (-python3_5%*)" 0 KiB
[ebuild   R    ] app-crypt/gnupg-2.2.19::gentoo  USE="bzip2 doc nls readline ssl tofu tools usb -ldap (-selinux) -smartcard -user-socket -wks-server" 0 KiB
[ebuild   R    ] app-portage/gemato-14.3::gentoo  USE="blake2 bzip2 gpg -lzma -sha3 -test -tools" PYTHON_TARGETS="python2_7 python3_6 python3_7 -pypy3 -python3_8 (-pypy%) (-python3_5%*)" 0 KiB
[ebuild   R   *] sys-apps/portage-9999::gentoo  USE="(ipc) native-extensions rsync-verify xattr -build -doc -epydoc -gentoo-dev (-selinux)" PYTHON_TARGETS="python2_7 python3_6 python3_7 -python3_8 (-pypy%) (-python3_5%*)" 0 KiB




On the host2 where I cannot update my portage tree I have:

host2# emerge -pv gemato gnupg openpgp-keys-gentoo-release portage
[ebuild     U  ] app-crypt/openpgp-keys-gentoo-release-20191030::gentoo [20190427::gentoo] USE="-test" 24 KiB
[ebuild     U  ] app-crypt/gnupg-2.2.19::gentoo [2.2.17::gentoo] USE="bzip2 doc nls readline smartcard ssl tools usb -ldap (-selinux) -tofu -user-socket -wks-server" 6 597 KiB
[ebuild     U  ] app-portage/gemato-14.3::gentoo [14.1::gentoo] USE="blake2 bzip2 gpg -lzma -sha3 -test -tools" PYTHON_TARGETS="python2_7 python3_6 python3_7 -pypy -pypy3% -python3_5 -python3_8%" 70 KiB
[ebuild     U  ] sys-apps/portage-2.3.82::gentoo [2.3.75-r1::gentoo] USE="(ipc) native-extensions rsync-verify xattr -build -doc -epydoc -gentoo-dev (-selinux)" PYTHON_TARGETS="python2_7 python3_6 python3_7 -pypy -python3_5 -python3_8%" 1 019 KiB

I am not showing the many python2_7 issues,.

host2# gzip -dc /proc/config.gz |  grep IPV6
# CONFIG_IPV6 is not set
host2#

I bet if I enable CONFIG_IPV6 there it will start working. But that is not what I want to. ;)
Comment 30 Martin Mokrejš 2020-01-01 10:53:03 UTC
After reading more similar bugs I tried dig on both hosts:


The one with working syncing due to IPV6 enabled in kernel:

host1# dig srv _pgpkey-http._tcp.eu.pool.sks-keyservers.net

; <<>> DiG 9.14.8 <<>> srv _pgpkey-http._tcp.eu.pool.sks-keyservers.net
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 28696
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 11, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;_pgpkey-http._tcp.eu.pool.sks-keyservers.net. IN SRV

;; ANSWER SECTION:
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3597 IN SRV 0 628 11371 keys2.kfwebs.net.
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3597 IN SRV 0 578 11371 sks.pod01.fleetstreetops.com.
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3597 IN SRV 0 573 11371 sks.mj2.uk.
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3597 IN SRV 0 500 11371 sks.hnet.se.
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3597 IN SRV 0 481 11371 zuul.rediris.es.
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3597 IN SRV 0 566 11371 pgp.cyberbits.eu.
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3597 IN SRV 0 556 11371 sks.infcs.de.
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3597 IN SRV 0 624 11371 pgpkeys.co.uk.
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3597 IN SRV 0 556 11371 pgpkeys.eu.
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3597 IN SRV 0 678 11371 pgpkeys.uk.
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3597 IN SRV 0 626 11371 sks.pod02.fleetstreetops.com.

;; Query time: 4 msec
;; SERVER: 192.168.0.1#53(192.168.0.1)
;; WHEN: St led 01 11:41:58 CET 2020
;; MSG SIZE  rcvd: 462



The one having a problem, with IPV6 disabled in kernel:

host2# dig srv _pgpkey-http._tcp.eu.pool.sks-keyservers.net

; <<>> DiG 9.14.5 <<>> srv _pgpkey-http._tcp.eu.pool.sks-keyservers.net
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1923
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 11, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;_pgpkey-http._tcp.eu.pool.sks-keyservers.net. IN SRV

;; ANSWER SECTION:
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3600 IN SRV 0 628 11371 keys2.kfwebs.net.
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3600 IN SRV 0 578 11371 sks.pod01.fleetstreetops.com.
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3600 IN SRV 0 573 11371 sks.mj2.uk.
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3600 IN SRV 0 500 11371 sks.hnet.se.
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3600 IN SRV 0 481 11371 zuul.rediris.es.
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3600 IN SRV 0 566 11371 pgp.cyberbits.eu.
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3600 IN SRV 0 556 11371 sks.infcs.de.
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3600 IN SRV 0 624 11371 pgpkeys.co.uk.
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3600 IN SRV 0 556 11371 pgpkeys.eu.
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3600 IN SRV 0 678 11371 pgpkeys.uk.
_pgpkey-http._tcp.eu.pool.sks-keyservers.net. 3600 IN SRV 0 626 11371 sks.pod02.fleetstreetops.com.

;; Query time: 1767 msec
;; SERVER: 192.168.0.1#53(192.168.0.1)
;; WHEN: Wed Jan 01 11:41:55 CET 2020
;; MSG SIZE  rcvd: 462

host2# strace -a 256 -v -f emerge --sync 1>/tmp/strace.txt 2>&1


Router 192.168.0.1 with dd-wrt does not do any IPv6-to-IPv4 translation, upstream provider does not provide IPv6 at all.
Comment 31 Martin Mokrejš 2020-01-01 10:55:14 UTC
Created attachment 602154 [details]
strace -a 256 -v -f emerge --sync
Comment 32 Zac Medico gentoo-dev 2020-01-01 11:19:59 UTC
(In reply to Martin Mokrejš from comment #30)
> The one having a problem, with IPV6 disabled in kernel:

Please open a bug for app-crypt/gnupg if you think it handles this case incorrectly.
Comment 33 Martin Mokrejš 2020-01-01 13:41:37 UTC
I opened bug #704456 for the very same error, but traced it does to "search foo.bar" in /etc/resolv.conf gettting appedned to DNS queries (for some cases it fails somewhat different downstream path). All the tests I did on the host with only IPv4.

I cannot exclude that along enabling the "IPv6" in the other host I did not tweak my router settings to fix that inadverently. Actually, I was fiddling with my router, so it could be the cause. However, I agree with others that my laptop(s) work behind some router but do not behind another. The bug #704456 could lead to the answer how to make gnupg and gemato more resistant to "mis?"configured routers, DHCP servers and DNS servers in home routers.
Comment 34 Alex Efros 2020-08-14 09:27:51 UTC
I've just updated server which was updated about month ago and got this issue.
I.e. about month ago `emerge --sync` works, but after updating these (listed below) packages it breaks.
I've gnupg-2.2.20 installed several months ago, so probably it breaks because of updating some of it dependencies.

I was able to fix this by enabled CONFIG_IPV6 in kernel together with disabling IPv6 using sysctl:

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1

List of updated packages:

2020-08-13T05:46:30 >>> sys-libs/glibc-2.31-r6
2020-08-13T05:52:58 >>> dev-libs/libffi-3.3-r2
2020-08-13T05:53:19 >>> acct-group/fetchmail-0
2020-08-13T05:53:25 >>> app-arch/rar-5.9.1_p20200625
2020-08-13T05:53:34 >>> sys-kernel/linux-firmware-20200721
2020-08-13T05:54:41 >>> sys-devel/binutils-config-5.3.2
2020-08-13T05:54:58 >>> sys-devel/gcc-config-2.3.1
2020-08-13T05:55:10 >>> acct-user/fetchmail-0
2020-08-13T05:55:16 >>> dev-libs/mpfr-4.1.0
2020-08-13T05:55:40 >>> sys-libs/libseccomp-2.4.3
2020-08-13T05:55:56 >>> net-misc/socat-1.7.3.4
2020-08-13T05:56:31 >>> net-misc/rsync-3.2.2-r1
2020-08-13T05:57:02 >>> media-libs/freetype-2.10.2-r1
2020-08-13T05:57:23 >>> net-libs/libtirpc-1.2.6
2020-08-13T05:57:41 >>> sys-apps/pciutils-3.6.4
2020-08-13T05:57:53 >>> net-libs/libnsl-1.3.0-r1
2020-08-13T05:58:08 >>> virtual/perl-CPAN-Meta-YAML-0.18.0-r6
2020-08-13T05:58:16 >>> virtual/perl-ExtUtils-Manifest-1.720.0-r1
2020-08-13T05:58:23 >>> virtual/perl-version-0.992.400-r1
2020-08-13T05:58:31 >>> virtual/perl-Digest-SHA-6.20.0-r1
2020-08-13T05:58:38 >>> virtual/
2020-08-13T06:49:09 >>> app-misc/tmux-3.1b
2020-08-13T06:49:31 >>> app-portage/eix-0.34.4
2020-08-13T06:50:15 >>> net-mail/fetchmail-6.4.8
2020-08-13T06:50:48 >>> net-misc/ntp-4.2.8_p15
2020-08-13T06:53:04 >>> sys-apps/usbutils-012
2020-08-13T06:53:29 >>> dev-lang/python-3.7.8-r2
2020-08-13T06:56:31 >>> dev-lang/python-3.6.11-r2
2020-08-13T06:58:48 >>> dev-lang/python-2.7.18-r1
2020-08-13T07:00:58 >>> dev-lang/python-3.8.4-r1
2020-08-13T07:03:21 >>> dev-libs/apr-util-1.6.1-r6
2020-08-13T07:03:41 >>> net-libs/gnutls-3.6.14
2020-08-13T07:05:17 >>> app-shells/zsh-5.8
2020-08-13T07:07:23 >>> dev-db/sqlite-3.32.3-r1
2020-08-13T07:08:28 >>> dev-python/certifi-10001
2020-08-13T07:08:42 >>> dev-libs/jsoncpp-1.9.3
2020-08-13T07:08:57 >>> app-admin/apache-tools-2.4.46
2020-08-13T07:09:21 >>> sys-devel/gdb-9.2
2020-08-13T07:13:45 >>> dev-python/setuptools-46.4.0-r1
2020-08-13T07:14:17 >>> dev-util/cmake-3.16.5
2020-08-13T07:16:18 >>> dev-python/six-1.15.0
2020-08-13T07:16:35 >>> dev-python/idna-2.10
2020-08-13T07:16:50 >>> dev-python/pytz-2020.1
2020-08-13T07:17:06 >>> dev-db/mysql-connector-c-8.0.21
2020-08-13T07:19:06 >>> dev-python/zope-interface-5.1.0
2020-08-13T07:19:32 >>> app-portage/gemato-14.4
2020-08-13T07:19:45 >>> dev-python/configargparse-1.2.3
2020-08-13T
2020-08-13T08:05:58 >>> virtual/rust-1.44.1
2020-08-13T08:06:07 >>> dev-python/requests-2.24.0
2020-08-13T08:06:23 >>> dev-python/requests-toolbelt-0.9.1
2020-08-13T08:06:39 >>> app-crypt/acme-1.6.0
2020-08-13T08:06:56 >>> app-crypt/certbot-1.6.0
2020-08-13T08:07:16 >>> dev-libs/cyrus-sasl-2.1.27-r3
2020-08-13T08:07:58 >>> dev-lang/perl-5.30.3
2020-08-13T08:10:56 >>> virtual/perl-File-Spec-3.780.0-r1
2020-08-13T08:11:06 >>> virtual/perl-Data-Dumper-2.174.0-r1
2020-08-13T08:11:14 >>> virtual/perl-MIME-Base64-3.150.0-r7
2020-08-13T08:11:23 >>> virtual/perl-ExtUtils-ParseXS-3.400.0-r1
2020-08-13T08:11:31 >>> virtual/perl-Test-Harness-3.420.0-r3
2020-08-13T08:11:39 >>> virtual/perl-Parse-CPAN-Meta-2.150.10-r4
2020-08-13T08:11:47 >>> virtual/perl-Time-Local-1.280.0-r1
2020-08-13T08:11:54 >>> virtual/perl-ExtUtils-Install-2.140.0-r3
2020-08-13T08:12:03 >>> virtual/perl-Perl-OSType-1.10.0-r4
2020-08-13T08:12:10 >>> virtual/perl-Text-ParseWords-3.300.0-r7
2020-08-13T08:12:18 >>> media-gfx/imagemagick-7.0.10.23
2020-08-13T08:14:18 >>> net-analyzer/rrdtool-1.7.2
2020-08-13T08:15:05 >>> www-servers/apache-2.4.46
2020-08-13T08:16:03 >>> virtual/perl-File-Path-2.160.0-r1
2020-08-13T08:16:11 >>> virtual/perl-Memoize-1.30.100_rc-r10
2020-08-13T08:16:19 >>> virtual/perl-libnet-3.110.0-r3
2020-08-13T08:16:27 >>> sys-apps/help2man-1.47.15
2020-08-13T08:16:37 >>> sys-apps/man-db-2.8.7
2020-08-13T08:18:16 >>> dev-db/mysql-8.0.20-r1
2020-08-13T08:55:02 >>> net-libs/courier-authlib-0.69.0-r1
2020-08-13T08:56:16 >>> mail-client/mutt-1.14.4-r1
2020-08-13T08:56:59 >>> net-mail/courier-imap-5.0.7
Comment 35 Zac Medico gentoo-dev 2020-08-14 18:39:19 UTC
(In reply to Alex Efros from comment #34)
> I've gnupg-2.2.20 installed several months ago, so probably it breaks
> because of updating some of it dependencies.

Or maybe something changed with the network related to ipv6 support?

> I was able to fix this by enabled CONFIG_IPV6 in kernel together with
> disabling IPv6 using sysctl:
> 
> net.ipv6.conf.all.disable_ipv6 = 1
> net.ipv6.conf.default.disable_ipv6 = 1
> net.ipv6.conf.lo.disable_ipv6 = 1
> 
> List of updated packages:
> 
> 2020-08-13T07:03:41 >>> net-libs/gnutls-3.6.14

You could try rolling back that gnutls update to see if that's the trigger.
Comment 36 Martin 2020-08-14 23:15:22 UTC
Is this actually a recent consequence of:

bug https://bugs.gentoo.org/736756

?

And also that there is not an appropriate key available by hkps?


Regards,
Martin
Comment 37 Zac Medico gentoo-dev 2020-08-15 18:35:09 UTC
gnupg-2.2.20-r1 has a patch that hopefully solves ipv6 issues for some people:

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=f880165f3ad8531f8b185108094f46a47c9e2fb4