Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 690760 - gnutls without trusted system directory in stage3 tar ball
Summary: gnutls without trusted system directory in stage3 tar ball
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Release Media
Classification: Unclassified
Component: Stages (show other bugs)
Hardware: All Linux
: Normal normal with 1 vote (vote)
Assignee: Anthony Basile
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 650144
  Show dependency tree
 
Reported: 2019-07-26 12:05 UTC by okannen
Modified: 2020-02-06 13:45 UTC (History)
8 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge -pt output from stage3 merge of portage (portage-mergetree.txt,8.03 KB, text/plain)
2019-12-02 21:11 UTC, Jeremy Drake
Details
output of emerge of portage in stage3 (portage-merge.txt,7.08 KB, text/plain)
2019-12-02 22:10 UTC, Jeremy Drake
Details

Note You need to log in before you can comment on or make changes to this bug.
Description okannen 2019-07-26 12:05:18 UTC
This bug affects gnutls only inside the installation tar ball, the package net-libs/gnutls is not affected.

After gentoo installation using the tar ball:
 * stage3-amd64-20190724T214501Z.tar.xz

TLS certification is not functionning, web-clients complain.

The reason is that /usr/lib64/libgnutls.so.30.23.2 does not appear to have a configured default trusted system directory.

It can be checked using this command:

  $ strings /usr/lib64/libgnutls.so.30.23.2 | grep etc

The output message will be "/etc/gnutls/default-priority". 

Normaly, it should also output "/etc/ssl/certs/ca-certificates.crt".

The gentoo installation can be fixed by emerging gnutls:
  $ emerge net-libs/gnutls
Comment 1 Michael Weiser 2019-08-06 15:38:13 UTC
I can confirm that this issue is still present in 
https://ftp.fau.de/gentoo/releases/amd64/autobuilds/current-stage3-amd64-systemd/stage4-amd64-systemd-20190803.tar.xz.

What's more is that emaint sync -a / emerge --sync fail because the gemato GPG key update fails because gpg's dirmngr cannot verify the Let's Encrypt-issued server certificate of the WKD on https://www.gentoo.org/. This causes a fallback to keyservers, which in turn retries indefinitely with a non-descriptive error message:

$ emaint sync -a
>>> Syncing repository 'gentoo' into '/usr/portage'...
 * Using keys from /usr/share/openpgp-keys/gentoo-release.asc
 * Refreshing keys via WKD ...                                                                                                        [ !! ]
 * Refreshing keys from keyserver hkps://keys.gentoo.org ...OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://keys.gentoo.org
gpg: keyserver refresh failed: General error

OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://keys.gentoo.org
gpg: keyserver refresh failed: General error
[loops indefinitely]
Comment 2 Ben Kohler gentoo-dev 2019-08-06 18:31:32 UTC
The autobuild process is definitely producing a broken libgnutls right now.  

My own independent build box was also producing broken libgnutls as of last week, but seems to be producing a working one now (after purging binpkgs so gnutls is freshly built).

I have just purged the gnutls binpkgs from our main autobuild box so that it gets freshly rebuilt there as well.  I'm hoping that whatever caused the problem is fixed now, and it's just the fresh rebuild which is needed.

The next autobuild should happen tomorrow night, so look for new builds to appear in 36 hrs or so.
Comment 3 Ben Kohler gentoo-dev 2019-08-08 14:22:17 UTC
Ok I just tested on brand new stage3-amd64-20190807T214502Z.tar.xz and it's looking better:

test-chroot ~ # strings /usr/lib64/libgnutls.so.30.23.2 | grep etc
/etc/gnutls/default-priorities
/etc/ssl/certs/ca-certificates.crt
test-chroot ~ # emerge --sync
>>> Syncing repository 'gentoo' into '/var/db/repos/gentoo'...
 * Using keys from /usr/share/openpgp-keys/gentoo-release.asc
 * Refreshing keys via WKD ...                                           [ ok ]
>>> Starting rsync with rsync://91.186.30.235/gentoo-portage...
Comment 4 Ben Kohler gentoo-dev 2019-08-14 17:01:29 UTC
Reassigning to blueness to take care of systemd stages
Comment 5 Anthony Basile gentoo-dev 2019-08-15 20:35:29 UTC
(In reply to Ben Kohler from comment #4)
> Reassigning to blueness to take care of systemd stages

i've cleared out all the binpkgs, the next round of build should be good.
Comment 6 Ben Kohler gentoo-dev 2019-09-05 20:37:43 UTC
I think we have all of these fixed now, thanks
Comment 7 Simon Levermann (sonOfRa) 2019-10-08 19:51:08 UTC
This still seems to affect some stage3 archives. I installed a new system 2 days ago, using stage3-amd64-systemd-20191003.tar.bz2, downloaded from http://distfiles.gentoo.org/releases/amd64/autobuilds/current-stage3-amd64-systemd/
Comment 8 Ben Kohler gentoo-dev 2019-10-08 20:08:49 UTC
@blueness, please fix
Comment 9 Anthony Basile gentoo-dev 2019-10-28 17:07:21 UTC
(In reply to Ben Kohler from comment #8)
> @blueness, please fix

I thought it was because I purged all the broken binpkgs.  There have been many cycles of catalyst runs since then, how is it that this problem is still persisting?

Can someone tell me how to make sure I don't get a broken gnutls?
Comment 10 Anthony Basile gentoo-dev 2019-10-31 20:52:43 UTC
(In reply to Anthony Basile from comment #9)
> (In reply to Ben Kohler from comment #8)
> > @blueness, please fix
> 
> I thought it was because I purged all the broken binpkgs.  There have been
> many cycles of catalyst runs since then, how is it that this problem is
> still persisting?
> 
> Can someone tell me how to make sure I don't get a broken gnutls?

I just did another catalyst run and I can't seem to fix this.  I purged all binpkgs and started from fresh.  I still get the problem in comment #0.  When I *rebuild* gnutls within the chroot, libgnutls is fixed, but the catalyst run itself leads to a broken libgnutls.

I don't understand what gnutls is doing, so I'm cc-ing the crypto team.
Comment 11 Jeremy Drake 2019-11-04 20:23:34 UTC
I am also seeing this issue with my own catalyst builds on arm.  I do not recycle binpkgs between runs, but my last two build attempts (on 2019-10-22 and 2019-11-01) could not refresh gentoo keys via WKD or hkps until I re-merged net-libs/gnutls.  emerge-webrsync did work for me to get an initial tree so that I had an ebuild for gnutls to merge.

Maybe this is a dependency/ordering issue where the system certificate location does not yet exist when gnutls is built?
Comment 12 Anthony Basile gentoo-dev 2019-11-05 12:13:26 UTC
(In reply to Jeremy Drake from comment #11)

> Maybe this is a dependency/ordering issue where the system certificate
> location does not yet exist when gnutls is built?

I agree, the fix would have to be in gnutls ebuild.

@crypto.  Can you please look at this and advise.
Comment 13 Anthony Basile gentoo-dev 2019-12-01 17:53:15 UTC
(In reply to Anthony Basile from comment #12)
> (In reply to Jeremy Drake from comment #11)
> 
> > Maybe this is a dependency/ordering issue where the system certificate
> > location does not yet exist when gnutls is built?
> 
> I agree, the fix would have to be in gnutls ebuild.
> 
> @crypto.  Can you please look at this and advise.

@crypto team.  Help please.  I'm still not able to get past this.
Comment 14 Kristian Fiskerstrand gentoo-dev Security 2019-12-02 12:10:55 UTC
Does the catalyst run have /etc/ssl/certs/ca-certificates.crt available on gnutls compile time? ( app-misc/ca-certificates )

 This is tested for in configure.ac 
856 dnl auto detect https://lists.gnu.org/archive/html/help-gnutls/2012-05/msg     00004.html
 857 AC_ARG_WITH([default-trust-store-file],
 858   [AS_HELP_STRING([--with-default-trust-store-file=FILE],
 859     [use the given file default trust store])], with_default_trust_store_f     ile="$withval",
 860   [if test "$build" = "$host" && test x$with_default_trust_store_pkcs11 =      x && test x$with_default_trust_store_dir = x && test x$have_macosx = x;the     n
 861   for i in \
 862     /etc/ssl/ca-bundle.pem \
 863     /etc/ssl/certs/ca-certificates.crt \
 864     /etc/pki/tls/cert.pem \
 865     /usr/local/share/certs/ca-root-nss.crt \
 866     /etc/ssl/cert.pem
 867     do 
 868     if test -e "$i"; then
 869       with_default_trust_store_file="$i"
 870       break
 871     fi
 872   done
 873   fi]
Comment 15 Kristian Fiskerstrand gentoo-dev Security 2019-12-02 12:11:49 UTC
(In reply to Kristian Fiskerstrand from comment #14)
> Does the catalyst run have /etc/ssl/certs/ca-certificates.crt available on
> gnutls compile time? ( app-misc/ca-certificates )


... so one alternative is of course overriding with_default_trust_store_file="$i" to be static string for the purpose of catalyst, and see if that helps things along
Comment 16 Jeremy Drake 2019-12-02 21:08:24 UTC
(In reply to Kristian Fiskerstrand from comment #14)
> Does the catalyst run have /etc/ssl/certs/ca-certificates.crt available on
> gnutls compile time? ( app-misc/ca-certificates )

I just ran a test, and no, /etc/ssl does not exist at the time net-libs/gnutls is built.

It seems that gnutls is now an indirect dependency of portage (with default USE flags), so it is getting built really early in the stage3 process, before it even starts to emerge @system.  app-misc/ca-certificates are also part of portage's dependency tree, and is merged as part of that early process (apparently due to a PDEPEND in openssl).

> 
>  This is tested for in configure.ac 

That sounds to me like gnutls has an undeclared build-time dependency on app-misc/ca-certificates, and the solution should be to either declare that dependency, or to remove it as you suggested by overriding the configure check.
Comment 17 Jeremy Drake 2019-12-02 21:11:07 UTC
Created attachment 598292 [details]
emerge -pt output from stage3 merge of portage

This is the dependency tree being built when portage is being merged for stage3.
Comment 18 Jeremy Drake 2019-12-02 22:10:10 UTC
Created attachment 598294 [details]
output of emerge of portage in stage3

This is the (quiet) output of the actual merge of portage for stage3.
Comment 19 Kristian Fiskerstrand gentoo-dev Security 2019-12-03 08:30:03 UTC
(In reply to Jeremy Drake from comment #16)
> (In reply to Kristian Fiskerstrand from comment #14)


> 
> > 
> >  This is tested for in configure.ac 
> 
> That sounds to me like gnutls has an undeclared build-time dependency on
> app-misc/ca-certificates, and the solution should be to either declare that
> dependency, or to remove it as you suggested by overriding the configure
> check.

No, its not a build dep in general, only for the functionality wanted in this specific use case. Catalyst run can ensure this file exists (it doesn't even have to be populated) for the wanted functionality to present itself.
Comment 20 Larry the Git Cow gentoo-dev 2019-12-13 20:08:17 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=b10605ee20c39968635a530cd0830b53f425baf1

commit b10605ee20c39968635a530cd0830b53f425baf1
Author:     Thomas Deutschmann <whissi@gentoo.org>
AuthorDate: 2019-12-13 20:07:26 +0000
Commit:     Thomas Deutschmann <whissi@gentoo.org>
CommitDate: 2019-12-13 20:08:09 +0000

    net-libs/gnutls: use system certificate store by default
    
    Closes: https://bugs.gentoo.org/690760
    Acked-by: Kristian Fiskerstrand <k_f@gentoo.org>
    Package-Manager: Portage-2.3.81, Repoman-2.3.20
    Signed-off-by: Thomas Deutschmann <whissi@gentoo.org>

 net-libs/gnutls/{gnutls-3.6.10.ebuild => gnutls-3.6.10-r1.ebuild}   | 1 +
 net-libs/gnutls/{gnutls-3.6.11.1.ebuild => gnutls-3.6.11-r1.ebuild} | 1 +
 net-libs/gnutls/{gnutls-3.6.11.ebuild => gnutls-3.6.11.1-r1.ebuild} | 1 +
 net-libs/gnutls/{gnutls-3.6.7.ebuild => gnutls-3.6.7-r1.ebuild}     | 1 +
 4 files changed, 4 insertions(+)