Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 548436

Summary: app-misc/ca-certificates-20141019.3.17.4 - GTE CyberTrust Global Root is not trusted
Product: Gentoo Linux Reporter: Volker Hemmann <volkerarmin>
Component: Current packagesAssignee: Gentoo's Team for Core System packages <base-system>
Status: RESOLVED DUPLICATE    
Severity: normal CC: kensington, whissi
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
URL: http://civilwardailygazette.com/
Whiteboard:
Package list:
Runtime testing required: ---

Description Volker Hemmann 2015-05-02 22:27:45 UTC
Konqueror:
with app-misc/ca-certificates-20141019.3.17.4
the GTE CyberTrust Global Root
SHA1-Digest 4d34ea92764b3a3149119952f41930ca11348361

is seen as invalid.

Which is really bad because akamai among others is depending on it.

akamai.net: Cybertrust Public SureServer SV CA 
Sha1: f01a81f9c6c0a1ffb26b477fa38145ce428a4ff9
is trustworthy, next in chain:

Baltimore CyberTrust Root
Sha1: 30d1fd4a296ab1a8831cd56b4110a227f557bfff
is trustworthy, next in chain:

GTE CyberTrust Global Root
SHA1-Digest 4d34ea92764b3a3149119952f41930ca11348361
is not.

and bam, you are spammed with popups telling you that the site you are trying to visit is not trustworthy. 

app-misc/ca-certificates-20140927.3.17.2
does not show this behaviour

Reproducible: Always

Steps to Reproduce:
1. use konqueror to browse the web
2. emerge app-misc/ca-certificates-20141019.3.17.4
3. go to any site making use of akamai. Like http://civilwardailygazette.com or http://dlisted.com
Actual Results:  
spamfest of popups telling me that some connection is not trustworthy

Expected Results:  
no popups at all, like before.
Comment 1 Michael Palimaka (kensington) gentoo-dev 2015-05-03 12:58:18 UTC
I guess this is not restricted to konqueror.
Comment 2 Volker Hemmann 2015-05-03 14:37:41 UTC
chromium and firefox do not complain. I guess they either don't care or have that certificate in their own repository.
Comment 3 Thomas Deutschmann (RETIRED) gentoo-dev 2015-05-03 19:15:31 UTC
Could you please share in future the actual HTTPS URL which is causing the problem? The sites you mentioned don't use HTTPS itself and crawling for embedded resources makes everything harder than necessary. 

You are hit by https://bugzilla.mozilla.org/show_bug.cgi?id=1047011 (removal of 1024bit Root CA) which finally happen with NSS 3.17.3 [1].

I am unsure about the quoted SHA1 from you. But this will be the same reason like described in bug 544276 comment 3: The server is sending the wrong IM certificate so that the new Baltimore CyberTrust root ca won't be used.

Don't expect that Gentoo will undo the removal. Try contacting sites which are using an invalid IM.

If you don't care about security that much you can still re-install the removed CA for your own: Just place a copy of the certificate in /usr/local/share/ca-certificates and run update-ca-certificates.

But keep in mind that your are now trusting a 1024bit root again...


See also:
=========
[1] https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/NSS_3.17.3_release_notes
Comment 4 Volker Hemmann 2015-05-03 19:28:59 UTC
the 'sites' is akamai.

Plus pretty much a billion ad servers....
Comment 5 Thomas Deutschmann (RETIRED) gentoo-dev 2015-05-03 20:02:54 UTC
This bug report is filled against app-misc/ca-certificates.

Gentoo's trust store (that's /etc/ssl/certs, the directory you will get with app-misc/ca-certificates) is the same like if you would install a plain Firefox 37 on Windows or MacOS.
Remember, app-misc/ca-certificates-20141019.3.17.4 is in tree for 6 weeks, now.

If there would be a general problem with the certificates we ship there would be more bug reports, especially if this will affect any Akamai HTTPS like you said. Not?

So this is why I am saying this is not a general problem. Maybe you should fill a bug against konqueror if it expects something special. But to investigate what's the problem, we need to know the HTTPS URL which is triggering the problem for you... something we can use with "openssl s_client" or "gnutls-cli".
Comment 6 Volker Hemmann 2015-05-03 23:14:54 UTC
on loading civilwargazette I get the popup and these infos:
Adress: fbstatic-a.akamaihd.net
IP: 217.89.105.216

SSL-Version TLSv1.2
certificate chain a248.e.akamai.net

Name a248.e.akamai.net
Organsation Akamai Technologies, Inc.
Valid until 12.06.2014 20:35 bis 12.06.2015 20:35
Serial number 02:00:00:00:00:01:46:91:cb:0b:26:5b:ef:fe
md5-digest: 463fb080fd9dec6088c9df41d806d333
sha1:f01a81f9c6c0a1ffb26b477fa38145ce428a4ff9

trustworthy yes

next in chain is Cybertrust Public SureServer as posted already

next in chain is Baltimore CyberTrust Global Root

as posted already.

another popup with this info:
Address: fbcdn-profile-a.akamaihd.net
IP: 217.89.105.209

and then the same.

on phoronix the chain starts with
tags.bkrtx.com
IP:2.16.99.149
encryption:AES, bei Verwendung von 256 Bit eines Schlüssels mit 256 Bit
ssl version: SSLv3

*.bkrtx.com
Oracle Corporation
serial: 0c:35:7b:a5:6d:70:b5:b2:77:dc:c4:0a:21:22:81:05:35:8c:c9:20
md5: 85426615c5bcf7e5b8ad5ca564bbfb54
sha1: 1eda250e4fb3c7b5b3127c785c7fb1db0d3c3d2a

which is trustworthy

goes next to
Verizon Akamai SureServer CA G14-SHA1
serial: 120038506
md5: 89007e20a67eea4562770b25220e1257
sha1: 3209cc3ce4f1c22ab64b2e4284100b0022ad2739


that is all the information I get

gnutls-cli fbstatic-a.akamaihd.net
Processed 174 CA certificate(s).
Resolving 'fbstatic-a.akamaihd.net'...
Connecting to '217.89.105.216:443'...
- Certificate type: X.509
- Got a certificate list of 3 certificates.
- Certificate[0] info:
 - subject `C=US,ST=MA,L=Cambridge,O=Akamai Technologies\, Inc.,CN=a248.e.akamai.net', issuer `O=Cybertrust Inc,CN=Cybertrust Public SureServer SV CA', RSA key 2048 bits, signed using RSA-SHA1, activated `2014-06-12 20:35:45 UTC', expires `2015-06-12 20:35:45 UTC', SHA-1 fingerprint `f01a81f9c6c0a1ffb26b477fa38145ce428a4ff9'
        Public Key ID:
                729d81d2de120831624910be826e1ce8f306e4b2
        Public key's random art:
                +--[ RSA 2048]----+
                |o++.+.           |
                |.... o o .       |
                | .    o + .      |
                |o..    o + o     |
                |*o    . S +      |
                |*o.    o .       |
                |.B.              |
                |E o.             |
                |  ..             |
                +-----------------+

- Certificate[1] info:
 - subject `O=Cybertrust Inc,CN=Cybertrust Public SureServer SV CA', issuer `C=IE,O=Baltimore,OU=CyberTrust,CN=Baltimore CyberTrust Root', RSA key 2048 bits, signed using RSA-SHA1, activated `2010-09-08 17:35:16 UTC', expires `2020-09-08 17:34:08 UTC', SHA-1 fingerprint `30d1fd4a296ab1a8831cd56b4110a227f557bfff'
- Certificate[2] info:
 - subject `C=IE,O=Baltimore,OU=CyberTrust,CN=Baltimore CyberTrust Root', issuer `C=US,O=GTE Corporation,OU=GTE CyberTrust Solutions\, Inc.,CN=GTE CyberTrust Global Root', RSA key 2048 bits, signed using RSA-SHA1, activated `2012-04-18 16:36:18 UTC', expires `2018-08-13 16:35:17 UTC', SHA-1 fingerprint `4d34ea92764b3a3149119952f41930ca11348361'
- Status: The certificate is trusted. 
- Description: (TLS1.2)-(ECDHE-RSA-SECP256R1)-(AES-256-GCM)
- Session ID: 9D:AA:7C:A2:1E:7A:44:94:DB:6C:8C:EF:3D:BF:48:73:18:47:03:75:0B:2D:6A:38:F1:EB:0A:10:76:3C:47:23
- Ephemeral EC Diffie-Hellman parameters
 - Using curve: SECP256R1
 - Curve size: 256 bits
- Version: TLS1.2
- Key Exchange: ECDHE-RSA
- Server Signature: RSA-SHA256
- Cipher: AES-256-GCM
- MAC: AEAD
- Compression: NULL
- Options:
- Handshake was completed

- Simple Client Mode:

- Peer has closed the GnuTLS connection
Comment 7 Thomas Deutschmann (RETIRED) gentoo-dev 2015-05-04 00:22:58 UTC
That's not what I am asking for. However, based on your answer, I am seeing 2 requests to fbstatic-a.akamaihd.net.

Please open https://fbstatic-a.akamaihd.net/rsrc.php/v2/y1/r/LVx-xkvaJ0b.png in your konqueror browser. A sprite graphic (2x thumbs up, 2x FB icons) should appear. Do you see the image without a SSL warning/prompt? I expect that this will trigger a warning/prompt.


You can reproduce the same with wget.


It is exactly the problem from bug 544276 comment 3: fbstatic-a.akamaihd.net sends 3 certificates, the last one is the "GTE CyberTrust Global Root" (4D:34:EA:92:76:4B:3A:31:49:11:99:52:F4:19:30:CA:11:34:83:61) which can't be verified because it is already a root certificated (not issued by anything) and not present in the local trust store.
If the host would only send 2 certificates (=omit the GTE CyberTrust certificate) we could validate the 2nd certificate (Baltimore CyberTrust Root, 30:D1:FD:4A:29:6A:B1:A8:83:1C:D5:6B:41:10:A2:27:F5:57:BF:FF) because this certificate is in the trust store (=the 2nd certificate is a trusted ca). 


What we can do:

1) Wait until openssl-1.1.0 will be released. As written in bug 544276 comment 3, OpenSSL won't be that strict in the new version and accepts alternative cert paths per default.

2) Try to contact Facebook/Akamai. If they would "fix" their setup, this won't be a problem (but as shown in the other bug, there will be other sites affected by the same problem).

3) Maybe we shouldn't apply changes from NSS to the trust store. Then we would still ship the GTE CyberTrust certificate like Debian Jessie. It is NSS >=3.17.3 which will remove GTE CyberTrust from trust store.
One could argue, that NSS changes should only be used by Mozilla products (because Mozilla already supports alternate cert paths), not in general.
Comment 8 Thomas Deutschmann (RETIRED) gentoo-dev 2015-05-04 00:38:50 UTC
CC'ing SpanKY who added NSS usage in app-misc/ca-certificates (via bug 504670).
Comment 9 SpanKY gentoo-dev 2015-05-04 02:58:59 UTC
the point of moving to NSS was to avoid the customizations Debian makes since updates can be sporadic.  on the other hand, the nss store is the same thing that firefox uses.

*** This bug has been marked as a duplicate of bug 544276 ***
Comment 10 Volker Hemmann 2015-05-04 15:42:59 UTC
it is not 'just' facebook or akamai or rackcdn or bkrtx.com-.

There are hundreds. 

With some websites where I have to click away half a douzend of those annoying security popups.

So, could somebody please unbreak ca-certificates?
Comment 11 SpanKY gentoo-dev 2015-05-12 07:05:02 UTC
there's nothing to change in ca-certificates.  it is simply shipping the cert store from mozilla (via nss).  changes to the cert store should be reported to mozilla as we are not making any changes to it.

ca-certificates-20141019.3.19 is in the tree.  no idea if it makes a difference.
Comment 12 Thomas Deutschmann (RETIRED) gentoo-dev 2015-05-12 15:57:07 UTC
(In reply to SpanKY from comment #11)
> ca-certificates-20141019.3.19 is in the tree.  no idea if it makes a
> difference.

No, it doesn't.


> there's nothing to change in ca-certificates.  it is simply shipping the
> cert store from mozilla (via nss).  changes to the cert store should be
> reported to mozilla as we are not making any changes to it.

I think you miss the point:

Plain ca-certificate from Mozilla works. It contains missing certificate to validate the cert chain from fbstatic-a.akamaihd.net and others.

But on Gentoo we do _not_ use plain ca-certificate. We add changes from NSS on top. That's the problem: Because NSS supports alternate trust chains, this works for software which uses NSS. But software using pure OpenSSL (which doesn't support alternate trust chains yet, support is coming!) will fail.

Proof:
======
1) emerge app-misc/ca-certificates-20141019.3.19.

2) "openssl s_client -connect fbstatic-a.akamaihd.net:443 < /dev/null" will fail - the cert chain can't be validated.

3) Just rename ca-certificates-20141019.3.19.ebuild to ca-certificates-20141019.ebuild. This will disable NSS. Well, you also have to fix src_unpack(), i.e. disable "mv ${PN}-*/ ${PN} || die"). 

4) Now emerge your modified ca-certificates-20141019.ebuild

5) Rerun the openssl command from step 2 => Your are now able to connect to fbstatic-a.akamaihd.net - the cert chain can be validated.



My point is simple: Because /etc/ssl/certs is used by multiple application which don't support alternate trust chains (like OpenSSL), we have to use plain ca-certificates.

The changes from NSS to ca-certificate works only with NSS. So this is _not_ for general usage.
Comment 13 Volker Hemmann 2015-05-12 17:33:56 UTC
you can't even visit www.gentoo.org without being spammed with error messages.

So, can somebody please unbreak certificates?
Comment 14 SpanKY gentoo-dev 2015-05-13 02:49:33 UTC
(In reply to Thomas D. from comment #12)

i don't think you understand what ca-certificates is.  it is not from mozilla -- they have nothing to do with it actually.  it is from Debian, and they take Mozilla's work and modify it.
Comment 15 Thomas Deutschmann (RETIRED) gentoo-dev 2015-05-13 11:54:40 UTC
(In reply to SpanKY from comment #14)
> i don't think you understand what ca-certificates is.  it is not from
> mozilla -- they have nothing to do with it actually.  it is from Debian, and
> they take Mozilla's work and modify it.

app-misc/ca-certificates from Gentoo uses the ca-certificates package from Debian as source (but adds changes from Mozilla's NSS on top -- that's why on Debian Jessie with ca-certificates-20141019 everything works and on Gentoo not; only if you are also using an application using a SSL library which supports alternate cert chains like NSS or gnutls).

ca-certificates from Debian is based on http://hg.mozilla.org/mozilla-central/filelog/62d9b117c688/security/nss/lib/ckfw/builtins/certdata.txt

Again, if we wouldn't apply changes from NSS on top of Debian's package people using OpenSSL for SSL would _not_ have any problems.


With:

net-misc/wget-1.16.3::gentoo was built with the following:
USE="idn ipv6 nls pcre ssl zlib -debug -gnutls -ntlm -static -test -uuid" ABI_X86="64"

# wget -O /dev/null https://fbstatic-a.akamaihd.net/rsrc.php/v2/y1/r/LVx-xkvaJ0b.png
--2015-05-13 13:49:59--  https://fbstatic-a.akamaihd.net/rsrc.php/v2/y1/r/LVx-xkvaJ0b.png
Resolving fbstatic-a.akamaihd.net (fbstatic-a.akamaihd.net)... 2003:8:0:a::d959:69e1, 2003:8:0:a::d959:69e3, 2003:8:0:a::d959:69e9, ...
Connecting to fbstatic-a.akamaihd.net (fbstatic-a.akamaihd.net)|2003:8:0:a::d959:69e1|:443... connected.
ERROR: cannot verify fbstatic-a.akamaihd.net's certificate, issued by ‘CN=Cybertrust Public SureServer SV CA,O=Cybertrust Inc’:
  Unable to locally verify the issuer's authority.
To connect to fbstatic-a.akamaihd.net insecurely, use `--no-check-certificate'.


If you now rebuild wget with gnutls, which already support alternate cert chains:

net-misc/wget-1.16.3::gentoo was built with the following:
USE="gnutls idn ipv6 nls pcre ssl zlib -debug -ntlm -static -test -uuid" ABI_X86="64"

# wget -O /dev/null https://fbstatic-a.akamaihd.net/rsrc.php/v2/y1/r/LVx-xkvaJ0b.png
--2015-05-13 13:53:48--  https://fbstatic-a.akamaihd.net/rsrc.php/v2/y1/r/LVx-xkvaJ0b.png
Resolving fbstatic-a.akamaihd.net (fbstatic-a.akamaihd.net)... 2003:8:0:a::d959:69e1, 2003:8:0:a::d959:69d8, 62.156.209.49, ...
Connecting to fbstatic-a.akamaihd.net (fbstatic-a.akamaihd.net)|2003:8:0:a::d959:69e1|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1329 (1,3K) [image/png]
Saving to: ‘/dev/null’

/dev/null                     100%[===================================================>]   1,30K  --.-KB/s   in 0s

2015-05-13 13:53:48 (50,4 MB/s) - ‘/dev/null’ saved [1329/1329]




Please comment on that. Why do you apply changes from NSS on a cert store which isn't only used by NSS-enabled applications when these changes are not yet supported by non-NSS-enabled applications.

If /et/ssl/cert is now a NSS store - fine. But this is a change. Then we need another cert store for OpenSSL and other which don't support alternate cert chains yet (see http://rt.openssl.org/Ticket/Display.html?id=3637&user=guest&pass=guest).


BTW: The Debian maintainer is preparing a new release (20150426): http://anonscm.debian.org/cgit/collab-maint/ca-certificates.git/commit/?id=651de1c8439c6206ea76b7d502daa9de05b5a461

This release is updated to the latest certdata.txt (which is now in sync with NSS) so GTE CyberTrust will be missing on Debian, too.

Seems like the package wasn't uploaded yet. I'll contact the maintainer and ask him if he is aware of that problem (maybe that's the reason why the new package wasn't uploaded yet).
Comment 16 SpanKY gentoo-dev 2015-05-17 14:39:48 UTC
(In reply to Thomas D. from comment #15)

we aren't using ca-certificates for its cert db.  we're using it for its update-ca-certificates tool and config files and man pages.

we aren't applying updates "on top of Debian", we're completely ignoring their outdated snapshot and taking the latest from nss.

/etc/ssl/certs/ isn't an "nss store", it's a directory full of certificates.  you're wrongly conflating nss's binary format and the set of certs that nss itself bundles.  this is exactly what Debian is doing, except our ebuilds are more up-to-date than Debian by virtue of pulling from the latest nss release.
Comment 17 Volker Hemmann 2015-06-19 01:03:51 UTC
and one month since the last comment, the whole thing is still broken...
Comment 18 Thomas Deutschmann (RETIRED) gentoo-dev 2015-06-19 08:32:20 UTC
(In reply to SpanKY from comment #16)
> we aren't using ca-certificates for its cert db.  we're using it for its
> update-ca-certificates tool and config files and man pages.
> 
> we aren't applying updates "on top of Debian", we're completely ignoring
> their outdated snapshot and taking the latest from nss.

Only when the version number contains "nss".


> /etc/ssl/certs/ isn't an "nss store", it's a directory full of certificates.
> you're wrongly conflating nss's binary format and the set of certs that nss
> itself bundles.  this is exactly what Debian is doing, except our ebuilds
> are more up-to-date than Debian by virtue of pulling from the latest nss
> release.

It is a "nss" store because the certificate files included in upstream source package were selected on the basis whether they work with applications using the NSS lib or not.

Because NSS already supports alternative certification paths (like GnuTLS), they were able to get rid of roots like the GTE one.

But we are using the same certificates for every SSL lib, whether they support alternative paths already or not. And because OpenSSL doesn't do that at the moment, /etc/ssl/certs is incomplete (incompatible) for applications using OpenSSL because OpenSSL is strict and tries to resolve the path a SSL host is sending instead of checking wether the path could be resolved differently.
Comment 19 SpanKY gentoo-dev 2015-06-19 08:38:00 UTC
(In reply to Volker Hemmann from comment #17)

as has been stated multiple times, there is nothing to be fixed on Gentoo's side.  we ship the cert store that mozilla's nss does.  if you don't like that, take it up with mozilla.  we specifically are not maintaining changes on top of it so that we don't diverge from other systems.

then again, things seem to work fine for me.  i can `wget` and `openssl s_client` to fbstatic-a.akamaihd.net using openssl-1.0.1o and ca-certificates-20141019.3.19 w/out errors.
Comment 20 Thomas Deutschmann (RETIRED) gentoo-dev 2015-06-19 09:20:19 UTC
(In reply to SpanKY from comment #19)
> then again, things seem to work fine for me.  i can `wget` and `openssl
> s_client` to fbstatic-a.akamaihd.net using openssl-1.0.1o and
> ca-certificates-20141019.3.19 w/out errors.

You are right. Alternative chain support landed in 1.0.1 (first release was in openssl-1.0.1o):

https://github.com/openssl/openssl/commit/f7bf8e02dfcb2c02bc12a59276d0a3ba43e6c204

So this problem has a solution, openssl-1.0.1o is marked stable within Gentoo.
Comment 21 Thomas Deutschmann (RETIRED) gentoo-dev 2015-06-19 09:36:25 UTC
To prevent problems like this I am suggesting that >app-misc/ca-certificates-20140927.3.17.2 should RDEPEND on >=dev-libs/openssl-1.0.1o. See bug 552540.