Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 597800 - emerge-webrsync: Downloads .gpgsig by default, but doesn't verify it.
Summary: emerge-webrsync: Downloads .gpgsig by default, but doesn't verify it.
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Tools (show other bugs)
Hardware: All Linux
: High critical with 2 votes (vote)
Assignee: Portage team
URL:
Whiteboard: Fixed in portage-3.0.47
Keywords: PullRequest
Depends on: 597918 903973
Blocks: CVE-2016-20021
  Show dependency tree
 
Reported: 2016-10-22 15:56 UTC by Walter
Modified: 2023-08-19 15:33 UTC (History)
10 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Walter 2016-10-22 15:56:55 UTC
This is dangerous as it is insecure default behavior. There may be an argument (poor) for keeping insecure default behavior. However, it is also doubly dangerous because it may imply to the user that the operation has been cryptographically verified when in fact it has not.

Reproducible: Always

Steps to Reproduce:
1. emerge-webrsync
2. "Fetching file portage-20161021.tar.xz.gpgsig ..."
Actual Results:  
User things "cool, it's verified"... but it's not verified, and no warning is shown to this effect.

Expected Results:  
All downloads are cryptographically verified or, failing that, a big (preferably red, flashing) SCARY WARNING is shown.


Probably it should whinge immediately and require the addition of an --insecure command line flag.
Comment 1 Walter 2016-10-22 15:57:41 UTC
Right now one must follow the obtuse, well-buried instructions @ https://wiki.gentoo.org/wiki/Handbook:AMD64/Working/Features#Validated_Portage_tree_snapshots to get it to actually verify.
Comment 2 Walter 2016-10-22 15:58:50 UTC
Further, I believe *emerge-webrsync* is the currently recommended sync method in the handbook, and no mention is made of configuring it for secure operation.
Comment 3 Walter 2016-10-22 16:09:46 UTC
Further, I believe this may be a serious enough vulnerability to issue a security advisory over, since people with high-security requirement systems may not be aware that their syncs are not being validated.

As I see it, MITM on sync + subsequent MITM'd emerge means rooted box, since emerge relies on Manifests pulled in via sync, and in current default behavior, sync is not validated except for md5sum, which is also MITM'd at the same time as SYNC, and often pulled via HTTP (so not even HTTPS levels of assurance).
Comment 4 Zac Medico gentoo-dev 2016-10-22 21:33:06 UTC
(In reply to Walter from comment #0)
> This is dangerous as it is insecure default behavior. There may be an
> argument (poor) for keeping insecure default behavior.

I think the existing support should be improved before we make it default, since the existing support requires manual verification of the key fingerprint that is posted here:

https://wiki.gentoo.org/wiki/Project:RelEng#Keys

I think we need an automated way to establish the chain of trust (people can manually double-check the key fingerprints if they desire). In order to accomplish this, the bare minimum would be to automate fetch and verification of key fingerprints via https. Alternatively, we could have an installed bundle of fingerprints, or an installed bundle of trusted public keys in ASCII armor format.

> doubly dangerous because it may imply to the user that the operation has
> been cryptographically verified when in fact it has not.

Yes, it's possible that some users could gain false confidence from this.

(In reply to Walter from comment #3)
> Further, I believe this may be a serious enough vulnerability to issue a
> security advisory over, since people with high-security requirement systems
> may not be aware that their syncs are not being validated.

Perhaps. Adding security to CC so that they can evaluate it.
Comment 5 Walter 2016-10-23 01:45:23 UTC
Re: automated establishment of trust, see https://bugs.gentoo.org/show_bug.cgi?id=453620 which I was also pushing for (3 years now).

The key issue there is that apparently the decision was made to use a python-based mechanism for key acquisition, and the normal 'establish base trust' situation is a fresh install from arbitrary (eg. non-gentoo, python-unavailable) media.

Therefore I motion that, in light of the severity of the current situation, placing current keys in the stage3 archives is strongly considered. (Possibly/probably along with that python-based tool.)

This has, at this point, simply been too severe a problem for far too long, and would solve the default insecure behavior for emerge-webrsync as well as for older/traditional rsync-based syncs.
Comment 6 Walter 2016-10-23 02:01:00 UTC
Actually almost 4 years.

Thinking with regards to the danger of downloading a compromised stage3 image with compromised keys (as unfortunately it seems all mirrors right now are insecure http, rsync or ftp), the handbook should also be updated to include a recommendation to use an alternate device or internet connection to download the GPG signature of the installation media (or of the current keys file) from a fixed, gentoo-hosted HTTPS URL.

The use of an alternate device with its own certificate chain and potentially internet connection (eg. phone in mobile data mode) is far more secure than any same-system approach, and remains viable for most people most of the time.
Comment 7 Jason A. Donenfeld gentoo-dev 2016-10-23 09:02:15 UTC
Jeeze louise. This is a mess indeed. I'm traveling now but will look into fixing the default behavior when I'm home. At the very least, if the current keychain doesn't have the necessary keys for verification, a big ugly warning message should be posted, "WARNING WARNING: you've sync'd with webrsync-gpg, but your gpg has issues XYZ, so this was not verified.", with possibly a different exit status too.
Comment 8 Zac Medico gentoo-dev 2016-10-23 20:38:44 UTC
I've filed bug 597918 for automated establishment of trust.

(In reply to Walter from comment #6)
> Thinking with regards to the danger of downloading a compromised stage3
> image with compromised keys (as unfortunately it seems all mirrors right now
> are insecure http, rsync or ftp), the handbook should also be updated to
> include a recommendation to use an alternate device or internet connection
> to download the GPG signature of the installation media (or of the current
> keys file) from a fixed, gentoo-hosted HTTPS URL.

The fingerprints provided via https are sufficient to establish trust, given that the corresponding keys can be used to verify *.DIGESTS.asc file that's distributed with the stage3, and the verified digests can be used to verify the stage3 itself.
Comment 9 Brian Dolbec (RETIRED) gentoo-dev 2016-10-23 21:08:10 UTC
And having install media and stage3's or stage4's with gentoo-keys and gkeys installed is also planned.

Along with the new meta-manifest system which also gpg signs all manifests and will be gpg verifed at every sync (normal rsync, not webrsync).
Comment 10 Kristian Fiskerstrand (RETIRED) gentoo-dev 2016-10-29 18:25:38 UTC
> 
> (In reply to Walter from comment #3)
> > Further, I believe this may be a serious enough vulnerability to issue a
> > security advisory over, since people with high-security requirement systems
> > may not be aware that their syncs are not being validated.
> 
> Perhaps. Adding security to CC so that they can evaluate it.

This issue likely require a CVE as expectations are for the OpenPGP signature to be verified. This is used by a Gentoo developer as late as in [1]. That is is not by default signify a ... very interesting ... design

References:
[1] https://www.reddit.com/r/Gentoo/comments/59yvl9/where_is_the_gentoo_foundation_and_the_source/d9cpy3s/
Comment 11 Walter 2016-10-29 23:05:55 UTC
I have added a discussion item to the relevant Gentoo Handbook page requesting the addition of explicit directions to secure portage for new users. https://wiki.gentoo.org/wiki/Handbook_Talk:AMD64/Installation/Stage#Portage_security_recommendation
Comment 12 Walter 2016-12-28 02:34:15 UTC
Nudge. Nobody has bothered to respond to or integrate the handbook suggestions.
Comment 13 Matthew Marchese Gentoo Infrastructure gentoo-dev 2016-12-31 04:24:32 UTC
I'm looking into the Handbook suggestions found here:

https://wiki.gentoo.org/wiki/Handbook_Talk:AMD64/Installation/Stage#Portage_and_stage3_security_recommendations

Closing some other discussions up (these things need testing, so it takes quite a bit of time), then I'll make my rounds to this discussion.
Comment 14 Walter 2017-05-30 07:17:51 UTC
Nudge.
Comment 15 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-03-29 13:41:44 UTC
I'll get this sorted now.
Comment 16 Larry the Git Cow gentoo-dev 2023-04-09 06:52:06 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=28cd240fb23d880b8641a058831c6762db71c3e2

commit 28cd240fb23d880b8641a058831c6762db71c3e2
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2023-03-29 22:34:37 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2023-04-09 06:51:35 +0000

    emerge-webrsync: support PGP verification via gemato
    
    Introduce PGP verification of the webrsync snapshot tarballs
    using app-portage/gemato - which is already a dependency of Portage
    for verifying normal rsync.
    
    This is the same method Portage uses (see below).
    
    Technical changes before we dive into the rationale:
    - Use gemato for PGP verification just like Portage does for
      sync-type=webrsync, sync-type=rsync (although that uses a metamanifest),
      and sync-type=git (although that uses gemato for gpg-wrap, so works differently).
    
    - Use gentoo-functions automagically if available for better output
      functions.
    
    - Be more verbose about verification and various other operations,
      while also respecting --quiet if passed for misc. existing & new
      messages.
    
    - Make --verbose a no-op. There weren't enough output messages
      to justify three states (--quiet, normal, --verbose).
    
    - Bail out more aggressively in the event of errors or "warnings".
    
    - Use modern terminology for repository, etc (avoid overloading the
      "portage" term.)
    
    - Allow disabling PGP verification with --no-pgp-verify.
    
    Technically, the fix is very straightforward, but getting to
    the fix was the slightly painful bit. What I've concluded
    happened is:
    - Portage starts getting reworked to gain proper sync module support;
    
    - Someone gets the idea of implementing emerge-webrsync fully in Python
      as a Portage sync module (which is a not-unreasonable idea);
    
      [This ultimately hasn't gone anywhere, and in fact, while working on this
      bug, I ended up finding a bunch of typos that meant you couldn't even test it.
      But it's a stub anyway.]
    
    - The idea of deprecating emerge-webrsync is floated around. The idea
      being Portage should call it via its new sync module with sync-type=webrsync.
    
      This is presumably with the ultimate goal of it transparently one day
      using the aforementioned (yet-non-existent) Python implementation as its
      backend, and not the shell script.
    
      [To this day, Portage's webrsync implementation shells out to the emerge-webrsync
      shell script, but it has the abstraction to switch that out, in theory.]
    
    - At the time, PGP verification in general of the Gentoo
      repository is an active topic, especially now we'd migrated to git which makes
      it way easier, unlike CVS.
    
    - A bug is filed for PGP verification in emerge-webrsync.
    
      People decide it doesn't matter too much, because Portage is going to
      Real Soon Now (TM) have its own backend (replacing the shell script) and/or
      Portage's sync module support obsoletes emerge-webrsync entirely.
    
      The idea here, I think, being that nobody should call emerge-webrsync and
      everyone should just call emerge (or emaint) to sync as appropriate.
    
      [This isn't a terrible idea in a sense, but it needs a better basis:
      we should probably make emerge-webrsync a wrapper which creates a temporary
      repo config to forcefully webrsync a repository if the user asks us to. This
      is what people expect from emerge-webrsync with the default sync-type=rsync
      in repos.conf for ::gentoo.
    
      I actually started implementing this before I realised that emerge was
      shelling out to emerge-webrsync, so have postponed it.]
    
    - Then nothing happens with the "replacement" ideas and the good
      ol' trusty emerge-webrsync ends up with the same problems sitting
      there because nobody saw the point in working on it if it was to
      be replaced soon. But that didn't happen.
    
    The fix overall for this is pretty small, but the commit is larger
    than I'd like because I had to rework a few things to sensibly allow
    disabling PGP verification as well as follow the flow.
    
    (I did start splitting up this commit but ultimately it needs -w
    for best review even without the output tweaks in this commit and
    deconstructing this for atomic commits would end up being more brittle
    as I couldn't be as confident in the result.)
    
    Bug: https://bugs.gentoo.org/597800
    Signed-off-by: Sam James <sam@gentoo.org>

 NEWS                |   5 +
 bin/emerge-webrsync | 363 ++++++++++++++++++++++++++++++++++------------------
 2 files changed, 240 insertions(+), 128 deletions(-)
Comment 17 Larry the Git Cow gentoo-dev 2023-04-30 04:14:22 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=5b3c80502e96406b4b175e2ee79eb65f3f3cd9f6

commit 5b3c80502e96406b4b175e2ee79eb65f3f3cd9f6
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2023-04-30 04:13:31 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2023-04-30 04:13:36 +0000

    sys-apps/portage: add 3.0.47
    
    Bug: https://bugs.gentoo.org/597800
    Closes: https://bugs.gentoo.org/903973
    Closes: https://bugs.gentoo.org/875362
    Closes: https://bugs.gentoo.org/904895
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-3.0.47.ebuild | 285 +++++++++++++++++++++++++++++++++
 2 files changed, 286 insertions(+)