| Summary: | MITM on sync+emerge = root almost any gentoo system | ||
|---|---|---|---|
| Product: | Gentoo Security | Reporter: | Walter <walter> |
| Component: | Auditing | Assignee: | Gentoo Security Audit Team <security-audit> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | alarig, arthur, can.ecodo.nu.n.o+bugs.gentoo, ch4os, garftd, gkeys, hanno, jaak, jj, kensington, kfm, luke, mail, mlen, mrueg, O01eg, sakaki, steffen.weber, wizardedit |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: |
Strawman utility to create or validate signed "master" hashes of ebuild repository
Sample /etc/portage/repo.postsync.d hook, for use with porthash |
||
|
Description
Walter
2016-10-22 16:35:18 UTC
Please note it's past midnight so I could be totally wrong on this... but it seems to stand up to my past-midnight brain, and seems worrisome enough to nudge you all. Going to sleep now. Rest well, but not too well. Indeed if one MITMs rsync, all is lost. For infrastructure I run, all my sync scripts only use ordinary rsync if they're passed "--i-know-what-im-doing-and-it-kills-kittens-but-i-dont-care". To me, it's pretty clear that Gentoo needs a viable solution for signed ebuilds that are incrementally syncable. If I have time during the next quarter or two, I'll see if I can try to push the ball forward with this. Indeed it's very important. (In reply to Jason A. Donenfeld from comment #2) > Rest well, but not too well. Indeed if one MITMs rsync, all is lost. For > infrastructure I run, all my sync scripts only use ordinary rsync if they're > passed "--i-know-what-im-doing-and-it-kills-kittens-but-i-dont-care". To me, > it's pretty clear that Gentoo needs a viable solution for signed ebuilds > that are incrementally syncable. If I have time during the next quarter or > two, I'll see if I can try to push the ball forward with this. Indeed it's > very important. CCing gentoo-keys, things should be done after the latest GSoC so not sure what the holdup is Why not get away from rsync altogether? The point is to only send deltas across thw wire. There are other ways to get it done, for example git. Most of the layman overlays are already using git. "Git" with the program! (Sorry, couldn't resist.) (In reply to Carlos Konstanski from comment #4) > Why not get away from rsync altogether? The point is to only send deltas > across thw wire. There are other ways to get it done, for example git. Most > of the layman overlays are already using git. "Git" with the program! > (Sorry, couldn't resist.) using rsync and signing the metamanifest on rsync mirror using release engineering key is much more sane approach than verifying all developer signatures (in particular of historical commits). rsync is superior for load-balancing on different infrastructure and cheaper performance wise. One month on and no movement. Can someone at least mark this as CONFIRMED please. (In reply to Walter from comment #6) > One month on and no movement. > > Can someone at least mark this as CONFIRMED please. Feel free to work on the gentoo-keys project or engineer a viable solution to address the issue. Countless bugs being opened isn't going to make anyone work faster. We get it... the risks are apparent and people are working (for free) to fix it. The key problem with gentoo-keys is that it doesn't work for python-unavailable install environments. This is why I have always felt (4 years now) that it was not a sane approach (I get the sense 'have a hammer and everything looks like a nail' psychology is sort of going on within Gentoo re: python, certain eselect functions, etc.), and despite waiting it doesn't seem to have become deployed even post-install. That's OK. Simple class of solution IMHO: 1. A static HTTPS URL with a machine-readable list of the keys that should currently be trusted (this handles expiry) and their public key contents (could be ASCII, JSON, XML, anything... just needs to be a single point of truth) 2. Sign all portage snapshots and incremental updates (squashfs). 3. Include the HTTPS URL's contents at the root of the portage tree. 4. Also include checksums and signature file for the same. 5. Include a gzipped file at the root of the portage tree with the output of "tree -fins /usr/portage" (note that this only includes size and name / location of files, not contents, but that's generally OK at this high level, as long as the overall trusted keys and individual file contents are separately and redundantly verified) 6. Add a signature file for that file that includes a series of checksums and a signature of its contents. This is validated after each 'emerge --sync' (or equivalent). Additionally, the HTTPS URL is checked against the emerge-distributed version of the same key list. 7. Individual files are still validated by their directory-specific Manifest files and checksums/signatures. Note that in this case, mirrors should be notified that care should be taken - where possible - not to expose partial syncs to the general public. Total additional code - a tiny bit in portage. Total additional webbery - one static file. Total hassle for mirror operators - low to none. See also https://wiki.gentoo.org/wiki/Handbook_Talk:AMD64/Installation/Stage#Portage_and_stage3_security_recommendations which deals with proposed install-time process for the handbook and has had zero comment. How can we have a team of 9 security people, however many on portage, however many on infrastructure, and no motion on something so critical that essentially represents a failure of coordination between all three teams and critically affects all of them? This is insane. As a user without Gentoo developer status I feel totally powerless to do anything but raise bugs. So I am just going to keep posting until something happens. (In reply to Walter from comment #9) > See also > https://wiki.gentoo.org/wiki/Handbook_Talk:AMD64/Installation/ > Stage#Portage_and_stage3_security_recommendations which deals with proposed > install-time process for the handbook and has had zero comment. How can we > have a team of 9 security people, however many on portage, however many on > infrastructure, and no motion on something so critical that essentially > represents a failure of coordination between all three teams and critically > affects all of them? This is insane. As a user without Gentoo developer > status I feel totally powerless to do anything but raise bugs. So I am just > going to keep posting until something happens. You could become a developer. You could also submit patches to fix the issues. You could do a lot of things, but if ranting continuously across many bugs is your course of action then so be it. You do remember this is open source and no one is paying us? Did I mention this is open source and no one is paying us to do this? Don't make it personal by saying a team of 9 *volunteers* is not meeting your standard as a user. Well that's a lot of angst. I wasn't attacking anyone's character, but you attacked mine. I would prefer that we stay on topic, but since you asked why I hadn't become a developer, I will gladly take this opportunity to tell you that the barriers to developer status are, from my perspective, frankly rather ridiculous. Instead of reviewing people's work (who cares who they are, it's open source, all contributions are welcome), instead we generally have a 'submit an ebuild and watch it languish in bugzilla while nobody cares' system that has gotten so bad that it seems the *default* for many single new projects that actually want to see the immediate light of day in portage is to start their own layman respository (a system whose configuration has changed so many times that its userbase is only a fraction of Gentoo's). It goes without saying that if there was a public portage tree where one could submit a pull request and actually receive a response, there would likely be more community contribution. Right now and for the last four years this issue is frankly a complete embarrassment to the entire Gentoo project. I take issue with your implication that I have done nothing. If the thanks users get for raising issues is to be lambasted by self-important developers, then how many users do you think Gentoo will gain? Over the last four years I have raised the issue of cryptographic validation from multiple angles, spelled out documentation changes, re-raised issues, even created a proposed solution that actually takes in to account real world usage. How much response have I got? 1) "Wait for the gentoo-keys project." (Even though it doesn't support the use case I had) 2) "Yes, shit, that's quite bad" / "Oh, yeah, it's an open secret the sync system is insecure and crap" -- multiple others 3) "Stop expecting us to do anything [even though its been 4 years], you over-demanding and useless user" -- one security dev Notice how absolutely NONE of those responses focus on the ATTEMPTED CONTRIBUTIONS. I can only wonder why? I suspect it is because the current division of teams in Gentoo is somewhat arbitrary and creates a "not my department" sort of resistance to changing fundamentals that affect multiple areas. A great example would be this issue, which essentially points out that the entire *infrastructure* meets *portage* meets *installation/handbook* meets *security* posture of Gentoo is utterly, utterly screwed, a fact apparently broadly known before it was raised here, yet never acted upon. (Oh and don't forget that to actually implement any useful fix we'll also need a *web* team person too.) How do we resolve this? Why not start by having an occasional meeting between different teams on higher-level/architectural issues such as this one, seeking proposals for solutions and volunteers for motion. Maybe insist that multi-team projects are maintained on a public git server with pull requests from anyone considered. Maybe to add visibility/stimulate contributions, add a separate chunk on the sidebar of the website explaining whats going on that affects everyone? These are just ideas. From where I'm sitting, it looks like I'm not "in the club" so I can't "do" anything else. You can publicly contribute to Portage [1] and the gkeys project [2]. I did not intend to diminish your documentation by any means, but do not call out the security team members as not meeting your particular goals. Thank you for documentating the items that you have so far. [1]: https://github.com/gentoo/gentoo [2]: https://wiki.gentoo.org/wiki/Project:Gentoo-keys Regardless of how to fix this issue or if it will be fixed at all, I wonder why it is at least not marked as "confirmed". Are there any special criteria that have to be met to get this status? Created attachment 494384 [details] Strawman utility to create or validate signed "master" hashes of ebuild repository Simple script to create a signed 'master' hash of the (recursive) contents of the /usr/portage directory (and its 'stat' metadata, under LC_ALL="C"). May be reviewed with manpage on GitHub at https://github.com/sakaki-/porthash/blob/master/porthash. For further info, please see comment to follow. As a strawman/stopgap for this issue (until migration to a fully signed-commit, git-based system or similar is complete, and the handbook default), how about using something like the attached 'porthash' script? I have had this deployed now for a while (for users of my gentoo-on-rpi3-64bit project), where it covers a weekly-gated copy of the Gentoo tree, and while it will win no programming awards, it appears to be working fine. This script is run (on the server side, with an appropriate private key and the "-c" option specified, see 'help' text below) whenever the repo is changed. It takes around 5 seconds wall time to complete on my Xeon E5-2696v4 machine. Two additional files are created (both <1KiB), /usr/portage/repo.hash and /usr/portage/repo.hash.asc. It is then run again (without -c) by an /etc/portage/repo.postsync.d hook on each client machine whenever a sync of the main repo is performed, to verify the signed master hash (and bail out with a vocal error if a problem is detected). The signed 'master' hash itself covers the (recursive) contents of the /usr/portage directory (and its 'stat' metadata, under LC_ALL="C"). The script (with a full manpage) is available on GitHub, at https://github.com/sakaki-/porthash Could something similar to this not be adopted for Gentoo's main rsync repo (using release engineering signatures, of course)? It would add only a small additional computational load to the root server (invoked whenever one or more files was changed) and would be backwards compatible with existing practice (users not wanting to check the repo master hash would only have two small additional files in their /usr/portage directory). Mirrors would reflect the /usr/portage/repo.hash and /usr/portage/repo.hash.asc automatically; no additional work would be created for them. Brief help text for porthash is below: Usage: /usr/bin/porthash [options] Create, or verify (default), a signed hash of full Portage tree The hash is saved to repo.hash, the signature to repo.hash.asc Useful when distributing snapshots via e.g. rsync Hash covers all file contents (recursively), and also the ownership, filetype and permissions of all files and directories. The hash does _not_ cover the distfiles, packages or .git directories, nor the repo.hash or repo.hash.asc files themselves. It will also exclude the metadata directory, unless the -m option is given. Options: -c, --create Create a new master hash (and sign it) (default is to verify an existing hash) -h, --help Show this help screen and exit --homedir=HOMEDIR Use HOMEDIR as gpg's home (default: /root/.gnupg) --key=KEYID Use key KEYID to verify/sign (default: 5D90CAF4) -m, --metadata Include metadata directory when creating hash (omitted by default) --repo=RDIR Repository tree location (default: /usr/portage) -v, --version Show version number of porthash and exit Best, sakaki Created attachment 494492 [details]
Sample /etc/portage/repo.postsync.d hook, for use with porthash
This is the repo.postsync.d hook used with the gentoo-on-rpi3-64bit project / porthash.
Obviously, the rsync server etc. is specific (and it uses the default GPG key ID), but I include it here for completeness.
saw your hackernews post :P https://www.gentoo.org/support/news-items/2018-01-30-portage-rsync-verification.html we are now sync'd securely by default, it was removed and readded for a bug, but seems to be enabled by default for 2.3.40. going to close this, but let me know if it needs additional clarification. Yay :) |
