Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 349026 - MetaManifest validation of tree distribution
Summary: MetaManifest validation of tree distribution
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Unspecified (show other bugs)
Hardware: All Linux
: High enhancement (vote)
Assignee: Robin Johnson
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-12-18 18:56 UTC by Richard
Modified: 2020-12-07 07:26 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Richard 2010-12-18 18:56:32 UTC
At the moment, portage does md5 hashes to verify that downloads are authentic and it also has manifests to verify that the files are authentic. I was thinking about this and I realized that there is no present mechanism to verify that the portage tree's manifests are authentic.

With that in mind, I think a potential attacker could take control of a Gentoo Linux system by doing a MITM attack when the user is doing updates. They just need to go between portage and the rsync server (or take over the rsync server). Then they could serve a modified version of the tree that has an update to a package that usually runs as root (e.g. udev) and that package would then act as a carrier for whatever arbitrary code that the attacker wants to run, with full root privileges.

Reproducible: Didn't try

Steps to Reproduce:




A way to fix this would be to make some sort of master Manifest file that has hashes for all of the Manifest files in the portage tree. Then a SHA1 hash of it could be taken, which would then be encrypted with a 2048-bit secret key, known only inside the Gentoo Foundation. The encrypted SHA1 has could then be stored in a file included in the portage tree.

A package could be introduced with the core system that basically puts a file into the file system containing the public key. Finally, whenever portage does updates, it will take a SHA1 hash of the manifest file, decrypt the encrypted hash with the public key and verify that the two are the same. If they are not the same, then it should refuse to do anything until a sync fixes the problem or it is given a FEATURES flag that tells it to override the check.

Overlays have the potential to suffer from the same problem and they could be modified to use a similar mechanism, although since not all overlays are officially hosted by Gentoo Linux, third party would need to be provided some sort of mechanism by which they could manage their own secret keys. That would likely involve obtaining a public key for the overlay when adding the overlay through layman. In the case of local overlays, the user is responsible for them, so portage should not require any sort of verification for them beyond what it already does.
Comment 1 Petteri Räty (RETIRED) gentoo-dev 2010-12-18 19:02:52 UTC
http://www.gentoo.org/proj/en/glep/

Take a look at "Accepted but not implemented gleps (Accepted)". I am not really sure how to properly mark this. robbat2: Do we have a bug to track implementation?
Comment 2 Richard 2010-12-18 19:10:37 UTC
GLEP-58 sounds exactly like what I described, except with more detail in terms of its implementation:

http://www.gentoo.org/proj/en/glep/glep-0058.html

At the same time, it does not seem to describe how specifically third party and local overlays would fit into this framework.
Comment 3 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2010-12-19 00:35:51 UTC
GLEP58's MetaManifest does cover this problem for the main tree yes.
I did provide zmedico with previous patches for the implementation, not sure why they never got merged. Everybody should have a sufficiently new Portage version now per the original timeline proposal.

Richard:
1. For the meantime, FEATURES=severe will require that ALL manifests are signed by trusted keys. FEATURES=strict is probably useful at the same time, however not all of the tree is signed by developers...

2. I _strongly_ suggest you go and read the Cappos et al papers linked in GLEP58. Your proposed scheme CAN still be used to get vulnerabilities on a target machine (specifically, send the target machine a completely valid, but OLD snapshot of the tree, containing the exploits you want them to have exposed). Modifying the tree directly will get them noticed by FEATURES=severe.

3. Regarding the overlays and third-party trees: The main Portage tree is more exposed because it is known to be available from mirrors other than the original sources. If you fetch directly from the original source, you're just at the point of requiring the original source to not be tampered with. And presumably, if they can tamper with the original source, they can tamper with the key distribution mechanism, so there is no added source.


regarding overlays, the problem is that you need to be able to fetch a validation timestamp
Comment 4 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2010-12-19 00:38:31 UTC
(In reply to comment #3)
> regarding overlays, the problem is that you need to be able to fetch a
> validation timestamp
Ignore these two lines, editor fail.
Comment 5 Richard 2010-12-19 01:22:48 UTC
> 2. I _strongly_ suggest you go and read the Cappos et al papers linked in
> GLEP58. Your proposed scheme CAN still be used to get vulnerabilities on a
> target machine (specifically, send the target machine a completely valid, but
> OLD snapshot of the tree, containing the exploits you want them to have
> exposed). Modifying the tree directly will get them noticed by FEATURES=severe.

I believe the issue with old rsync mirrors was fixed in bug #21794. Now whenever you try to synchronize with a mirror that is older than your local snapshot, portage will quit and display a message saying that you need to delete the timestamp.chk file to force synchronization.

I have to finish debugging a non-Gentoo related thing, but I skimmed through the paper. I am not sure how much damage a MITM attacker could do by providing an old version of the portage tree given the timestamp check and the fact that having portage try to downgrade many packages at once should cause the root user to investigate.

This sort of issue could likely be mitigated by including in the portage tree a file containing the public keys of the official rsync servers, which was digitally signed by the hypothetical Gentoo Linux master key. Then the clients could by default establish a SSL connection with the mirror before transferring files.

The official mirrors have to obtain updates from the main update servers, which only talk to official mirrors, so the only way of doing a MITM on the portage would be to go through the process of making an official mirror, in which case they would both need the victim to configure portage to talk to their mirror, which I imagine would make it difficult to successfully deny access to updates through the use of old snapshots.
Comment 6 Richard 2010-12-19 01:30:59 UTC
It seems I have editing issues too. That last paragraph should have been:

The official mirrors obtain updates from the main update servers and if the MetaManifest feature were deployed in addition to SSL connections, an attacker would both need to establish an official mirror and have the victim configure portage to update from it. In that case, the attacker could only deny the victim access to updated versions of the tree without being detected immediately and if the attacker did that, he would likely not go undetected for very long considering that portage receives daily updates and having them stop would be cause for the system administrator to investigate. With that in mind, I think it would be difficult to do a successful attack through the use of old snapshots, provided that these things are implemented and that the root user did not disable the security features.
Comment 7 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2010-12-19 02:30:20 UTC
(In reply to comment #5)
> I believe the issue with old rsync mirrors was fixed in bug #21794. Now
> whenever you try to synchronize with a mirror that is older than your local
> snapshot, portage will quit and display a message saying that you need to
> delete the timestamp.chk file to force synchronization.
Portage only enforces that the rsync mirror you are fetching from is NEWER or the same age. Delaying it a few hours would still provide them enough window to extend their attack. I'm not saying the attacker needs to hand out a much older tree, but just not give one that is any newer than what the target client has. So there are no downgrades, but no upgrades to the vulnerable package either.

> I have to finish debugging a non-Gentoo related thing, but I skimmed through
> the paper. I am not sure how much damage a MITM attacker could do by providing
> an old version of the portage tree given the timestamp check and the fact that
> having portage try to downgrade many packages at once should cause the root
> user to investigate.
This paper is the most useful in the set:
http://www.cs.arizona.edu/stork/packagemanagersecurity/ccs2008.pdf



> This sort of issue could likely be mitigated by including in the portage tree a
> file containing the public keys of the official rsync servers, which was
> digitally signed by the hypothetical Gentoo Linux master key. Then the clients
> could by default establish a SSL connection with the mirror before transferring
> files.
It would be wonderful if rsync supported proper TLS (with SNI) or even SSL, so we can use the existing validation mechanisms in SSL. We don't need to distribute specific CA keys, we have CACert already.

Note that only the cryptographically signed hash of the MetaManifest needs to be fetched securely, everything else can be validated from that root trust link.

> 
> The official mirrors have to obtain updates from the main update servers, which
> only talk to official mirrors,


> so the only way of doing a MITM on the portage
> would be to go through the process of making an official mirror, in which case
> they would both need the victim to configure portage to talk to their mirror,
> which I imagine would make it difficult to successfully deny access to updates
> through the use of old snapshots.

(In reply to comment #6)
> The official mirrors obtain updates from the main update servers
Machines in the rsync.g.o rotation are run only be the Gentoo infrastructure team. It's the rsync.$CC.g.o machines that an attacker would try to target, and getting SSL deployed there would be much harder.
Already all of the rsync.$CC.g.o machines talk to our solely to our {masterrsync,rsync1}.g.o server via a limited IP ACL.

> and if the
> MetaManifest feature were deployed in addition to SSL connections, an attacker
> would both need to establish an official mirror and have the victim configure
> portage to update from it. In that case, the attacker could only deny the
> victim access to updated versions of the tree without being detected
> immediately and if the attacker did that, he would likely not go undetected for
> very long considering that portage receives daily updates and having them stop
> would be cause for the system administrator to investigate.
No. You're missing that the malicious attacker running the mirror is handing out a specific out of date tree to JUST the victim. To all other requests, he hands out the updated tree.

> With that in mind,
> I think it would be difficult to do a successful attack through the use of old
> snapshots, provided that these things are implemented and that the root user
> did not disable the security features.
The delay window does NOT need to be very long. The attacker's malicious server can detect how old the client's tree is, by what rsync requests (the hash of the timestamp files specifically).
Call the latest tree available: T_latest.
Call the tree presently on the client: T_client
The tree right after the vuln was fix is: T_fixed

If there is a vulnerability that appeared between T_client and T_latest, the attacker can hand out the version of the tree immediately before T_fixed, or even just the same version T_client. He now has a window of time before the sysadmin notices the failure, and tries to sync again. It doesn't need to be large, he can initiate a probe of the victim (to use the vuln that may exist, depending what the client has installed) as soon as he knows which tree the victim has.

The remote Exim root vulnerability from recently would be a good candidate, because SMTP connections are fairly quick, and more importantly, very common, so it would simply vanish in the noise if the system was running something else.
Comment 8 Richard 2010-12-19 04:09:15 UTC
(In reply to comment #7)
> No. You're missing that the malicious attacker running the mirror is handing
> out a specific out of date tree to JUST the victim. To all other requests, he
> hands out the updated tree.

I did not miss that, but executing such an attack requires:

1. Setting up a rotation server well in advance.
2. Getting the victim to use it exclusively out of all of his other options. Geographic proximity might help with this, but this is the biggest problem I see with this. People using the rotations to synchronize such as myself would be probalistically affected by this, so it would require the equivalent of winning the lottery for it to happen.
3. That the window of the min of (the system's update period and about a day) and at worse nearly immediately that handing out old versions of the tree would buy before suspicions occur is significant enough for the attacker to produce a working exploit for the vulnerability and use it. Given that Gentoo is undermanned enough that things can go unpatched in portage for a significant amount of time, I do not think that the additional time that this buys is worth the effort considering that the time involved in doing this could probably be better spent in finding a zero-day vulnerability, which is something that would bypass all of these protections.
4. That the attacker be on top of things such that he can stop handing out newer snapshots after a vulnerability becomes known and before the vulnerability is patched in the tree. Assuming he learns of the hole before the tree is patched, he use a script to stop distributing updated versions to the victim the moment the next update has a newer patched ebuild.

The big issue with portage at present is that it appears that any attacker can hijack it to execute arbitrary code. If the MetaManifest is implemented, that would no longer be possible. Being able to walk into a system the next time it does updates and having to fulfill a set of conditions that are so difficult to meet that they are almost never met in practice are two vastly different risk levels.

Since FEATURES="severe" is neither used by default nor in a position to be used by default, nearly all Gentoo Linux users fall into the category of being at risk of receiving an update that contains arbitrary code whenever they do a system update.

By the way, is there any documentation for how ebuilds can be signed and how one would designate signatures as being trusted or untrusted?
Comment 9 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2010-12-19 06:15:28 UTC
(In reply to comment #8)
> 1. Setting up a rotation server well in advance.
> 2. Getting the victim to use it exclusively out of all of his other options.
> Geographic proximity might help with this, but this is the biggest problem I
> see with this. People using the rotations to synchronize such as myself would
> be probalistically affected by this, so it would require the equivalent of
> winning the lottery for it to happen.
Neither of these are actually required. You just need to ensure that the victim's traffic ends up at your rsync server. Hijack local DNS, MITM on the network, compromise an existing rsync.$CC.g.o mirror (this HAS happened before, but nothing of Gentoo was changed, they were after other distros).

> 3. That the window of the min of (the system's update period and about a day)
> and at worse nearly immediately that handing out old versions of the tree would
> buy before suspicions occur is significant enough for the attacker to produce a
> working exploit for the vulnerability and use it. Given that Gentoo is
> undermanned enough that things can go unpatched in portage for a significant
> amount of time, I do not think that the additional time that this buys is worth
> the effort considering that the time involved in doing this could probably be
> better spent in finding a zero-day vulnerability, which is something that would
> bypass all of these protections.
Yes, the attacker would do better just to research 0day vuln with our present update rate. The real value from this security hole is ease of target availability.

> 4. That the attacker be on top of things such that he can stop handing out
> newer snapshots after a vulnerability becomes known and before the
> vulnerability is patched in the tree. Assuming he learns of the hole before the
> tree is patched, he use a script to stop distributing updated versions to the
> victim the moment the next update has a newer patched ebuild.

> The big issue with portage at present is that it appears that any attacker can
> hijack it to execute arbitrary code. If the MetaManifest is implemented, that
> would no longer be possible. Being able to walk into a system the next time it
> does updates and having to fulfill a set of conditions that are so difficult to
> meet that they are almost never met in practice are two vastly different risk
> levels.
Even MetaManifest itself will NOT prevent malicious authors of overlays.

> By the way, is there any documentation for how ebuilds can be signed and how
> one would designate signatures as being trusted or untrusted?
FEATURES=sign generates the signature, read the Portage source for that and how to specify what key to sign with, but there's nothing unfortunately for the present verification.

Relatedly, for security discussion, the MOST secure you can be right now is using the emerge-webrsync with FEATURES=webrsync-gpg, because it DOES have validation (against any key in the keyring located in /etc/portage/gnupg, which isn't great still).
Comment 10 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2020-12-07 07:26:30 UTC
closing out old tickets implemented long ago