Greetings, I found some weaknesses in the Gentoo Portage sandbox with consequences similar to the recent posting about the critical issue with Portage and /tmp. FYI, Chris P. posted an selinux one-liner solution for the latter =) > create_dir_notdevfile(portage_t,{ file_type -user_tmp_t }) > Therefore, portage_t can modify all files, except those created by users, in /tmp. I hope the discussion below about Portage is constructive, informative, and encouraging, without diminishing either the value and accomplishments of the Portage team or the relevance/importance of weaknesses (discussed below) to security-conscious Gentoo users and administrators. I have invested a great deal of effort in switching our software from Red Hat to Gentoo, and have a vested interest in improving the security of Gentoo. I welcome all feedback. Thanks, Gavin Reproducible: Always Steps to Reproduce: Steps to reproduce are presented and explained in examples below. Abstract ========= I reviewed some of Gentoo's sandbox code in preparation for integrating an enhanced compiler-caching technology derived from ccache. Although I personally believe the Portage sandbox is extremely helpful and useful, I found that the sandbox is not completely self-enforcing and relies on the cooperation of the installation code in source packages to maintain the integrity of the sandbox during an emerge. Using the 'sandbox' FEATURE in /etc/make.conf without the FEATURES userpriv, and usersandbox (i.e. emerging a package as the root user) is not sufficient to prevent an ebuild from modifying sensitive files (e.g. /etc/passwd) outside the sandbox while performing an ebuild. Similarly, the use of all three FEATURES does not prevent an ebuild from writing to any file writable by the portage user (subject to security context limitations, etc. established by selinux/grsecurity/etc.). Conclusion ========== 1) The attached patch helps detect and prevent "accidental" breakage of the sandbox by "whitehat" package/ebuild authors. 2) Hardened Gentoo (either grsecurity or selinux) provides a kernel-based capability to enforcing the sandbox. After this article has been edited appropriately, perhaps the hardened team might want to contribute some advice? 3) I suggest either updating Portage to eliminate the documented weaknesses, or changing Portage documentation to reflect the weaknesses, documented below, contradicting documentation claims like, ".. as long as the package isn't installed on your system, the package cannot touch any file outside the sandbox." [http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=2&chap=3] Personal Opinions ================ I've found the sandbox an invaluable part of Gentoo and whole-heartedly recommend its continued use via FEATURES="sandbox userpriv usersandbox", but with an understanding for security-conscious administrators that emerge'ing packages poses more risks than implied by the documentation (e.g. "cannot touch any file outside the sandbox"). However, a virtual sandbox enforced by the kernel might provide an appealing alternative to reliance on the current pre-loaded library method. I believe the Portage team has already created a "best in class" solution. Weaknesses in Portage's Sandbox ================================================ The following weaknesses allow for the possibility of a similar type of exploit as: http://bugs.gentoo.org/show_bug.cgi?id=21923 Essentially, a blackhat might be able to trick portage into installing some form of a rootkit that the root user eventually executes unknowingly. More specifically, a blackhat who is aware of the following weakness might be able to find an existing ebuild that scribbles outside the sandbox to some pre-determined location (e.g. /tmp), and then hardlink/symlink that location to somewhere else under their control, thus allowing them to insert code (i.e. a rootkit) during the emerge'ing of a package. Regardless, such writable, vulnerable locations are not supposed to exist at all on a secured/hardened system. First, since the correct operation of the sandbox depends on the presence and definition of the LD_PRELOAD(7) and SANDBOX_* environment variables to enable the sandbox and control its functioning, package code executed during emerge can easily escape the sandbox by unsetting these variables (e.g. SANDBOX_ON), as libsandbox.so lacks enforcement (except for LD_PRELOAD with execve, which is practically useless if the SANDBOX_* variables have been unset). Second, when the execve wrapper finds that libsandbox.so is not present in LD_PRELOAD, it appends this library, instead of pre-pending it, potentially allowing interception of further system calls by a library other than libsandbox.so(6). Example(8): - - - b e g i n e x a m p l e m a k e f i l e - - - export LD_PRELOAD=whitehat_lib.so breaksandbox: env /bin/echo 'misc. conf. stuff' > /etc/whitehat.conf - - - e n d e x a m p l e m a k e f i l e - - - Third, when using a version of libc (e.g. recent versions of glibc) that implements execv, execl, execlp, or execvp without calling execve, the execve wrapper code in libsandbox.c is bypassed(2) by some executables (e.g. GNU's make). Note that glibc's implementation of __execve defines a weak alias, "weak_alias (__execve, execve)". When we provide our own execve in libsandbox.so, glibc's execve is no longer "available for reference from other files" [http://gcc.gnu.org/onlinedocs/gccint/]. However, execl, execle, execv, and execvp (but not execlp which calls execvp) all call __execve (glibc's "true" execve). When libsandbox.c replaces execve, these other functions know nothing about the libsandbox.so execve, and continue using __execve. From a practical perspective, GNU's make uses execvp, while gcc's frontend uses both execv and execvp. In contrast, the exec family in uClibc is written using execvp and execve. Modifying libsandbox.c to wrap __execve might work with glibc, but not with uClibc, unless additional changes are made. Consider that glibc's implementation of __execve defines a weak alias, "weak_alias (__execve, execve)", and implements execv as follows: int execv (const char *path, char *const argv[]) { return __execve (path, argv, __environ); } Thus, when libsandbox.so provides its own execve, execv continues to use glibc's __execve(). The attached patch adds support for wrapping execvp [the others are left as an "exercise" for the reader ;) ], factors functionality common to the execve and execvp wrappers, and ensures that libsandbox.so is listed first in LD_PRELOAD. As an example demonstrating the consequences of a missing wrapper for execvp in libsandbox.c, please consider a hypothetical Makefile in a package with a corresponding ebuild: - - - b e g i n e x a m p l e m a k e f i l e - - - export LD_PRELOAD=whitehat_lib.so breaksandbox: /bin/echo 'blackhat was here' > /etc/passwd /bin/echo 'misc. conf. stuff' > /etc/whitehat.conf env - ./scribble_anywhere_portage_user_can_scribble.sh - - - e n d e x a m p l e m a k e f i l e - - - The script "scribble_anywhere_portage_user_can_scribble.sh" runs as the portage user ('root', unless FEATURES="sandbox usersandbox userpriv ..." in /etc/make.conf). The first and second rules for the breaksandbox(1) target above exploit the missing execvp wrapper weakness, while the third rule exploits the lack of enforcement of the sandbox due to missing "SANDBOX_*" environment variables. The second rule also highlights how easily a package author might accidentally break the sandbox. Proposed Changes ===================== The attached example patch(5) addresses the execvp weakness (the first of the two example rules above), and ensures that libsandbox.so is listed first in LD_PRELOAD. The "env - script" weakness might be avoided by storing the data in the SANDBOX_* environment variables somewhere that the portage user lacks write access to. However, that would necessitate other changes totaling a non-trivial effort. Although one might endeavor to enforce application of sandbox protections requested via the SANDBOX_* environment variables, other weaknesses in relying on the LD_PRELOAD tactic might ameliorate the value of such endeavors. Since the hardened Gentoo project team provides hardened solutions that specifically include the capability of isolating a user using ACLs enforced by the kernel, leveraging their work might prove more efficient in seeking a self-enforcing sandbox capable of preventing one emerge from affecting unrelated files owned by the portage user. Notes ======= 1) The "breaksandbox" Makefile rule also breaks anything depending on other preloaded shared objects (via LD_PRELOAD), but the code in libsandbox.c was already broken anyway for IA32/gcc 3.3/glibc 2.3 combo with multiple shared objects in LD_PRELOAD [50-r3 just fixed that bug]. 2) The execvp problem can be hidden in a "generation skip", where a process uses execvp (e.g. GNU make), but its child uses one of the exec's implemented by execve. In the latter case, the sandbox resumes functioning normally, even though the sandbox execve wrapper code was skipped by the parent using execvp. The use of the terms 'child' and 'parent' are inappropriate to examples where there was a pure exec without a fork, but this is already too wordy. 3) Unlike glibc, uClibc implements execvp using execve. I have not tried to determine whether versions of glibc older than glibc-2.3.3 implement execvp using execve. 4) Why hasn't this problem been noticed before? Perhaps it has, and I haven't heard. Also, see #2 above. Perhaps most programs that use execvp during an ebuild don't alter the environment (i.e. clean up the environment / alter LD_PRELOAD environment variable) before executing execvp? Perhaps the practical consequences are minimal. I'm not aware of any practical, already-existing examples exploiting these sandbox weaknesses. 5) The patch contains some code inappropriate for most any purpose other than testing/evaluating the sandbox system. However, it shouldn't require much effort to integrate and adapt the patch to one's own preferences and style. Also, some debugging statements left in the patch require /tmp/jlog to exist before emerg'ing and belong to the portage user and group. These debugging statements are left intentionally in the patch to highlight how much GNU's make uses execvp (see the log file /tmp/jlog after performing an emerge using a patched version of libsandbox using the supplied patch). The list of SANDBOX_* environment variables checked by the patch for non-empty definitions should be tuned, and possibly extended to check for unreasonable combinations. Also what is reported and how needs "adjusting" to suit the typical Gentoo user. The patch also includes an experimental performance tweak for x86 architectures that is automatically selected if ARCH == 'x86'. The patch uses a different (experimental) method of wrapping execve and execvp using assembly. I've only tested the patch on a single system (x86, the latest version of almost everything). #include <std_disclaimer.txt> 6) Prior to 50-r3, this weakness was limited to special cases where an ebuild explicitly set 'LD_PRELOAD=/mylib.so /lib/libsandbox.so', since libsandbox.so was partly broken when using multiple libraries in LD_PRELOAD, at least for the current version of GNU's ld that requires spaces between libs(prior versions of libsandbox.c used a ':' instead of ' ' when appending). Thus, prior to 50-r3, 'export LD_PRELOAD=/mylib.so' resulted in something like 'LD_PRELOAD=/mylib.so:/lib/libsandbox.so', causing ld to report an error. 7) Actually, an LD_SO_PRELOAD option exists in sandbox.c that tries to enforce the preloading of libsandbox.so via /etc/ld.so.preload. However, if the sandbox executable is interrupted at an inopportune time, the contents of ld.so.preload could be corrupted. Also, negative consequences might arise for unrelated system processes that happen to start while the sandbox exists. Perhaps if ld.so were tweaked to provide for enforced, per-user versions of ld.so.preload ... Regardless, the portage ebuild states this approach has been deprecated, and it still doesn't prevent installation code from unsetting SANDBOX_ON. 8) If we append ' /lib/libsandbox.so' to the 'export LD_PRELOAD=whitehat_lib.so' in the first makefile example, the supplied patch does not check to make sure that libsandbox.so is listed first. I consider that a "todo" item. Actual Results: I can create ebuilds that modify files outside the sandbox. When emerging without 'userpriv' (i.e. running portage sandbox as the root user), I could overwrite any file on the system from within a source package's Makefile, during an emerge with sandbox included in FEATURES from /etc/make.conf. Expected Results: I expected Portage to report sandbox violations. Please refer to configuration information presented in the article copied above. I submitted the patch mentioned above to Daniel, Martin, and Nicholas for their consideration.
Created attachment 29233 [details, diff] patches libsandbox.c to correct for sequencing LD_PRELOAD and execvp weaknesses The patch modifies libsandbox.c per the detailed explanation in my bug description.
Given the post http://www.securityfocus.com/bid/10060/discussion/ for a bug with similar exploits, it seems prudent to also make a post regarding the weaknesses discussed above.
I hope 6 days was adequate lead time. I sent all of this information to: To: "\"Daniel Robbins\"" <drobbins@gentoo.org>, "\"Martin Schlemmer\"" <azarah@gentoo.org>, "\"Nicholas Jones\"" <carpaski@gentoo.org> Subject: How to bypass the Gentoo Portage Sandbox Date: Wed, 7 Apr 2004 16:30:56 -0700 and: To: <gentoo-hardened@lists.gentoo.org>, "\"Martin Schlemmer\"" <azarah@gentoo.org>, "\"Nicholas Jones\"" <carpaski@gentoo.org> Cc: <golden_moon_rising@yahoo.com> Subject: Hardening the Gentoo Portage Sandbox [rev 2 - edits re: glibc's use of weak aliases] Date: Mon, 12 Apr 2004 10:46:22 -0700
The author of grsecurity just sent an email relating to the possibility of using it to create a virtual sandbox (i.e. the kernel-enforced sandbox I previously suggested). From: <spender@grsecurity.net> To: <grsecurity@grsecurity.net> Sent: Saturday, April 17, 2004 6:04 AM Subject: [grsec] grsecurity 2.0, 1.9.15 released . . > One of the things I plan to write over the summer > is a completely transparent method of sandboxing an application that > essentially gives the application its own private version of the > filesystem (and thus it cannot affect the public version of the > filesystem so as to compromise it).
This bug has been here for some time and I was wondering why a developer has not commented on its merits. It seems to raise some legitimate concerns and provides constructive feedback for improving the sandbox. I am not in a position to evaluate all the points made but it would be nice if a developer addresses all the issues reported in the bug.
The LD_PRELOAD should be fine in the current sandbox. I'm not sure if the variations on execvp are covered or not, but I don't believe so. These variations should be fixed though, as many developers assume that anything the sandbox doesn't show as a violation is acceptable. It's well-known (I hope!) at least among developers that the sandbox is far from infallible. As noted, any process can unset SANDBOX* and then do whatever they like in a subprocess, so there is no real urgency in fixing security holes related to the *process that calls* sandbox. Having said that, I'll test and do execvp -> __execvp in the next couple of days.
Thanks Jason for your time and effort to consider this matter. Is there any official or unofficial Gentoo documentation regarding the "fallability" of the Portage sandbox? I have never seen or heard any, and could not find any Googling gentoo.org, but I have not read all of the arhives of email list posts by all of the Gentoo developers. These official web pages make false claims regarding the sandbox: o http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=2&chap=3 o http://www.gentoo.org/proj/en/portage/ o http://www.gentoo.org/proj/en/devrel/handbook/handbook.xml?part=1&chap=4 o http://www.gentoo.org/security/en/glsa/glsa-200404-01.xml - Even this widely published GLSA announcement makes false claims about the sandbox o I have not checked the non-English language versions of the pages above. Per the examples in prior posts, the process calling the sandbox is not the only process affected by the various issues mentioned above. P.S. I suggest following up with Brad regarding his progress on a kernel-enforced sandbox before investing too much time reworking a userland sandbox.
Reassigning to security. Jstubbs any progress on this one?
ping to myself
We can merge the exec* changes if they aren't already merged...\ We'll look into it. Portage is, by it's nature, cooperative... It's not a security tool althought it could by chance work that way some times. Since it's cooperative, it's very easy to bypass. We could try and fix all the potential holes but it would be an unworldly investment of manpower. I was looking for a kernel sandbox at one time, but there wasn't a whole lot in the way of options. Userland can only do so much to save userland. The patch does seem to try to follow the exec calls to prevent environment stripping, but there are a lot of potential weaknesses. We'd also have to port for every libc that we wanted to support _and_ ensure that the specs we code by do not change. If someone can point the way to a kernel-level sandbox, it would be an enourmous amount better than this scrapped together sandbox.
Thx Nicholas, is this fixed?
After discussion, the decision was that this won't be fixed. The sandbox can never be secure by itself. Userpriv in conjunction with fakeroot is 99% there. However, the only real solution can be at the kernel level using something like fuse. As far as sandbox goes, seeing that it can bypassed by way of its own config (export SANDBOX_WRITE="/") and offers absolutely no protection against staticly linked executables, attempting to remain "secure" against calls to internal glibc functions is not really constructive. Until a solution such as thosementioned above does come into place, we will support/fix sandbox with regard to any public glibc APIs.
Closing (see comment #12)
"Userpriv in conjunction with fakeroot is 99% there." With the current implementation of userpriv in portage, I don't agree. An ebuild has complete control over files and directories owned or writable by the portage user. Thus, one ebuild can easily run amok (whether as a trojan or accidentally). To perform an ebuild, processes running as the "userpriv" user only require write access to a subset of what they currently have write access to.
