Created attachment 272019 [details] kernel config for hardened 2.6.38-r2 Me and some other people over in #gentoo-hardened@FreeNode seems to have problems with sandbox. Way to test: $ echo "main(){}" > test.c && gcc -o test test.c && sandbox /lib/ld-linux.so.2 --verify ./test On this system without sandbox this works fine on any setup. With sandbox with vanilla-sources-2.6.38.5, gentoo-sources-2.6.38 and hardened-sources-2.6.27-r7 this just returns without any output. However with hardened-sources-2.6.38-r2 I get: $ echo "main(){}" > test.c && gcc -o test test.c && sandbox /lib/ld-linux.so.2 --verify ./test /usr/lib64/libsandbox.so(+0x36b2)[0x6f768735a6b2] /usr/lib64/libsandbox.so(+0x3743)[0x6f768735a743] /usr/lib64/libsandbox.so(+0x565d)[0x6f768735c65d] /usr/lib64/libsandbox.so(+0x5fba)[0x6f768735cfba] /usr/lib64/libsandbox.so(+0x6a98)[0x6f768735da98] /usr/lib64/libsandbox.so(execve+0x63)[0x6f7687361bc3] /bin/bash(shell_execve+0x43)[0x17543937b63] /bin/bash(execute_command_internal+0x25ee)[0x1754393a60e] /bin/bash(parse_and_execute+0x200)[0x1754397f730] /bin/bash(+0x2aaf6)[0x17543922af6] /proc/5052/cmdline: /lib/ld-linux.so.2 --verify ./test Sandboxed process killed by signal: Aborted I have tested different versions of gcc and of sandbox, but only the kernel seems to have a influence. At least one have reported not having this with hardened-sources-2.6.38, so it may be either a regression within the 2.6.38 release serie, or it could be configuration specific. I will test 2.6.38 when I have time This seems to be the same that kills the build of glibc and wine (which both do similar calls with ld-linux.so due to their multilib nature). Portage 2.2.0_alpha30 (hardened/linux/amd64, gcc-4.5.2, libc-0-r0, 2.6.38-hardened-r2 x86_64) ================================================================= System uname: Linux-2.6.38-hardened-r2-x86_64-Intel-R-_Core-TM-_i7_CPU_920_@_2.67GHz-with-gentoo-2.0.2 Timestamp of tree: Tue, 03 May 2011 12:30:01 +0000 distcc 3.1 x86_64-pc-linux-gnu [disabled] ccache version 3.1.4 [disabled] app-shells/bash: 4.1_p9 dev-java/java-config: 2.1.11-r3 dev-lang/python: 2.7.1-r1, 3.1.3-r1 dev-util/ccache: 3.1.4 dev-util/cmake: 2.8.4-r1 sys-apps/baselayout: 2.0.2 sys-apps/openrc: 0.8.2-r1 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.68 sys-devel/automake: 1.9.6-r3, 1.11.1-r1 sys-devel/binutils: 2.21 sys-devel/gcc: 4.5.2 sys-devel/gcc-config: 1.4.1-r1 sys-devel/libtool: 2.4-r1 sys-devel/make: 3.82 sys-kernel/linux-headers: 2.6.38 sys-libs/glibc: 2.12.2 virtual/os-headers: 0 Repositories: gentoo gamerlay-stable x11 xake-overlay Mine Installed sets: @system ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -O2 -pipe -ggdb -mtune=native" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /usr/share/maven-bin-3.0/conf /var/lib/hsqldb" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=native -O2 -pipe -ggdb -mtune=native" DISTDIR="/var/portage/distfiles" FEATURES="assume-digests binpkg-logs buildpkg distlocks fixlafiles fixpackages metadata-transfer news parallel-fetch preserve-libs protect-owned sandbox sfperms splitdebug strict test unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox" FFLAGS="" GENTOO_MIRRORS="ftp://ftp.sunet.se/pub/os/Linux/distributions/gentoo" LANG="en_US.utf-8" LC_ALL="C" LDFLAGS="-Wl,--as-needed -Wl,-O1 -Wl,--sort-common -Wl,--warn-once,--hash-style=gnu" LINGUAS="sv en" MAKEOPTS="-j16 -l15" PKGDIR="/var/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/var/portage" PORTDIR_OVERLAY="/var/overlays/layman/gamerlay /var/overlays/layman/x11 /var/overlays/layman/xake-overlay /var/overlays/mine" SYNC="rsync://liten.csbnet.se/gentoo-portage" USE="X a52 aac accessibility acl acpi alsa amd64 amr amrnb amrwb apng applet archive asyncns autoipd avahi bash-completion bluetooth branding btrfs bzip2 cairo caps ccache cdaudio cdda cdr cleartype cli clutter connection-sharing consolekit coverart cracklib crypt cups cxx dbus device-mapper devicekit devkit dhcpcd digitalradio djvu dri dts dvd dvdr dvi eds enca encode eselect evo exif fat fbcondecor ffmpeg fftw flac fluidsynth fontconfig fuse gdbm gdm gdu geoip gif gimp glib gmp gnome gnome-keyring gphoto2 gpm grammar graphite gsf gsm gstreamer gtk gtk3 gtkstyle gudev hardened hires-icons hpn ical iconv iconvacl icq icu id3tag idn ieee1394 iptc ipv6 jabber jack java6 jingle jpeg jpeg2k justify kate kvm lcms libffi libnotify libsamplerate lm_sensors logrotate lvm lzma mad maps math matroska md mdadm midi mms mmx mmxext mng moonlight mp2 mp3 mpeg mpfr mpi msn mtp mudflap multilib musepack musicbrainz natspec nautilus ncurses network-cron networkmanager nfs nls nntp nptl nptlonly ntfs ntp nut offensive ogg openal opencore-amr opengl openmp openntpd ots pam pango parted pcre pdf perl pidgin playlist png policykit pppd pulseaudio python qt3support quicktime quvi raw readline realtime rrdcgi rtmp samba schroedinger seed sensord session smp sms speex spell sse sse2 sse3 ssl ssse3 startup-notification subversion svg sysfs test tex theora thesaurus threads tiff totem truetype udev unicode upnp urandom usb userlocales vaapi vhook videos vim-syntax vorbis webkit wmf x264 xattr xcb xcomposite xinerama xml xmp xmpp xorg xrandr xscreensaver xulrunner xv xvid xvmc zeroconf zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DRACUT_MODULES="lvm mdraid plymouth syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="sv en" NETBEANS_MODULES="cnd profiler dlight harness ide java websvccommon apisupport nb" PHP_TARGETS="php5-3" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="nouveau" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
hardened-2.6.38 seems to work, 2.6.38-r1 seems broken.
Maybe this backtrace says something to someone, but it seems like sandbox uses ptrace(PTRACE_PEEKUSER) to find out what "personality" a process have (or rather if the process is 32-bit or 64-bit), and seems to think in this case that the process is 32-bit (which it is not) and calls sb_abort. So what makes this happends with newer patches? grsec returning something wrong, or denies some access that sandbox does not handle? Tested the latest, grsecurity-2.2.2-2.6.38.4-201105021909.patch, and got the same result. #0 0x00006cd6b89d9655 in raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 resultvar = 0 pid = <value optimized out> selftid = <value optimized out> #1 0x00006cd6b89da955 in abort () at abort.c:92 save_stage = 2 act = {__sigaction_handler = {sa_handler = 0x1071, sa_sigaction = 0x1071}, sa_mask = {__val = {7811907410160, 119669484148651, 0, 1, 119669486273440, 136569913344912, 7811904476699, 0, 12678774269484397441, 119669486264512, 7811904482013, 4, 12678774269484397441, 2, 119669484148651, 0}}, sa_flags = -641029235, sa_restorer = 0x6cd6b93820c0 <path.7201>} sigs = {__val = {32, 0 <repeats 15 times>}} #2 0x00006cd6b9173758 in sb_abort () at ../../sandbox-2.5/libsandbox/libsandbox.c:490 No locals. #3 0x00006cd6b917565d in pers_is_32 () at ../../sandbox-2.5/libsandbox/trace/linux/x86_64.c:21 No locals. #4 0x00006cd6b9175fba in trace_check_personality (filename=<value optimized out>, argv=<value optimized out>) at ../../sandbox-2.5/libsandbox/trace/linux/x86_64.c:27 No locals. #5 lookup_syscall (filename=<value optimized out>, argv=<value optimized out>) at ../../sandbox-2.5/libsandbox/trace.c:203 No locals. #6 trace_loop (filename=<value optimized out>, argv=<value optimized out>) at ../../sandbox-2.5/libsandbox/trace.c:425 ret = <value optimized out> nr = <value optimized out> se = <value optimized out> tbl_at_fork = 0x6cd6b9381820 regs = {r15 = 0, r14 = 0, r13 = 0, r12 = 0, rbp = 0, rbx = 0, r11 = 512, r10 = 0, r9 = 0, r8 = 0, rax = 0, rcx = 0, rdx = 0, rsi = 0, rdi = 0, orig_rax = 59, rip = 3832113568, cs = 35, eflags = 512, rsp = 4117279968, ss = 43} before_syscall = true exec_state = 2 #7 trace_main (filename=<value optimized out>, argv=<value optimized out>) at ../../sandbox-2.5/libsandbox/trace.c:491 sa = {__sigaction_handler = {sa_handler = 0x6cd6b9175b70 <trace_child_signal>, sa_sigaction = 0x6cd6b9175b70 <trace_child_signal>}, sa_mask = {__val = {12678774269484397441, 0, 119669488293544, 0, 12678774269484397441, 7, 0, 7811907490112, 12678774269484397441, 7811907465040, 0, 7811907490112, 119669484124042, 0, 7811904476699, 119669484148935}}, sa_flags = 268435460, sa_restorer = 0} old_sa = {__sigaction_handler = {sa_handler = 0, sa_sigaction = 0}, sa_mask = {__val = {0, 7811904508352, 119669476417548, 7811907479616, 119669479501408, 7811907479664, 7811907376120, 7811907446192, 7811907452672, 7811904508352, 119669479501408, 7811907381392, 0, 7811907446192, 119669479501408, 7811907381392}}, sa_flags = 335544320, sa_restorer = 0x6cd6b89d96d0 <__restore_rt>} __func__ = "trace_main" #8 0x00006cd6b9176a98 in sb_check_exec (filename=0x71ad9f89140 "/lib/ld-linux.so.2", argv=0x71ad9f82f50) at ../../sandbox-2.5/libsandbox/wrapper-funcs/__wrapper_exec.c:68 fd = 3 elf = <value optimized out> st = {st_dev = 64769, st_ino = 4718674, st_nlink = 1, st_mode = 33261, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 0, st_size = 117948, st_blksize = 4096, st_blocks = 232, st_atim = {tv_sec = 1304450545, tv_nsec = 615956084}, st_mtim = {tv_sec = 1304359127, tv_nsec = 0}, st_ctim = {tv_sec = 1304359180, tv_nsec = 591716768}, __unused = {0, 0, 0}} #9 0x00006cd6b917abc3 in execve_DEFAULT (path=0x71ad9f89140 "/lib/ld-linux.so.2", argv=0x71ad9f82f50, envp=0x71ad9f7eef0) at ../../sandbox-2.5/libsandbox/wrapper-funcs/__wrapper_exec.c:214 result = -1 my_env = 0x71ad9f7eef0 old_errno = 0 check_path = 0x71ad9f89140 "/lib/ld-linux.so.2"
Confirmed that this happens with 2.6.32-hardened-r45 as well which is based on grsecurity-2.2.2-2.6.32.39-201104232142. I'll try to bisect this later, in the mean time, cc-ing upstream. They may see what's going on just from the bt.
does this happen only on 64 bit kernels?
This seems to be affecting multilib only then? I've tested on two amd64 no-multilib systems, one with .38-hardened and the other with .38-hardened-r2 and I can't reproduce the issue.
(In reply to comment #4) > does this happen only on 64 bit kernels? We need a better testcase for this, since what sandbox does is essentially: switch (do_peekuser(8 * CS)) { case 0x23: return true; case 0x33: return false; default: sb_abort(); } Where do_peekuser essentially does ptrace(PTRACE_PEEKUSER, 8 * CS, trace_pid, NULL) Now these functions are never executed on a no-multilib system (the bool pers_is_32 is forced to false), and on a x86 the file containing them should not even be launched. So we need a testcase that always runt a ptrace. I have not had the time to do this yet, tho.
@pax, spender Have you done anything with permissions wrt ptrace? It seems like I get "permission denied" for different stuff when I try to launch ptrace against it (have yet to try it on a non-hardened kernel).
I think this is related: While playing around a bit with this I noticed some things, and tried run "strace /lib/ld-linux.so.2" only to find strange messages. With old grsecurity patch I more or less just get [ Process PID=... runs in 32 bit mode. ] before the output from ld-linux.so.2, and after that I get exit_group(127). With newer grsecurity patches I get the first output, but before ld-linux.so.2 prints its output I also get some: "Unknown value CS=0x... while detecting personlaity of process PID..." and after the output from ld-linux.so.2 I do not get exit_group(127), but instead two more of those messages. For the first to strace reports a value of CS being 0x282, while for the rest the value is 0x246. Since sandbox seems to mess with ptrace and CS to when trying to figure out personality, I thought this may be of importance.
(In reply to comment #8) > With newer grsecurity patches I get the first output, but before ld-linux.so.2 > prints its output I also get some: "Unknown value CS=0x... while detecting > personlaity of process PID..." and after the output from ld-linux.so.2 I do not > get exit_group(127), but instead two more of those messages. > For the first to strace reports a value of CS being 0x282, while for the rest > the value is 0x246. ok, this makes me think that the recent changes related to RANDKSTACK on amd64 are causing this, i'll invesitage.
Here's what clues I have: 1) I confirmed as radegand saw that it only affects 64-bit multilib. Running echo "main(){}" > test.c && gcc -o test test.c && sandbox /lib/ld-2.11.3.so --verify ./test on nomultilib works. 2) You can turn off all GRSEC/PAX features and you still hit this error on recent patches >=201104191737, both for the .32 and .38 branches. You do not hit this error on previous. (My usual technique is to isolate which feature causes the problem before jumping into the ifnef maze.) Not sure where to start with this.
(In reply to comment #10) > Here's what clues I have: > > 1) I confirmed as radegand saw that it only affects 64-bit multilib. Running > > echo "main(){}" > test.c && gcc -o test test.c && sandbox /lib/ld-2.11.3.so > --verify ./test > > on nomultilib works. > That is as I pointed out because on anything but multilib this logic is turned off. "strace /lib/ld-linux.so.2" seems to be a better indicator.
see also [url=https://bugs.gentoo.org/show_bug.cgi?id=365915]#365915[/url]
(In reply to comment #8) > With newer grsecurity patches I get the first output, but before ld-linux.so.2 > prints its output I also get some: "Unknown value CS=0x... while detecting > personlaity of process PID..." and after the output from ld-linux.so.2 I do not > get exit_group(127), but instead two more of those messages. > For the first to strace reports a value of CS being 0x282, while for the rest > the value is 0x246. > > Since sandbox seems to mess with ptrace and CS to when trying to figure out > personality, I thought this may be of importance. see https://bugs.gentoo.org/attachment.cgi?id=272157
i fixed the 32 bit userland breakage, the next grsec patch will have the fix.
(In reply to comment #14) > i fixed the 32 bit userland breakage, the next grsec patch will have the fix. Thank you.:-) We close this when blueness has bumped the sources in portage and confirmed the fix.
Okay confirmed that hardened-sources-2.6.38-r4 is fixed. I have removed -r2. I will test -r3 in a minute and remove it too if its also broken. I'll close this for now. Thanks pipacs :)
hardened-sources-2.6.38-r3 is broken too. Its out.
grsecurity-2.2.2-2.6.38.6-201105111839 please add this and retest.
