Bug 381833 – media-libs/mesa-7.11 will kill xorg on hardened kernels

• Product: Gentoo Linux
• Status: RESOLVED TEST-REQUEST

[Description Francisco Blas Izquierdo Riera (RETIRED) 2011-09-04 22:12:58 UTC]

Running mesa-7.11 with the Radeon VIDEO_CARD and llvm (which is required for it to build) creates a library which attempts to do RWX mmaping and kills the programs afterwards.

Reproducible: Always

Steps to Reproduce:
1. Run an opengl application
2. See xorg dying

Actual Results:  
[  953.188449] [drm:drm_mode_getfb] *ERROR* invalid framebuffer id
[  959.156324] grsec: denied RWX mmap of <anonymous mapping> by /usr/lib64/kde4/libexec/kwin_opengl_test[kwin_opengl_tes:9442] uid/euid:1000/1000 gid/egid:1000/1000, parent /usr/bin/kwin[kwin:9437] uid/euid:1000/1000 gid/egid:1000/1000
[  959.170061] grsec: denied RWX mmap of <anonymous mapping> by /usr/bin/Xorg[X:7127] uid/euid:0/0 gid/egid:0/0, parent /usr/bin/kdm[kdm:6668] uid/euid:0/0 gid/egid:0/0
[  959.536709] [drm:drm_mode_getfb] *ERROR* invalid framebuffer id


Expected Results:  
Things work smoothly

# emerge -vp mesa

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] media-libs/mesa-7.11  USE="egl gallium llvm* nptl pax_kernel shared-dricore shared-glapi -bindist -classic -debug -gbm -gles -motif -openvg -pic (-selinux)" VIDEO_CARDS="radeon* -intel -mach64 -mga -nouveau -r128 -savage -sis -tdfx -via -vmware" 0 kB

# emerge --info
Portage 2.1.10.11 (hardened/linux/amd64, gcc-4.4.5, glibc-2.12.2-r0, 2.6.39-hardened-r8 x86_64)
=================================================================
System uname: Linux-2.6.39-hardened-r8-x86_64-Intel-R-_Core-TM-_i5_CPU_M_430_@_2.27GHz-with-gentoo-2.0.3
Timestamp of tree: Sun, 04 Sep 2011 18:15:01 +0000
app-shells/bash:          4.1_p9
dev-java/java-config:     2.1.11-r3
dev-lang/python:          2.6.6-r2, 2.7.1-r1, 3.1.3-r1
dev-util/cmake:           2.8.4-r1
dev-util/pkgconfig:       0.26
sys-apps/baselayout:      2.0.3
sys-apps/openrc:          0.8.3-r1
sys-apps/sandbox:         2.4
sys-devel/autoconf:       2.13, 2.68
sys-devel/automake:       1.9.6-r3, 1.11.1
sys-devel/binutils:       2.21.1-r1
sys-devel/gcc:            4.4.5
sys-devel/gcc-config:     1.4.1-r1
sys-devel/libtool:        2.4-r1
sys-devel/make:           3.82-r1
sys-kernel/linux-headers: 2.6.36.1 (virtual/os-headers)
sys-libs/glibc:           2.12.2
Repositories: gentoo verlihub x11 armagetron hardened-dev overlay-dev-blueness gentoo-haskell scarabeus_local_overlay klondike avr
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA dlj-1.1 AdobeFlash-10.1 AdobeFlash-10 spin-commercial skype-eula"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -fforce-addr -fomit-frame-pointer -funswitch-loops -fbranch-target-load-optimize --param max-gcse-passes=3 -fweb -frename-registers -freorder-blocks-and-partition -fpredictive-commoning -fira-region=all -fira-coalesce -fgcse-sm -fgcse-las -fgcse-after-reload -freschedule-modulo-scheduled-loops  -fipa-struct-reorg -fipa-matrix-reorg -fno-sched-stalled-insns -ftree-loop-im -ftree-loop-linear -ftree-loop-distribution -ftree-vectorize -fvect-cost-model -fivopts -fmodulo-sched -fmodulo-sched-allow-regmoves -fmerge-all-constants -march=native"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/themes/oxygen-gtk/gtk-2.0"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -pipe -fforce-addr -fomit-frame-pointer -funswitch-loops -fbranch-target-load-optimize --param max-gcse-passes=3 -fweb -frename-registers -freorder-blocks-and-partition -fpredictive-commoning -fira-region=all -fira-coalesce -fgcse-sm -fgcse-las -fgcse-after-reload -freschedule-modulo-scheduled-loops  -fipa-struct-reorg -fipa-matrix-reorg -fno-sched-stalled-insns -ftree-loop-im -ftree-loop-linear -ftree-loop-distribution -ftree-vectorize -fvect-cost-model -fivopts -fmodulo-sched -fmodulo-sched-allow-regmoves -fmerge-all-constants -march=native -fno-reorder-blocks-and-partition"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests binpkg-logs distlocks ebuild-locks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
FFLAGS=""
GENTOO_MIRRORS="ftp://darkstar.ist.utl.pt/pub/gentoo/"
LANG="es_ES.UTF-8@euro"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="es es_ES en sv"
MAKEOPTS="-j4"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/var/lib/layman/verlihub /var/lib/layman/x11 /var/lib/layman/armagetron /var/lib/layman/hardened-development /var/lib/layman/blueness /var/lib/layman/haskell /var/lib/layman/scarabeus /usr/local/portage/local-portage /usr/local/portage/avr"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="acl alsa amd64 bash-completion bzip2 cli consolekit cracklib crypt cups cxx dbus dri fam gdbm gpm handbook hardened iconv idn ipv6 justify kde lcms libnotify mmap mmx modules mudflap multilib ncurses nls nptl nptlonly ogg opengl openmp optimized-qmake pam pax_kernel pcre policykit pppd qt4 readline semantic-desktop session speex sse sse2 sse3 sse4 ssl ssse3 sysfs theora threads udev unicode urandom vorbis xattr xorg xv zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="es es_ES en sv" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="radeon" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

[Comment 1 Francisco Blas Izquierdo Riera (RETIRED) 2011-09-04 22:17:53 UTC]

As commented by Aleister seems that this happens when gallium is there, but it worked well on 7.10

[Comment 2 Anton Bolshakov 2011-09-19 12:57:38 UTC]

It could be related,  although I'm not sure if it's the best solution:

Try to enable PAX_MPROTECT_COMPAT:

http://forums.grsecurity.net/viewtopic.php?f=3&t=2441
bug #366167

[Comment 3 Magnus Granberg 2012-02-15 15:28:25 UTC]

*** Bug 403865 has been marked as a duplicate of this bug. ***

[Comment 4 Aaron W. Swenson 2012-03-19 17:20:44 UTC]

(In reply to comment #2)
> It could be related,  although I'm not sure if it's the best solution:
> 
> Try to enable PAX_MPROTECT_COMPAT:
> 
> http://forums.grsecurity.net/viewtopic.php?f=3&t=2441
> bug #366167

This worked around the issue for me.

[Comment 5 Nikoli 2013-12-31 10:56:48 UTC]

Seems this bug is fixed long time ago, mesa-9.2.5 works fine for me with and without llvm enabled:
USE="egl gallium gbm llvm llvm-shared-libs nptl openvg pax_kernel r600-llvm-compiler vdpau xa -bindist -classic -debug -gles1 -gles2 -opencl -osmesa -pic (-selinux) -wayland -xorg -xvmc" ABI_X86="64 -32 -x32" VIDEO_CARDS="r600 radeon -freedreno -i915 -i965 -ilo -intel -nouveau -r100 -r200 -r300 -radeonsi -vmware"

But when nouveau is used, i need to 'pax-mark m' every bin using opengl. Even when llvm is disabled in mesa, see bug #432520