sys-devel/llvm-2.9-r2 fails test with 3.1.10-hardened kernel. Expected Passes : 3827 Expected Failures : 20 Unsupported Tests : 1932 Unexpected Failures: 63 make[1]: *** [check-local-lit] Error 1 make[1]: Leaving directory `/var/tmp/portage/sys-devel/llvm-2.9-r2/work/llvm-2.9/test' make: *** [check] Error 2 Example of possible fix is in www-client/chromium/chromium-16.0.912.75.ebuild: src_compile() { ... if use test; then ... pax-mark m out/Release/{base,crypto,googleurl,net}_unittests fi Portage 2.1.10.41 (hardened/linux/amd64, gcc-4.5.3, glibc-2.13-r4, 3.1.10-hardened x86_64) ================================================================= Timestamp of tree: Mon, 23 Jan 2012 12:15:01 +0000 app-shells/bash: 4.1_p9 dev-lang/python: 2.7.2-r3 dev-util/pkgconfig: 0.26 sys-apps/baselayout: 2.0.3 sys-apps/openrc: 0.9.4 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.68 sys-devel/automake: 1.11.1 sys-devel/binutils: 2.21.1-r1 sys-devel/gcc: 4.5.3-r1 sys-devel/gcc-config: 1.4.1-r1 sys-devel/libtool: 2.4-r1 sys-devel/make: 3.82-r1 sys-kernel/linux-headers: 3.1 (virtual/os-headers) sys-libs/glibc: 2.13-r4 Repositories: gentoo nikoli ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=amdfam10 -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/openvpn/easy-rsa /usr/share/themes/oxygen-gtk/gtk-2.0" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-march=amdfam10 -O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="assume-digests binpkg-logs distlocks ebuild-locks fixlafiles news parallel-fetch protect-owned sandbox sfperms strict test unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync" FFLAGS="" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LINGUAS="ru en" MAKEOPTS="-j3" PKGDIR="/usr/portage/packages/amdfam10" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/var/lib/layman/nikoli" USE="3dnow 3dnowext 7zip X a52 aac acl acpi alsa amd64 amr atm audiofile bash-completion bluetooth bzip2 cairo caps cdda cddb cdio cdparanoia cdr celt cli consolekit cracklib crypt css cups cxx dbus djvu dri dts dv dvd dvdr encode exif fat ffmpeg flac fluidsynth fontconfig fortran gd geoip gif gimp gmp gnutls gphoto2 gpm graphviz gsm gstreamer gtk handbook hardened iconv icu id3tag idn ilbc imagemagick imap imlib ios ipod ipv6 jbig jpeg jpeg2k justify kde kipi lame laptop lcms libnotify libproxy libsamplerate lm_sensors lzma lzo mac mad matroska mikmod mmx mmxext mng modplug modules mp3 mp4 mpeg mtp mudflap multilib musepack musicbrainz ncurses nls nptl nptlonly ntfs ogg openal openexr opengl openmp pam pango pax_kernel pcre pdf pg-intdatetime phonon plasma pm-utils png policykit pppd qt3support qt4 quicktime rar raw readline reiserfs replaygain rtmp sasl scanner schroedinger session smp sndfile socks5 speex spell sqlite sse sse2 sse3 sse4a ssl startup-notification svg symlink sysfs taglib theora threads thumbnail tiff truetype tta udev unicode upnp usb v4l v4l2 vcd vorbis vpx wavpack webkit wifi wma wmf wps x264 xattr xcb xcomposite xface xinerama xml xmp xorg xpm xscreensaver xv xvid zip zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan stage tables krita karbon braindump" CAMERAS="*" COLLECTD_PLUGINS="df interface load memory rrdtool swap syslog hddtemp sensors cpu cpufreq disk entropy network processes" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="ru en" NGINX_MODULES_HTTP="access auth_basic autoindex fastcgi gzip rewrite" PHP_TARGETS="php5-3" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="radeon vesa" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Created attachment 299611 [details] dmesg
llvm + hardened = welcome to a world of pain! Okay, joking aside, here's what's going. On pax hardened kernel with CONFIG_PAX_MPROTECT enable, RWX mmappings are going to lead to the process being killed. You have one of two choices: 1) either use paxctl to exempt that binary from having mprotect enforced (in an ebuild inherit pax-utils and pax-mark -m) or 2) disable CONFIG_PAX_MPROTECT. The former is the better solution because it relaxes security on just one/some binaries rather than the whole system. However, the story does not end with just pax marking llvm's binaries, because binaries generated by llvm may themselves contain RWX mmappings in which case they will also be killed under MPROTECT. This is the case with mesa/gallium.
Just to add on what blueness said, here is the status with LLVM. LLVM (and its libraries) does RWX mappings to support JIT code execution of the code being compiled, this is the case for example with gallium shaders. LLVM has code, for Mac OS X systems, to do instead RW then RX following the restrictions of the W^X applied on some *BSD kernels. This code could be used if Pax had a way to support JIT code, but Pax explicitly disables the execution of any JIT code, so any mapping which was once writable cannot be turned executable. Finally on systems (and for users) not enforced by grsec TPE. You can try to modify the code to mmap the pages from a file (maybe on shm) and when changing permissions unmap them and mmap them again (which means they may end up in other address) I'm not sure of how well can LLVM cope with this address change and I still think this could be a performance killer if linux tries to copy the pages on kernel space from either disk or the memory device. Then we also have the fact that the TLB is flushed each time a system call is done on a hardened kernel with amd64 which means we want to reduce those to reduce pressure on memory.
llvm on hardened definitely sounds fun :) On llvm itself, I'm not sure we can easily tweak the test system to separate build and run steps (so we can do pax-marking) Though from klondike's comment, the pax-marking may be required on llvm library and binaries themselves?
(In reply to comment #4) > llvm on hardened definitely sounds fun :) > > On llvm itself, I'm not sure we can easily tweak the test system to separate > build and run steps (so we can do pax-marking) > > Though from klondike's comment, the pax-marking may be required on llvm library > and binaries themselves? For now only on binaries, once revdep-pax is out the library will need to be marked too.
Someone in IRC informed me that this can no longer be reproduced. Please try to reproduce it with the latest software and let us know if this is still a problem.
That person informed me that this bug is still valid. I am reopening it.
That was me. I can confirm that this still happens with llvm versions: 2.9-r2 ~3.0-r2 ~3.1 on both kernels: - linux-3.2.2-hardened-r1 - linux-3.4.2-hardened Errors are like: LLVM ERROR: Allocation failed when allocating new memory in the JIT Can't allocate RWX Memory: Operation not permitted From my understanding the original poster Nikoli is correct: Calls to JIT do not work unless MPROTECT is disabled. Something like he suggested might fix this.
I found that adding the following to src_compile() fixes the test-phase: pax-mark m Release/bin/lli if use test; then pax-mark m unittests/ExecutionEngine/JIT/Release/JITTests fi This works on amd64 for ebuilds: - llvm-2.9-r2.ebuild - llvm-3.0-r2.ebuild - llvm-3.1-r1.ebuild - llvm-9999.ebuild Does not work on llvm-2.8-r2.ebuild. I not bored enough to provide a patch for this also unless specifically asked for. Adding diff files for the mentioned versions next.
Created attachment 316967 [details, diff] llvm-2.9-r2.ebuild.diff
Created attachment 316969 [details] llvm-3.0-r2.ebuild.diff
Created attachment 316971 [details] llvm-3.1-r1.ebuild.diff
Created attachment 316973 [details] llvm-9999.ebuild.diff
Thanks! Don't worry about 2.8, going back to 2.9 is more than enough. I've updated the other versions
All tests work fine for sys-devel/llvm-3.1, thank you :)
