KVM gets killed by the stack smashing protection of gcc-4.3.4 as the (Windows XP) virtual machine begins to boot. It works as expected when KVM is compiled with vanilla GCC. $ kvm /mnt/storage/kvm/wxp-devel.img *** stack smashing detected ***: kvm - terminated kvm: stack smashing attack in function <unknown> - terminated Report to http://bugs.gentoo.org/ Killed $ emerge --info Portage 2.1.6.13 (hardened/linux/amd64/10.0, gcc-4.3.4, glibc-2.9_p20081201-r4, 2.6.31-gentoo x86_64) ================================================================= System uname: Linux-2.6.31-gentoo-x86_64-AMD_Phenom-tm-_II_X4_920_Processor-with-gentoo-1.12.11.1 Timestamp of tree: Mon, 21 Sep 2009 17:45:01 +0000 app-shells/bash: 4.0_p28 dev-lang/python: 2.6.2-r1 dev-util/cmake: 2.6.4 sys-apps/baselayout: 1.12.11.1 sys-apps/sandbox: 1.6-r2 sys-devel/autoconf: 2.13, 2.63-r1 sys-devel/automake: 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.2 sys-devel/binutils: 2.18-r3 sys-devel/gcc-config: 1.4.1 sys-devel/libtool: 2.2.6a virtual/os-headers: 2.6.27-r2 ACCEPT_KEYWORDS="amd64" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -O2 -pipe -fomit-frame-pointer" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/eselect/postgresql /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /etc/udev/rules.d" CXXFLAGS="-march=native -O2 -pipe -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="distlocks fixpackages parallel-fetch protect-owned sandbox sfperms strict unmerge-orphans userfetch" GENTOO_MIRRORS="http://distfiles.gentoo.org/ http://gentoo.osuosl.org/" LANG="en_US.utf8" LDFLAGS="-Wl,-O1" LINGUAS="en" MAKEOPTS="-j5" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage /home/mjo/src/hardened-development" SYNC="rsync://rsync.us.gentoo.org/gentoo-portage" USE="3dnow X acl acpi alsa amd64 bash-completion bzip2 cairo cli cracklib crypt cups curl dbus djvu doc dri dv dvd dvdr emacs encode exif ffmpeg flac foomaticdb fuse gif gnutls gstreamer gtk hal hardened iconv imap imlib isdnlog jpeg jpeg2k justify latex lcms matroska mmx mng modules mp3 mpeg mudflap multilib ncurses nptl nptlonly offensive ofx ogg openal openexr opengl pam pcre pdf pic png ppds pppd quicktime readline reflection ruby schroedinger sdl session smp speex spell spl sse sse2 ssl startup-notification svg sysfs tcpd theora threads tiff truetype unicode urandom usb vorbis wmf x264 xorg xpm xvid zlib" ALSA_CARDS="emu10k1" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" ELIBC="glibc" INPUT_DEVICES="evdev keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en" SANE_BACKENDS="plustek" USERLAND="GNU" VIDEO_CARDS="nv nvidia vesa" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
For me it works, so my question is: are you sure that all needed modules for proper hardware support is loaded? It is still a bug in the ebuild that needs to be fixed, but I can only see this problem if I use softmmu, and I think kvm falls back on it if the proper hardware support cannot be used. `
Change filter-flags -fPIE -fstack-protector to filter-flags -fPIE filter-flags -fstack-protector in the ebuild if that fix it.
You can also try to just remove -fpie from the filter-flags, since kvm still builds without it.
I rebuilt kvm using all of the filter-flags suggestions with no success. What modules should I be checking for? How can I tell if kvm is falling back to softmmu?
$ ls -l /sys/devices/system/kvm/ IIRC this should not exist or be empty if not everything kvm-wise is loaded
It contains one directory, kvm0, which is empty. $ ls -l /sys/devices/system/kvm/ total 0 drwxr-xr-x 2 root root 0 2009-09-27 13:35 kvm0
If you with gcc-config choose -vanilla and rebuilds qemu, does it stack smash or just segfaults when you try to run it?
also are you using the nvidia-binary driver? if so try to "kvm -nographic /mnt/storage/kvm/wxp-devel.img" and tells us if that also segfaults/stacksmash
It works fine when I compile kvm with vanilla GCC. I am using nvidia-drivers, but -nographic didn't help: $ kvm -nographic /mnt/storage/kvm/wxp-devel.img *** stack smashing detected ***: kvm - terminated kvm: stack smashing attack in function <unknown> - terminated Report to http://bugs.gentoo.org/ Killed
Ok, now we take this slow and careful: Are your user in group kvm? I can only reproduce your runtime problems if I run 'kvm -no-kvm', so I have a feeling your kvm-installation does not use the hardware. Also do "ls -l /dev/kvm". If this does not work: First remove qemu from all you overlays and make sure that you build qemu from portage. Then do the following as root gcc-config x86_64-pc-linux-gnu-4.3.4 && source /etc/profile && emerge -1q qemu try run qemu as your regular user. gcc-config x86_64-pc-linux-gnu-4.3.4-hardenednopie && source /etc/profile && emerge -1q qemu try run qemu as your regular user. gcc-config x86_64-pc-linux-gnu-4.3.4-hardenednossp && source /etc/profile && emerge -1q qemu try run qemu as your regular user. From all three tries, post the output here.
(In reply to comment #10) Of course I mean kvm here instead of qemu. I forgot to say post the output from ls -l /dev/kvm What groups you are currently in you see with 'groups'.
$ ls -l /dev/kvm crw-rw---- 1 root kvm 10, 232 2009-09-27 13:35 /dev/kvm $ groups audio cdrom video games postgres cdrw scanner vmware mjo kvm The two other less-hardened compilers work as well. Both, x86_64-pc-linux-gnu-4.3.4-hardenednopie x86_64-pc-linux-gnu-4.3.4-hardenednossp compile KVM and run the VM just fine. x86_64-pc-linux-gnu-4.3.4 still dies with the SSP error, though.
(In reply to comment #12) > The two other less-hardened compilers work as well. Both, > > x86_64-pc-linux-gnu-4.3.4-hardenednopie > x86_64-pc-linux-gnu-4.3.4-hardenednossp > > compile KVM and run the VM just fine. > > x86_64-pc-linux-gnu-4.3.4 still dies with the SSP error, though. > And this really really confuses me, because the ebuild from portage may fail to filter SSP, but it should still filter -fPIE, and you would automatically end up with choice 1. I am pretty confused, and also to why I cannot reproduce the fault... Does kvm work with any other OS? Does kvm die while booting windows, or before it has shown its BIOS-screen? Post this info more for completness, but I have actually no idea how to proceed. Zorry: maybe you have some thoughts about the filtering at least?
It does work with another OS: I was able to install and run the latest Zenwalk ISO, even after compiling KVM with the default hardened GCC. I have three closely-related Windows XP VMs: one base image, and two derived from that base image, all of which cause the SSP error. I'm as confused as you. They start to boot, i.e. they make it past the BIOS screen, but then die as soon as Windows starts to load. On the reboot, it makes it to the "...Start Windows in Safe Mode..." screen, and crashes only when I make a selection. I did copy those three images from another machine -- a Gentoo laptop using kvm-intel instead of kvm-amd. Did I do something dumb there?
(In reply to comment #14) > I did copy those three images from another machine -- a Gentoo laptop using > kvm-intel instead of kvm-amd. Did I do something dumb there? This should afaik work. I suspect currently that the kvm_amd and kvm_intel may do some things differently, and kvm uses softmmu to compensate, but since the compilation of softmmu is broken this fails on gentoo hardened. And it should not.
Ok, I want you to try something. In the original ebuild, do the following change: exchange filter-flags -fpie -fstack-protector with filter-flags -fpie filter-flags -fstack-protector append-ldflags -nopie recompile, and try again.
That didn't work either. I also tried re-copying the VM images, in case they were corrupted. Didn't help. They still run fine on the Intel machine, with the same version of KVM, only compiled with gcc-4.4.1.
(In reply to comment #17) > That didn't work either. > > I also tried re-copying the VM images, in case they were corrupted. Didn't > help. They still run fine on the Intel machine, with the same version of KVM, > only compiled with gcc-4.4.1. > Did you get an segmentationfault or an stack smash?
Stack smash.
Ok, do you want to try the qemu-0.10.6 and kvm-88-r1 from the hardened overay, and try your images against both? With kvm both with and without -no-kvm and for qemu both with and without -enable-kvm?
kvm : Killed by SSP kvm -no-kvm : Works qemu : Works qemu -enable-kvm: Works Both KVM and qemu from the overlay were compiled with x86_64-pc-linux-gnu-4.3.4.
Ok, now we know that filterflags works most of the time, now we just need to figure out what we are missing. I have updated the kvm patch with some new CFLAGS-work. Would you mind git pull the hardened overlay and recompile kvm and retry it again (no need for -no-kvm or any other test)?
Just tried it; same results for kvm/no-kvm.
(In reply to comment #23) > Just tried it; same results for kvm/no-kvm. > So it stopped working with -no-kvm? Think I have to dig around in the Makefiles some more.:-(
No, sorry. It worked with -no-kvm, but still died from SSP without it. I'm going to try with gcc-4.4.x here in a minute.
No change when KVM is compiled with gcc-4.4.1-r1. I guess I'll try to recompile the kernel later when I'm physically at the machine.
Ok, a status update on this bug. -fPIE breaks everything softmmu, but not really much kvm-wise. Therefore I think something in your image requires a bit of softmmu on your amd system, probably becouse kvm has to interact diffrently with the diffrent hardware (i.e. intel/amd wise). The problem is afaics that qemu buildsystems are not the most sane around - and kvms' is even unsaner - and for the filter-flags to work we need to have *FLAGS set where they should. I have made some hunting but I am not really sure any longer where it breaks. @reporter Michael, could you try to add -v to your CFLAGS and then run V=1 emerge -1 kvm > build.log 2>&1 and attach build.log here?
(In reply to comment #27) > @reporter > > Michael, could you try to add -v to your CFLAGS and then run > V=1 emerge -1 kvm > build.log 2>&1 > and attach build.log here? > Scratch that, I just found a new level off CFLAGS/configure that makes this mess just more fun... I be back with a new ebuild/patch.
Could you do me a favor? Can you do the following? Remerge glibc with debuggingsymbols and USE="debug". Also remerge kvm with its debuggingsymbols. Then attach a backtrace of the crash in kvm as a comment or if it is big, as a textfile. This will tell us where kvm segfaults and help us very much to nail where the problem is. If you do not know how to obtain a backtrace or what I am talking about, the following page has good descriptions, just remember that as a hardened user you NEED glibc compiled with USE="debug", else the backtrace (as it is considered a security concern) is rendered useless. http://www.gentoo.org/proj/en/qa/backtraces.xml Please tell if you feel up for it!
GDB has not once ever done anything I've asked it to. Any idea what I should do here? $ gdb kvm GNU gdb 6.8 Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu"... (gdb) run /mnt/storage/kvm/wxp-devel.img Starting program: /usr/bin/kvm /mnt/storage/kvm/wxp-devel.img [Thread debugging using libthread_db enabled] [New Thread 0x7fccf361f6f0 (LWP 7959)] [New Thread 0x7fccefcc2910 (LWP 7963)] [New Thread 0x7fcce6aae910 (LWP 7964)] [Thread 0x7fcce6aae910 (LWP 7964) exited] [New Thread 0x7fcce6aae910 (LWP 7973)] *** stack smashing detected ***: kvm - terminated kvm: stack smashing attack in function <unknown> - terminated Report to http://bugs.gentoo.org/ Program terminated with signal SIGKILL, Killed. The program no longer exists. (gdb) bt full No stack.
(In reply to comment #30) > [New Thread 0x7fcce6aae910 (LWP 7973)] > *** stack smashing detected ***: kvm - terminated > kvm: stack smashing attack in function <unknown> - terminated > Report to http://bugs.gentoo.org/ > Are you sure you have glibc compiled with USE="debug"?
Yep. I switched CFLAGS to "-ggdb" too. $ emerge -pv glibc These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild R ] sys-libs/glibc-2.10.1 USE="hardened (multilib) -debug* -gd -glibc-omitfp -nls -profile (-selinux) -vanilla" 0 kB
That last revision to the hardened-development overlay seems to have fixed it. Fingers crossed. Thanks for all your work.
This is really not fixed in the overlay, what we have fixed so far is that flags are actually handled correctly, so filter-flags works. Next step is for someone to produce a backtrace of this bug since AFAIK the stack smash should never happen any way. So the question is, should we go for filter stack-protector and close this bug with that, or should we have this open and try to get someone to produce a backtrace? Me myself have made the hoops default on my system, so I do actually not currently remember the hoops to produce a backtrace with hardened/PAX/GRSec... I am going to push the filter-flags fix in another bugreport.
I don't mind trying to debug this if anyone has an idea how to get a backtrace from a process that was killed by SSP. I am able to get them otherwise with the hardened toolchain (segfaults, manually killing the process, etc.), just not when SSP is involved.
(In reply to comment #35) > I don't mind trying to debug this if anyone has an idea how to get a backtrace > from a process that was killed by SSP. I am able to get them otherwise with the > hardened toolchain (segfaults, manually killing the process, etc.), just not > when SSP is involved. On hardened, you really need to do "USE=debug emerge sys-libs/glibc", else your program would be terminated from __stack_chk_fail inside glibc by signal 9, and consequently without dumping core, and without gdb being able to intercept. From your comment #32 I see you actually didn't compile with USE=debug. It's simply not enough to have -g in CFLAGS/CXXFLAGS. See BUG #297467 reporting kstars crashing due to SSP for some background information.
I did compile both glibc and kvm with -ggdb and USE=debug. The output in comment #32 just shows that, if I were to re-emerge it, debug would be (newly) disabled. But, this bug is so old that I don't think it matters. I honestly don't remember most of the details anymore.
This issue is about a version of qemu that has been dropped from the tree sometime ago. I'd suggest closing this bug or requesting someone to test it with the latest qemu-kvm version in the tree.
This report is about an ebuild no longer in tree. Additionally, i dont have any issues with latest qemu-kvm. If you still have a problem with latest qemu-kvm, please open a new bug with the details. Closing this bug with the aggreement of Zorry
