Created attachment 270061 [details] kernel messages I'm using qemu-kvm on hardened kernel app-shells/bash: 4.2_p8 dev-lang/python: 2.7.1-r1, 3.1.3-r1 dev-util/cmake: 2.8.4 sys-apps/baselayout: 2.0.2 sys-apps/openrc: 0.8.1 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.68 sys-devel/automake: 1.11.1-r1 sys-devel/binutils: 2.21 sys-devel/gcc: 4.5.2 sys-devel/gcc-config: 1.4.1 sys-devel/libtool: 2.4-r1 sys-devel/make: 3.82 sys-kernel/linux-headers: 2.6.38 virtual/os-headers: 2.6.38 (sys-kernel/linux-headers) ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=core2 -mtune=generic -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /var/bind" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-march=core2 -mtune=generic -O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="assume-digests binpkg-logs distlocks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch" FFLAGS="" GENTOO_MIRRORS="ftp://rush.tisys.org/pub/gentoo/" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j9" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="" SYNC="rsync://rush.tisys.org/gentoo-portage" Portage 2.1.9.46 (hardened/linux/amd64/no-multilib, gcc-4.5.2, glibc-2.13-r2, 2.6.38-hardened x86_64) ================================================================= System uname: Linux-2.6.38-hardened-x86_64-Intel-R-_Core-TM-_i7_CPU_930_@_2.80GHz-with-gentoo-2.0.2 Timestamp of tree: Fri, 15 Apr 2011 09:45:01 +0000 USE="acl amd64 berkdb bzip2 cli cracklib crypt cvs cxx dri gdbm git gnutls gpm hardened iconv ipv6 justify lighttpd mercurial mmx modules mudflap ncurses nls nptl nptlonly openmp pam pcre perl php postgresql pppd python readline sbcl session sse sse2 sse4 ssl ssse3 subversion sysfs tcpd unicode urandom xml xmlrpc xsl zlib" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="ncurses text" PHP_TARGETS="php5-3" USERLAND="GNU" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Created attachment 270063 [details] kernel config
Created attachment 270065 [details] lspci verbose output
Created attachment 270067 [details] dmidecode output
Created attachment 270069 [details] superiotool output
Created attachment 270071 [details] biosdecode output
Created attachment 270073 [details] output of virsh sysinfo
here is gdb output when connect to qemu: (gdb) target remote localhost:1234 Remote debugging using localhost:1234 Ignoring packet error, continuing... warning: unrecognized item "timeout" in "qSupported" response Ignoring packet error, continuing... Ignoring packet error, continuing... warning: Invalid remote reply: timeout warning: Invalid remote reply: PacketSize=1000 warning: Invalid remote reply: PacketSize=1000 warning: Invalid remote reply: PacketSize=1000 warning: Invalid remote reply: PacketSize=1000 [Switching to Thread 1] 0x0000000000000000 in ?? () (gdb) i r rax 0x180000000 6442450944 rbx 0x8000005800000cff -9223371658897650433 rcx 0xcff00006eec 14289356222188 rdx 0xc800000038000 3518437209112576 rsi 0x6fffe73a5 30064669605 rdi 0x1000000008 68719476744 rbp 0x1000000010 0x1000000010 rsp 0x1000000010 0x1000000010 r8 0x0 0 r9 0x0 0 r10 0x0 0 r11 0x0 0 r12 0x0 0 r13 0x0 0 r14 0x0 0 r15 0x0 0 rip 0x0 0 eflags 0x0 [ ] cs 0x0 0 ss 0x37f 895 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0 (gdb) i r rax 0x180000000 6442450944 rbx 0x8000005800000cff -9223371658897650433 rcx 0xcff00006eec 14289356222188 rdx 0xc800000038000 3518437209112576 rsi 0x6fffe73a5 30064669605 rdi 0x1000000008 68719476744 rbp 0x1000000010 0x1000000010 rsp 0x1000000010 0x1000000010 r8 0x0 0 r9 0x0 0 r10 0x0 0 r11 0x0 0 r12 0x0 0 r13 0x0 0 r14 0x0 0 r15 0x0 0 rip 0x0 0 eflags 0x0 [ ] cs 0x0 0 ss 0x37f 895 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0 (gdb) i r rax 0x180000000 6442450944 rbx 0x8000005800000cff -9223371658897650433 rcx 0xcff00006eec 14289356222188 rdx 0xc800000038000 3518437209112576 rsi 0x6fffe73a5 30064669605 rdi 0x1000000008 68719476744 rbp 0x1000000010 0x1000000010 rsp 0x1000000010 0x1000000010 r8 0x0 0 r9 0x0 0 r10 0x0 0 r11 0x0 0 r12 0x0 0 r13 0x0 0 r14 0x0 0 r15 0x0 0 rip 0x0 0 eflags 0x0 [] ] cs 0x0 0 ss 0x37f 895 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0 and here is qemu output (qemu) info roms fw=genroms/vapic.bin size=0x002400 name="vapic.bin" addr=00000000fffe0000 size=0x020000 mem=rom name="bios.bin" (qemu) info irq irq statistic code not compiled. (qemu) info pci Bus 0, device 0, function 0: Host bridge: PCI device 8086:1237 id "" Bus 0, device 1, function 0: ISA bridge: PCI device 8086:7000 id "" Bus 0, device 1, function 1: IDE controller: PCI device 8086:7010 BAR4: I/O at 0xffffffffffffffff []0x000e]. id "" Bus 0, device 1, function 3: Bridge: PCI device 8086:7113 IRQ 0. id "" Bus 0, device 2, function 0: VGA controller: PCI device 1013:00b8 BAR0: 32 bit prefetchable memory at 0xffffffffffffffff []0x01fffffe]. BAR1: 32 bit memory at 0xffffffffffffffff []0x00000ffe]. BAR6: 32 bit memory at 0xffffffffffffffff []0x0000fffe]. id "" (qemu) info mem PG disabled (qemu) info cpus * CPU #0: pc=0x00000000fffe73a5 thread_id=10182 (qemu) info network VLAN 0 devices: tap.0: ifname=tap1,script=no,downscript=/etc/qemu-ifdown Devices not on any VLAN: (qemu) info registers EAX=80000000 EBX=80000058 ECX=00000001 EDX=00000cff ESI=00038000 EDI=000c8000 EBP=00000cff ESP=00006eec EIP=fffe73a5 EFL=00000006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0010 00000000 ffffffff 00c09300 DPL=0 DS [-WA] CS =0008 00000000 ffffffff 00c09b00 DPL=0 CS32 [-RA] SS =0010 00000000 ffffffff 00c09300 DPL=0 DS [-WA] DS =0010 00000000 ffffffff 00c09300 DPL=0 DS [-WA] FS =0010 00000000 ffffffff 00c09300 DPL=0 DS [-WA] GS =0010 00000000 ffffffff 00c09300 DPL=0 DS [-WA] LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS32-busy GDT= 000fce38 00000037 IDT= 000fdcf0 00000000 CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000000 FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80 FPR0=0000000000000000 0000 FPR1=0000000000000000 0000 FPR2=0000000000000000 0000 FPR3=0000000000000000 0000 FPR4=0000000000000000 0000 FPR5=0000000000000000 0000 FPR6=0000000000000000 0000 FPR7=0000000000000000 0000 XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000 XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000 XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000 XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000
starting qemu as qemu-kvm -net tap,ifname=tap1,script=no -net nic -monitor stdio -m 256 -d cpu,in_asm,exec -s -boot d -cdrom debian-minimal.iso -hda debian.qcow2
is known that virt does not work well with hardened
you need to turn on CONFIG_GRKERNSEC_HARDENED_VIRTUALIZATION, not a CONFIG_GRKERNSEC_HARDENED_SERVER
Created attachment 270611 [details] kernel config Grecurity -> Security level -> Virtualization enabled
(qemu) info kvm kvm support: enabled (qemu) info cpus * CPU #0: pc=0x000000000010017c (halted) thread_id=4688 (qemu) info pci Bus 0, device 0, function 0: Host bridge: PCI device 8086:1237 id "" Bus 0, device 1, function 0: ISA bridge: PCI device 8086:7000 id "" Bus 0, device 1, function 1: IDE controller: PCI device 8086:7010 BAR4: I/O at 0xc000 [0xc00f]. id "" Bus 0, device 1, function 3: Bridge: PCI device 8086:7113 IRQ 9. id "" Bus 0, device 2, function 0: VGA controller: PCI device 1013:00b8 BAR0: 32 bit prefetchable memory at 0xf0000000 [0xf1ffffff]. BAR1: 32 bit memory at 0xf2000000 [0xf2000fff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe]. id "" (qemu) info status VM status: running (qemu) info roms fw=genroms/vapic.bin size=0x002400 name="vapic.bin" addr=00000000fffe0000 size=0x020000 mem=rom name="bios.bin" (qemu) info registers EAX=00000000 EBX=00187130 ECX=00187130 EDX=00000000 ESI=00000000 EDI=00000000 EBP=00000000 ESP=0ffcfeac EIP=0010017c EFL=00000246 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=1 ES =0028 00000000 ffffffff 00c09300 DPL=0 DS [-WA] CS =0020 00000000 ffffffff 00c09b00 DPL=0 CS32 [-RA] SS =0028 00000000 ffffffff 00c09300 DPL=0 DS [-WA] DS =0028 00000000 ffffffff 00c09300 DPL=0 DS [-WA] FS =0000 00000000 ffffffff 00000000 GS =0000 00000000 ffffffff 00000000 LDT=0000 00000000 ffffffff 00000000 TR =0008 00000580 00000067 00008b00 DPL=0 TSS32-busy GDT= 0000ab80 0000002f IDT= 000030b8 000007ff CR0=00000013 CR2=00000000 CR3=00000000 CR4=00000000 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000000 FCW=037f FSW=0020 [ST=0] FTW=00 MXCSR=00001f80 FPR0=f44d002c60000000 400d FPR1=80847fe700000000 400e FPR2=fa007fa240000000 400e FPR3=80e88055f0000000 400e FPR4=ea61009c40000000 400d FPR5=ea62009c40000000 400c FPR6=bb7fffb9b0000000 400b FPR7=bb83ffb9b0000000 400b XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000 XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000 XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000 XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000
Additional discussion in qemu-devel mailing list http://lists.nongnu.org/archive/html/qemu-devel/2011-04/msg01547.html
Added bug in kernel kvm bugtracker https://bugzilla.kernel.org/show_bug.cgi?id=33762
(In reply to comment #12) > EIP=0010017c EFL=00000246 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=1 can you 'objdump -drw vmlinux' and show me what kind of code is at 0x10017c?
not found 0x10017c line here is full objdump output http://ompldr.org/vOG82ag/vmlinux_objdump.txt.bz2
Do you still have this issue with kernel hardened-sources-2.6.38-r6?
This was a strange bug. It give me infinite loop, when i try load any x86_64 guest in qemu (qemu-system-x86_64, btw), and works ok, when load 32-bit guest. So, you can close this bug for kernel. Looks like this is internal bug in qemu. Because I have 64-bit only system, it looks strange...
Are you still experiencing the issue though?
Yes, when trying to loading x86_64 guest. On 32-bit all ok.
Well, in that case it's up to the kernel guys whether they want to drop it.
(In reply to comment #21) > Well, in that case it's up to the kernel guys whether they want to drop it. I have not hit this and I've been waiting to see if the PaX team has any suggestions. In the mean time, it might help narrow down if you try earlier and later kernels to see if you hit it there.
(In reply to comment #22) > I have not hit this and I've been waiting to see if the PaX team has any > suggestions. i've been debugging and fixing several issues but i don't know about this one really, at most i can think of some toolchain issue (e.g., recently someone reported a problem on the grsec forums that was fixed by switching to gcc 4.5 and binutils 2.21). or another bug with paravirt support on 32 bit that i thought i'd managed to reproduce but then on the next recompilation the code looked fine - black magic or i don't know what ;). so for this bug i don't know what else we can do, maybe Anton could send me a failing bzImage and i can try to reproduce it here...
All solved, looks like this is qemu bug - even if qemu built for x86-64 target it can load only 32-bit guests.
(In reply to comment #24) > All solved, looks like this is qemu bug - even if qemu built for x86-64 target > it can load only 32-bit guests. Can you post a link to the solution so we have a record for this bug.
(In reply to comment #25) > (In reply to comment #24) > > All solved, looks like this is qemu bug - even if qemu built for x86-64 target > > it can load only 32-bit guests. > > Can you post a link to the solution so we have a record for this bug. There is no solution - just load 32-bit guests, instead of x86-64 on qemu-kvm on hardened kernel (at least in my case). Looks like this is qemu-kvm bug, and after inspesting the sources i can't find it.
