Summary: | sys-kernel/hardened-sources-2.6.39-r8 and above: workstation get rebooted during vmware/virtualbox start up | ||
---|---|---|---|
Product: | Gentoo Linux | Reporter: | Anton Bolshakov <anton.bugs> |
Component: | Hardened | Assignee: | The Gentoo Linux Hardened Kernel Team (OBSOLETE) <hardened-kernel+disabled> |
Status: | RESOLVED CANTFIX | ||
Severity: | normal | CC: | pageexec, powerman-asdf, prometheanfire, pva, spender, underling, zerochaos |
Priority: | Normal | ||
Version: | 10.0 | ||
Hardware: | All | ||
OS: | Linux | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Bug Depends on: | |||
Bug Blocks: | 350475 | ||
Attachments: |
2.6.37-r7 working config
3.0.4 config, almost without any changes paxtest output with 2.6.37 paxtest output with 2.6.39 and above lspci output vmware vmnet patch for gcc-4.5.x |
Description
Anton Bolshakov
2011-09-13 08:55:23 UTC
Created attachment 286311 [details]
2.6.37-r7 working config
Created attachment 286313 [details]
3.0.4 config, almost without any changes
Created attachment 286321 [details]
paxtest output with 2.6.37
Created attachment 286323 [details]
paxtest output with 2.6.39 and above
It might be something to do with pax settings. This is the only difference I noticed partial "diff -u paxtest-2.6.37.log paxtest-3.0.4.log" output:
+Executable anonymous mapping : Killed
+Executable bss : Killed
+Executable data : Killed
+Executable heap : Killed
+Executable stack : Killed
+Executable shared library bss : Killed
+Executable shared library data : Killed
Created attachment 286325 [details]
lspci output
Yep confirmed. I'm busy these days, do you think you can get a kernel panic with netconsole. If not, bug me and I'll do it. Well, I haven't got much from the netconsole output: Sep 17 03:48:07 pt /dev/vmmon[3856]: PTSC: initialized at 2393999000 Hz using TSC Sep 17 03:48:07 pt /dev/vmmon[3856]: HV check: anyNotCapable=0 anyUnlocked=0 anyEnabled=0 anyDisabled=1 Sep 17 03:48:25 pt Program vmware-vmx tried to access /dev/mem between 0->100000. Sep 17 03:48:25 pt Program vmware-vmx tried to access /dev/mem between 0->100000. Sep 17 03:48:25 pt Program vmware-vmx tried to access /dev/mem between 0->100000. Sep 17 03:48:25 pt Program vmware-vmx tried to access /dev/mem between 0->100000. the setting "deny writting to /dev/mem" is not enabled in my kernel. Any ideas? (In reply to comment #7) > Sep 17 03:48:25 pt Program vmware-vmx tried to access /dev/mem between > 0->100000. > > the setting "deny writting to /dev/mem" is not enabled in my kernel. > > Any ideas? what about CONFIG_STRICT_DEVMEM? (In reply to comment #8) > what about CONFIG_STRICT_DEVMEM? I also disable it but the problem is still the same. Is any other mechanisms are there for crash catching or any traces of the problem? (In reply to comment #9) > Is any other mechanisms are there for crash catching or any traces of the problem? since the problem (probably a triple fault) is triggered by the hypervisor code, not some guest code, you'd have to debug this under another hypervisor that supports nested virtualization and use *its* debug facilities to get some information about where exactly the (nested) hypervisor code fails... Hi, I'm experiencing same problems on Gentoo Hardened. VMware Workstation 8.0.x is rebooting my PC when trying to run VM with GRSEC patched kernel. Tested on: hardened-sources-3.1.10 hardened-sources-3.2.2-r1 hardened-sources-3.2.6 Also with vanilla kernel 3.2.7 + latest grsecurity patch and with gentoo-sources-3.2.1-r2 + grsecurity patch (<- without GRSEC patch, VMware works fine) Looks like the kernel just have to be patched with grsecurtity to make VMware able to reboot the PC. All GRSEC/PAX kernel options can be disabled - that wont help anyway. VirtualBox-bin works, no reboot (however VM wont start with "Virtualization" profile in kernel config) In addition, someone recently writed about this also on gentoo-hardened mailing list: http://archives.gentoo.org/gentoo-hardened/msg_e3b8a52d0a853cb747b7aa7e73f7a210.xml vmware-modules doesn't even install for me. You have a patch for that so I can reproduce the issue? It looks like there is not a way to get vmware-modules installed on hardened-amd64. I did document this on 2011-10-31 http://git.overlays.gentoo.org/gitweb/?p=proj/hardened-docs.git;a=blob_plain;f=html/hardened-virtualization.html;hb=877d7d12b2d0d0431a6c137012181cee4742e1ba >>> Source configured. >>> Compiling source in /var/tmp/portage/app-emulation/vmware-modules-264.1/work ... * Preparing vmblock module make -j5 HOSTCC=x86_64-pc-linux-gnu-gcc CROSS_COMPILE=x86_64-pc-linux-gnu- 'LDFLAGS=-m elf_x86_64' auto-build KERNEL_DIR=/usr/src/linux KBUILD_OUTPUT=/lib/modules/3.2.6-hardened/build Using 2.6.x kernel build system. make -C /lib/modules/3.2.6-hardened/build SUBDIRS=$PWD SRCROOT=$PWD/. \ MODULEBUILDDIR= modules make[1]: Entering directory `/usr/src/linux-3.2.6-hardened' CC [M] /var/tmp/portage/app-emulation/vmware-modules-264.1/work/vmblock-only/linux/filesystem.o CC [M] /var/tmp/portage/app-emulation/vmware-modules-264.1/work/vmblock-only/linux/dentry.o CC [M] /var/tmp/portage/app-emulation/vmware-modules-264.1/work/vmblock-only/linux/dbllnklst.o cc1: error: incompatible gcc/plugin versions cc1: error: Fail to initialize plugin /usr/src/linux-3.2.6-hardened/tools/gcc/constify_plugin.so cc1: error: incompatible gcc/plugin versions cc1: error: Fail to initialize plugin /usr/src/linux-3.2.6-hardened/tools/gcc/stackleak_plugin.so CC [M] /var/tmp/portage/app-emulation/vmware-modules-264.1/work/vmblock-only/linux/file.o CC [M] /var/tmp/portage/app-emulation/vmware-modules-264.1/work/vmblock-only/linux/block.o make[3]: *** [/var/tmp/portage/app-emulation/vmware-modules-264.1/work/vmblock-only/linux/filesystem.o] Error 1 make[3]: *** Waiting for unfinished jobs.... cc1: error: incompatible gcc/plugin versions cc1: error: Fail to initialize plugin /usr/src/linux-3.2.6-hardened/tools/gcc/constify_plugin.so cc1: error: incompatible gcc/plugin versions cc1: error: Fail to initialize plugin /usr/src/linux-3.2.6-hardened/tools/gcc/stackleak_plugin.so cc1: error: incompatible gcc/plugin versions cc1: error: Fail to initialize plugin /usr/src/linux-3.2.6-hardened/tools/gcc/constify_plugin.so cc1: error: incompatible gcc/plugin versions cc1: error: Fail to initialize plugin /usr/src/linux-3.2.6-hardened/tools/gcc/stackleak_plugin.so cc1: error: incompatible gcc/plugin versions cc1: error: Fail to initialize plugin /usr/src/linux-3.2.6-hardened/tools/gcc/constify_plugin.so cc1: error: incompatible gcc/plugin versions cc1: error: Fail to initialize plugin /usr/src/linux-3.2.6-hardened/tools/gcc/stackleak_plugin.so cc1: error: incompatible gcc/plugin versions cc1: error: Fail to initialize plugin /usr/src/linux-3.2.6-hardened/tools/gcc/constify_plugin.so cc1: error: incompatible gcc/plugin versions cc1: error: Fail to initialize plugin /usr/src/linux-3.2.6-hardened/tools/gcc/stackleak_plugin.so make[3]: *** [/var/tmp/portage/app-emulation/vmware-modules-264.1/work/vmblock-only/linux/block.o] Error 1 make[3]: *** [/var/tmp/portage/app-emulation/vmware-modules-264.1/work/vmblock-only/linux/file.o] Error 1 make[3]: *** [/var/tmp/portage/app-emulation/vmware-modules-264.1/work/vmblock-only/linux/dbllnklst.o] Error 1 make[3]: *** [/var/tmp/portage/app-emulation/vmware-modules-264.1/work/vmblock-only/linux/dentry.o] Error 1 make[2]: *** [_module_/var/tmp/portage/app-emulation/vmware-modules-264.1/work/vmblock-only] Error 2 make[1]: *** [sub-make] Error 2 make[1]: Leaving directory `/usr/src/linux-3.2.6-hardened' make: *** [vmblock.ko] Error 2 emake failed * ERROR: app-emulation/vmware-modules-264.1 failed (compile phase): * Unable to emake HOSTCC=x86_64-pc-linux-gnu-gcc CROSS_COMPILE=x86_64-pc-linux-gnu- LDFLAGS=-m elf_x86_64 auto-build KERNEL_DIR=/usr/src/linux KBUILD_OUTPUT=/lib/modules/3.2.6-hardened/build * * Call stack: * ebuild.sh, line 85: Called src_compile * environment, line 3450: Called linux-mod_src_compile * environment, line 2580: Called die * The specific snippet of code: * eval "emake HOSTCC=\"$(tc-getBUILD_CC)\" CROSS_COMPILE=${CHOST}- LDFLAGS=\"$(get_abi_LDFLAGS)\" ${BUILD_FIXES} ${BUILD_PARAMS} ${BUILD_TARGETS} " || die "Unable to emake HOSTCC="$(tc-getBUILD_CC)" CROSS_COMPILE=${CHOST}- LDFLAGS="$(get_abi_LDFLAGS)" ${BUILD_FIXES} ${BUILD_PARAMS} ${BUILD_TARGETS}"; * * If you need support, post the output of 'emerge --info =app-emulation/vmware-modules-264.1', * the complete build log and the output of 'emerge -pqv =app-emulation/vmware-modules-264.1'. * The complete build log is located at '/usr/portage/log/app-emulation:vmware-modules-264.1:20120226-204519.log'. * The ebuild environment file is located at '/var/tmp/portage/app-emulation/vmware-modules-264.1/temp/environment'. * S: '/var/tmp/portage/app-emulation/vmware-modules-264.1/work' I'm still with gcc-4.4.5 and no problems with compilation. It looks like your kernel wasn't compiled with gcc-4.5.x which support plugins like stackleak/constify and you are trying to compile modules with gcc-4.5.x. you should be missing "constify_plugin.so" and "stackleak_plugin.so" in "/usr/src/linux-3.2.6-hardened/tools/gcc/". anyway, I was playing with gcc-4.5.x too - these modules wont compile with 4.5.x even if plugins are on the place because of this error(s): ---- /var/tmp/portage/app-emulation/vmware-modules-264.1/work/vmci-only/linux/driver.c:388:4: error: assignment of read-only variable ‘vmuser_fops’ /var/tmp/portage/app-emulation/vmware-modules-264.1/work/vmci-only/linux/driver.c:389:4: error: assignment of read-only variable ‘vmuser_fops’ ---- which leads us to https://bugs.gentoo.org/show_bug.cgi?id=386721 where Andrew Dean posted a patch for one module (vmci driver.c patch) which resolves compilation for VMCI. Still we will get errors for VMNET: ---- /opt/vmware/lib/vmware/modules/source/vmnet-only/userif.c: In function ‘VNetCsumCopyDatagram’: /opt/vmware/lib/vmware/modules/source/vmnet-only/userif.c:520: error: incompatible type for argument 1 of ‘kmap’ include/linux/highmem.h:48: note: expected ‘struct page *’ but argument is of type ‘const struct <anonymous>’ /opt/vmware/lib/vmware/modules/source/vmnet-only/userif.c:523: error: incompatible type for argument 1 of ‘kunmap’ ---- so I made a patch (dunno if this one is good-I can attach it here If u want), which is based on Andrew's patch, to solve VMNET... but yet again, it is needed only in case of using gcc-4.5.x. The kernel was compiled with 4.5 and the plugins are there, can you attach your patch? (In reply to comment #13) > I'm still with gcc-4.4.5 and no problems with compilation. > > It looks like your kernel wasn't compiled with gcc-4.5.x which support plugins > like stackleak/constify and you are trying to compile modules with gcc-4.5.x. > > you should be missing "constify_plugin.so" and "stackleak_plugin.so" in > "/usr/src/linux-3.2.6-hardened/tools/gcc/". > > anyway, I was playing with gcc-4.5.x too - these modules wont compile with > 4.5.x even if plugins are on the place because of this error(s): > ---- > /var/tmp/portage/app-emulation/vmware-modules-264.1/work/vmci-only/linux/driver.c:388:4: > error: assignment of read-only variable ‘vmuser_fops’ > /var/tmp/portage/app-emulation/vmware-modules-264.1/work/vmci-only/linux/driver.c:389:4: > error: assignment of read-only variable ‘vmuser_fops’ > ---- > which leads us to https://bugs.gentoo.org/show_bug.cgi?id=386721 > where Andrew Dean posted a patch for one module (vmci driver.c patch) which > resolves compilation for VMCI. Still we will get errors for VMNET: > ---- > /opt/vmware/lib/vmware/modules/source/vmnet-only/userif.c: In function > ‘VNetCsumCopyDatagram’: > /opt/vmware/lib/vmware/modules/source/vmnet-only/userif.c:520: error: > incompatible type for argument 1 of ‘kmap’ > include/linux/highmem.h:48: note: expected ‘struct page *’ but argument is of > type ‘const struct <anonymous>’ > /opt/vmware/lib/vmware/modules/source/vmnet-only/userif.c:523: error: > incompatible type for argument 1 of ‘kunmap’ > ---- > so I made a patch (dunno if this one is good-I can attach it here If u want), > which is based on Andrew's patch, to solve VMNET... but yet again, it is needed > only in case of using gcc-4.5.x. Created attachment 303443 [details, diff]
vmware vmnet patch for gcc-4.5.x
Sure, patch on the way.
BTW: I reproduced these errors with 3.2.6-hardened:
----
cc1: error: incompatible gcc/plugin versions
cc1: error: Fail to initialize plugin
/usr/src/linux-3.2.6-hardened/tools/gcc/constify_plugin.so
cc1: error: incompatible gcc/plugin versions
cc1: error: Fail to initialize plugin
/usr/src/linux-3.2.6-hardened/tools/gcc/stackleak_plugin.so
----
kernel which wasnt compiled with gcc-4.5.x because then I wont have "stackleak_plugin.so" and constify plugin in /tools/gcc - with 4.5.3 got them.
oh, kk, I see you also have these plugins - nvm. also, see the bug #384739 for more 3.x.x patches. sorry for ACKing but is there any progress? Been a while, but no progress has been made, a while ago I tried to get it working with pipacs but I think the way we are going is to support kvm for server virt and virtualbox for desktop virt (on the hardened project). Looks like this won't be ever fixed. :( As was recommended in hardened maillist, I'm switching to qemu. Got Win7x64 working by converting VMware image, handcrafting&compiling custom BIOS, switching windows to "Testing mode" to load unsigned qemu drivers, patching spice-gtk to not reset Xorg's DPI after entering fullscreen into guest OS… Finally, it's usable now, works at similar speed to VMware, except lack of video 3D/acceleration. I suppose other Win and *nix also should work. Main problem with qemu-kvm is lack of modern MacOSX versions support. (In reply to comment #20) > Looks like this won't be ever fixed. :( > > As was recommended in hardened maillist, I'm switching to qemu. > Got Win7x64 working by converting VMware image, handcrafting&compiling > custom BIOS, switching windows to "Testing mode" to load unsigned qemu > drivers, patching spice-gtk to not reset Xorg's DPI after entering > fullscreen into guest OS… Finally, it's usable now, works at similar speed > to VMware, except lack of video 3D/acceleration. > I suppose other Win and *nix also should work. > Main problem with qemu-kvm is lack of modern MacOSX versions support. I'm sorry but it looks like virtualbox + pax is a lost cause :( I've been using qemu + hardened for a while, so I think that's going to be the only path we can support. I would like to have had virtualbox for the reasons you state but ... (In reply to comment #21) > (In reply to comment #20) > > Looks like this won't be ever fixed. :( > > > > As was recommended in hardened maillist, I'm switching to qemu. > > Got Win7x64 working by converting VMware image, handcrafting&compiling > > custom BIOS, switching windows to "Testing mode" to load unsigned qemu > > drivers, patching spice-gtk to not reset Xorg's DPI after entering > > fullscreen into guest OS… Finally, it's usable now, works at similar speed > > to VMware, except lack of video 3D/acceleration. > > I suppose other Win and *nix also should work. > > Main problem with qemu-kvm is lack of modern MacOSX versions support. > > I'm sorry but it looks like virtualbox + pax is a lost cause :( I've been > using qemu + hardened for a while, so I think that's going to be the only > path we can support. I would like to have had virtualbox for the reasons > you state but ... Just for reference, I use vbox every day on hardened with no issues an this config: https://code.google.com/p/pentoo/source/browse/livecd/trunk/amd64/kernel/config-3.8.6 *** Bug 404155 has been marked as a duplicate of this bug. *** (In reply to Rick Farina (Zero_Chaos) from comment #22) > Just for reference, I use vbox every day on hardened with no issues an this > config: > > https://code.google.com/p/pentoo/source/browse/livecd/trunk/amd64/kernel/ > config-3.8.6 I've just took your 3.9.9 config, copied a lot of your settings to my config, and got some effect! Previously when I was trying to use VirtualBox I had two options: 1) with disabled VT-x/AMD-V it was able to boot Ubuntu install .iso, but only in 32-bit mode by unknown reason (my Gentoo host is 64-bit) 2) with enabled VT-x/AMD-V it fail to start virtual machine and I see several kernel BUG reports in kernel log With current settings VirtualBox successfully started 64-bit Ubuntu with VT-x/AMD-V enabled! Sadly, but attempt to run Win7 install result in early crash (I'll google for shown error code later, maybe I'll find some tweaks for VirtualBox settings to avoid that error). VMware still reset host os when I try to start any virtual machine, nothing was changed. Later I'll try to bisect differences between my original kernel config and current one to find which setting affect VirtualBox so critically. Thanks! Okay. To have VirtualBox working on hardened nowadays all we need is switch off these two PaX options: [ ] Enforce non-executable kernel pages [ ] Randomize kernel stack base I'm using these packages from main portage without any extra patches: sys-kernel/hardened-sources-3.9.9 app-emulation/virtualbox-bin-4.1.26 app-emulation/virtualbox-modules-4.1.26 This situation differs from 2 years old one, because at that time VirtualBox didn't work on hardened-sources even when all GrSecurity and PaX options was switched off! So that I can add a VirtualBox option to the automatic configuration: With grsec as the host, what features need to be disabled for VirtualBox to work? With grsec as the guest, what features need to be disabled for the kernel to boot? Thanks, -Brad (In reply to Brad Spengler from comment #26) > With grsec as the host, what features need to be disabled for VirtualBox to > work? Two options listed in my previous comment. > With grsec as the guest, what features need to be disabled for the kernel to > boot? Don't know, I didn't tried to install Gentoo guest yet. BTW, just installed MacOSX 10.8 guest (using iAtkos ML2) - works ok. Huh. |