I've no previous virtualbox experience (always used VMware before). I've installed it, reboot to load modules (hardened kernel configured to prevent loading modules after boot), created few new virtual machines (including completely default ones, just by clicking next-next-next in wizard), but when I try to start any of these virtual machines, both with and without using VT-x (I've Core2Duo 6600), result is same: empty black window with title "starting" and kernel bug details in log. I've tried -bin and -ose, stable and ~x86, I've even tried to completely disable PaX and GrSecurity in kernel - no way, it doesn't work on my system. I'll attach logs with kernel bug details. I usually don't use udev, and just do `mknod /dev/vboxnetctl c 10 56` and `mknod /dev/vboxdrv c 10 57`, but while experimenting with this I've also tried to boot with udev - no luck, same kernel bug happens again. Reproducible: Always Portage 2.1.7.17 (hardened/linux/x86/10.0, gcc-4.3.4, glibc-2.10.1-r1, 2.6.28-hardened-r9 i686) ================================================================= System uname: Linux-2.6.28-hardened-r9-i686-Intel-R-_Core-TM-2_CPU_6600_@_2.40GHz-with-gentoo-1.12.13 Timestamp of tree: Fri, 02 Apr 2010 23:00:01 +0000 app-shells/bash: 4.0_p37 dev-java/java-config: 2.1.10 dev-lang/python: 2.6.4-r1 dev-util/cmake: 2.6.4-r3 sys-apps/baselayout: 1.12.13 sys-apps/sandbox: 1.6-r2 sys-devel/autoconf: 2.13, 2.63-r1 sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.1 sys-devel/binutils: 2.18-r3 sys-devel/gcc: 4.3.4 sys-devel/gcc-config: 1.4.1 sys-devel/libtool: 2.2.6b virtual/os-headers: 2.6.30-r1 ACCEPT_KEYWORDS="x86" ACCEPT_LICENSE="*" CBUILD="i686-pc-linux-gnu" CFLAGS="-march=prescott -O2 -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /service /usr/inferno/keydb /usr/inferno/lib /usr/inferno/services /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config /var/log /var/qmail/alias /var/qmail/control" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-march=prescott -O2 -pipe" DISTDIR="/usr/portage-distfiles" EMERGE_DEFAULT_OPTS="--with-bdeps=y" FEATURES="assume-digests distlocks fixpackages news parallel-fetch protect-owned sandbox sfperms strict unmerge-logs unmerge-orphans userfetch userpriv usersandbox" GENTOO_MIRRORS="http://ftp.lug.ro/gentoo/ http://mirror.qubenet.net/mirror/gentoo/" LANG="ru_RU.UTF-8" LDFLAGS="-Wl,-O1" LINGUAS="en ru" MAKEOPTS="-j3" PKGDIR="/usr/portage-packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_EXTRA_OPTS="--exclude ChangeLog --delete-excluded" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage/layman/powerman /usr/local/portage/layman/sunrise /usr/local/portage/layman/mozilla /usr/local/portage/layman/kde-sunset /usr/local/portage/layman/vmware /usr/local/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="X Xaw3d a52 aac acl acpi aim alsa apache2 asf avi bash-completion berkdb bitmap-fonts bzip2 cddb cdr chm cli cracklib crypt cscope cue curl cxx dbus dga divx4linux djvu dlloader dri dts dvd dvdr dvdread encode fastcgi ffmpeg flac flash gd gdbm gif gnutls gpg gtk gtk2 hardened hddtemp iconv icq idn imagemagick imap imlib irc jabber javascript jpeg kde lm_sensors lzo mad mailbox mbox mmx mng modules motif mp3 mpeg msn mudflap musepack mysql ncurses nls nptl nptlonly ogg opengl openmp oss pam pcre perl pic png pppd pwdb python qt qt3support qt4 quicktime readline reflection rss rtc samba sdl session spell spl sse sse2 ssl ssse3 svg sysfs tcltk tcpd theora tiff truetype truetype-fonts type1-fonts unicode urandom vim-pager vim-syntax vim-with-x vorbis wavpack win32codecs x86 xinetd xorg xv xvid yahoo zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES=" log_config vhost_alias autoindex alias rewrite dir deflate filter mime negotiation auth_basic authn_file authz_host authz_user authz_groupfile cgi actions headers env setenvif " ELIBC="glibc" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en ru" LIRC_DEVICES="serial" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="vesa fbdev nv nvidia" Unset: CPPFLAGS, CTARGET, FFLAGS, INSTALL_MASK, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS
Created attachment 226801 [details] -ose-3.0.12, using VT-x, with PaX&GrSecurity: kernel BUG at mm/mmap.c:1746
Created attachment 226803 [details] -bin-3.0.12, using VT-x, without PaX&GrSecurity: general protection fault
Created attachment 226805 [details] -bin-3.0.12, without VT-x, without PaX&GrSecurity: general protection fault
Created attachment 226807 [details] -bin-3.1.6, using VT-x, without PaX&GrSecurity: general protection fault
Created attachment 226809 [details] -bin-3.1.6, using VT-x, with PaX&GrSecurity: kernel BUG at mm/mmap.c:1746
does this still happen with the latest vbox/PaX kernel combos?
(In reply to comment #6) > does this still happen with the latest vbox/PaX kernel combos? Yeah. I've just tested kernel 2.6.32-hardened-r22 with virtual-box versions -bin-3.1.8, -bin-3.2.10-r1 and -ose-3.1.8: all hangs on starting new virtual machine and in all tests I see same kernel bug in log: <0>------------[ cut here ]------------ kern.crit: kernel BUG at mm/mmap.c:1774! <0>invalid opcode: 0000 [#1] SMP <0>last sysfs file: /sys/devices/pci0000:00/0000:00:1f.2/host4/target4:0:0/4:0:0:0/block/sr1/dev kern.warn: Modules linked in: act_police cls_fw cls_u32 sch_ingress sch_tbf sch_sfq sch_prio sch_cbq sch_htb nvidia(P) vboxnetadp vboxnetflt vboxdrv sky2 8139too skge kern.warn: kern.warn: Pid: 4162, comm: VirtualBox Tainted: P (2.6.32-hardened-r22 #1) System Product Name kern.warn: EIP: 0060:[<c1084c70>] EFLAGS: 00010206 CPU: 0 kern.warn: EAX: 00000000 EBX: 00051000 ECX: 00000000 EDX: ee3f453c kern.warn: ESI: 02080000 EDI: edc39140 EBP: ec4bfc24 ESP: ec4bfc18 kern.warn: DS: 0068 ES: 0068 FS: 00d8 GS: 00e0 SS: 0068 <0>Process VirtualBox (pid: 4162, ti=ec4be000 task=ec598ce0 task.ti=ec4be000) <0>Stack: kern.warn: ec9a6b9c ef85189c 4352d000 ec4bfc68 c108681c 00000000 ec4bfca0 00000246 kern.warn: <0> 00000001 00000041 ffffffff 00000000 0000618f 000fffff 00000000 ef85189c kern.warn: <0> 4350d000 ef8518bc 4350d000 00000020 ec4bfcc4 c108731c 4352d000 000000ff <0>Call Trace: kern.warn: [<c108681c>] ? kern.warn: [<c108731c>] ? kern.warn: [<c1087a9c>] ? kern.warn: [<f83a6d5e>] ? kern.warn: [<f83a78d7>] ? ... kern.warn: [<c1003645>] ? <0>Code: 75 29 5b 89 d0 5e 5f c9 c3 66 90 8b 70 5c 85 f6 75 15 5b 31 d2 5e 89 d0 5f c9 c3 0f 0b eb fe 0f 0b eb fe 0f 0b eb fe 0f 0b eb fe <0f> 0b eb fe 0f 0b eb fe 81 fb 00 00 00 60 76 8a 0f 0b eb fe 0f <0>EIP: [<c1084c70>] SS:ESP 0068:ec4bfc18 kern.warn: ---[ end trace 0eb47bba3f532979 ]--- Also I've just tested on same workstation windows versions of both VirtualBox (VirtualBox-3.2.10-66523-Win.exe) and VMware (VMware-workstation-full-7.1.2-301548.exe). WinXP SP3 32-bit host run both 32-bit and 64-bit guests in VirtualBox and VMware without issues - so this isn't hardware issue with my CPU of something. No sure is this related, but in addition to this issue I've another one with VMware (only when using Hardened Gentoo 32-bit as host os): while it perfectly run 32-bit guests, attempt to run 64-bit guest lead to _host_os_reset_ (guest BIOS pass ok, guest os start booting, and shortly after that, I suppose when guest os switch CPU to 64-bit mode, host reset to BIOS without doing usual linux shutdown procedure). This happens only when VT-x (I've Core2Duo 6600) enabled in BIOS - without it VMware refuse to start 64-bit guests on 32-bit host. I'm now going to test virtualbox with hardened-sources-2.6.35-r5...
I forget to add: virtualbox hangs in all configurations (32-bit and 64-bit guests; with enabled/disabled VT-x in virtualbox's guest config; with enabled/disabled VT-x in BIOS).
(In reply to comment #7) > I'm now going to test virtualbox with hardened-sources-2.6.35-r5... Yeah, in 2.6.35-hardened-r5 I got same kernel bug, only line number changed: mm/mmap.c:1884.
Switching off CONFIG_PAX_MEMORY_UDEREF fixed this bug! Both for VirtualBox and VMware.
yesterday i finally had time to investigate these crashes and here's the summary. there are actually 3 bugs here, not just one. 1. vbox does some funky things with the userland address space from the kernel, such as creating rwx mappings and then populating the page tables from the kernel as well. the bug in PaX was that SEGMEXEC wasn't aware of these rwx mappings and the mirroring logic detected it as an inconsistency. this one is fixed in the latest PaX test patches for .35 and .36, i'll update .32 later today. 2. vbox has its own kernel module loader implementation which is incompatible with KERNEXEC/i386 (possibly amd64 too, i didn't try there) and can't be fixed on the PaX side. 3. vboxdrv (or more likely, some runtime generated code from there) accesses userland directly without using the proper accessors which is then caught by UDEREF. this is a vbox bug (probably a security bug actually since the logged in user must be able to access the underlying device and issue the ioctl that triggers UDEREF), so again i can't fix in PaX.
pageexec would you be so kind and report this upstream? I doubt I understand the whole matter enough to submit the correct information to VBox-upstream.
Please reopen if this is still an issue with latest virtualbox versions in tree.
(In reply to comment #13) > Please reopen if this is still an issue with latest virtualbox versions in > tree. Sorry, I can't test that. Some time ago I've moved to 64-bit, and since that time neither VMware nor VirtualBox work (on hardened) at all - see bug 404155 and bug 382793.
