I've just converted my system from x86 to amd64 (Core i7), and one of things which become broken because of this is vmware. When I start any guest my host immediately reset, and after booting I didn't see anything in logs - neither in kernel nor in vmware's logs. I've experimented with different kernels, and here is what I found: - hardened-sources-3.2.2-r1 work ok on x86 - gentoo-sources-3.2.1-r2 work ok on amd64 - no one hardened-sources since 2.6.39-r8 work on amd64 (I didn't tried older versions) Disabling both GRSEC and PAX in hardened kernels doesn't solve this issue, so this bug probably in that part of hardened patches which is active even with disabled GRSEC and PAX config options. I've tested gentoo-sources and hardened-sources with exactly same vmware-modules-264.1, using these 3 patches (required for hardened): https://384739.bugs.gentoo.org/attachment.cgi?id=295017 https://384739.bugs.gentoo.org/attachment.cgi?id=295019 https://384739.bugs.gentoo.org/attachment.cgi?id=295021 I've also tried hardened-sources-3.2.1, both x86 and amd64 - vmware work on x86 and didn't work on amd64. I've tried to keep .config same, but there are a lot of differences anyway (I suppose they all should be related to 32/64-bit). So, here is diff between -gentoo and -hardened on amd64: --- /tmp/config-amd64-gentoo 2012-02-14 20:33:31.579285488 +0200 +++ /tmp/config-amd64-hardened 2012-02-14 20:33:40.383285603 +0200 @@ -179,6 +179,7 @@ CONFIG_X86_L1_CACHE_SHIFT=6 CONFIG_X86_XADD=y CONFIG_X86_WP_WORKS_OK=y +CONFIG_X86_ALIGNMENT_16=y CONFIG_X86_INTEL_USERCOPY=y CONFIG_X86_USE_PPRO_CHECKSUM=y CONFIG_X86_P6_NOP=y @@ -599,7 +600,6 @@ CONFIG_NTFS_FS=y CONFIG_PROC_FS=y CONFIG_PROC_SYSCTL=y -CONFIG_PROC_PAGE_MONITOR=y CONFIG_SYSFS=y CONFIG_TMPFS=y CONFIG_CONFIGFS_FS=y @@ -647,6 +647,7 @@ CONFIG_IO_DELAY_TYPE_NONE=3 CONFIG_IO_DELAY_0X80=y CONFIG_DEFAULT_IO_DELAY_TYPE=0 +CONFIG_TASK_SIZE_MAX_SHIFT=47 CONFIG_SECURITY_DMESG_RESTRICT=y CONFIG_SECURITY=y CONFIG_DEFAULT_SECURITY_DAC=y And here is diff between -hardened x86 and -hardened amd64: --- /tmp/config-x86 2012-02-14 20:31:08.183283609 +0200 +++ /tmp/config-amd64 2012-02-14 20:30:53.192283412 +0200 @@ -1,26 +1,31 @@ -CONFIG_X86_32=y +CONFIG_64BIT=y +CONFIG_X86_64=y CONFIG_X86=y CONFIG_INSTRUCTION_DECODER=y -CONFIG_OUTPUT_FORMAT="elf32-i386" -CONFIG_ARCH_DEFCONFIG="arch/x86/configs/i386_defconfig" +CONFIG_OUTPUT_FORMAT="elf64-x86-64" +CONFIG_ARCH_DEFCONFIG="arch/x86/configs/x86_64_defconfig" CONFIG_GENERIC_CMOS_UPDATE=y CONFIG_CLOCKSOURCE_WATCHDOG=y CONFIG_GENERIC_CLOCKEVENTS=y +CONFIG_ARCH_CLOCKSOURCE_DATA=y CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y CONFIG_LOCKDEP_SUPPORT=y CONFIG_STACKTRACE_SUPPORT=y CONFIG_HAVE_LATENCYTOP_SUPPORT=y CONFIG_MMU=y CONFIG_ZONE_DMA=y +CONFIG_NEED_DMA_MAP_STATE=y CONFIG_NEED_SG_DMA_LENGTH=y CONFIG_GENERIC_ISA_DMA=y CONFIG_GENERIC_IOMAP=y CONFIG_GENERIC_BUG=y +CONFIG_GENERIC_BUG_RELATIVE_POINTERS=y CONFIG_GENERIC_HWEIGHT=y CONFIG_ARCH_MAY_HAVE_PC_FDC=y CONFIG_RWSEM_XCHGADD_ALGORITHM=y CONFIG_ARCH_HAS_CPU_IDLE_WAIT=y CONFIG_GENERIC_CALIBRATE_DELAY=y +CONFIG_GENERIC_TIME_VSYSCALL=y CONFIG_ARCH_HAS_CPU_RELAX=y CONFIG_ARCH_HAS_DEFAULT_IDLE=y CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y @@ -29,13 +34,14 @@ CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK=y CONFIG_ARCH_HIBERNATION_POSSIBLE=y CONFIG_ARCH_SUSPEND_POSSIBLE=y +CONFIG_ZONE_DMA32=y CONFIG_ARCH_POPULATES_NODE_MAP=y +CONFIG_AUDIT_ARCH=y CONFIG_ARCH_SUPPORTS_OPTIMIZED_INLINING=y CONFIG_ARCH_SUPPORTS_DEBUG_PAGEALLOC=y -CONFIG_X86_32_SMP=y +CONFIG_X86_64_SMP=y CONFIG_X86_HT=y -CONFIG_ARCH_HWEIGHT_CFLAGS="-fcall-saved-ecx -fcall-saved-edx" -CONFIG_KTIME_SCALAR=y +CONFIG_ARCH_HWEIGHT_CFLAGS="-fcall-saved-rdi -fcall-saved-rsi -fcall-saved-rdx -fcall-saved-rcx -fcall-saved-r8 -fcall-saved-r9 -fcall-saved-r10 -fcall-saved-r11" CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config" CONFIG_HAVE_IRQ_WORK=y CONFIG_IRQ_WORK=y @@ -131,7 +137,6 @@ CONFIG_HAVE_PERF_EVENTS_NMI=y CONFIG_HAVE_ARCH_JUMP_LABEL=y CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG=y -CONFIG_HAVE_GENERIC_DMA_COHERENT=y CONFIG_SLABINFO=y CONFIG_RT_MUTEXES=y CONFIG_BASE_SMALL=0 @@ -140,9 +145,9 @@ CONFIG_MODULE_FORCE_UNLOAD=y CONFIG_STOP_MACHINE=y CONFIG_BLOCK=y -CONFIG_LBDAF=y CONFIG_BLK_DEV_BSG=y CONFIG_BLK_DEV_THROTTLING=y +CONFIG_BLOCK_COMPAT=y CONFIG_IOSCHED_NOOP=y CONFIG_IOSCHED_DEADLINE=y CONFIG_IOSCHED_CFQ=y @@ -174,26 +179,24 @@ CONFIG_X86_L1_CACHE_SHIFT=6 CONFIG_X86_XADD=y CONFIG_X86_WP_WORKS_OK=y -CONFIG_X86_INVLPG=y -CONFIG_X86_BSWAP=y -CONFIG_X86_POPAD_OK=y CONFIG_X86_ALIGNMENT_16=y CONFIG_X86_INTEL_USERCOPY=y CONFIG_X86_USE_PPRO_CHECKSUM=y +CONFIG_X86_P6_NOP=y CONFIG_X86_TSC=y CONFIG_X86_CMPXCHG64=y CONFIG_X86_CMOV=y -CONFIG_X86_MINIMUM_CPU_FAMILY=5 +CONFIG_X86_MINIMUM_CPU_FAMILY=64 CONFIG_X86_DEBUGCTLMSR=y CONFIG_CPU_SUP_INTEL=y -CONFIG_CPU_SUP_CYRIX_32=y CONFIG_CPU_SUP_AMD=y CONFIG_CPU_SUP_CENTAUR=y -CONFIG_CPU_SUP_TRANSMETA_32=y -CONFIG_CPU_SUP_UMC_32=y CONFIG_HPET_TIMER=y CONFIG_HPET_EMULATE_RTC=y CONFIG_DMI=y +CONFIG_GART_IOMMU=y +CONFIG_SWIOTLB=y +CONFIG_IOMMU_HELPER=y CONFIG_NR_CPUS=8 CONFIG_SCHED_MC=y CONFIG_PREEMPT_VOLUNTARY=y @@ -201,27 +204,25 @@ CONFIG_X86_IO_APIC=y CONFIG_X86_MCE=y CONFIG_X86_MCE_INTEL=y -CONFIG_X86_MCE_AMD=y CONFIG_X86_MCE_THRESHOLD=y CONFIG_X86_THERMAL_VECTOR=y -CONFIG_VM86=y CONFIG_X86_MSR=y CONFIG_X86_CPUID=y -CONFIG_HIGHMEM64G=y -CONFIG_PAGE_OFFSET=0xC0000000 -CONFIG_HIGHMEM=y -CONFIG_X86_PAE=y CONFIG_ARCH_PHYS_ADDR_T_64BIT=y CONFIG_ARCH_DMA_ADDR_T_64BIT=y -CONFIG_ARCH_FLATMEM_ENABLE=y +CONFIG_DIRECT_GBPAGES=y CONFIG_ARCH_SPARSEMEM_ENABLE=y +CONFIG_ARCH_SPARSEMEM_DEFAULT=y CONFIG_ARCH_SELECT_MEMORY_MODEL=y -CONFIG_ILLEGAL_POINTER_VALUE=0 +CONFIG_ILLEGAL_POINTER_VALUE=0xdead000000000000 CONFIG_SELECT_MEMORY_MODEL=y -CONFIG_FLATMEM_MANUAL=y -CONFIG_FLATMEM=y -CONFIG_FLAT_NODE_MEM_MAP=y -CONFIG_SPARSEMEM_STATIC=y +CONFIG_SPARSEMEM_MANUAL=y +CONFIG_SPARSEMEM=y +CONFIG_HAVE_MEMORY_PRESENT=y +CONFIG_SPARSEMEM_EXTREME=y +CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y +CONFIG_SPARSEMEM_ALLOC_MEM_MAP_TOGETHER=y +CONFIG_SPARSEMEM_VMEMMAP=y CONFIG_HAVE_MEMBLOCK=y CONFIG_PAGEFLAGS_EXTENDED=y CONFIG_SPLIT_PTLOCK_CPUS=4 @@ -247,7 +248,7 @@ CONFIG_HZ=1000 CONFIG_SCHED_HRTICK=y CONFIG_PHYSICAL_START=0x1000000 -CONFIG_PHYSICAL_ALIGN=0x400000 +CONFIG_PHYSICAL_ALIGN=0x1000000 CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y CONFIG_PM_RUNTIME=y CONFIG_PM=y @@ -266,8 +267,6 @@ CONFIG_CPU_IDLE_GOV_LADDER=y CONFIG_INTEL_IDLE=y CONFIG_PCI=y -CONFIG_PCI_GOANY=y -CONFIG_PCI_BIOS=y CONFIG_PCI_DIRECT=y CONFIG_PCI_MMCONFIG=y CONFIG_PCI_DOMAINS=y @@ -282,8 +281,12 @@ CONFIG_ISA_DMA_API=y CONFIG_AMD_NB=y CONFIG_BINFMT_ELF=y -CONFIG_HAVE_AOUT=y -CONFIG_HAVE_ATOMIC_IOMAP=y +CONFIG_COMPAT_BINFMT_ELF=y +CONFIG_IA32_EMULATION=y +CONFIG_IA32_AOUT=y +CONFIG_COMPAT=y +CONFIG_COMPAT_FOR_U64_ALIGNMENT=y +CONFIG_SYSVIPC_COMPAT=y CONFIG_HAVE_TEXT_POKE_SMP=y CONFIG_NET=y CONFIG_PACKET=y @@ -351,6 +354,7 @@ CONFIG_RPS=y CONFIG_RFS_ACCEL=y CONFIG_XPS=y +CONFIG_HAVE_BPF_JIT=y CONFIG_FIB_RULES=y CONFIG_NET_9P=y CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug" @@ -547,7 +551,6 @@ CONFIG_USB_STORAGE=y CONFIG_USB_UAS=y CONFIG_EDAC=y -CONFIG_EDAC_DECODE_MCE=y CONFIG_EDAC_MM_EDAC=y CONFIG_RTC_LIB=y CONFIG_RTC_CLASS=y @@ -559,7 +562,6 @@ CONFIG_RTC_INTF_DEV_UIE_EMUL=y CONFIG_RTC_DRV_CMOS=y CONFIG_DMADEVICES=y -CONFIG_CLKSRC_I8253=y CONFIG_CLKEVT_I8253=y CONFIG_I8253_LOCK=y CONFIG_CLKBLD_I8253=y @@ -638,7 +640,6 @@ CONFIG_STRICT_DEVMEM=y CONFIG_X86_VERBOSE_BOOTUP=y CONFIG_EARLY_PRINTK=y -CONFIG_DOUBLEFAULT=y CONFIG_HAVE_MMIOTRACE_SUPPORT=y CONFIG_IO_DELAY_TYPE_0X80=0 CONFIG_IO_DELAY_TYPE_0XED=1 @@ -646,7 +647,7 @@ CONFIG_IO_DELAY_TYPE_NONE=3 CONFIG_IO_DELAY_0X80=y CONFIG_DEFAULT_IO_DELAY_TYPE=0 -CONFIG_PAX_ENABLE_PAE=y +CONFIG_TASK_SIZE_MAX_SHIFT=47 CONFIG_SECURITY_DMESG_RESTRICT=y CONFIG_SECURITY=y CONFIG_DEFAULT_SECURITY_DAC=y @@ -687,7 +688,6 @@ CONFIG_CRC_ITU_T=y CONFIG_CRC32=y CONFIG_LIBCRC32C=y -CONFIG_AUDIT_GENERIC=y CONFIG_ZLIB_INFLATE=y CONFIG_ZLIB_DEFLATE=y CONFIG_HAS_IOMEM=y Maybe this is same as bug 382793. I've just tried virtualbox-bin-4.1.8 on 3.2.2-hardened-r1 (with enabled GRSEC and PAX) - it doesn't reset host, but refused to run as non-root, and even as root it didn't work anyway: when I try to start new just created guest it says 'some error happens, see logs' and do nothing. And it logs are huge and I can't find actual error message. Portage 2.1.10.44 (hardened/linux/amd64, gcc-4.5.3, glibc-2.13-r4, 3.2.2-hardened-r1 x86_64) ================================================================= System uname: Linux-3.2.2-hardened-r1-x86_64-Intel-R-_Core-TM-_i7-2600K_CPU_@_3.40GHz-with-gentoo-2.0.3 Timestamp of tree: Wed, 15 Feb 2012 15:15:01 +0000 app-shells/bash: 4.1_p9 dev-java/java-config: 2.1.11-r3 dev-lang/python: 2.7.2-r3, 3.1.4-r3 dev-util/cmake: 2.8.6-r4 dev-util/pkgconfig: 0.26 sys-apps/baselayout: 2.0.3 sys-apps/openrc: 0.9.8.4 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.68 sys-devel/automake: 1.11.1 sys-devel/binutils: 2.21.1-r1 sys-devel/gcc: 4.5.3-r1 sys-devel/gcc-config: 1.4.1-r1 sys-devel/libtool: 2.4-r1 sys-devel/make: 3.82-r1 sys-kernel/linux-headers: 3.1 (virtual/os-headers) sys-libs/glibc: 2.13-r4 Repositories: gentoo perl-experimental vmware powerman local ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /opt/upsmon-usb/EXT/DownOS /opt/upsmon-usb/EXT/JSystem /service /usr/inferno/keydb /usr/inferno/lib /usr/inferno/services /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/openvpn/easy-rsa /var/log /var/qmail/alias /var/qmail/control" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-march=native -O2 -pipe" DISTDIR="/usr/portage-distfiles" EMERGE_DEFAULT_OPTS="--with-bdeps=y" FEATURES="assume-digests binpkg-logs distlocks ebuild-locks fixlafiles news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync webrsync-gpg" FFLAGS="-march=native -O2 -pipe" GENTOO_MIRRORS="http://portage.org.ua/ http://gentoo.iteam.net.ua/ http://mirror.mdfnet.se/gentoo http://gentoo.mneisen.org/ http://gentoo.wheel.sk/" LANG="ru_RU.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LINGUAS="en ru" MAKEOPTS="-j9" PKGDIR="/usr/portage-packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_EXTRA_OPTS="--exclude ChangeLog --delete-excluded" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/var/lib/layman/perl-experimental /var/lib/layman/vmware /var/lib/layman/powerman /usr/local/portage" SYNC="rsync://rsync4.ua.gentoo.org/gentoo-portage" USE="X a52 aac acl alac alsa amd64 avx bash-completion berkdb bzip2 cdda cddb cli cracklib crypt cxx dbus dri dts dvd flac gdbm gif gnutls gpg gpm hardened iconv id3tag idn jpeg jpeg2k justify libnotify mac mad matroska mbox mmx mng modules mp3 mpeg mudflap multilib musepack mysql ncurses network-cron nls nptl nptlonly nsplugin ogg opengl openmp pam pax_kernel pcre perl png pppd qt3support readline session spell sse sse2 sse3 sse4_1 sse4_2 ssl ssse3 svg sysfs tcpd theora tiff truetype unicode urandom vdpau vim-syntax vorbis wavpack x264 xorg xosd xv xvid xvmc zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="log_config vhost_alias autoindex alias rewrite dir deflate filter mime negotiation auth_basic authn_file authz_host authz_user authz_groupfile cgi actions headers env setenvif" CALLIGRA_FEATURES="kexi words flow plan stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en ru" PHP_TARGETS="php5-3" RUBY_TARGETS="ree18 ruby18" USERLAND="GNU" VIDEO_CARDS="nvidia nv nouveau" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS
> - no one hardened-sources since 2.6.39-r8 work on amd64 (I didn't tried > older versions) Since I got busy with other bugs, I didn't really keep track of vmware or virtualbox on hardened sources since about 2.6.39-r8. BTW, try to keep the bugs separate, ie open one for virtualbox and another for vmware. Let's keep this one for vwmare. I know they're probably related, but they may not be. And yes, I think this is closely related to, or maybe a dup of bug #382793. For upstream, hardened-sources-2.6.39-r8 is based on grsecurity-2.2.2-2.6.39.3-201107191826
hardened-sources-3.2.11 - vmware still doesn't work.
Any chance vmware/virtualbox will be supported on hardened amd64 ever again?
(In reply to comment #3) > Any chance vmware/virtualbox will be supported on hardened amd64 ever again? i'm personnaly only interested in configs where PaX is enabled and virtualbox will probably never be compatible with such setups, so that leaves vmware but it's still low on my todo list.
this is a dup of the bug #382793
*** This bug has been marked as a duplicate of bug 382793 ***
