When booting a hardened kernel (also with GRKERNSEC off) on a Windows Server 2012 Core Hyper-V (6.2.9200.16384) with hyperv drivers compiled in, the CPU stalls and the boot fails. A non-hardened kernel boots perfectly. Hyperv drivers are needed for better interoperability, performance and in our case for VLAN tagging support in the guest OS. With acpi=off the hardened kernel boots up, however hyperv drivers are not loaded. (config HYPERV depends on X86 && ACPI && PCI && X86_LOCAL_APIC) Affected: - hardened-sources 3.2.46-r1 - hardened-sources 3.8.6 - hardened-sources 3.9.5 Other sources: - "18. Emerge a kernel source; You need gentoo-sources for the hyper-V kernel drivers to load correctly. NB: hardened-sources cause your VPS hang when loading hv_vmbus driver. emerge gentoo-sources" Source: http://glennaitchison.blogspot.hu/2013/01/howto-install-gentoo-121-on-lvm2-root.html - http://bugs.alpinelinux.org/issues/1021 Steps taken: - Tried on Hyper-V server: Disabling processor C sleep states in BIOS - Tried on Hyper-V server: Switching Windows power profile to 'high performance' on Hyper-V server - Tired various Hyper-V settings, no luck - Can boot with acpi=off, however hyper-v drivers are not loaded (config HYPERV depends on X86 && ACPI && PCI && X86_LOCAL_APIC) - If "acpi=off" allows the system to boot, try to isolate the ACPI issue with the following boot parameters Try booting with "acpi=ht" This disables all of ACPI except just enough to enable Hyper Threading. If acpi=off works and acpi=ht fails, then the issue is in the ACPI table parsing code itself, or perhaps the SMP code. - Booting with "acpi=ht" fails, cpu stalls - Tried compiling with SMP off, boot process stalls - Tried reverting hardened changes to hyper-v drivers manually, boot process stalls I attach a working non-hardened config, a hardened config causing the cpu stall, a diff and the error messages with acpi=on and acpi=off.
Created attachment 351418 [details] Hardened kernel boot fails
Created attachment 351420 [details] Error when booting hardened kernel with acpi=off
Created attachment 351422 [details] Working gentoo-sources
Created attachment 351424 [details] Failing hardened-sources
Created attachment 351426 [details, diff] Config diff
Created attachment 351428 [details] Emerge info
in drivers/hv/hv.c:hv_init can you replace virtaddr = __vmalloc(PAGE_SIZE, GFP_KERNEL, PAGE_KERNEL_EXEC); with virtaddr = module_alloc_exec(PAGE_SIZE); and see if hv works better?
CC drivers/hv/hv.o drivers/hv/hv.c: In function 'hv_init': drivers/hv/hv.c:179:2: error: implicit declaration of function 'module_alloc_exec' [-Werror=implicit-function-declaration] drivers/hv/hv.c:179:11: warning: assignment makes pointer from integer without a cast [enabled by default] cc1: some warnings being treated as errors make[2]: *** [drivers/hv/hv.o] Error 1 make[1]: *** [drivers/hv] Error 2 make: *** [drivers] Error 2
add #include <linux/moduleloader.h>
Added. LD vmlinux.o MODPOST vmlinux.o WARNING: modpost: Found 844 section mismatch(es). To see full details build your kernel with: 'make CONFIG_DEBUG_SECTION_MISMATCH=y' GEN .version CHK include/generated/compile.h UPD include/generated/compile.h CC init/version.o LD init/built-in.o LD .tmp_vmlinux1 drivers/built-in.o: In function `hv_init': (.text+0x16875c): undefined reference to `module_alloc_exec' make: *** [.tmp_vmlinux1] Error 1 # # ls -la arch/x86/kernel/module* -rw-r--r-- 1 root root 7179 Jun 19 09:41 arch/x86/kernel/module.c -rw-r--r-- 1 root root 0 Jun 20 18:46 arch/x86/kernel/modules.order #
Also, our kernel has: # CONFIG_MODULES is not set
(In reply to Lorand Kelemen from comment #11) > Also, our kernel has: > # CONFIG_MODULES is not set yeah, that'll be a problem, can you enable modules for now? there's no need to make any actual modules, it's just for making this particular function available and i'll figure out something for the real fix if this works.
Successful boot with CONFIG_MODULES set! I used hardened-sources-3.2.46-r1, the hyperv network driver does not work yet, but that will be another story. Will test more...
Just for information: tested gentoo-sources-3.2.46 without modifications with the same config, hyperv network driver does not work. I guess changes were not backported to hyperv drivers to 3.2 LTS.
Successful boot with hardened-source-3.9.5 applying the hv.c modification and enabling MODULES. Hyperv drivers seem to work.
Hyper-v LIS features (live migration, shutdown) seem to work with the modified 3.9.5 hardened kernel. Only need to test VLAN tagging. I will wait for the final patch to re-test, a really big thank you for the quick help!
this should be now fixed in the latest PaX patches, can you test them (or the next grsec that should be out soon)?
Of course, I will wait for the next grsec release, test and report back!
Strange. I use the config from Comment 15 (only MODULES is not set), with grsecurity-2.9.1-3.9.7-201306220939 and I get the error below. With gentoo-sources-3.9.7 the same config compiles fine. RELOCS arch/x86/realmode/rm/realmode.relocs OBJCOPY arch/x86/realmode/rm/realmode.bin AS arch/x86/realmode/rmpiggy.o LD arch/x86/realmode/built-in.o CC arch/x86/vdso/vma.o LDS arch/x86/vdso/vdso.lds AS arch/x86/vdso/vdso-note.o CC arch/x86/vdso/vclock_gettime.o CC arch/x86/vdso/vgetcpu.o VDSO arch/x86/vdso/vdso.so.dbg OBJCOPY arch/x86/vdso/vdso.so AS arch/x86/vdso/vdso.o VDSOSYM arch/x86/vdso/vdso-syms.lds LD arch/x86/vdso/built-in.o LD arch/x86/built-in.o CC kernel/fork.o CC kernel/exec_domain.o CC kernel/panic.o CC kernel/printk.o kernel/printk.c: In function 'devkmsg_open': kernel/printk.c:678:53: error: 'SYSLOG_FROM_FILE' undeclared (first use in this function) kernel/printk.c:678:53: note: each undeclared identifier is reported only once for each function it appears in make[1]: *** [kernel/printk.o] Error 1 make: *** [kernel] Error 2 #
Created attachment 351730 [details] Failing grsec-3.9.7 sources
linux-3.9.5-hardened # grep -R 'SYSLOG_FROM_FILE' * fs/proc/kmsg.c: return do_syslog(SYSLOG_ACTION_OPEN, NULL, 0, SYSLOG_FROM_FILE); fs/proc/kmsg.c: (void) do_syslog(SYSLOG_ACTION_CLOSE, NULL, 0, SYSLOG_FROM_FILE); fs/proc/kmsg.c: !do_syslog(SYSLOG_ACTION_SIZE_UNREAD, NULL, 0, SYSLOG_FROM_FILE)) fs/proc/kmsg.c: return do_syslog(SYSLOG_ACTION_READ, buf, count, SYSLOG_FROM_FILE); fs/proc/kmsg.c: if (do_syslog(SYSLOG_ACTION_SIZE_UNREAD, NULL, 0, SYSLOG_FROM_FILE)) include/linux/syslog.h:#define SYSLOG_FROM_FILE 1 kernel/printk.c: err = check_syslog_permissions(SYSLOG_ACTION_OPEN, SYSLOG_FROM_FILE); linux-3.9.5-hardened # linux-3.9.7-gentoo # grep -R 'SYSLOG_FROM_FILE' * linux-3.9.7-gentoo # linux-3.9.7-hardened # grep -R 'SYSLOG_FROM_FILE' * kernel/printk.c: err = check_syslog_permissions(SYSLOG_ACTION_OPEN, SYSLOG_FROM_FILE); linux-3.9.7-hardened #
In kernel/printk.c:devkmsg_open - err = check_syslog_permissions(SYSLOG_ACTION_OPEN, SYSLOG_FROM_FILE); + err = check_syslog_permissions(SYSLOG_ACTION_OPEN, SYSLOG_FROM_PROC); seems to solve it, but the modification is based on logic, not knowledge :) Next error: CC mm/memory.o CC mm/mincore.o CC mm/mlock.o CC mm/mmap.o CC mm/mprotect.o CC mm/mremap.o CC mm/msync.o CC mm/rmap.o CC mm/vmalloc.o mm/vmalloc.c: In function '__get_vm_area_node': mm/vmalloc.c:1369:14: error: 'VM_KERNEXEC' undeclared (first use in this function) mm/vmalloc.c:1369:14: note: each undeclared identifier is reported only once for each function it appears in mm/vmalloc.c: In function 'vmap': mm/vmalloc.c:1620:12: error: 'VM_KERNEXEC' undeclared (first use in this function) mm/vmalloc.c: In function '__vmalloc_node_range': mm/vmalloc.c:1726:65: error: 'VM_KERNEXEC' undeclared (first use in this function) make[1]: *** [mm/vmalloc.o] Error 1 make: *** [mm] Error 2 linux-3.9.7-hardened #
there's a new grsec patch out, it should fix the compile errors.
Indeed. Updated own overlay with 4420_grsecurity-2.9.1-3.9.7-201306231443.patch, kernel compiled and booted fine. Thank you!
