Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 642268

Summary: gentoo-sources-4.14.9: doesn't boot on hardened profiles
Product: Gentoo Linux Reporter: Stijn Tintel <stijn+gentoo>
Component: Current packagesAssignee: Gentoo Kernel Bug Wranglers and Kernel Maintainers <kernel>
Status: RESOLVED FIXED    
Severity: critical CC: alexander, jaak, jdavid.ibp, nholland, pageexec, phantom4, phils, robbat2, slyfox, tamiko, tka, tsmksubc
Priority: High    
Version: unspecified   
Hardware: AMD64   
OS: Linux   
URL: https://bugzilla.kernel.org/show_bug.cgi?id=198263
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 637152    

Description Stijn Tintel 2017-12-26 09:13:06 UTC
gentoo-sources-4.14.9 doesn't boot on my machines that use hardened profiles (both glibc and musl). See upstream report. Can someone confirm this is indeed hardened related? Maybe it should be masked.

Reproducible: Always

Steps to Reproduce:
1. build and install gentoo-sources-4.14.9 on machine with hardened profile
2. reboot
Comment 1 Alexander Tsoy 2017-12-26 22:04:10 UTC
(In reply to Stijn Tintel from comment #0)
> Can someone confirm this is indeed hardened related?
I don't think so. I have the same problem on an old laptop with Core 2 Duo, but my AMD system and KVM VMs are fine.
Comment 2 Matthias Maier gentoo-dev 2017-12-26 22:46:28 UTC
Neither hardened settings for glibc/musl (the kernel doesn't use the c library), nor default PIE/SSP and FORTIFY_SOURCE have an effect on the kernel build - meaning this is very likely not a hardened problem.
Comment 3 Matthias Maier gentoo-dev 2017-12-26 22:50:16 UTC
Further, we have a couple of more issues [1]

Kernel team, what about masking 4.14.9 for now?

Given the fact that this version has seen quite a heavy changeset for preparatory changes for the KAISER patches (for page table separation) as well, I guess it is simply too experimental at the moment.

[1] https://bugzilla.kernel.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&cf_kernel_version=4.14.9&cf_kernel_version_type=allwordssubstr&order=Importance&query_format=advanced
Comment 4 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2017-12-27 00:05:40 UTC
@stijn: what GCC version did you use?
Comment 5 Alice Ferrazzi Gentoo Infrastructure gentoo-dev 2017-12-27 00:53:13 UTC
Author: Alice Ferrazzi <alicef@gentoo.org>
Date:   Wed Dec 27 00:46:04 2017 +0000

    package.mask: Mask gentoo-sources-4.14.9 due to issue
Comment 6 Stijn Tintel 2017-12-27 06:37:50 UTC
(In reply to Robin Johnson from comment #4)
> @stijn: what GCC version did you use?

6.4.0
Comment 7 Stijn Tintel 2017-12-27 13:29:35 UTC
Starting to suspect this might be a compiler bug in GCC 6.4.0. Since recompiling one of my KVM hosts (the physical host where I run into the problem this bug is about) with GCC 6.4.0, I have random crashes of the KVM guest that runs on it as well. I have configured a serial console that outputs to a file, and the backtrace there reminds me of the one from this problem:

[ 9568.238587] PANIC: double fault, error_code: 0x0
[ 9568.246195] Kernel panic - not syncing: Machine halted.
[ 9568.247047] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.4.0-104-generic #127-Ubuntu
[ 9568.247760] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1.fc27 04/01/2014
[ 9568.248615]  0000000000000086 8e0d95a609959fba ffff88013fc04e80 ffffffff813fb523
[ 9568.249367]  ffffffff81cb9d95 ffff88013fc04f18 ffff88013fc04f08 ffffffff8118e087
[ 9568.250120]  0000000000000008 ffff88013fc04f18 ffff88013fc04eb0 8e0d95a609959fba
[ 9568.250885] Call Trace:
[ 9568.251130]  <#DF>  [<ffffffff813fb523>] dump_stack+0x63/0x90
[ 9568.251711]  [<ffffffff8118e087>] panic+0xd3/0x215
[ 9568.252172]  [<ffffffff81060d5d>] df_debug+0x2d/0x30
[ 9568.252645]  [<ffffffff8102fb8c>] do_double_fault+0x7c/0xf0
[ 9568.253173]  [<ffffffff818463f8>] double_fault+0x28/0x30
[ 9568.253677]  [<ffffffff81064636>] ? native_safe_halt+0x6/0x10
[ 9568.254223]  <<EOE>>  [<ffffffff81038e5e>] default_idle+0x1e/0xe0
[ 9568.254824]  [<ffffffff8103966f>] arch_cpu_idle+0xf/0x20
[ 9568.255334]  [<ffffffff810c478a>] default_idle_call+0x2a/0x40
[ 9568.255886]  [<ffffffff810c4af1>] cpu_startup_entry+0x2f1/0x350
[ 9568.256453]  [<ffffffff818380cc>] rest_init+0x7c/0x80
[ 9568.256956]  [<ffffffff81f5e011>] start_kernel+0x481/0x4a2
[ 9568.257484]  [<ffffffff81f5d120>] ? early_idt_handler_array+0x120/0x120
[ 9568.258131]  [<ffffffff81f5d339>] x86_64_start_reservations+0x2a/0x2c
[ 9568.258744]  [<ffffffff81f5d485>] x86_64_start_kernel+0x14a/0x16d
[ 9568.259460] Kernel Offset: disabled
[ 9568.263295] Rebooting in 5 seconds..
Comment 8 Stijn Tintel 2017-12-27 13:57:07 UTC
To confirm my suspicion, I have reinstalled GCC 5.4.0 on the VM where I ran into this problem, and rebuilt 4.14.9 with it. It now boots fine.
Comment 9 Michael Cook 2017-12-27 18:15:53 UTC
(In reply to Stijn Tintel from comment #8)
> To confirm my suspicion, I have reinstalled GCC 5.4.0 on the VM where I ran
> into this problem, and rebuilt 4.14.9 with it. It now boots fine.

I booted fine compiling with 7.2.0, so maybe whatever bug it is, is fixed in 7.2?
Comment 10 Jaak Ristioja 2017-12-27 18:17:26 UTC
*** Bug 642384 has been marked as a duplicate of this bug. ***
Comment 11 Jaak Ristioja 2017-12-27 18:38:45 UTC
(In reply to Michael Cook from comment #9)
> I booted fine compiling with 7.2.0, so maybe whatever bug it is, is fixed in
> 7.2?

For duplicate bug #642384, I tried compiled with GCC 7.2.0(In reply to Michael Cook from comment #9)
> (In reply to Stijn Tintel from comment #8)
> > To confirm my suspicion, I have reinstalled GCC 5.4.0 on the VM where I ran
> > into this problem, and rebuilt 4.14.9 with it. It now boots fine.
> 
> I booted fine compiling with 7.2.0, so maybe whatever bug it is, is fixed in
> 7.2?

Still fails to boot on my laptop when compiled with GCC 7.2.0 (https://bugs.gentoo.org/642384#c7).
Comment 12 Michael Cook 2017-12-27 18:56:05 UTC
(In reply to Jaak Ristioja from comment #11)
> (In reply to Michael Cook from comment #9)
> > I booted fine compiling with 7.2.0, so maybe whatever bug it is, is fixed in
> > 7.2?
> 
> For duplicate bug #642384, I tried compiled with GCC 7.2.0(In reply to
> Michael Cook from comment #9)
> > (In reply to Stijn Tintel from comment #8)
> > > To confirm my suspicion, I have reinstalled GCC 5.4.0 on the VM where I ran
> > > into this problem, and rebuilt 4.14.9 with it. It now boots fine.
> > 
> > I booted fine compiling with 7.2.0, so maybe whatever bug it is, is fixed in
> > 7.2?
> 
> Still fails to boot on my laptop when compiled with GCC 7.2.0
> (https://bugs.gentoo.org/642384#c7).

I'm also still on 13.0 (so no pie yet) are you on a 17.0 profile yet?
Comment 13 Jaak Ristioja 2017-12-27 19:15:17 UTC
(In reply to Michael Cook from comment #12)
> I'm also still on 13.0 (so no pie yet) are you on a 17.0 profile yet?

According to `eselect profile show` I'm using the hardened/linux/amd64/no-multilib profile.
Comment 14 Alexander Tsoy 2017-12-27 19:25:50 UTC
(In reply to Michael Cook from comment #12)
> I'm also still on 13.0 (so no pie yet) are you on a 17.0 profile yet?

Kernel build system pass -fno-PIE to compiler.
Comment 15 Phil Stracchino (Unix Ronin) 2017-12-27 20:42:23 UTC
FWIW, it's working fine for me on two non-hardened, non-selinux AMD64 systems (dual Intel Xeon E5620) with profile 17.0 and gcc-6.4.0.
Comment 16 Alexander Tsoy 2017-12-27 21:24:13 UTC
OK, I found a temporary workaround. Switching from CONFIG_MCORE2 to CONFIG_GENERIC_CPU fixed this problem for me. Can anybody confirm this?

On my AMD system and KVM VMs I'm using CONFIG_MK8 and CONFIG_GENERIC_CPU accordingly. That's why I don't have this issue on them.
Comment 17 Herbert Wantesh 2017-12-27 22:47:57 UTC
(In reply to Alexander Tsoy from comment #16)
> OK, I found a temporary workaround. Switching from CONFIG_MCORE2 to
> CONFIG_GENERIC_CPU fixed this problem for me. Can anybody confirm this?
> 
> On my AMD system and KVM VMs I'm using CONFIG_MK8 and CONFIG_GENERIC_CPU
> accordingly. That's why I don't have this issue on them.

i can confirm this. the kernel boots with CONFIG_GENERIC_CPU. so the bug seems to be related to CONFIG_MCORE2
Comment 18 Alexander Tsoy 2017-12-29 12:50:20 UTC
https://lkml.org/lkml/2017/12/29/153
Comment 19 Alexander Tsoy 2017-12-30 01:46:40 UTC
It turned out this bug caused by -fstack-check. Linus proposed to add -fno-stack-check to the KBUILD_CFLAGS:
https://lkml.org/lkml/2017/12/29/488
Comment 20 Octavian 2017-12-30 08:33:54 UTC
I presume 4.14.10 should also be masked since it does not have the patch in https://lkml.org/lkml/2017/12/29/488
Comment 21 Alice Ferrazzi Gentoo Infrastructure gentoo-dev 2017-12-30 11:31:10 UTC
someone can try this patch https://lkml.org/lkml/2017/12/29/488 to gentoo-sources-4.14.10 

if it works, I will make a 4.14.10 revision
Comment 22 Alexander Tsoy 2017-12-30 11:33:30 UTC
(In reply to Alice Ferrazzi from comment #21)

Yes, it works.
Comment 23 Alice Ferrazzi Gentoo Infrastructure gentoo-dev 2017-12-30 11:37:30 UTC
@Alexander Tsoy thanks!
Comment 24 Herbert Wantesh 2017-12-30 12:35:08 UTC
(In reply to Alice Ferrazzi from comment #21)
> someone can try this patch https://lkml.org/lkml/2017/12/29/488 to
> gentoo-sources-4.14.10 
> 
> if it works, I will make a 4.14.10 revision

works here too
Comment 25 Alexander Tsoy 2017-12-30 12:41:34 UTC
Please also make this bug block bug 637152
Comment 26 Alice Ferrazzi Gentoo Infrastructure gentoo-dev 2017-12-30 12:53:50 UTC
From 8eb2eb4711f9d696f3cb19fef852f4063220b8d3 Mon Sep 17 00:00:00 2001
From: Alice Ferrazzi <alicef@gentoo.org>
Date: Sat, 30 Dec 2017 12:50:19 +0000
Subject: [PATCH] sys-kernel/gentoo-sources: genpatches-4.14-13 Make sure
 -fstack-check isn't enabled bug #642268
Comment 27 Alice Ferrazzi Gentoo Infrastructure gentoo-dev 2017-12-30 12:55:14 UTC
someone can check 4.14.10-r1 ?
Comment 28 Alice Ferrazzi Gentoo Infrastructure gentoo-dev 2017-12-30 12:56:02 UTC
this is affecting also gentoo-sources-4.14.8-r1 ?
Comment 29 Matthias Maier gentoo-dev 2017-12-30 16:03:31 UTC
(In reply to Alice Ferrazzi from comment #28)
> this is affecting also gentoo-sources-4.14.8-r1 ?

4.14.8-gentoo-r1 works fine. Currenlty checking 4.14.10-r1.
Comment 30 Matthias Maier gentoo-dev 2017-12-30 17:05:22 UTC
(In reply to Alice Ferrazzi from comment #27)
> someone can check 4.14.10-r1 ?

I can confirm that 4.14.10-r1 works as expected.
Comment 31 Alice Ferrazzi Gentoo Infrastructure gentoo-dev 2017-12-31 04:47:38 UTC
thanks masked 4.14.9 and 4.14.10

please move to 4.14.10-r1 or 4.14.8-r1

From 2e8205879c38ddbc2f4033b324672ec014affec5 Mon Sep 17 00:00:00 2001
From: Alice Ferrazzi <alicef@gentoo.org>
Date: Sun, 31 Dec 2017 04:44:29 +0000
Subject: [PATCH] updated package mask for gentoo-sources 4.14.10 and 4.14.9
 doesn't boot on hardened profiles  -fstack-check isn't enabled bug #642268
Comment 32 matt black 2017-12-31 14:44:12 UTC
I just synced and got the 4.14.10-r1 ebuild but no genpatches-4.14-13. Nor do I see it in https://dev.gentoo.org/~mpagano/genpatches/tarballs/

genpatches-4.14.12 isn't there either. Where are you hiding these at?
Comment 33 Thomas Deutschmann (RETIRED) gentoo-dev 2017-12-31 16:05:43 UTC
(In reply to matt black from comment #32)
> I just synced and got the 4.14.10-r1 ebuild but no genpatches-4.14-13. Nor
> do I see it in https://dev.gentoo.org/~mpagano/genpatches/tarballs/
> 
> genpatches-4.14.12 isn't there either. Where are you hiding these at?

Learn to read ebuilds, the files are on our mirrors ;)

SRC_URI=mirror://kernel/linux/kernel/v4.x/linux-4.14.tar.xz mirror://gentoo/genpatches-4.14-13.base.tar.xz mirror://gentoo/genpatches-4.14-13.extras.tar.xz experimental? ( mirror://gentoo/genpatches-4.14-13.experimental.tar.xz )


`emerge --fetchonly '=sys-kernel/gentoo-sources-4.14.10-r1'` will bring you the patches.

You can also grab from git, https://gitweb.gentoo.org/proj/linux-patches.git/