I know that in the past this was moved back to testing due to some bugs... but I think that those bugs were fixed, right? In my case I am running 4.14.x in all my computers without issue for a long time Thanks
Same here,100+ machines on 4.14.
opening a stabilization bug
Alice, could you please let us know (when you know) what is the targetted 4.14.x version? I'm currently running 4.14.14 on all of my configurations, but would be happy to test whichever version is chosen, prior to stabilization.
I would like to go with most recent one, if there are no objections and no problems.
I've been on 4.14.13 for a long time without any problems. Current run is few hours shy of 60d.
I agree with Alice,we should stabilize the latest. Hundreds of patches appeared after 4.14.14.
Agreed, but 'latest' is a moving target. For now, I have tested 4.14.27 on armv7a, aarch64, x86, and several amd64 and xen machines without issues here.
(In reply to David Flogeras from comment #7) > Agreed, but 'latest' is a moving target. Well of course, this is how it works, the kernel changes all the time :) I was just trying to say that we should not stick to 4.14.13 or something like that but to get a newer one stabilized.
I have gentoo-sources 4.14.11-r2 running in my sytem from 08 jan 2018 without issues.
Now that 644942 is closed and the resolution is to use >=4.14.31, I'd like to chime in that I hit a bug on my Xen machines that can crash the networking stack, and it appears to have been fixed in 4.14.32, so I'd like to propose at least that version. The bug I mention was the NULL pointer deref mentioned in this commit https://www.systutorials.com/linux-kernels/334880/tcp-reset-sk_send_head-in-tcp_write_queue_purge-linux-4-14-32/
(In reply to David Flogeras from comment #10) > Now that 644942 is closed and the resolution is to use >=4.14.31, I'd like > to chime in that I hit a bug on my Xen machines that can crash the > networking stack, and it appears to have been fixed in 4.14.32, so I'd like > to propose at least that version. It's not a general xen issue as we have been using the 4.14 series since 4.14.0 without issues, but I can vote for 4.14.32 as it's working fine.
No, we still cannot recommend kernel 4.14.x. Gentoo infra for example hit > [ 26.300254] BUG: unable to handle kernel NULL pointer dereference at (null) > [ 26.316952] IP: (null) > [ 26.320531] PGD 0 P4D 0 > [ fmt_misc bridge stp llc vport_geneve geneve vport_gre ip_gre gre ip_tunnel vport_vxlan openvswitch nf_conntrack_ipv6 nf_nat_ipv6 nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_defrag_ipv6 nf_nat nf_conntrack tun veth drbd lru_cache configs sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel iTCO_wdt iTCO_vendor_support kvm irqbypass crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel intel_cstate mei_me ses intel_rapl_perf evdev input_leds i2c_i801 lpc_ich mei enclosure ioatdma wmi button ipmi_ssif acpi_pad acpi_power_meter aesni_intel crypto_simd glue_helper cryptd aes_x86_64 crc32_generic ixgb ixgbe dca samsung_sxgbe tulip cxgb3 cxgb mdio cxgb4 vxge bonding vxlan ip6_udp_tunnel udp_tunnel macvlan vmxnet3 qemu_fw_cfg msdos fat configfs overlay > [ 26.401501] fuse reiserfs btrfs zstd_compress multipath dm_zero dm_verity reed_solomon dm_thin_pool dm_switch dm_snapshot dm_log_writes dm_log_userspace cn dm_integrity dm_flakey dm_era dm_delay dm_crypt dm_cache_smq dm_cache dm_persistent_data dm_bufio dm_bio_prison uas usb_storage sr_mod cdrom sg pata_serverworks virtio_crypto crypto_engine virtio_rng rng_core > [ 26.435126] CPU: 12 PID: 7053 Comm: ovs-vswitchd Not tainted 4.14.34-gentoo-infra38 #2 > [ 26.443674] Hardware name: Supermicro SYS-2028TP-DECTR/X10DRT-PT, BIOS 2.0 12/18/2015 > [ 26.452115] task: ffff9b46ba90a140 task.stack: ffffb11e898bc000 > [ 26.458371] RIP: 0010: (null) > [ 26.462457] RSP: 0018:ffffb11e898bf698 EFLAGS: 00010286 > [ 26.468014] RAX: ffffffffbdecf800 RBX: ffff9b46ba979e00 RCX: 00000000000005aa > [ 26.475491] RDX: ffff9b46bad2b600 RSI: 0000000000000000 RDI: ffff9b46bae84600 > [ 26.482960] RBP: ffffb11e898bf790 R08: 0000000000000006 R09: 0000000000000000 > [ 26.490431] R10: 0000000000000002 R11: 0000000000000001 R12: ffff9b46ba979e00 > [ 26.497902] R13: ffff9b46bae846a8 R14: ffff9b46bad2b600 R15: ffff9b36ae5f8000 > [ 26.505374] FS: 00007fe71efe1400(0000) GS:ffff9b46bf300000(0000) knlGS:0000000000000000 > [ 26.514074] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 26.520155] CR2: 0000000000000000 CR3: 000000201d6a6001 CR4: 00000000003606e0 > [ 26.527631] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 > [ 26.535101] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 > [ 26.542572] Call Trace: > [ 26.545359] ? vxlan_xmit_one+0x6f5/0xa70 [vxlan] > [ 26.550404] ? validate_and_copy_set_tun+0x104/0x360 [openvswitch] > [ 26.556922] vxlan_xmit+0x5fe/0xe30 [vxlan] > [ 26.561441] ? vxlan_xmit+0x5fe/0xe30 [vxlan] > [ 26.566137] dev_hard_start_xmit+0xa2/0x1f0 > [ 26.570655] ? dev_hard_start_xmit+0xa2/0x1f0 > [ 26.575354] __dev_queue_xmit+0x720/0x7a0 > [ 26.579701] dev_queue_xmit+0x10/0x20 > [ 26.583697] ? dev_queue_xmit+0x10/0x20 > [ 26.587877] ovs_vport_send+0xc4/0x150 [openvswitch] > [ 26.593182] do_output+0x57/0x160 [openvswitch] > [ 26.598052] do_execute_actions+0x1373/0x14e0 [openvswitch] > [ 26.603966] ? __kmalloc+0x226/0x290 > [ 26.607876] ovs_execute_actions+0x4f/0x130 [openvswitch] > [ 26.613609] ? ovs_execute_actions+0x4f/0x130 [openvswitch] > [ 26.619520] ovs_packet_cmd_execute+0x1d8/0x240 [openvswitch] > [ 26.625609] genl_family_rcv_msg+0x1c1/0x380 > [ 26.630213] genl_rcv_msg+0x4c/0x90 > [ 26.634042] ? genl_family_rcv_msg+0x380/0x380 > [ 26.638820] netlink_rcv_skb+0xd7/0x110 > [ 26.642995] genl_rcv+0x28/0x40 > [ 26.646469] netlink_unicast+0x16c/0x1e0 > [ 26.650727] netlink_sendmsg+0x2f9/0x3a0 > [ 26.654992] sock_sendmsg+0x38/0x50 > [ 26.658821] ___sys_sendmsg+0x2f8/0x310 > [ 26.662989] ? ___cache_free+0x116/0x300 > [ 26.667248] ? destroy_inode+0x3e/0x60 > [ 26.671335] ? evict+0x131/0x1b0 > [ 26.674898] ? dentry_free+0x47/0x70 > [ 26.678814] ? __dentry_kill+0x10a/0x170 > [ 26.683072] ? dput+0x220/0x230 > [ 26.686552] __sys_sendmsg+0x45/0x80 > [ 26.690463] ? __sys_sendmsg+0x45/0x80 > [ 26.694546] SyS_sendmsg+0x12/0x20 > [ 26.698291] do_syscall_64+0x6d/0x130 > [ 26.702298] entry_SYSCALL_64_after_hwframe+0x3d/0xa2 > [ 26.707690] RIP: 0033:0x7fe71e2c47bb > [ 26.711601] RSP: 002b:00007ffce02fe0a0 EFLAGS: 00000293 ORIG_RAX: 000000000000002e > [ 26.719783] RAX: ffffffffffffffda RBX: 00007ffce02ff0d0 RCX: 00007fe71e2c47bb > [ 26.727252] RDX: 0000000000000000 RSI: 00007ffce02fe130 RDI: 0000000000000018 > [ 26.734723] RBP: 00007ffce02fedf0 R08: 0000000000000000 R09: 00007ffce02ff128 > [ 26.742192] R10: 0000000000000000 R11: 0000000000000293 R12: 00007ffce02fef40 > [ 26.749663] R13: 000055de1be47be0 R14: 000055de1be47be0 R15: 00007ffce02fe130 > [ 26.757135] Code: Bad RIP value. > [ 26.760792] RIP: (null) RSP: ffffb11e898bf698 > [ 26.766352] CR2: 0000000000000000 > [ 26.770018] ---[ end trace afa92eb4f357b936 ]--- > [ 26.777155] Kernel panic - not syncing: Fatal exception in interrupt > [ 26.936988] Kernel Offset: 0x3b000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) > [ 26.950861] Rebooting in 180 seconds..
(In reply to Thomas Deutschmann from comment #12) > No, we still cannot recommend kernel 4.14.x. Gentoo infra for example hit Any steps to reproduce that? I'm using different 4.14.x on different machines (both x86 and amd64) without any issues.
This is probably already fixed in newer kernels by https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=f15ca723c1, we are awaiting backport to 4.14. It is probably the same like https://github.com/coreos/bugs/issues/2382
(In reply to Thomas Deutschmann from comment #14) > This is probably already fixed in newer kernels by > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/ > ?id=f15ca723c1, we are awaiting backport to 4.14. It is probably the same > like https://github.com/coreos/bugs/issues/2382 Should be there now: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?h=v4.14.40&id=6a3c946b205fd5bc3be583a1f3adbda11128e7ed
we could target latest 4.14.x then, right?
(In reply to Thomas Deutschmann from comment #12) > No, we still cannot recommend kernel 4.14.x. Gentoo infra for example hit > Hi, Thomas, Has infra updated to later 4.14.X kernels? Any outstanding issues you can report ?
*** Bug 659208 has been marked as a duplicate of this bug. ***
Added for bug 659208 - Duplicate. Make sure whatever kernel we all agree on meets these requirements too: 1. Mitigates Variant 4 of Spectre - See https://www.us-cert.gov/ncas/alerts/TA18-141A https://access.redhat.com/security/vulnerabilities/ssbd https://www.suse.com/support/kb/doc/?id=7022937&add&title=Security+Vulnerability%3A+Spectre+Variant+4+%28Speculative+Store+Bypass%29+aka+CVE-2018-3639. 2. #journalctl -b Jun 26 00:55:40 hostname kernel: cgroup: cgroup2: unknown option "nsdelegate" ============================================== Note that in my bug, I updated the title, as condition 1 is met with a kernel >=4.13.xx, but condition 2 is met with >=4.15.xx(at least according to Ubuntu's version at https://people.canonical.com/~ubuntu-security/cve/2018/CVE-2018-3639.html along with upgrading affected applications)
(In reply to Carter Young from comment #19) > Added for bug 659208 - Duplicate. Make sure whatever kernel we all agree on > meets these requirements too: > > 1. Mitigates Variant 4 of Spectre - See > https://www.us-cert.gov/ncas/alerts/TA18-141A > https://access.redhat.com/security/vulnerabilities/ssbd > https://www.suse.com/support/kb/doc/ > ?id=7022937&add&title=Security+Vulnerability%3A+Spectre+Variant+4+%28Speculat > ive+Store+Bypass%29+aka+CVE-2018-3639. > > 2. #journalctl -b > Jun 26 00:55:40 hostname kernel: cgroup: cgroup2: unknown option "nsdelegate" > > ============================================== > > Note that in my bug, I updated the title, as condition 1 is met with a > kernel >=4.13.xx, but condition 2 is met with >=4.15.xx(at least according > to Ubuntu's version at > https://people.canonical.com/~ubuntu-security/cve/2018/CVE-2018-3639.html > along with upgrading affected applications) Sorry, I got that backwards, Condition 1 is >=4.15.xx and app updates according to ubuntu, and condition 1 is >=4.13.xx. If Stabilizing 4.14.xx meets the minimum for condition 2, I'm good with that.
(In reply to Carter Young from comment #20) > Sorry, I got that backwards, Condition 1 is >=4.15.xx and app updates > according to ubuntu, and condition 1 is >=4.13.xx. If Stabilizing 4.14.xx > meets the minimum for condition 2, I'm good with that. That should read: If Stabilizing 4.14.xx meets the minimum for condition 1, I'm good with that. Sorry for the bug spam. I'm too busy thinking about the trivial "nsdelegate" warning I initially reported, so that came out first, but I wrote them in order of importance(Spectre/Meltdown trumps a warning), but it still came out backwards.
Remember that 4.14.x is a LTS release upstream. So it makes sense to stabilize it, even if it does not provide all of the recent developments upstream. If you are in need of a specific feature, you can always unmask one of the more recent kernel versions. But it would really be nice to finally have a stable 4.14.x release.
Carter, for the systemd bug from #659208 kernel 4.14 is fine. And regarding spectre, this is the output from a 4.14 machine: CVE-2018-3639 [speculative store bypass] aka 'Variant 4' * Mitigated according to the /sys interface: YES (Mitigation: Speculative Store Bypass disabled via prctl and seccomp) * Kernel supports speculation store bypass: YES (found in /proc/self/status) > STATUS: NOT VULNERABLE (Mitigation: Speculative Store Bypass disabled via prctl and seccomp) The spectre-meltdown checker shows a machine not vulnerable with kernel 4.14 and updated microcode.
(In reply to Tomáš Mózes from comment #23) > Carter, for the systemd bug from #659208 kernel 4.14 is fine. And regarding > spectre, this is the output from a 4.14 machine: > > CVE-2018-3639 [speculative store bypass] aka 'Variant 4' > * Mitigated according to the /sys interface: YES (Mitigation: Speculative > Store Bypass disabled via prctl and seccomp) > * Kernel supports speculation store bypass: YES (found in > /proc/self/status) > > STATUS: NOT VULNERABLE (Mitigation: Speculative Store Bypass disabled via prctl and seccomp) > > The spectre-meltdown checker shows a machine not vulnerable with kernel 4.14 > and updated microcode. Great, then I'll add my thumbs up for 4.14.xx!!
Please try to run 4.14 if it works for you and report back,so that the maintainers have bigger confidence in stabilizing it.
4.14.44 is working properly for a month on my ASUS P4540UQ
(In reply to Tomáš Mózes from comment #25) > Please try to run 4.14 if it works for you and report back,so that the > maintainers have bigger confidence in stabilizing it. 4.14.51 Status Report: 1. Spectre and Meltdown mitigation detection tool v0.37+ Checking for vulnerabilities on current system Kernel is Linux 4.14.51-gentoo #1 SMP Thu Jun 28 19:56:50 CDT 2018 x86_64 CPU is Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz Hardware check * Hardware support (CPU microcode) for mitigation techniques * Indirect Branch Restricted Speculation (IBRS) * SPEC_CTRL MSR is available: YES * CPU indicates IBRS capability: YES (SPEC_CTRL feature bit) * Indirect Branch Prediction Barrier (IBPB) * PRED_CMD MSR is available: YES * CPU indicates IBPB capability: YES (SPEC_CTRL feature bit) * Single Thread Indirect Branch Predictors (STIBP) * SPEC_CTRL MSR is available: YES * CPU indicates STIBP capability: YES (Intel STIBP feature bit) * Speculative Store Bypass Disable (SSBD) * CPU indicates SSBD capability: YES (Intel SSBD) * Enhanced IBRS (IBRS_ALL) * CPU indicates ARCH_CAPABILITIES MSR availability: NO * ARCH_CAPABILITIES MSR advertises IBRS_ALL capability: NO * CPU explicitly indicates not being vulnerable to Meltdown (RDCL_NO): NO * CPU explicitly indicates not being vulnerable to Variant 4 (SSB_NO): NO * CPU microcode is known to cause stability problems: NO (model 0x9e family 0x6 stepping 0x9 ucode 0x8e cpuid 0x906e9) * CPU vulnerability to the speculative execution attack variants * Vulnerable to Variant 1: YES * Vulnerable to Variant 2: YES * Vulnerable to Variant 3: YES * Vulnerable to Variant 3a: YES * Vulnerable to Variant 4: YES CVE-2017-5753 [bounds check bypass] aka 'Spectre Variant 1' * Mitigated according to the /sys interface: YES (Mitigation: __user pointer sanitization) * Kernel has array_index_mask_nospec (x86): UNKNOWN (couldn't check (couldn't find your kernel image in /boot, if you used netboot, this is normal)) * Kernel has the Red Hat/Ubuntu patch: UNKNOWN (couldn't check (couldn't find your kernel image in /boot, if you used netboot, this is normal)) * Kernel has mask_nospec64 (arm): UNKNOWN (couldn't check (couldn't find your kernel image in /boot, if you used netboot, this is normal)) * Checking count of LFENCE instructions following a jump in kernel... UNKNOWN (couldn't check (couldn't find your kernel image in /boot, if you used netboot, this is normal)) STATUS: NOT VULNERABLE (Mitigation: __user pointer sanitization) CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2' * Mitigated according to the /sys interface: YES (Mitigation: Full generic retpoline, IBPB, IBRS_FW) * Mitigation 1 * Kernel is compiled with IBRS support: YES * IBRS enabled and active: YES (for kernel and firmware code) * Kernel is compiled with IBPB support: YES * IBPB enabled and active: YES * Mitigation 2 * Kernel has branch predictor hardening (arm): NO * Kernel compiled with retpoline option: YES * Kernel compiled with a retpoline-aware compiler: YES (kernel reports full retpoline compilation) * Kernel supports RSB filling: UNKNOWN (kernel image missing) STATUS: NOT VULNERABLE (IBRS + IBPB are mitigating the vulnerability) CVE-2017-5754 [rogue data cache load] aka 'Meltdown' aka 'Variant 3' * Mitigated according to the /sys interface: YES (Mitigation: PTI) * Kernel supports Page Table Isolation (PTI): YES * PTI enabled and active: YES * Reduced performance impact of PTI: YES (CPU supports INVPCID, performance impact of PTI will be greatly reduced) * Running as a Xen PV DomU: NO STATUS: NOT VULNERABLE (Mitigation: PTI) CVE-2018-3640 [rogue system register read] aka 'Variant 3a' * CPU microcode mitigates the vulnerability: YES STATUS: NOT VULNERABLE (your CPU microcode mitigates the vulnerability) CVE-2018-3639 [speculative store bypass] aka 'Variant 4' * Mitigated according to the /sys interface: YES (Mitigation: Speculative Store Bypass disabled via prctl and seccomp) * Kernel supports speculation store bypass: YES (found in /proc/self/status) STATUS: NOT VULNERABLE (Mitigation: Speculative Store Bypass disabled via prctl and seccomp) A false sense of security is worse than no security at all, see --disclaimer 2.systemd warning regarding "nsdelegate" gone ==================================================== I'm good with this. If you'd like me to test anything else let me know... +1 for stabilizing
Archs, please stablize gentoo-sources-4.14.52
amd64 stable
Looking good on ppc64.
Created attachment 538078 [details] dmesg (ppc64)
x86 stable
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=7d21b2ccbbed896602998b5001de512a1ed3b1e8 commit 7d21b2ccbbed896602998b5001de512a1ed3b1e8 Author: Sergei Trofimovich <slyfox@gentoo.org> AuthorDate: 2018-07-07 10:07:46 +0000 Commit: Sergei Trofimovich <slyfox@gentoo.org> CommitDate: 2018-07-07 10:07:46 +0000 sys-kernel/gentoo-sources: stable 4.14.52 for ppc64, bug #649198 Tested-by: ernsteiswuerfel Bug: https://bugs.gentoo.org/649198 Package-Manager: Portage-2.3.41, Repoman-2.3.9 RepoMan-Options: --include-arches="ppc64" sys-kernel/gentoo-sources/gentoo-sources-4.14.52.ebuild | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=5254e1f8f56e6dcaf361cf78fd31cc20e683a20b commit 5254e1f8f56e6dcaf361cf78fd31cc20e683a20b Author: Sergei Trofimovich <slyfox@gentoo.org> AuthorDate: 2018-07-07 19:39:44 +0000 Commit: Sergei Trofimovich <slyfox@gentoo.org> CommitDate: 2018-07-07 19:39:53 +0000 sys-kernel/gentoo-sources: stable 4.14.52 for ia64, bug #649198 Bug: https://bugs.gentoo.org/649198 Package-Manager: Portage-2.3.41, Repoman-2.3.9 RepoMan-Options: --include-arches="ia64" sys-kernel/gentoo-sources/gentoo-sources-4.14.52.ebuild | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Stable on alpha.
arm stable
stabilized 4.14.65 on ppc for bug 663744. closing.
