Bug 686094 – sys-fs/zfs-kmod-0.7.12 with >=sys-kernel/gentoo-sources-4.19.38 - In file included from .../work/zfs-0.7.12/module/zcommon/zfs_fletcher_intel.c:45: .../work/zfs-0.7.12/include/linux/simd_x86.h:98:10: fatal error: asm/i387.h: No such file or directory

• Status: RESOLVED FIXED
• Product: Gentoo Linux

[Description Simon Levermann (sonOfRa) 2019-05-16 11:32:43 UTC]

This is a bug originally introduced into kernel version 5.0.0, but it appears to have been backported to 4.19.38 and upwards.

https://github.com/zfsonlinux/zfs/issues/8259 has a description of the underlying problem.

The bug appears to be fixed in version 0.7.13, so maybe a stabilization would be in order soon.

Reproducible: Always

[Comment 1 Simon Levermann (sonOfRa) 2019-05-16 11:33:36 UTC]

Created attachment 576910 [details]
build.log

[Comment 2 Simon Levermann (sonOfRa) 2019-05-16 11:37:03 UTC]

It should be noted that Linux-4.19.43 introduces mitigations for a new class of hardware attacks, see:

https://wiki.gentoo.org/wiki/Project:Security/Vulnerabilities/MDS_-_Microarchitectural_Data_Sampling_aka_ZombieLoad

I encountered this bug when upgrading to 4.19.43 in order to enable these mitigations, which is where the build failure occurred.

[Comment 3 Georgy Yakovlev 2019-05-16 12:15:55 UTC]

Thanks,

I think 0.7.13 includes fix for this, can you try it? I was going to stabilize it anyway.

I’ll be able to check myself, but later.

[Comment 4 Simon Levermann (sonOfRa) 2019-05-16 12:18:02 UTC]

Compiles and works with 0.7.13 on 4.19.43!

[Comment 5 Larry the Git Cow 2019-05-16 12:30:30 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ead386260ab0d14008e5f27b369d09d6fa98feda

commit ead386260ab0d14008e5f27b369d09d6fa98feda
Author:     Georgy Yakovlev <[e-mail]>
AuthorDate: 2019-05-16 12:29:06 +0000
Commit:     Georgy Yakovlev <[e-mail]>
CommitDate: 2019-05-16 12:30:10 +0000

    sys-fs/zfs: 0.7.13 stable amd64, builds on linux-4.19.43
    
    Bug: https://bugs.gentoo.org/686094
    Package-Manager: Portage-2.3.66, Repoman-2.3.12
    Signed-off-by: Georgy Yakovlev <[e-mail]>

 sys-fs/zfs/zfs-0.7.13.ebuild | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=def11dde72f64a993751b1bf082d35347b7f1d39

commit def11dde72f64a993751b1bf082d35347b7f1d39
Author:     Georgy Yakovlev <[e-mail]>
AuthorDate: 2019-05-16 12:27:34 +0000
Commit:     Georgy Yakovlev <[e-mail]>
CommitDate: 2019-05-16 12:30:09 +0000

    sys-fs/zfs-kmod: 0.7.13 stable amd64, builds on linux-4.19.43
    
    Bug: https://bugs.gentoo.org/686094
    Package-Manager: Portage-2.3.66, Repoman-2.3.12
    Signed-off-by: Georgy Yakovlev <[e-mail]>

 sys-fs/zfs-kmod/zfs-kmod-0.7.13.ebuild | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=d54e484c852e10255bd0abc6003f72bb8e148ba4

commit d54e484c852e10255bd0abc6003f72bb8e148ba4
Author:     Georgy Yakovlev <[e-mail]>
AuthorDate: 2019-05-16 12:27:06 +0000
Commit:     Georgy Yakovlev <[e-mail]>
CommitDate: 2019-05-16 12:30:08 +0000

    sys-kernel/spl: 0.7.13 stable amd64, builds on linux-4.19.43
    
    Bug: https://bugs.gentoo.org/686094
    Package-Manager: Portage-2.3.66, Repoman-2.3.12
    Signed-off-by: Georgy Yakovlev <[e-mail]>

 sys-kernel/spl/spl-0.7.13.ebuild | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[Comment 6 Georgy Yakovlev 2019-05-16 12:34:16 UTC]

leaving this open for a while for visibility, will close later

[Comment 7 Simon Levermann (sonOfRa) 2019-05-16 14:00:58 UTC]

NixOS uses the following patch https://raw.githubusercontent.com/NixOS/nixpkgs/master/pkgs/os-specific/linux/kernel/export_kernel_fpu_functions.patch 

in order to restore the kernel symbols that got removed. ZFS uses quite a few vectorizable operations, and without these symbols, they cannot use extensions like AVX and AES-NI, which may reduce performance noticeably.

The patch applies successfully for me on 4.19.43 and in the configure stage of zfs-kmod-0.7.13 there should be a line

"checking which kernel_fpu function to use... __kernel_fpu_*" if patching was successful.

Phoronix has an article that this performance drop is definitely noticeable, at least when running the later 0.8 versions that support encryption:

https://www.phoronix.com/scan.php?page=news_item&px=NixOS-Linux-5.0-ZFS-FPU-Drop

[Comment 8 Georgy Yakovlev 2019-05-16 15:39:22 UTC]

I use similar pathch for zfs-0.8.0 with encryption, works great. but I don’t think it’s needed with 0.7.x, as there is no encryption support in it. Maybe checksum will benefit from it, idk.

Don’t think it’s legal to distribute this patch with gentoo-sources and doubt kernel team will ever consider it.
Legal matters is not my area, but I believe as long as one uses it personally on home machine and doesn’t distribute binaries they are ok.

[Comment 9 Simon Levermann (sonOfRa) 2019-05-16 16:15:34 UTC]

It should be noticeable (obviosuly not quite as much as encryption) even for checksumming. On the legal matters or bundling it with gentoo-sources, I'm also not certain. NixOS seems to think it's acceptable to bundle it. I don't know if other distros are also bundling it.

[Comment 10 Georgy Yakovlev 2019-05-16 19:03:40 UTC]

<offtopic>
you may be interested comparing output of this:

grep '' /proc/spl/kstat/zfs/*_bench

before and after the patch with affected kernels =)
those files contain zfs microbenchmarks module does at load time to select optimal algorithm.


going to build a new without the patch and see how different it is on microbenchmark level.
</offtopic>

[Comment 11 Simon Levermann (sonOfRa) 2019-05-17 09:04:21 UTC]

Depending on the values, the AVX2 implementations is generally somewhere between 3 and 9 times faster than the fastest non-SIMD version. I'm not sure if it would make a difference on a RAID-1 on 7200RPM spinning drives, but the difference is definitely there.

[Comment 12 Larry the Git Cow 2019-06-23 08:00:38 UTC]

The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=5603412b88016f7349a4909f701f13916eef970a

commit 5603412b88016f7349a4909f701f13916eef970a
Author:     Georgy Yakovlev <[e-mail]>
AuthorDate: 2019-06-23 07:56:56 +0000
Commit:     Georgy Yakovlev <[e-mail]>
CommitDate: 2019-06-23 08:00:13 +0000

    sys-fs/zfs-kmod: drop 0.7.12
    
    Closes: https://bugs.gentoo.org/686094
    Package-Manager: Portage-2.3.67, Repoman-2.3.15
    Signed-off-by: Georgy Yakovlev <[e-mail]>

 sys-fs/zfs-kmod/Manifest               |   1 -
 sys-fs/zfs-kmod/zfs-kmod-0.7.12.ebuild | 146 ---------------------------------
 2 files changed, 147 deletions(-)