Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 718026 - sys-fs/zfs: Add support for Intel QuickAssist (QAT) acceleration
Summary: sys-fs/zfs: Add support for Intel QuickAssist (QAT) acceleration
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal enhancement (vote)
Assignee: Georgy Yakovlev
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-04-18 10:20 UTC by grixfosforito
Modified: 2021-03-01 11:00 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description grixfosforito 2020-04-18 10:20:11 UTC
ZFS has support for Intel QuickAssist (QAT) acceleration during compression, encryption and checksum. 

http://open-zfs.org/wiki/ZFS_Hardware_Acceleration_with_QAT

If I understand correctly, it is not included in the current ebuild. Has it been left out due to any bug, instability, etc? Could it be included in the ebuild for the current version of ZFS?

I can offer my current hardware for testing.
Comment 1 Georgy Yakovlev gentoo-dev 2020-04-20 07:42:15 UTC
I have compatible hardware and can investigate it.

Will need to do some research on it, on licensing and packaging QAT libraries, so it will not happen soon, but It's something I wanted to do anyway, so thanks for reminder =)
Comment 2 Georgy Yakovlev gentoo-dev 2020-10-01 22:35:45 UTC
situation with QAT is rather awful and to build on modern kernels it requires patching the kernel.

and you can't use in-kernel QAT, because of licensing reasons, it has to be external one, provided as a separate package/source at build time.

zfs-2.0.0-rc3 includes patches and general readme in contrib/intel_qat

so this will not work out of the box.

I'll leave bug open for now, but QAT support via ebuild not happening for now.
Comment 3 grixfosforito 2020-10-08 15:42:04 UTC
Thanks a lot for looking into this. If I may ask: why is QAT support not happening via ebuild? Is kernel patching or other limitation not acceptable by Gentoo? Or is not happening because we need a maintainer to step in and have the time to do it?

I am asking because I've been using some of my spare time to create a private ebuild to install QAT, patching kernels 5.8.x. It is working fine, except because the qat_service is installed, by the driver, in /etc/init.d, so no systemd support under Gentoo:

https://wiki.gentoo.org/wiki/Systemd#OpenRC_services

I am trying, to the best of my abilities, to install QAT support in my Gentoo server, so I can share my work and you could decide if it is usable for sys-fs/zfs. Of course, I don't want to overstep here.
Comment 4 grixfosforito 2020-10-09 06:32:10 UTC
I forgot to add the github link to my ebuild in my previous message:

https://github.com/bugalo/ZFS-QAT-gentoo
Comment 5 Georgy Yakovlev gentoo-dev 2020-10-09 17:04:03 UTC
oh I misunderstood something.
I thought that kernel needs patching to be compatible with QAT
but it's the other way, QAT needs patching to be compatible with certain kernels.


btw, zfs2 itself now includes those patches (just for convenience, it does not apply them, just contains)

https://github.com/openzfs/zfs/pull/10962/files



it that case I can look at it again, you ebuild is a good starting point.
my QAT-capable hardware is running FreeBSD(FreeNAS) now, but I wanted to migrate it to gentoo anyway, another reason to start this project.

No promises, but this looks doable now, sorry for confusion.

To address build issues I need to poke it myself, addwrite to root filesystem directories is absolutely no-go in gentoo repo, this will need a workaround or fix.

I will poke it, maybe this weekend, maybe not, bit busy lately.
Comment 6 grixfosforito 2020-10-10 08:05:36 UTC
I'm happy I was of some assistance. 

Regarding the patches provided in the ZFS repo, I read it in your comment, but then forgot it, so I found them in the source, SPDK's github. However, keep in mind that they are not complete. When compiling the driver with --enable-qat-lkcf, the driver requires the "ablkcipher" by the kernel, but support was dropped in 5.6 in favor of "skcipher". Thus the big patch to migrate all the calls to "ablkcipher" to "skcypher". I don't know if the ZFS developers solve this issue in a different way.
Comment 7 Georgy Yakovlev gentoo-dev 2020-10-20 09:26:16 UTC
good news, got my nas migrated to gentoo from freeNAS, so I now have a machine to test this on. will poke as soon as I have more time.