Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 716520 - sys-kernel/gentoo-sources-5.6.2+: Add support for ZSTD-compressed kernel and initramfs
Summary: sys-kernel/gentoo-sources-5.6.2+: Add support for ZSTD-compressed kernel and ...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal enhancement (vote)
Assignee: Gentoo Kernel Bug Wranglers and Kernel Maintainers
URL:
Whiteboard: gentoo-sources-5.6.4
Keywords: PATCH
Depends on:
Blocks:
 
Reported: 2020-04-07 06:54 UTC by Klemen Mihevc
Modified: 2020-04-13 14:50 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Full combined v4 patch (v4-add-support-for-ZSTD-compressed-kernel-and-initramfs.patch,26.26 KB, patch)
2020-04-07 06:55 UTC, Klemen Mihevc
Details | Diff
lib: prepare zstd for preboot environment (5000_ZSTD-v4-1-8-prepare-zstd-for-preboot-env.patch,2.94 KB, patch)
2020-04-08 14:53 UTC, Mike Pagano
Details | Diff
lib: prepare xxhash for preboot environment (5001_ZSTD-v4-2-8-prepare-xxhash-for-preboot-env.patch,2.53 KB, patch)
2020-04-08 14:53 UTC, Mike Pagano
Details | Diff
lib: add zstd support to decompress (5002_ZSTD-v4-3-8-add-zstd-support-to-decompress.patch,12.46 KB, patch)
2020-04-08 14:54 UTC, Mike Pagano
Details | Diff
init: add support for zstd compressed kernel (5003_ZSTD-v4-4-8-add-support-for-zstd-compres-kern.patch,2.43 KB, patch)
2020-04-08 14:55 UTC, Mike Pagano
Details | Diff
usr: add support for zstd compressed initramfs (5004_ZSTD-v4-5-8-add-support-for-zstd-compressed-initramfs.patch,1.60 KB, patch)
2020-04-08 14:55 UTC, Mike Pagano
Details | Diff
x86: bump ZO_z_extra_bytes margin for zstd (5005_ZSTD-v4-6-8-bump-ZO-z-extra-bytes-margin.patch,819 bytes, patch)
2020-04-08 14:56 UTC, Mike Pagano
Details | Diff
x86: Add support for ZSTD compressed kernel (5006_ZSTD-v4-7-8-support-for-ZSTD-compressed-kernel.patch,3.30 KB, patch)
2020-04-08 14:57 UTC, Mike Pagano
Details | Diff
.gitignore: add ZSTD-compressed files (5007_ZSTD-v4-8-8-gitignore-add-ZSTD-compressed-files.patch,209 bytes, patch)
2020-04-08 14:57 UTC, Mike Pagano
Details | Diff
build log from gentoo-sources-5.6.3 (build.log,6.91 KB, application/octet-stream)
2020-04-08 15:17 UTC, Klemen Mihevc
Details
build.log with v5 patches (build.log,6.82 KB, application/octet-stream)
2020-04-12 10:15 UTC, Klemen Mihevc
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Klemen Mihevc 2020-04-07 06:54:15 UTC
There is (again) push to add support for ZSTD compressed kernel in to mainline:

https://lore.kernel.org/lkml/20200401053913.216783-1-nickrterrell@gmail.com/
https://lore.kernel.org/patchwork/project/lkml/list/?submitter=27840

Not sure if it will get adopted or not, since if i know correctly they try to include this patch for 3+ years... 

However it would be nice anyway if this patch would be added in to gentoo-sources, if not in main patch, you could have it in experimental for a while... Im using ZSTD kernel for couple of days and everything works, smaller size then LZO that i used before + faster decompression time. There isnt really any downside, if its added i think...

Reproducible: Always
Comment 1 Klemen Mihevc 2020-04-07 06:55:08 UTC
Created attachment 630946 [details, diff]
Full combined v4 patch
Comment 2 Mike Pagano gentoo-dev 2020-04-08 14:53:06 UTC
Created attachment 631330 [details, diff]
lib: prepare zstd for preboot environment
Comment 3 Mike Pagano gentoo-dev 2020-04-08 14:53:53 UTC
Created attachment 631332 [details, diff]
lib: prepare xxhash for preboot environment
Comment 4 Mike Pagano gentoo-dev 2020-04-08 14:54:42 UTC
Created attachment 631334 [details, diff]
lib: add zstd support to decompress
Comment 5 Mike Pagano gentoo-dev 2020-04-08 14:55:24 UTC
Created attachment 631336 [details, diff]
init: add support for zstd compressed kernel
Comment 6 Mike Pagano gentoo-dev 2020-04-08 14:55:55 UTC
Created attachment 631338 [details, diff]
usr: add support for zstd compressed initramfs
Comment 7 Mike Pagano gentoo-dev 2020-04-08 14:56:29 UTC
Created attachment 631340 [details, diff]
x86: bump ZO_z_extra_bytes margin for zstd
Comment 8 Mike Pagano gentoo-dev 2020-04-08 14:57:07 UTC
Created attachment 631342 [details, diff]
x86: Add support for ZSTD compressed kernel
Comment 9 Mike Pagano gentoo-dev 2020-04-08 14:57:42 UTC
Created attachment 631344 [details, diff]
.gitignore: add ZSTD-compressed files
Comment 10 Mike Pagano gentoo-dev 2020-04-08 14:59:09 UTC
Can you apply and test patches 5000-5007 that I just attached?
Comment 11 Klemen Mihevc 2020-04-08 15:17:59 UTC
Created attachment 631346 [details]
build log from gentoo-sources-5.6.3

build.log attached from gentoo-sources-5.6.3, patches are working as they suppose to and ZSTD compressed kernel works as well.
Comment 12 Mike Pagano gentoo-dev 2020-04-08 16:52:59 UTC
(In reply to Klemen Mihevc from comment #11)
> Created attachment 631346 [details]
> build log from gentoo-sources-5.6.3
> 
> build.log attached from gentoo-sources-5.6.3, patches are working as they
> suppose to and ZSTD compressed kernel works as well.

Thanks for that quick response.
I'll add it to experimental and you'll see it in gentoo-sources-5.6.4

Feel free to ping me on IRC in gentoo-kernel if they release new versions and I haven't updated them yet
Comment 13 Klemen Mihevc 2020-04-08 17:10:09 UTC
Thanks and i will...
Comment 14 Mike Pagano gentoo-dev 2020-04-08 17:40:43 UTC
commit 1925b0520f1735eb1c30313f518c521fc5478adf (HEAD -> 5.6, origin/5.6)
Author: Mike Pagano <mpagano@gentoo.org>
Date:   Wed Apr 8 13:37:44 2020 -0400

    Add support for ZSTD-compressed kernel and initramfs (use=experimental)
    
    Signed-off-by: Mike Pagano <mpagano@gentoo.org>
Comment 15 Klemen Mihevc 2020-04-12 08:05:33 UTC
gonna ping here, since bug is currently still open... v5 released...
Comment 16 Michael 'veremitz' Everitt 2020-04-12 08:10:47 UTC
(In reply to Klemen Mihevc from comment #15)
> gonna ping here, since bug is currently still open... v5 released...

Unless I'm mistaken, the latest release of the 5.6 branch is still 5.6.3, and as mpagano has pointed out above, the patches you require will not be released until 5.6.4 is out. So I'm afraid you'll have to wait a bit longer!
Comment 17 Klemen Mihevc 2020-04-12 08:28:25 UTC
(In reply to Michael 'veremitz' Everitt from comment #16)
> (In reply to Klemen Mihevc from comment #15)
> > gonna ping here, since bug is currently still open... v5 released...
> 
> Unless I'm mistaken, the latest release of the 5.6 branch is still 5.6.3,
> and as mpagano has pointed out above, the patches you require will not be
> released until 5.6.4 is out. So I'm afraid you'll have to wait a bit longer!

obviously... but better that its released with new v5 version then with old v4 right? :) i know bug is still open just because 5.6.4 was not released yet, but he said to ping him if version gets updated and since it is ~20 years since i used IRC last time, i rather do it here ;P
Comment 18 Klemen Mihevc 2020-04-12 10:15:31 UTC
Created attachment 632454 [details]
build.log with v5 patches

kernel works with with v5 patches, as far as i noticed there isnt any significant change anyway...
Comment 19 Mike Pagano gentoo-dev 2020-04-12 15:01:52 UTC
It's all good. Let me try to update to v5 before 5.6.4 is out, which I expect later today.
Comment 20 Mike Pagano gentoo-dev 2020-04-12 15:29:56 UTC
commit a47c371bfe46821952d584e7c0102948682a4602 (HEAD -> 5.6, origin/5.6)
Author: Mike Pagano <mpagano@gentoo.org>
Date:   Sun Apr 12 11:28:25 2020 -0400

    Bump ZSTD Patchset to V5
    
    Closes: https://bugs.gentoo.org/716520
    
    Signed-off-by: Mike Pagano <mpagano@gentoo.or
Comment 21 Mike Pagano gentoo-dev 2020-04-13 14:50:46 UTC
commit 7d026fa69906949e376c76dd73c7b665f45fba88 (HEAD -> master, origin/master, origin/HEAD)
Author: Mike Pagano <mpagano@gentoo.org>
Date:   Mon Apr 13 10:23:21 2020 -0400

    sys-kernel/gentoo-sources: Linux patch 5.6.4. ZSTD patch.
    
    Add support for v5 of the ZSTD-compressed kernel and initramfs.
    
    Package-Manager: Portage-2.3.89, Repoman-2.3.20
    Signed-off-by: Mike Pagano <mpagano@gentoo.org>