TLDR: Kernel modules build by portage and XZ compressed fail to load with latest kmod tools. Long version: Kernel modules when XZ compressed need specific XZ compression parameters to be decompressible by the Linux kernel. Kmod tools have switched to use the kernel-provided XZ decompressor. https://lore.kernel.org/all/3d34a965-ab9c-d549-0c63-c717ab5d2edc@tweek.dk/ See above link, the kernel build system compresses in scripts/Makefile.modinst with cmd_xz = $(XZ) --check=crc32 --lzma2=dict=1MiB -f $< Using portage to compile and install third party kernel modules, for example virtualbox-modules, calls in ebuild the function linux-mod-r1_src_install Following this call to eclass/linux-mod-r1.class, portage compresses with compress=(xz -qT"$(makeopts_jobs)" --memlimit-compress=50%) This produces xz compressed kernel modules, but they fail to load via modprobe/insmod with: decompression failed with status 6 ...because the XZ compression parameters are not supported by the kernel XZ decompressor. I believe the fix is to add the same parameters in eclass/linux-mod-r1.class, but this is just my guess after brute-force grepping this problem in the portage code, without any deeper understanding... Reproducible: Always
Hm, from a quick try I can't seem to reproduce? Originally I thought I couldn't because I was testing with kernel 6.1.x (gentoo-kernel-bin in a VM), and kmod-31 NEWS says: - Use in-kernel decompression <snip> requires a recent kernel (>= 6.4) But after updating to gentoo-kernel-bin-6.6.8, modules still load fine? I tried to add a (unsupported) --check=sha256 --lzma2=dict=64MiB (default is crc32 when unspecified like in the eclass) but it doesn't make them fail either. I tested with nvidia-drivers fwiw: nvidia.ko.xz: XZ compressed data, checksum SHA-256 Maybe still missing some condition to trigger usage of the kernel decompressor, may have to dig in the kmod sources. At least the >6.4 bit does mean stable LTS users are likely not affected even though kmod-31 was stabled. What's your emerge --info? It could help.
(In reply to Ionen Wolkens from comment #1) > Maybe still missing some condition to trigger usage of the kernel > decompressor, may have to dig in the kmod sources. Ah, thought having a kernel with CONFIG_XZ_DEC=y and then compressing (only) the out-of-tree modules would be enough, but it checks: static enum kmod_file_compression_type get_kernel_compression(struct kmod_ctx *ctx) { const char *path = "/sys/module/compression"; Meaning I need full on CONFIG_MODULE_COMPRESS, guess I'll build a new kernel and try again.
(In reply to Ionen Wolkens from comment #2) > Meaning I need full on CONFIG_MODULE_COMPRESS, guess I'll build a new kernel > and try again. Or rather, what I was missing is CONFIG_MODULE_DECOMPRESS. This is off-by-default (description has a "If unsure, say N."), and is also disabled in gentoo-kernel-bin (so couldn't reproduce with that). Now got dropped to the console when it failed to load my nvidia.ko.xz, rebuilding drivers with the noted options worked again. Think in reality we have very few affected users, so can probably skip revbumps after fixing the eclass and modules will get naturally rebuilt on kernel >6.4 kernel upgrades for stable users. Probably not worth a ML review (limited xz-only scope), I'll fix in a bit after testing a bit more.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=44db9a32adb0632ae6ae77dd63c83c010387bb5d commit 44db9a32adb0632ae6ae77dd63c83c010387bb5d Author: Ionen Wolkens <ionen@gentoo.org> AuthorDate: 2023-12-28 05:04:14 +0000 Commit: Ionen Wolkens <ionen@gentoo.org> CommitDate: 2023-12-28 06:29:00 +0000 linux-mod-r1.eclass: use kernel compatible xz settings for >=kmod-31 When the non-default CONFIG_KERNEL_DECOMPRESS option is set in kernel >6.4, >=kmod-31 will let the kernel handle decompression rather than use xz-utils. ...but kernel XZ decompressor is more limited, so need to use the same options the kernel does. -T/--threads fortunately is harmless to keep on top. Note crc32 is already the default and main problem was dict size, but still specify in case someone has e.g. XZ_DEFAULTS="--check=sha256". Closes: https://bugs.gentoo.org/920837 Signed-off-by: Ionen Wolkens <ionen@gentoo.org> eclass/linux-mod-r1.eclass | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
(In reply to Larry the Git Cow from comment #4) > Note crc32 is already the default and main problem was dict size, but > still specify in case someone has e.g. XZ_DEFAULTS="--check=sha256". err, actually crc64 is the default, not sure why I thought that. Maybe I checked the wrong file. Anyway, option is passed.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=eaefe0f676c685a0e52682eb125db34a1f54da9b commit eaefe0f676c685a0e52682eb125db34a1f54da9b Author: Michał Górny <mgorny@gentoo.org> AuthorDate: 2023-12-28 08:51:02 +0000 Commit: Michał Górny <mgorny@gentoo.org> CommitDate: 2023-12-30 16:19:09 +0000 linux-mod.eclass: Fix xz compression options Match xz compression options to the ones used by the kernel, as the xz decoder used by the kernel supports only a subset of the xz format. Bug: https://bugs.gentoo.org/920837 Signed-off-by: Michał Górny <mgorny@gentoo.org> eclass/linux-mod.eclass | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
