When using `/usr/libexec/xen/bin/grub-x86_64-xen.bin` to boot a Xen domU with a ZSTD-compressed kernel, GRUB will fail to recognize the kernel, and will fail to boot with an extremely unclear boot message: ``` Loading Linux linux ... error: not xen image. Loading initial ramdisk ... error: you need to load the kernel first. Press any key to continue... ``` It turns out that Arch Linux (domU distro), with the release of their 5.9 series kernel packages, converted to compressing the kernel with ZSTD by default: https://github.com/archlinux/svntogit-packages/commit/5c7d8268bbddfe55aba9e587658592069fdb24af#diff-3e341d2d9c67be01819b25b25d5e53ea3cdf3a38d28846cda85a195eb9b7203a As an interim hack, I've added an ALPM hook to the domU Arch systems to unpack the kernel if it detects ZSTD. This uses the scripts from the linux kernel tree `scripts/extract-vmlinux`. This works, but is kind of silly. Note that `app-emulation/grub-xen-host` is really just a wrapper around the image build command an already installed grub. But I've no idea where or how this bug is in grub since it seems that normal, non-virtualization grub isn't affected by it (well, I presume, since I've not found any bug reports and I don't think the kernel community would have accepted the ZSTD patchset if grub failed to boot it). For reference, my more colorful rant is here: https://bugs.archlinux.org/task/68445
Also, one of the other fruits of my search / debugging. GRUB already has code for ZSTD, but it seems to only be wired up for btrfs zstd-compressed file systems. AFAICT, the grub codebase isn't aware that ZSTD can be used in the kernel.
Just to sum all this up: I think the bug is either in the `grub-xen-host` package in that it appears to build a grub boot loader that inspects the kernel to determine if it's a valid kernel. Which, imho, it shouldn't do. However, the problem **may** be in the grub code itself. Regardless, it's silly and should be happening.
The release notes of xen 4.15 says: "To enable the new support for zstd-compressed guest kernels, zstd-devel needs to be installed. If it isn't, the support for zstd is disabled at build-time." [0] Maybe this fixes the problem? I'm currently not sure if the gentoo Xen is build with zstd support. [0] https://wiki.xenproject.org/wiki/Xen_Project_4.15_Release_Notes
Hm, the update didn't help for me. On the Xen devel Mailinglist, they say it should work with grub and type Xen domU type PVH [0]. So zstd compressed Kernel with Type PVH works for me. Currently app-emulation/grub-xen-host has no support for pvh, PR is open [1]. [0] https://lists.xenproject.org/archives/html/xen-devel/2020-11/msg01073.html [1] https://github.com/gentoo/gentoo/pull/13496
Related gentoo-dist kernel bug that disabled ZSTD compression: bug #784599.
