Bug 738160 – sys-kernel/genkernel-4.1.0 - ERROR: Invalid action specified: --integrated-initramfs option requires action "all", i.e. building of kernel and initramfs at the same time!

• Status: RESOLVED INVALID
• Product: Gentoo Hosted Projects

[Description cazzantonio 2020-08-20 08:12:54 UTC]

I use genkernel to build and integrate the initramfs to my already (manually) built kernel, as part of an efi stub kernel:
genkernel --keymap --btrfs --no-ssh --disklabel --luks --gpg --integrated-initramfs --postclear --no-ramdisk-modules initramfs

Until 4.0.X everything works, then 4.1 came out with this error:
* Gentoo Linux Genkernel; Version 4.1.0
* Using genkernel configuration from '/etc/genkernel.conf' ...
* Running with options: --keymap --btrfs --no-ssh --disklabel --luks --gpg --integrated-initramfs --postclear --no-ramdisk-modules initramfs
* ERROR: Invalid action specified: --integrated-initramfs option requires action "all", i.e. building of kernel and initramfs at the same time!
* Please consult '/var/log/genkernel.log' for more information and any
* errors that were reported above.
* 
* Report any genkernel bugs to bugs.gentoo.org and
* assign your bug to [e-mail]. Please include
* as much information as you can in your bug report; attaching
* '/var/log/genkernel.log' so that your issue can be dealt with effectively.
* 
* Please do *not* report kernel compilation failures as genkernel bugs!
* 

Please, if you radically change genkernel behaviour between minior versions, please advertise it!! And also possibly keep compatibility with previous versions!
I don't know why someone suddenly woke up and decided that I can't use --integrated-initramfs with initramfs action, I'm sure there is a reason, but please at least explain it and provide an alternative!

/* complain mode on: I love gentoo, but sometimes it seems anything but professional. Remember that developers are working for users, not just for themselves...) */

Thanks

Reproducible: Always




genkernel.log:

>>> Started genkernel v4.1.0 on: 2020-08-20 09:55:11
* CMD_KEYMAP: yes
* CMD_BTRFS: yes
* CMD_SSH: no
* CMD_DISKLABEL: yes
* CMD_LUKS: yes
* CMD_GPG: yes
* CMD_INTEGRATED_INITRAMFS=yes
* CMD_POSTCLEAR: yes
* CMD_RAMDISKMODULES: no
* Gentoo Linux Genkernel; Version 4.1.0
* Using genkernel configuration from '/etc/genkernel.conf' ...
* Running with options: --keymap --btrfs --no-ssh --disklabel --luks --gpg --integrated-initramfs --postclear --no-ramdisk-modules initramfs
* Resolving config file, command line, and arch default settings.
*   TMPDIR defaulted to "/var/tmp/genkernel".
*   LOGFILE set in config file to "/var/log/genkernel.log".
*   KERNEL_DIR defaulted to "/usr/src/linux".
*   KERNEL_MODULES_PREFIX not set.
*   KERNEL_SOURCES defaulted to "yes".
*   INITRAMFS_FILENAME defaulted to "initramfs-%%KV%%.img".
*   INITRAMFS_SYMLINK_NAME defaulted to "initramfs".
*   KERNEL_CONFIG_FILENAME defaulted to "kernel-config-%%KV%%".
*   KERNEL_FILENAME defaulted to "vmlinuz-%%KV%%".
*   KERNEL_SYMLINK_NAME defaulted to "kernel".
*   SYSTEMMAP_FILENAME defaulted to "System.map-%%KV%%".
*   SYSTEMMAP_SYMLINK_NAME defaulted to "System.map".
*   CHECK_FREE_DISK_SPACE_BOOTDIR not set.
*   CHECK_FREE_DISK_SPACE_KERNELOUTPUTDIR not set.
*   COMPRESS_INITRD defaulted to "yes".
*   COMPRESS_INITRD_TYPE defaulted to "best".
*   CHOST defaulted to "x86_64-pc-linux-gnu".
*   MAKEOPTS defaulted to "-j7".
*   NICE defaulted to "10".
*   KERNEL_MAKE defaulted to "make".
*   UTILS_CFLAGS defaulted to "-Os -pipe -fomit-frame-pointer".
*   UTILS_MAKE defaulted to "make".
*   KERNEL_CC defaulted to "gcc".
*   KERNEL_LD defaulted to "ld".
*   KERNEL_AS defaulted to "as".
*   UTILS_CC defaulted to "gcc".
*   UTILS_CXX defaulted to "g++".
*   UTILS_LD defaulted to "ld".
*   UTILS_AS defaulted to "as".
*   CROSS_COMPILE not set.
*   BOOTDIR defaulted to "/boot".
*   KERNEL_APPEND_LOCALVERSION not set.
*   KERNEL_LOCALVERSION defaulted to "-%%ARCH%%".
*   MODPROBEDIR defaulted to "/etc/modprobe.d".
*   SPLASH defaulted to "no".
*   CLEAR_CACHEDIR defaulted to "no".
*   POSTCLEAR set on command line to "yes".
*   MRPROPER defaulted to "yes".
*   MENUCONFIG defaulted to "no".
*   GCONFIG defaulted to "no".
*   NCONFIG defaulted to "no".
*   XCONFIG defaulted to "no".
*   CLEAN defaulted to "yes".
*   MINKERNPACKAGE not set.
*   MODULESPACKAGE not set.
*   MODULEREBUILD defaulted to "yes".
*   MODULEREBUILD_CMD defaulted to "emerge --ignore-default-opts --buildpkg=n --usepkg=n --quiet-build=y @module-rebuild".
*   KERNCACHE not set.
*   RAMDISKMODULES set on command line to "no".
*   ALLRAMDISKMODULES defaulted to "no".
*   INITRAMFS_OVERLAY not set.
*   LINUXRC not set.
*   MOUNTBOOT defaulted to "yes".
*   BUILD_STATIC defaulted to "no".
*   SAVE_CONFIG defaulted to "yes".
*   SYMLINK defaulted to "no".
*   OLDCONFIG defaulted to "yes".
*   SANDBOX defaulted to "yes".
*   SSH set on command line to "no".
*   SSH_AUTHORIZED_KEYS_FILE defaulted to "/etc/dropbear/authorized_keys".
*   SSH_HOST_KEYS defaulted to "create".
*   STRACE defaulted to "no".
*   BCACHE defaulted to "no".
*   LVM defaulted to "no".
*   DMRAID defaulted to "no".
*   ISCSI defaulted to "no".
*   HYPERV defaulted to "no".
*   BOOTFONT defaulted to "none".
*   BOOTLOADER defaulted to "no".
*   B2SUM defaulted to "no".
*   BUSYBOX defaulted to "yes".
*   BUSYBOX_CONFIG not set.
*   NFS defaulted to "yes".
*   MICROCODE defaulted to "all".
*   MICROCODE_INITRAMFS defaulted to "no".
*   UNIONFS defaulted to "no".
*   NETBOOT defaulted to "no".
*   REAL_ROOT not set.
*   DISKLABEL set on command line to "yes".
*   LUKS set on command line to "yes".
*   GPG set on command line to "yes".
*   MDADM defaulted to "no".
*   MDADM_CONFIG not set.
*   E2FSPROGS defaulted to "no".
*   XFSPROGS defaulted to "no".
*   ZFS defaulted to "no".
*   BTRFS set on command line to "yes".
*   VIRTIO defaulted to "no".
*   MULTIPATH defaulted to "no".
*   FIRMWARE defaulted to "no".
*   FIRMWARE_DIR defaulted to "/lib/firmware".
*   FIRMWARE_FILES not set.
*   FIRMWARE_INSTALL defaulted to "no".
*   INTEGRATED_INITRAMFS set on command line to "yes".
*   WRAP_INITRD defaulted to "no".
*   GENZIMAGE defaulted to "no".
*   KEYMAP set on command line to "yes".
*   DOKEYMAPAUTO defaulted to "no".
*   BUSYBOX_CONFIG not set.
*   STRIP_TYPE defaulted to "modules".
*   INSTALL defaulted to "yes".
*   CLEANUP defaulted to "yes".
*   KERNEL_OUTPUTDIR defaulted to "/usr/src/linux".

* Read 'x86_64' from 'uname -m' ...
* ARCH 'x86_64' detected ...
* CBUILD set to 'x86_64-pc-linux-gnu' ...
* CHOST set to 'x86_64-pc-linux-gnu' ...

* ERROR: Invalid action specified: --integrated-initramfs option requires action "all", i.e. building of kernel and initramfs at the same time!
* Please consult '/var/log/genkernel.log' for more information and any
* errors that were reported above.
* 
* Report any genkernel bugs to bugs.gentoo.org and
* assign your bug to [e-mail]. Please include
* as much information as you can in your bug report; attaching
* '/var/log/genkernel.log' so that your issue can be dealt with effectively.
* 
* Please do *not* report kernel compilation failures as genkernel bugs!
* 

* mount: >> Boot partition state on '/boot' was not changed; Skipping restore boot partition state ...
* Running final cache/tmp cleanup ...
* CACHE_DIR: /var/cache/genkernel
* Clearing cache dir contents from /var/cache/genkernel ...
* TMPDIR: /tmp/genkernel
* Removing tmp dir contents
>>> Ended on: 2020-08-20 09:55:11 (after 0 days 0 hours 00 minutes 00 seconds)

[Comment 1 Thomas Deutschmann (RETIRED) 2020-08-20 11:57:04 UTC]

Sorry if that change broke your workflow. Genkernel has so many options and it isn't really clear which options are supposed to work/getting used in which workflow. The recent change wasn't supposed as a breaking change and therefore wasn't announced as such. It was seen as a bug fix/clarification:

Genkernel provides switches like --lvm, --luks, --mdadm... which are supposed to add support for LVM, LUKS, MDRAID. See past cryptsetup format change example:

People don't really know what is required to unlock a LUKS2 volume. Most of them even don't know if they are using LUKS1 or LUKS2 format -- the used cryptsetup version from install media they used will decide that.

Now you can argue that this doesn't match up with Gentoo's idea. When you are using Gentoo you are supposed to know that and do stuff on your own. Sure, but genkernel was created as tool to help in this situation because if you don't even get your Gentoo system booted you have no chance to get started with Gentoo at all. That's why passing options mentioned above will ensure that both, kernel and initramfs, will support selected features.

Of course this will create some other problems advanced users don't like, i.e. there are multiple RAID levels. Using --mdadm option will enable all of them even if you know for sure you will only ever use RAID1. That's also the case for --integrated-initramfs option: It will require a prepared kernel. But what should we do if you told genkernel to also compress stuff using FOO compression but your kernel lacks decompression support for that compression algorithm? You will end up with an unbootable system -- our top priority to avoid at all costs.

When compression feature was recently reworked because it was found it didn't always do what it was supposed to do we changed stuff. It became critical that we are working with correct kernel config at this point. But how can we be sure that we are using correct kernel config if kernel wasn't build during that run?

To avoid an undefined behavior I decided to force building kernel in that case which will solve and help us to avoid multiple pitfalls and is aligned with our top priority to always ensure that if we create something, it is working (bootable) or to error out to allow user to investigate before losing access to the system.

I am sorry if this broke your workflow. I hope this explanation will help you to understand that this wasn't a random change and not intended as breaking change -- just a bug fix to address a previously undefined behavior.

PS: If you want to keep using genkernel all you have to do is switching action to "all" -- genkernel can also build EFI stub kernel with integrated initramfs if you enabled EFI stub support in your kernel config.

[Comment 2 cazzantonio 2020-08-21 17:54:11 UTC]

Thank you for the explanation.
It's a standard, in the software industry, to start to deprecate features before dropping them. This means that if I start seeing a message "warning: this feature will be deprecated in a future release" I have some time to change my automated scripts before everything breaks down.

That is if you consider your software "professional". If it's an amateur project you are free to do as you please.

Of course you have all your reasons to deprecate things, but users have the right to be warned for at least a minimum of time to prepare themselves... unless the feature you are dropping is a serious major bug that must be addressed immediately.

I understand that I can start using the all action, but it has some, IMHO, major issues:
Here is a diff between my kernel config and genkernel's config

< CONFIG_LOCALVERSION=""
---
> CONFIG_LOCALVERSION="stub"
359,360c355,356
< # CONFIG_MICROCODE_AMD is not set
< # CONFIG_MICROCODE_OLD_INTERFACE is not set
---
> CONFIG_MICROCODE_AMD=y
> CONFIG_MICROCODE_OLD_INTERFACE=y
4302c4298
< # CONFIG_CRYPTO_USER_API_AEAD is not set
---
> CONFIG_CRYPTO_USER_API_AEAD=y

1) It forces me to select a localversion unless it adds its "-x86-64" string. passing an empty string --kernel-localversion="" does not work
2) It arbitrarily add things to the config file, e.g. AMD microcode support (I'm on intel, thanks) and CONFIG_CRYPTO_USER_API_AEAD that I don't know of but I'm pretty sure I can live without.

The point is trust. You ask me to trust your program but I see that it's making changes to the .config despite me kindly asking to use mine 
This is the full string:
genkernel --kernel-config=/usr/src/linux/.config --no-mrproper --no-clean --no-oldconfig --kernel-localversion="stub" --keymap --btrfs --no-ssh --disklabel --luks --gpg --integrated-initramfs --postclear --no-ramdisk-modules all

The point isn't "the changes are small", the point is it's changing something without asking. How can I trust it to respect my config file without issues? Should I check everytime to see if it doesn't add some arbitrary thing inside? Maybe some "new brad feature" added with a minor update?

Thanks

[Comment 3 Thomas Deutschmann (RETIRED) 2020-08-21 19:28:42 UTC]

You cannot avoid that genkernel will grab CONFIG_LOCALVERSION. This is the only *reliable* and *transparent* way to add stuff like ARCH information we are aware of (feel free to suggest or even submit a patch if you know about a better way!). IMHO this is something you really want because you want that your /lib/modules matches 'uname -r'. However, this is fully configurable, see --kernel-localconfig parameter. For example, you could restore genkernel-3 naming if you really want to.

> 2) It arbitrarily add things to the config file, e.g. AMD microcode support
> (I'm on intel, thanks)
Notice, that will only happen if you ask genkernel to do add support for microcode loading (--microcode). Please get familiar with genkernel-4 -- this is now configurable: From /etc/genkernel.conf,

> # Add in early microcode support: this sets the kernel options for early microcode loading
> # Possible values: empty/"no", "all", "intel", "amd"
> #MICROCODE="all"
So you maybe want to set this to "intel" (or pass --microcode=amd via command-line).

> and CONFIG_CRYPTO_USER_API_AEAD that I don't know of but
> I'm pretty sure I can live without.
Maybe you are right. But this is getting set because you are building with "--luks" -- you will need that once you are dealing with LUKS2 format which is the default for some time now. Of course, if you have created your system 5 years ago you will probably never need this on that specific system.

> The point is trust. You ask me to trust your program but I see that
> it's making changes to the .config despite me kindly asking to use mine 
Maybe there is a general misunderstanding: We don't ask you for anything. :) And Gentoo is about choices...

Also, like explained in my previous comment, genkernel is a helper tool which will allow people to get started with Gentoo -- i.e. getting a kernel which will allow them to boot their system even if they use things like MDRAID, LVM, LUKS... there are soooo many possible configurations. Like said, cryptsetup's default format is LUKS2 now which is requiring new kernel features -- but if you are only dealing with LUKS1 format... yeah, you don't care.

If this is a problem for you, maybe genkernel isn't the tool you are looking for and you should do everything on your own so you know exactly what's in your kernel/initramfs. But when you see all the bugs which were filed when LUKS2 for example became a thing and people using genkernel just expected that genkernel will create a bootable kernel/initramfs when using --luks because they didn't even know that there are multiple LUKS formats...