Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 642096 - sys-kernel/genkernel-next-68: compressed kernel modules not supported
Summary: sys-kernel/genkernel-next-68: compressed kernel modules not supported
Status: RESOLVED OBSOLETE
Alias: None
Product: Gentoo Hosted Projects
Classification: Unclassified
Component: genkernel-next (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Ettore Di Giacinto (RETIRED)
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-12-23 10:20 UTC by Morton Pellung
Modified: 2020-08-20 12:51 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
Patch for gen_initramfs.sh and gen_moddeps.sh (genkernel-next-68-gen_initramfs-gen_moddeps.patch,2.41 KB, patch)
2019-02-07 13:58 UTC, segmentation fault
Details | Diff
Patch for modules_load in arch/x86_64 (genkernel-next-68-modules_load.patch,341 bytes, patch)
2019-02-07 14:00 UTC, segmentation fault
Details | Diff
ebuild that uses the above two patches (genkernel-next-68-r1.ebuild,1.32 KB, text/plain)
2019-02-07 14:02 UTC, segmentation fault
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Morton Pellung 2017-12-23 10:20:52 UTC
when building a kernel with compressed kernel modules the kernel modules have a different file extension than ".ko". For example, when compressed with xz they are now named ".ko.xz". When genkernel-next assembles the initramfs it collects the kernel modules by extension .ko and consequently fails to copy the modules to the intramfs when they have a different extension. The generated initramfs fails boot.

My workaround is to extend the search file extension in /usr/share/genkernel/gen_initramfs.sh:

--- gen_initramfs.sh.orig       2017-12-22 22:58:37.053028762 +0100
+++ gen_initramfs.sh    2017-12-23 01:35:24.524016660 +0100
@@ -766,7 +766,7 @@
 append_modules() {
     local group
     local group_modules
-    local MOD_EXT=".ko"
+    local MOD_EXT=".ko.*"
 
     print_info 2 "initramfs: >> Searching for modules..."
     if [ "${INSTALL_MOD_PATH}" != '' ]

...so the "find" command a few lines below finds the kernel modules again.

This is my workaround for the case of *.ko.xz modules, I don't know whether this works in all cases or if there are other side-effects to consider.


Reproducible: Always
Comment 1 segmentation fault 2019-02-07 13:55:28 UTC
I have the same problem. Not being able to include compressed AND uncompressed modules in the initramfs results in an unusable initramfs for me. I will attach an ebuild and patches that solve this problem once and for all.


The situation
=============

I don't know if it's only me, but even though I have set 

CONFIG_MODULE_COMPRESS=y
CONFIG_MODULE_COMPRESS_GZIP=y
# CONFIG_MODULE_COMPRESS_XZ is not set

in my 

/usr/src/linux/.config

file, external drivers - notably nvidia-drivers - do NOT honour this setting! The result is (and this has been this way for *decades*) that I have a mix of 

- .ko.gz modules (from kernel)
- .ko modules (from nvidia-drivers and virtualbox-modules)

Since I want to make use of Kernel Mode Setting (KMS, see: https://wiki.gentoo.org/wiki/Xorg/Guide#Kernel_modesetting), I *need* the nvidia modules in my initramfs.

However, genkernel-next-68 (and 69) will not put them in. :-(((


Reason
------

There are two reasons involved here:

1. genkernel-next assumes that CONFIG_MODULE_COMPRESS_* will be honoured by ALL parts involved, so it consults the kernel .config file and takes into account only the ending that results from it. Although in theory correct, in practice it fails - see above.

2. There is no possibility to add video modules in genkernel.conf - unless, possibly, by adding them to a group they do not belong, e.g.

AMODULES_NET='nvidia nvidia-modeset nvidia-uvm'

This is a) misleading (video modules in the NET group?) and b) will not work, because of 1), i.e. the modules will be present as nvidia.ko etc. but genkernel-next will be looking for nvidia.ko.gz etc.


Solution
--------

You have to give the user the chance to define the module extensions present in his system - there is no other way around it, except, perhaps, searching the kernel tree for all known extensions after '.ko' in filenames.

The attached patches do exactly that, the attached ebuild is the same as the one for version 68 - it just applies the patches too.
Comment 2 segmentation fault 2019-02-07 13:58:49 UTC
Created attachment 564038 [details, diff]
Patch for gen_initramfs.sh and gen_moddeps.sh
Comment 3 segmentation fault 2019-02-07 14:00:49 UTC
Created attachment 564040 [details, diff]
Patch for modules_load in arch/x86_64
Comment 4 segmentation fault 2019-02-07 14:02:02 UTC
Created attachment 564042 [details]
ebuild that uses the above two patches
Comment 5 segmentation fault 2019-02-07 14:35:35 UTC
O.K. let's see what the above ebuild and patches do in detail...

The easy part first:

The ebuild is just a copy of 

genkernel-next-68.ebuild

with just the extra lines:

PATCHES=(
    "${FILESDIR}"/${PN}-${PV}-gen_initramfs-gen_moddeps.patch
    "${FILESDIR}"/${PN}-${PV}-modules_load.patch
)

i.e. it works the same plus it applies the above patches.

The modules_load patch is also very simple: it adds a new modules group, VIDEO:

# Video
MODULES_VIDEO="nvidia nvidia-modeset nvidia-drm nvidia-uvm"

I have put the modules I need there, but for general use, you might want to leave MODULES_VIDEO just empty and instruct the user to add the modules he needs through the AMODULES mechanism in genkernel.conf, i.e. add a line 

AMODULES_VIDEO="nvidia nvidia-modeset nvidia-drm nvidia-uvm"

in /etc/genkernel.conf. But without a MODULES_VIDEO line in modules_load, AMODULES_VIDEO in /etc/genkernel.conf will not work.

Now on to the main patch, genkernel-next-68-gen_initramfs-gen_moddeps.patch:

I introduce a new loop

for KEXT in ${MOD_EXT_LIST}; do
...

which means: 'for each extension in MOD_EXT_LIST, do...'. That is, the user is supposed to enter the list of module extensions (endings) in the variable MOD_EXT_LIST in genkernel.conf.  For example, I now have in my /etc/genkernel.conf:

# List of module extensions to search for.
# MOD_EXT_LIST='.ko .ko.gz .ko.xz'
MOD_EXT_LIST='.ko .ko.gz'

Theoretically, you could put in as many extensions as you like (as you can see in the commented line) - I need .ko and .ko.gz, so I enter them there.

The rest of the patch is actually work needed in order to go through the extensions in MOD_EXT_LIST, find modules with that extension (ending) and add them to whatever list we process.

Example:

    for i in $(gen_dep_list)
    do
        for KEXT in ${MOD_EXT_LIST}; do
            mymod=`find ./lib/modules/${KV} -name "${i}${KEXT}" 2>/dev/null| head -n 1`
            if [ -z "${mymod}" ]
            then
                print_warning 2 "Warning :: ${i}${KEXT} not found; skipping..."
                continue;
            fi

            print_info 2 "initramfs: >> Copying ${i}${KEXT}..."
            cp -ax --parents "${mymod}" "${TEMP}/initramfs-modules-${KV}-temp"
        done
    done


Now, certainly this is not the most efficient way to search through /lib/modules/${KV} for files with various endings from a list - one could concatenate the endings into a regular expression to pass to 'find'. I preferred to leave it as is  

a) for better readability, to understand the idea and not break anything on the way (KISS principle)

b) because it does not take *that* much time to loop once (or twice) again - after the first loop, the files are probably all in the file cache already. YMMV,

One last note: I have replaced 

for i in `gen_dep_list`

with

for i in $(gen_dep_list)

Looking at it now, this shouldn't be strictly necessary (as it looks irrelevant to our problem), but I do remember that I printed $i and got the literal 'gen_dep_list'! Please do your checks at that point, just in case...
Comment 6 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-08-20 12:51:04 UTC
Package removed.