Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 922663 - sys-kernel/genkernel-4.3.10 modinfo error
Summary: sys-kernel/genkernel-4.3.10 modinfo error
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Hosted Projects
Classification: Unclassified
Component: genkernel (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Genkernel Maintainers
URL:
Whiteboard:
Keywords: PATCH
Depends on:
Blocks:
 
Reported: 2024-01-22 03:56 UTC by Rin Cat
Modified: 2024-05-22 15:20 UTC (History)
8 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
moddeps (moddeps,2.20 KB, text/plain)
2024-01-22 16:33 UTC, Rin Cat
Details
genkernel.log (genkernel.log,96.71 KB, text/plain)
2024-01-22 16:36 UTC, Rin Cat
Details
genkernel.log from 6.7.1-r1 build (gzipped) (671r1.genkernel.log.gz,435.73 KB, application/gzip)
2024-01-22 18:40 UTC, BJP
Details
modinfo output saved with >& to include errors (modinfo-output.txt,20.65 KB, text/plain)
2024-01-22 22:30 UTC, BJP
Details
genkernel.conf (genkernel.conf,12.95 KB, text/plain)
2024-01-23 07:30 UTC, Rin Cat
Details
fix KEXT (draft) (kext-fix.patch,1.62 KB, patch)
2024-01-25 11:30 UTC, Dmitriy Baranov
Details | Diff
manually revised patch (modinfo.patch,1.62 KB, patch)
2024-01-26 00:51 UTC, Jens Koegler
Details | Diff
genkernel.conf Computer A (genkernel.conf,1.16 KB, text/plain)
2024-01-26 22:39 UTC, Jens Koegler
Details
genkernel.conf Computer B (genkernel.conf,1.31 KB, text/plain)
2024-01-26 22:41 UTC, Jens Koegler
Details
Fix modinfo error patch (Fix-modinfo-error.patch,6.51 KB, patch)
2024-03-06 09:13 UTC, Dmitriy Baranov
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Rin Cat 2024-01-22 03:56:21 UTC
After updating from 4.3.6 to 4.3.10, I got a "modinfo error!" while creating initramfs, which seems to be firmware related. But no configuration was changed in the genkernel. However, the kernel was upgraded from 6.1 to 6.6.

Reproducible: Always




*         >> Appending modules cpio data ...
* Checking for binpkg(s) required for kmod-30 (L0) ...
* > zlib-1.2.13 has no dependencies. (L1)
* > GKBUILD '/usr/share/genkernel/gkbuilds/zlib-1.2.13.gkbuild' does NOT exist; Skipping ...
* > Existing zlib-1.2.13 binpkg is newer than '/usr/share/genkernel/gkbuilds/zlib.gkbuild'; Skipping ...
* > Can keep using existing zlib-1.2.13 binpkg from '/var/cache/genkernel/4.3.10/zlib-1.2.13-x86_64.tar.xz'!
* > Binpkg of zlib-1.2.13 is ready; Adding to list 'CHECK_L0_REQUIRED_BINPKGS' ...
* > xz-5.4.3 has no dependencies. (L1)
* > GKBUILD '/usr/share/genkernel/gkbuilds/xz-5.4.3.gkbuild' does NOT exist; Skipping ...
* > Existing xz-5.4.3 binpkg is newer than '/usr/share/genkernel/gkbuilds/xz.gkbuild'; Skipping ...
* > Can keep using existing xz-5.4.3 binpkg from '/var/cache/genkernel/4.3.10/xz-5.4.3-x86_64.tar.xz'!
* > Binpkg of xz-5.4.3 is ready; Adding to list 'CHECK_L0_REQUIRED_BINPKGS' ...
* > zstd-1.5.5 has no dependencies. (L1)
* > GKBUILD '/usr/share/genkernel/gkbuilds/zstd-1.5.5.gkbuild' does NOT exist; Skipping ...
* > Existing zstd-1.5.5 binpkg is newer than '/usr/share/genkernel/gkbuilds/zstd.gkbuild'; Skipping ...
* > Can keep using existing zstd-1.5.5 binpkg from '/var/cache/genkernel/4.3.10/zstd-1.5.5-x86_64.tar.xz'!
* > Binpkg of zstd-1.5.5 is ready; Adding to list 'CHECK_L0_REQUIRED_BINPKGS' ...
* GKBUILD '/usr/share/genkernel/gkbuilds/kmod-30.gkbuild' does NOT exist; Skipping ...
* Existing kmod-30 binpkg is newer than '/usr/share/genkernel/gkbuilds/kmod.gkbuild'; Skipping ...
* Existing kmod-30 binpkg is newer than 'zlib-1.2.13-x86_64.tar.xz'; Skipping ...
* Existing kmod-30 binpkg is newer than 'xz-5.4.3-x86_64.tar.xz'; Skipping ...
* Existing kmod-30 binpkg is newer than 'zstd-1.5.5-x86_64.tar.xz'; Skipping ...
* Existing kmod-30 binpkg is newer than '/usr/share/genkernel/patches/kmod/30/kmod-29-static.patch'; Skipping ...
* Can keep using existing kmod-30 binpkg from '/var/cache/genkernel/4.3.10/kmod-30-x86_64.tar.xz'!
*                 kmod: >> Using kmod-30 binpkg ...
*                 modules: >> Searching modules in '/lib/modules/6.6.13-x86_64' ...
* ERROR: modinfo error!
* Please consult '/var/log/genkernel.log' for more information and any
* errors that were reported above.
*
Comment 1 Dmitriy Baranov 2024-01-22 14:14:45 UTC
Do you have any modules in /lib/modules/6.6.13-x86_64 ?
Can you show /var/tmp/genkernel/gk_*/moddeps ?
Comment 2 Rin Cat 2024-01-22 15:47:07 UTC
(In reply to Dmitry Baranov from comment #1)
> Do you have any modules in /lib/modules/6.6.13-x86_64 ?
> Can you show /var/tmp/genkernel/gk_*/moddeps ?

# ls /lib/modules/6.6.13-x86_64
./  ../  build@  kernel/  modules.alias  modules.alias.bin  modules.builtin  modules.builtin.alias.bin  modules.builtin.bin  modules.builtin.modinfo  modules.dep  modules.dep.bin  modules.devname  modules.order  modules.softdep  modules.symbols  modules.symbols.bin  source@

However, /var/tmp/genkernel/gk_*/moddeps does not exist.

# find /var/tmp/genkernel/
/var/tmp/genkernel/
/var/tmp/genkernel/initramfs-x86_64-6.6.13-x86_64
Comment 3 Dmitriy Baranov 2024-01-22 16:01:56 UTC
(In reply to Rin Cat from comment #2)
> However, /var/tmp/genkernel/gk_*/moddeps does not exist.

genkernel --no-cleanup --no-postclear --no-lvm --no-luks ramdisk
Comment 4 Ben Kohler gentoo-dev 2024-01-22 16:30:13 UTC
Can you share the full genkernel.log?
Comment 5 Rin Cat 2024-01-22 16:33:29 UTC
Created attachment 882858 [details]
moddeps
Comment 6 Rin Cat 2024-01-22 16:34:34 UTC
(In reply to Dmitry Baranov from comment #3)
> (In reply to Rin Cat from comment #2)
> > However, /var/tmp/genkernel/gk_*/moddeps does not exist.
> 
> genkernel --no-cleanup --no-postclear --no-lvm --no-luks ramdisk

This command had a long list in the moddeps. But the command I use shows this file is empty.

genkernel initramfs --kernel-config=/usr/src/linux/.config --loglevel=5 --no-cleanup --no-postclear
Comment 7 Rin Cat 2024-01-22 16:36:40 UTC
Created attachment 882859 [details]
genkernel.log
Comment 8 Rin Cat 2024-01-22 16:37:09 UTC
(In reply to Ben Kohler from comment #4)
> Can you share the full genkernel.log?

Added
Comment 9 Ben Kohler gentoo-dev 2024-01-22 16:56:21 UTC
Is /lib/modules/6.6.13-x86_64 fully populated?  I'm guessing not
Comment 10 Rin Cat 2024-01-22 17:03:41 UTC
(In reply to Ben Kohler from comment #9)
> Is /lib/modules/6.6.13-x86_64 fully populated?  I'm guessing not

It has the same files count of current running kernel. So I think it is fully populated.

# find /lib/modules/6.6.13-x86_64/|wc -l
1790
               
# find /lib/modules/6.1.67-x86_64/|wc -l
1790

Also the modinfo seems fine when running it manually.

# modinfo /lib/modules/6.6.13-x86_64/kernel/fs/btrfs/btrfs.ko
filename:       /lib/modules/6.6.13-x86_64/kernel/fs/btrfs/btrfs.ko
alias:          fs-btrfs
alias:          char-major-10-234
alias:          devname:btrfs-control
license:        GPL
softdep:        pre: crc32c
softdep:        pre: xxhash64
softdep:        pre: sha256
softdep:        pre: blake2b-256
vermagic:       6.6.13-x86_64 SMP mod_unload modversions 
name:           btrfs
intree:         Y
retpoline:      Y
depends:        lzo_compress
srcversion:     6BA9E1F41410AEE48430752
sig_id:         PKCS#7
signer:         
sig_key:        
sig_hashalgo:   unknown
signature:
Comment 11 BJP 2024-01-22 18:27:01 UTC
I can confirm that I see the same modinfo error reported on my system today after installing genkernel-4.3.10, while building kernel 6.7.1-r1.
Comment 12 Ben Kohler gentoo-dev 2024-01-22 18:36:39 UTC
(In reply to BJP from comment #11)
> I can confirm that I see the same modinfo error reported on my system today
> after installing genkernel-4.3.10, while building kernel 6.7.1-r1.

Can you share your genkernel.log as well?
Comment 13 BJP 2024-01-22 18:40:03 UTC
Created attachment 882863 [details]
genkernel.log from 6.7.1-r1 build (gzipped)
Comment 14 BJP 2024-01-22 18:42:53 UTC
(In reply to Ben Kohler from comment #12)
> (In reply to BJP from comment #11)
> > I can confirm that I see the same modinfo error reported on my system today
> > after installing genkernel-4.3.10, while building kernel 6.7.1-r1.
> 
> Can you share your genkernel.log as well?

Attached (sorry for the verbose log). In case it is relevant - I discarded the new /etc/genkernel.conf file to be installed with the genkernel update and kept my previously working config, so if there are any config file changes required for the new version I may not have implemented them.
Comment 15 Ben Kohler gentoo-dev 2024-01-22 19:02:02 UTC
Most likely this commit broke it but I'm not exactly sure what is going wrong:
https://github.com/gentoo/genkernel/commit/08b4f191c9fb064f8564d888e3969a02b0384a32
Comment 16 Dmitriy Baranov 2024-01-22 19:41:37 UTC
Looks like some missing module in moddeps, but not sure. I'll take a look soon.
Comment 17 Dmitriy Baranov 2024-01-22 19:58:01 UTC
(In reply to Rin Cat from comment #5)
> Created attachment 882858 [details]
> moddeps

You can try to find missing module by command:

modinfo -k "6.6.13-x86_64" -F filename mod_from_moddeps
Comment 18 Ben Kohler gentoo-dev 2024-01-22 20:04:28 UTC
(In reply to BJP from comment #14)
> (In reply to Ben Kohler from comment #12)
> > (In reply to BJP from comment #11)
> > > I can confirm that I see the same modinfo error reported on my system today
> > > after installing genkernel-4.3.10, while building kernel 6.7.1-r1.
> > 
> > Can you share your genkernel.log as well?
> 
> Attached (sorry for the verbose log). In case it is relevant - I discarded
> the new /etc/genkernel.conf file to be installed with the genkernel update
> and kept my previously working config, so if there are any config file
> changes required for the new version I may not have implemented them.

Can you try this? 

modinfo -b '' -k 6.7.1-gentoo-r1-x86_64 -F filename $(cat /var/tmp/genkernel/gk_5SAYdB3b/moddeps)

And see if it produces errors, or a nice list of module paths?
Comment 19 Rin Cat 2024-01-22 20:07:02 UTC
(In reply to Dmitry Baranov from comment #17)
> (In reply to Rin Cat from comment #5)
> > Created attachment 882858 [details]
> > moddeps
> 
> You can try to find missing module by command:
> 
> modinfo -k "6.6.13-x86_64" -F filename mod_from_moddeps

The file /var/tmp/genkernel/gk_XXXXX/moddeps is empty for me. So I am not sure what happened.
Comment 20 BJP 2024-01-22 20:20:17 UTC
(In reply to Ben Kohler from comment #18)
> (In reply to BJP from comment #14)
> > (In reply to Ben Kohler from comment #12)
> > > (In reply to BJP from comment #11)
> > > > I can confirm that I see the same modinfo error reported on my system today
> > > > after installing genkernel-4.3.10, while building kernel 6.7.1-r1.
> > > 
> > > Can you share your genkernel.log as well?
> > 
> > Attached (sorry for the verbose log). In case it is relevant - I discarded
> > the new /etc/genkernel.conf file to be installed with the genkernel update
> > and kept my previously working config, so if there are any config file
> > changes required for the new version I may not have implemented them.
> 
> Can you try this? 
> 
> modinfo -b '' -k 6.7.1-gentoo-r1-x86_64 -F filename $(cat
> /var/tmp/genkernel/gk_5SAYdB3b/moddeps)
> 
> And see if it produces errors, or a nice list of module paths?

genkernel wiped out the /var/tmp/genkernel/gk_* directory after my build attempt. I tried again with POSTCLEAR="no" and it did the same thing. Is there a commandline flag or config file setting that will prevent this so I can provide the result of this command?
Comment 21 Dmitriy Baranov 2024-01-22 20:22:16 UTC
(In reply to Rin Cat from comment #6)
> (In reply to Dmitry Baranov from comment #3)
> > (In reply to Rin Cat from comment #2)
> > > However, /var/tmp/genkernel/gk_*/moddeps does not exist.
> > 
> > genkernel --no-cleanup --no-postclear --no-lvm --no-luks ramdisk
> 
> This command had a long list in the moddeps. But the command I use shows
> this file is empty.
> 
> genkernel initramfs --kernel-config=/usr/src/linux/.config --loglevel=5
> --no-cleanup --no-postclear

Sorry, I haven't noticed this. Moddeps shouldn't be empty. I guess there is wrong path to modules dir or smth in the gen_moddeps.sh. in case "--kernel-config".
Comment 22 Rin Cat 2024-01-22 21:54:50 UTC
(In reply to Dmitry Baranov from comment #21)
> (In reply to Rin Cat from comment #6)
> > (In reply to Dmitry Baranov from comment #3)
> > > (In reply to Rin Cat from comment #2)
> > > > However, /var/tmp/genkernel/gk_*/moddeps does not exist.
> > > 
> > > genkernel --no-cleanup --no-postclear --no-lvm --no-luks ramdisk
> > 
> > This command had a long list in the moddeps. But the command I use shows
> > this file is empty.
> > 
> > genkernel initramfs --kernel-config=/usr/src/linux/.config --loglevel=5
> > --no-cleanup --no-postclear
> 
> Sorry, I haven't noticed this. Moddeps shouldn't be empty. I guess there is
> wrong path to modules dir or smth in the gen_moddeps.sh. in case
> "--kernel-config".

Without "--kernel-config", which using the default config file, it works. But that may break the initramfs due to different configuration.
Comment 23 BJP 2024-01-22 22:30:01 UTC
Created attachment 882904 [details]
modinfo output saved with >& to include errors
Comment 24 BJP 2024-01-22 22:30:20 UTC
(In reply to BJP from comment #20)
> (In reply to Ben Kohler from comment #18)
> > (In reply to BJP from comment #14)
> > > (In reply to Ben Kohler from comment #12)
> > > > (In reply to BJP from comment #11)
> > > > > I can confirm that I see the same modinfo error reported on my system today
> > > > > after installing genkernel-4.3.10, while building kernel 6.7.1-r1.
> > > > 
> > > > Can you share your genkernel.log as well?
> > > 
> > > Attached (sorry for the verbose log). In case it is relevant - I discarded
> > > the new /etc/genkernel.conf file to be installed with the genkernel update
> > > and kept my previously working config, so if there are any config file
> > > changes required for the new version I may not have implemented them.
> > 
> > Can you try this? 
> > 
> > modinfo -b '' -k 6.7.1-gentoo-r1-x86_64 -F filename $(cat
> > /var/tmp/genkernel/gk_5SAYdB3b/moddeps)
> > 
> > And see if it produces errors, or a nice list of module paths?

I tried again with the --no-cleanup and --no-postclear commandline options and I was able to run this. I've attached the output as modinfo-output.txt.

It returns errors to stderr and module paths to stdout, with an exit status code of 1.
Comment 25 Dmitriy Baranov 2024-01-23 07:27:34 UTC
(In reply to Rin Cat from comment #22)
> Without "--kernel-config", which using the default config file, it works.
> But that may break the initramfs due to different configuration.

Can you please publish genkernel.conf? Or:

cat /etc/genkernel.conf | grep "FIRMWARE\|RAMDISK"
Comment 26 Rin Cat 2024-01-23 07:30:17 UTC
Created attachment 882910 [details]
genkernel.conf
Comment 27 Dmitriy Baranov 2024-01-23 10:44:29 UTC
(In reply to BJP from comment #24)
> I tried again with the --no-cleanup and --no-postclear commandline options
> and I was able to run this. I've attached the output as modinfo-output.txt.
> 
> It returns errors to stderr and module paths to stdout, with an exit status
> code of 1.

Can you please publish genkernel.conf? Or:

cat /etc/genkernel.conf | grep "FIRMWARE\|RAMDISK"
Comment 28 Dmitriy Baranov 2024-01-23 11:47:00 UTC
(In reply to Rin Cat from comment #6)
> (In reply to Dmitry Baranov from comment #3)
> > (In reply to Rin Cat from comment #2)
> > > However, /var/tmp/genkernel/gk_*/moddeps does not exist.
> > 
> > genkernel --no-cleanup --no-postclear --no-lvm --no-luks ramdisk
> 
> This command had a long list in the moddeps. But the command I use shows
> this file is empty.
> 
> genkernel initramfs --kernel-config=/usr/src/linux/.config --loglevel=5
> --no-cleanup --no-postclear

What do you expect with custom kernel-config on initramfs gen? What differences should there be without this option? Also I still can't understand how it brokes ramdisk build.
Comment 29 Dmitriy Baranov 2024-01-23 13:39:06 UTC
(In reply to Ben Kohler from comment #15)
> Most likely this commit broke it but I'm not exactly sure what is going
> wrong:
> https://github.com/gentoo/genkernel/commit/
> 08b4f191c9fb064f8564d888e3969a02b0384a32

I found the problem, but it's not in the commit above directly:

https://github.com/gentoo/genkernel/blob/d6a77d90fd511b04b12bd7ae40d710d3d144c077/gen_moddeps.sh#L63

$KEXT is equal ".ko.xz", but it should be just ".ko" (because only ".ko" in modules.dep). I have no idea how it worked before my changes... And I'm not sure how to fix this properly. We need the dev of this stuff:

https://github.com/gentoo/genkernel/blob/d6a77d90fd511b04b12bd7ae40d710d3d144c077/gen_determineargs.sh#L4-L37
Comment 30 BJP 2024-01-23 13:42:28 UTC
(In reply to Dmitry Baranov from comment #27)
> (In reply to BJP from comment #24)
> > I tried again with the --no-cleanup and --no-postclear commandline options
> > and I was able to run this. I've attached the output as modinfo-output.txt.
> > 
> > It returns errors to stderr and module paths to stdout, with an exit status
> > code of 1.
> 
> Can you please publish genkernel.conf? Or:
> 
> cat /etc/genkernel.conf | grep "FIRMWARE\|RAMDISK"

xps ~ # cat /etc/genkernel.conf | grep "FIRMWARE\|RAMDISK"
#FIRMWARE_INSTALL="no"
FIRMWARE="yes"
#FIRMWARE_DIR="/lib/firmware"
# relative to FIRMWARE_DIR.  If empty or unset, the full contents of 
# FIRMWARE_DIR will be included (if FIRMWARE option above is set to YES).
#FIRMWARE_FILES=""
FIRMWARE_FILES="iwlwifi-QuZ-a0-hr-b0-77.ucode,iwlwifi-QuZ-a0-hr-b0-74.ucode,iwlwifi-QuZ-a0-hr-b0-73.
ucode,iwlwifi-QuZ-a0-hr-b0-72.ucode,iwlwifi-QuZ-a0-hr-b0-66.ucode,regulatory.db,regulatory.db.p7s,i9
15/kbl_dmc_ver1_04.bin,intel/ibt-19-0-4.sfi,intel/ibt-19-0-4.ddc"
ALLRAMDISKMODULES="yes"
#RAMDISKMODULES="yes"
Comment 31 Dmitriy Baranov 2024-01-23 13:59:19 UTC
(In reply to BJP from comment #30)
> xps ~ # cat /etc/genkernel.conf | grep "FIRMWARE\|RAMDISK"
> #FIRMWARE_INSTALL="no"
> FIRMWARE="yes"
> #FIRMWARE_DIR="/lib/firmware"
> # relative to FIRMWARE_DIR.  If empty or unset, the full contents of 
> # FIRMWARE_DIR will be included (if FIRMWARE option above is set to YES).
> #FIRMWARE_FILES=""
> FIRMWARE_FILES="iwlwifi-QuZ-a0-hr-b0-77.ucode,iwlwifi-QuZ-a0-hr-b0-74.ucode,
> iwlwifi-QuZ-a0-hr-b0-73.
> ucode,iwlwifi-QuZ-a0-hr-b0-72.ucode,iwlwifi-QuZ-a0-hr-b0-66.ucode,regulatory.
> db,regulatory.db.p7s,i9
> 15/kbl_dmc_ver1_04.bin,intel/ibt-19-0-4.sfi,intel/ibt-19-0-4.ddc"
> ALLRAMDISKMODULES="yes"
> #RAMDISKMODULES="yes"

Ok. Your case is here:

https://github.com/gentoo/genkernel/blob/d6a77d90fd511b04b12bd7ae40d710d3d144c077/gen_moddeps.sh#L20

moddeps is not empty, but the basename doesn't cut the .ko.xz.
BTW, you can use just FIRMWARE="yes" without FIRMWARE_FILES.
Comment 32 BJP 2024-01-23 18:04:26 UTC
(In reply to Dmitry Baranov from comment #31)
> (In reply to BJP from comment #30)
> > xps ~ # cat /etc/genkernel.conf | grep "FIRMWARE\|RAMDISK"
> > #FIRMWARE_INSTALL="no"
> > FIRMWARE="yes"
> > #FIRMWARE_DIR="/lib/firmware"
> > # relative to FIRMWARE_DIR.  If empty or unset, the full contents of 
> > # FIRMWARE_DIR will be included (if FIRMWARE option above is set to YES).
> > #FIRMWARE_FILES=""
> > FIRMWARE_FILES="iwlwifi-QuZ-a0-hr-b0-77.ucode,iwlwifi-QuZ-a0-hr-b0-74.ucode,
> > iwlwifi-QuZ-a0-hr-b0-73.
> > ucode,iwlwifi-QuZ-a0-hr-b0-72.ucode,iwlwifi-QuZ-a0-hr-b0-66.ucode,regulatory.
> > db,regulatory.db.p7s,i9
> > 15/kbl_dmc_ver1_04.bin,intel/ibt-19-0-4.sfi,intel/ibt-19-0-4.ddc"
> > ALLRAMDISKMODULES="yes"
> > #RAMDISKMODULES="yes"
> 
> Ok. Your case is here:
> 
> https://github.com/gentoo/genkernel/blob/
> d6a77d90fd511b04b12bd7ae40d710d3d144c077/gen_moddeps.sh#L20
> 
> moddeps is not empty, but the basename doesn't cut the .ko.xz.
> BTW, you can use just FIRMWARE="yes" without FIRMWARE_FILES.

It's not clear to me if you're asking me to test a change at this line to see if it helps. If there is, please let me know and I can try.

I'm not sure if it makes a difference or not, but I have kernel modules that end in both .ko and .ko.xz. In-kernel modules are compressed to .ko.xz but external modules (r8168, nvidia-drivers) are not compressed and are left as .ko.
Comment 33 Dmitriy Baranov 2024-01-23 20:37:51 UTC
(In reply to BJP from comment #32)
> I'm not sure if it makes a difference or not, but I have kernel modules that
> end in both .ko and .ko.xz. In-kernel modules are compressed to .ko.xz but
> external modules (r8168, nvidia-drivers) are not compressed and are left as
> .ko.

Can you provide full and clear info? How was compiled "In-kernel" modules genkernel/dist-kernel? What contains modules.dep - both or one extension type? I don't see any .ko.xz, even with CONFIG_MODULE_COMPRESS_XZ=y (sys-kernel/gentoo-kernel).
Comment 34 BJP 2024-01-23 20:47:18 UTC
(In reply to Dmitry Baranov from comment #33)
> (In reply to BJP from comment #32)
> > I'm not sure if it makes a difference or not, but I have kernel modules that
> > end in both .ko and .ko.xz. In-kernel modules are compressed to .ko.xz but
> > external modules (r8168, nvidia-drivers) are not compressed and are left as
> > .ko.
> 
> Can you provide full and clear info? How was compiled "In-kernel" modules
> genkernel/dist-kernel? What contains modules.dep - both or one extension
> type? I don't see any .ko.xz, even with CONFIG_MODULE_COMPRESS_XZ=y
> (sys-kernel/gentoo-kernel).

I use CONFIG_MODULE_COMPRESS_XZ=y in my kernel .config (sys-kernel/gentoo-sources-6.1.7-r1) and compile the kernel with "genkernel all". This results in compressed .ko.xz modules in /lib/modules/6.7.1-gentoo-r1-x86_64.

I also have two external modules (net-misc/r8168, x11-drivers/nvidia-drivers) that are compiled automatically by genkernel during the "Compiling out-of-tree module(s)" step. I did not have the 'modules-compress' USE flag set on these packages. They produced .ko files, uncompressed, in /lib/modules/6.7.1-gentoo-r1-x86_64.

However with that said this turns out to be irrelevant because I tried again with USE=modules-compress set and it did not change the "modinfo error!" from genkernel. Now all modules are compressed and have a .ko.xz suffix.

modules.dep contains the correct strings in all cases.
Comment 35 Jens Koegler 2024-01-24 21:09:39 UTC
I can also confirm that the bug occurs on all the servers and workstations I operate (a little under 100). Unfortunately not only since the last genkernel version but already with an earlier unstable gentoo-sources update. The log only repeats the error "modinfo error!" in the standard settings.

The workaround to get a working RamDisk after all is to select the option "Module compression mode (None)" in the kernel configuration. Neither zstd nor gzip were successful. Additional info: all modules are signed with:

CONFIG_MODULE_SIG_SHA512=y
CONFIG_MODULE_SIG_HASH="sha512"
Comment 36 Jens Koegler 2024-01-24 21:38:07 UTC
I took another quick look at the logs of various productive virtual machines:
The combination genkernel-4.3.6 with gentoo-sources:6.1.67 with modules-compress zstd but without microcodes update still worked.

I also remember that I still have to run a manual 'depmod -a' after the reboot because of the new kernel. Otherwise the new modules are not available after the reboot. Perhaps this information will help to narrow down the error further.
Comment 37 Dmitriy Baranov 2024-01-25 07:21:23 UTC
I won't be able to fix this quickly in the next week.
Possible solution: get rid $KEXT in gen_moddeps.sh.
Another solution: create the common extension regexp: ".ko(.xz|.etc)?".
PR is welcome.
Comment 38 Dmitriy Baranov 2024-01-25 11:30:37 UTC
Created attachment 883090 [details, diff]
fix KEXT (draft)
Comment 39 Jens Koegler 2024-01-26 00:46:47 UTC
Unfortunately I get a reject in version 4.3.10 with your patch. I was free enough to install it manually and apply it via the user-patch option. On one system I cleaned everything in the kernel sources with "make distclean" and selected zstd as the compression method. And what can I say: it was successful. Thank you very much!


There is still a small limitation. No external modules such as zfs/nvidia/lkrg were/are installed on the system.


PS: Since I can't see so well anymore, I recommend checking my patch again
Comment 40 Jens Koegler 2024-01-26 00:51:08 UTC
Created attachment 883135 [details, diff]
manually revised patch
Comment 41 Dmitriy Baranov 2024-01-26 08:57:37 UTC
(In reply to Jens Koegler from comment #39)
> Unfortunately I get a reject in version 4.3.10 with your patch. I was free
> enough to install it manually and apply it via the user-patch option.
Can you provide more reproduce steps?
What command did you use: genkernel all/ramdisk/initramfs?

> On one
> system I cleaned everything in the kernel sources with "make distclean" and
> selected zstd as the compression method. And what can I say: it was
> successful.
Same questions here.
Comment 42 BJP 2024-01-26 17:01:56 UTC
(In reply to Jens Koegler from comment #40)
> Created attachment 883135 [details, diff] [details, diff]
> manually revised patch

I applied the patch from comment #40 as a user patch against genkernel-4.3.10 and I can confirm that my full kernel compilation of kernel 6.7.2 with compressed modules, initrd, and EFI stub completed successfully and booted to a working system.

I have the following kernel config options set:

CONFIG_INITRAMFS_COMPRESSION_XZ=y
CONFIG_MODULE_COMPRESS_XZ=y

In /etc/genkernel.conf:

ALLFIRMWARE="NO"
FIRMWARE="YES"

In USE flags:

USE=modules-compress
Comment 43 Jens Koegler 2024-01-26 22:33:39 UTC
(In reply to Dmitry Baranov from comment #41)
> (In reply to Jens Koegler from comment #39)
> > Unfortunately I get a reject in version 4.3.10 with your patch. I was free
> > enough to install it manually and apply it via the user-patch option.
> Can you provide more reproduce steps?

Of course I can.


I got "patch hunk succeeded with fuzz" and a reject when patching


I simply inserted your patch line by line in the appropriate place or deleted lines compared to a copy. Then I created a new patch with diff -burN, implemented it via user patch, rebuilt genkernel. Then called genkernel. But here I already had an abort at "Compiling 6.x.x-gentoo bzImage". For this reason I did a "make distclean" on the sources. I know that I can also do this in genkernel.conf with CLEAN. But here it gets a bit more complicated with my machines. Some time ago I automated the whole process of compiling and rolling out the kernel with Puppet. The module has a total of four triggers for when genkernel should do its work:

-new genkernel.conf, implemented as a template

- new kernel.config, as a file depending on hiera, e.g. all VMware machines are built the same at work, the baremetals are special. At home, apart from the KVMs, this is more of a colorful zoo.

- new gentoo-sources directly via the module

new gentoo-sources, if these are installed outside of Puppet. This is normally the weekly @world update. Here I have written my own facter in Puppet.


Further information for completion: If genkernel is successful, I set adotfile, which then executes a reboot with cron once with a freely configurable time and after a reboot deletes the dotfile, executes eclean-kernel -n1 and cleans up the gentoo-sources with emerge --prune.
But since I don't necessarily customize a normally working configuration in Puppet, which I will most likely have to roll back, I went the manual way and first cleaned up the kernel with make distclean, make with the desired kernel configuration, make install, make modules, make modules_install manually compiled.
In a second step, I then called genkernel with the option all, as described below. Since there was no CLEAN, this went very quickly and the initramfs point was easily tested.

> What command did you use: genkernel all/ramdisk/initramfs?

 I always use the command line for genkernel in the automation:

genkernel --config=/etc/genkernel.conf --kernel-config=/root/kernel.config all


> > On one
> > system I cleaned everything in the kernel sources with "make distclean" and
> > selected zstd as the compression method. And what can I say: it was
> > successful.
> Same questions here.

I think I have already summarized the answer here in point 1

Here are the respective genkernel.conf for 2 machines, because I think these two are quite different
Comment 44 Jens Koegler 2024-01-26 22:39:58 UTC
Created attachment 883198 [details]
genkernel.conf Computer A
Comment 45 Jens Koegler 2024-01-26 22:41:43 UTC
Created attachment 883199 [details]
genkernel.conf Computer B
Comment 46 Dmitriy Baranov 2024-01-27 18:37:47 UTC
(In reply to Rin Cat from comment #6)
> (In reply to Dmitry Baranov from comment #3)
> > (In reply to Rin Cat from comment #2)
> > > However, /var/tmp/genkernel/gk_*/moddeps does not exist.
> > 
> > genkernel --no-cleanup --no-postclear --no-lvm --no-luks ramdisk
> 
> This command had a long list in the moddeps. But the command I use shows
> this file is empty.
> 
> genkernel initramfs --kernel-config=/usr/src/linux/.config --loglevel=5
> --no-cleanup --no-postclear

Hi. Can you please test a patch on your setup?
Comment 47 Rin Cat 2024-01-28 18:16:46 UTC
(In reply to Dmitry Baranov from comment #46)
> (In reply to Rin Cat from comment #6)
> > (In reply to Dmitry Baranov from comment #3)
> > > (In reply to Rin Cat from comment #2)
> > > > However, /var/tmp/genkernel/gk_*/moddeps does not exist.
> > > 
> > > genkernel --no-cleanup --no-postclear --no-lvm --no-luks ramdisk
> > 
> > This command had a long list in the moddeps. But the command I use shows
> > this file is empty.
> > 
> > genkernel initramfs --kernel-config=/usr/src/linux/.config --loglevel=5
> > --no-cleanup --no-postclear
> 
> Hi. Can you please test a patch on your setup?

The patch works for me.
Comment 48 Dmitriy Baranov 2024-02-02 09:31:49 UTC
PR: https://github.com/gentoo/genkernel/pull/57

Those interested can test:
git clone https://github.com/reagentoo/genkernel.git
git checkout modinfo-error
git format-patch HEAD~3
sudo mkdir -p /etc/portage/patches/sys-kernel/genkernel/
sudo cp -r ./*.patch /etc/portage/patches/sys-kernel/genkernel/
sudo emerge -1 genkernel
sudo genkernel ramdisk
Comment 49 BJP 2024-02-07 15:34:51 UTC
(In reply to Dmitry Baranov from comment #48)
> PR: https://github.com/gentoo/genkernel/pull/57
> 
> Those interested can test:
> git clone https://github.com/reagentoo/genkernel.git
> git checkout modinfo-error
> git format-patch HEAD~3
> sudo mkdir -p /etc/portage/patches/sys-kernel/genkernel/
> sudo cp -r ./*.patch /etc/portage/patches/sys-kernel/genkernel/
> sudo emerge -1 genkernel
> sudo genkernel ramdisk

The patch from comment #40 worked for me as a user patch, do you want me to test the patch from git as you have described here? Or are you looking for confirmation from other users? I will do so if it would be helpful.
Comment 50 Dmitriy Baranov 2024-02-28 07:55:13 UTC
(In reply to Dmitry Baranov from comment #48)
> PR: https://github.com/gentoo/genkernel/pull/57
> 
> Those interested can test:
> git clone https://github.com/reagentoo/genkernel.git
> git checkout modinfo-error
> git format-patch HEAD~3
> sudo mkdir -p /etc/portage/patches/sys-kernel/genkernel/
> sudo cp -r ./*.patch /etc/portage/patches/sys-kernel/genkernel/
> sudo emerge -1 genkernel
> sudo genkernel ramdisk

Without testing and acknowledge from local participants this PR will not be merged...
Comment 51 Dmitriy Baranov 2024-03-06 09:13:17 UTC
Created attachment 886788 [details, diff]
Fix modinfo error patch

The PR was updated: https://github.com/gentoo/genkernel/pull/57
Please test it on your configurations.
Note:
added support for compressed modules/firmwares
auto use kernel-config from packaged kernels (sys-kernel/gentoo-kernel)
squashed patch attached
Comment 52 val 2024-03-29 20:41:28 UTC
Updated patch worked for me
Comment 53 Larry the Git Cow gentoo-dev 2024-05-20 12:59:12 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/genkernel.git/commit/?id=56c89013f80111143170e62c91dddd55184d6cdb

commit 56c89013f80111143170e62c91dddd55184d6cdb
Author:     Dmitriy Baranov <reagentoo@gmail.com>
AuthorDate: 2024-03-05 14:24:09 +0000
Commit:     Ben Kohler <bkohler@gentoo.org>
CommitDate: 2024-05-20 12:58:59 +0000

    gen_moddeps.sh: use KEXT along with default '.ko' extension to prevent modinfo error
    
    Also improved gen_dep_list() and get rid xbasename()
    
    Bug: https://bugs.gentoo.org/922663
    Closes: #57
    Signed-off-by: Dmitriy Baranov <reagentoo@gmail.com>
    Signed-off-by: Ben Kohler <bkohler@gentoo.org>

 gen_moddeps.sh | 33 +++++++++++++--------------------
 1 file changed, 13 insertions(+), 20 deletions(-)
Comment 54 Larry the Git Cow gentoo-dev 2024-05-22 15:20:09 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=63fb9e80f5d9b3d3bd9e07f239a1f9f355f1c053

commit 63fb9e80f5d9b3d3bd9e07f239a1f9f355f1c053
Author:     Ben Kohler <bkohler@gentoo.org>
AuthorDate: 2024-05-22 15:00:54 +0000
Commit:     Ben Kohler <bkohler@gentoo.org>
CommitDate: 2024-05-22 15:20:00 +0000

    sys-kernel/genkernel: add 4.3.15
    
    This version contains new bundled package versions plus some misc
    genkernel code fixes.  Some of the bundled package versions will fix
    known bugs.
    
    Closes: https://bugs.gentoo.org/932397
    Closes: https://bugs.gentoo.org/931324
    Closes: https://bugs.gentoo.org/928573
    Closes: https://bugs.gentoo.org/922663
    
    Signed-off-by: Ben Kohler <bkohler@gentoo.org>

 sys-kernel/genkernel/Manifest                      |   1 +
 ...el-4.3.15-fix-srcdir-for-new-bcache-tools.patch |  26 ++
 .../files/genkernel-4.3.15-mdadm-musl-fix.patch    |  14 +
 sys-kernel/genkernel/genkernel-4.3.15.ebuild       | 282 +++++++++++++++++++++
 4 files changed, 323 insertions(+)