Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 910597 - sys-kernel/gentoo-sources-6.4.4 does not install Intel microcode prepended to initrd
Summary: sys-kernel/gentoo-sources-6.4.4 does not install Intel microcode prepended to...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: AMD64 Linux
: Normal normal
Assignee: Gentoo Kernel Bug Wranglers and Kernel Maintainers
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-07-20 18:40 UTC by Stefan Trenker
Modified: 2023-07-23 22:01 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
dmesg 6.4.4 (dmesg-6.4.4,84.06 KB, application/x-troff-man)
2023-07-20 21:18 UTC, Stefan Trenker
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Trenker 2023-07-20 18:40:03 UTC
It seems that in sys-kernel/gentoo-sources-6.4.3 and 6.4.4 (this are the kernels i checked) Intel microcode is not installed.

I double checked the uncompressed microcode cpio is prepended to the compresed initrd-file:

# cpio -tv <initrd 
drwxr-xr-x   3 root     root            0 Jul 20 20:17 .
-rw-r--r--   1 root     root            2 Jul 20 20:17 early_cpio
drwxr-xr-x   3 root     root            0 Jul 20 20:17 kernel
drwxr-xr-x   3 root     root            0 Jul 20 20:17 kernel/x86
drwxr-xr-x   2 root     root            0 Jul 20 20:17 kernel/x86/microcode
-rw-r--r--   1 root     root       110592 Jul 20 20:17 kernel/x86/microcode/GenuineIntel.bin
218 Blöcke

but dmesg | grep -i microcde does not show any installed Intel microcode:

# dmesg | grep -i microcode
[    0.935381] microcode: Microcode Update Driver: v2.2.

Just the prompt of the microcode upload driver.

Booting e.g. 6.1.39 dmesg lists the microcode patch as expected.

When I compare the source-code of the microcode update module from 6.1.39 with 6.4.4 then i see the new kernel module code received a lot of rearranging and rewriting. Maybe the authors missed something?

Reproducible: Always
Comment 1 Stefan Trenker 2023-07-20 18:45:01 UTC
Unfortunately i do not know any means to check if microcode is properly loaded other than checking in dmesg output.
Comment 2 Mike Pagano gentoo-dev 2023-07-20 20:11:59 UTC
How are you building the initrd?
Are you using grub ? If os, do you have GRUB_EARLY_INITRD_LINUX_CUSTOM defined in /etc/default/grub ?
Comment 3 Stefan Trenker 2023-07-20 20:37:34 UTC
(In reply to Mike Pagano from comment #2)
> How are you building the initrd?
> Are you using grub ? If os, do you have GRUB_EARLY_INITRD_LINUX_CUSTOM
> defined in /etc/default/grub ?

I use dracut to build the initrd
My bootloader is systemd-boot since grub is a nightmare when you use secure boot with individual signing keys.

I use a set of ansible scripts to update, build, sign, and install my kernels. By this means all kernels are build the same way. Starting with 6.4.3/6.4.4 there is no evidence in dmesg output, that intel microcode is loaded even when it is prepended to the initrd build by dracut.

this is my dracut config:

compress=zstd
DRACUTMODULES=(
        base
        crypt
        dracut-systemd
        fs-lib
        i18n
        kernel-modules
        kernel-modules-extra
        lvm
        qemu
        resume
        rootfs-block
        shutdown
        systemd
        systemd-initrd
        terminfo
        udev-rules
        usrmount
)
dracutmodules+=" ${DRACUTMODULES[@]} "
early_microcode=yes
hostonly_cmdline=yes
hostonly=yes
install_optional_items+=" $(find /lib/firmware/ | grep intel) "
show_modules=yes
stdloglvl=6
sysloglvl=6
use_fstab=yes
lvmconf=no
DRIVERS=(
        i915
        ext4
)
drivers+=" ${DRIVERS[@]} "


and my kernel command line:
rd.info rd.udev.info rd.locale.LANG=de_DE.utf8 rd.vconsole.keymap=de rd.vconsole.unicode rd.shell rd.luks.uuid=luks-2cdef804-0332-4d3f-9867-93dee1204ef0 rd.lvm.lv=lvm-vg/root rd.lvm.lv=lvm-vg/swap rd.luks.allow-discards root=/dev/mapper/lvm--vg-root resume=/dev/mapper/lvm--vg-swap rootfstype=ext4 rootflags=noatime,discard systemd.show_status=1 loglevel=5 video=i915drmfb:1920x1200-24@60 i915.enable_guc=2 rw retbleed=stuff
Comment 4 Mike Pagano gentoo-dev 2023-07-20 20:48:04 UTC
so dracut is not building the microcode into the initrd ?
Comment 5 Stefan Trenker 2023-07-20 20:59:52 UTC
(In reply to Mike Pagano from comment #4)
> so dracut is not building the microcode into the initrd ?

Dracut does build / prepend the microcode to the initrd. When i check the initrd, the microcode is prepended:

~ $ cpio -tv < /boot/871fc930ac62412195ba1e43481bd3ee/6.4.4-gentoo-x86_64/initrd 
drwxr-xr-x   3 root     root            0 Jul 20 20:17 .
-rw-r--r--   1 root     root            2 Jul 20 20:17 early_cpio
drwxr-xr-x   3 root     root            0 Jul 20 20:17 kernel
drwxr-xr-x   3 root     root            0 Jul 20 20:17 kernel/x86
drwxr-xr-x   2 root     root            0 Jul 20 20:17 kernel/x86/microcode
-rw-r--r--   1 root     root       110592 Jul 20 20:17 kernel/x86/microcode/GenuineIntel.bin
218 Blöcke

But when i boot the kernel the first line of dmesg does not show that early microcode loading happened. In older kernels (6.1.x and earlier this is still the case). Are there any other ways to check/prove that the microcode patch has been applied?
Comment 6 Mike Pagano gentoo-dev 2023-07-20 21:15:19 UTC
Now I got it , thanks.  I see a bunch of messages in the code around this, can I see your full dmesg ?
Comment 7 Stefan Trenker 2023-07-20 21:18:37 UTC
Created attachment 865849 [details]
dmesg 6.4.4
Comment 8 Mike Pagano gentoo-dev 2023-07-20 21:24:55 UTC
thanks for being so responsive.

ok, this is what we're looking for.
Similar to this?

"microcode: CPU11: patch_level=0x08701030"

This is from my 6.4.4
Comment 9 Mike Pagano gentoo-dev 2023-07-20 21:26:31 UTC
or this one ?

microcode: updated early
Comment 10 Stefan Trenker 2023-07-20 21:36:21 UTC
(In reply to Mike Pagano from comment #9)
> or this one ?
> 
> microcode: updated early

I am looking for a line like:

[    1.111181] microcode: sig=0x806c1, pf=0x80, revision=0xac

... this is what dmesg of my 6.1.39 kernel shows as very first line.
Comment 11 Stefan Trenker 2023-07-20 21:48:53 UTC
(In reply to Stefan Trenker from comment #10)
> (In reply to Mike Pagano from comment #9)
> > or this one ?
> > 
> > microcode: updated early
> 
> I am looking for a line like:
> 
> [    1.111181] microcode: sig=0x806c1, pf=0x80, revision=0xac
> 
> ... this is what dmesg of my 6.1.39 kernel shows as very first line.

Ok ... it is not the first line in dmesg. It's the first of two appearances when i grep for "microcode" in dmesg. But its the only line in dmesg which gives an idea that a microcode patch has been applied and which patch has been applied.

I compare the sig=... with the output of 

/usr/sbin/iucode_tool -S -l /lib/firmware/intel-ucode/* 2>&1 | grep -v bundle

By this way i can check, if the proper microcode patch has been applied.
Comment 12 Mike Pagano gentoo-dev 2023-07-20 22:51:50 UTC
Never used systemd-boot, can I see your kernel entries in :

/boot/EFI/loader/entries/
Comment 13 Stefan Trenker 2023-07-21 05:43:03 UTC
(In reply to Mike Pagano from comment #12)
> Never used systemd-boot, can I see your kernel entries in :
> 
> /boot/EFI/loader/entries/

There is only a /boot/loader/entries directory (/boot is my UEFI-Partition)

# ls -l /boot/loader/entries
insgesamt 16
-rwxr-xr-x 1 root root 873  6. Jul 06:21 871fc930ac62412195ba1e43481bd3ee-6.1.38-gentoo-x86_64.conf
-rwxr-xr-x 1 root root 873 20. Jul 19:44 871fc930ac62412195ba1e43481bd3ee-6.1.39-gentoo-x86_64.conf
-rwxr-xr-x 1 root root 870 11. Jul 17:36 871fc930ac62412195ba1e43481bd3ee-6.4.3-gentoo-x86_64.conf
-rwxr-xr-x 1 root root 870 20. Jul 20:17 871fc930ac62412195ba1e43481bd3ee-6.4.4-gentoo-x86_64.conf


# cat //boot/loader/entries/871fc930ac62412195ba1e43481bd3ee-6.4.4-gentoo-x86_64.conf 
# Boot Loader Specification type#1 entry
# File created by /usr/lib/kernel/install.d/90-loaderentry.install (systemd 253)
title      Gentoo Linux
version    6.4.4-gentoo-x86_64
machine-id 871fc930ac62412195ba1e43481bd3ee
sort-key   gentoo
options    rd.info rd.udev.info rd.locale.LANG=de_DE.utf8 rd.vconsole.keymap=de rd.vconsole.unicode rd.shell rd.luks.uuid=luks-2cdef804-0332-4d3f-9867-93dee1204ef0 rd.lvm.lv=lvm-vg/root rd.lvm.lv=lvm-vg/swap rd.luks.allow-discards root=/dev/mapper/lvm--vg-root resume=/dev/mapper/lvm--vg-swap rootfstype=ext4 rootflags=noatime,discard systemd.show_status=1 loglevel=5 video=i915drmfb:1920x1200-24@60 i915.enable_guc=2 rw retbleed=stuff systemd.machine_id=871fc930ac62412195ba1e43481bd3ee
linux      /871fc930ac62412195ba1e43481bd3ee/6.4.4-gentoo-x86_64/linux
initrd     /871fc930ac62412195ba1e43481bd3ee/6.4.4-gentoo-x86_64/initrd


The given systemd-boot loader/entries/config is created using the command:

kernel-install --verbose add "Kernel Version" "/path/to/kernel-image"

When You do not provide a path to an initrd-image kernel-install calls dracut and creates one for you. kernel-install itself (and systemd-boot) comes with sys-apps/systemd when USE-flag gnuefi is enabled.


I understand that you are looking for a correct boot-setup. I use this setup which is maintained by a bunch of ansible scripts and roles already for a longer time on various systems and vms.

Starting with 6.4.4 dmesg output does not show up the line mentioned in comment#10.

So i am confused if microcode loading happened or not.

I compared using diff /usr/src/linux-6.4.4-/usr/src/linux-6.4.4-gentoo/arch/x86/kernel/cpu/microcode/core.cgentoo/arch/x86/kernel/cpu/microcode/intel.c and with the respective source code files of 6.1.39 and found out, that the 6.4.4 files differ a lot from the 6.1.39 ones. But i do not understand what is going on there. I am just a simple linux user. I guess during maintenance of this source files something went wrong.
Comment 14 Mike Pagano gentoo-dev 2023-07-21 11:00:36 UTC
what do the contents of this one look like?

871fc930ac62412195ba1e43481bd3ee-6.1.38-gentoo-x86_64.conf
Comment 15 Mike Pagano gentoo-dev 2023-07-21 11:11:23 UTC
If you think it's a kernel problem, you could do a bisect, if there's a problematic commit, that might show it.
Comment 16 Stefan Trenker 2023-07-21 21:31:04 UTC
(In reply to Mike Pagano from comment #15)
> If you think it's a kernel problem, you could do a bisect, if there's a
> problematic commit, that might show it.

bisecting is still in progress ...
Comment 17 Stefan Trenker 2023-07-22 08:43:28 UTC
I looks like i was chasing a ghost. No messages at all in early microcode loading imply no errors. I am not a fan of this, but such is live.

I apologize for consuming your time.

----------

commit a70210f41566131f88d31583f96e36cb7f5d2ad0
Merge: 3ef3ace4e2ec be1b670f6144
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Tue Dec 13 15:05:29 2022 -0800

    Merge tag 'x86_microcode_for_v6.2' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
    
    Pull x86 microcode and IFS updates from Borislav Petkov:
     "The IFS (In-Field Scan) stuff goes through tip because the IFS driver
      uses the same structures and similar functionality as the microcode
      loader and it made sense to route it all through this branch so that
      there are no conflicts.
    
       - Add support for multiple testing sequences to the Intel In-Field
         Scan driver in order to be able to run multiple different test
         patterns. Rework things and remove the BROKEN dependency so that
         the driver can be enabled (Jithu Joseph)
    
       - Remove the subsys interface usage in the microcode loader because
         it is not really needed
    
       - A couple of smaller fixes and cleanups"
    
    * tag 'x86_microcode_for_v6.2' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (24 commits)
      x86/microcode/intel: Do not retry microcode reloading on the APs
--->  x86/microcode/intel: Do not print microcode revision and processor flags
      platform/x86/intel/ifs: Add missing kernel-doc entry
      Revert "platform/x86/intel/ifs: Mark as BROKEN"
      Documentation/ABI: Update IFS ABI doc
      platform/x86/intel/ifs: Add current_batch sysfs entry
      platform/x86/intel/ifs: Remove reload sysfs entry
      platform/x86/intel/ifs: Add metadata validation
      platform/x86/intel/ifs: Use generic microcode headers and functions
      platform/x86/intel/ifs: Add metadata support
      x86/microcode/intel: Use a reserved field for metasize
      x86/microcode/intel: Add hdr_type to intel_microcode_sanity_check()
      x86/microcode/intel: Reuse microcode_sanity_check()
      x86/microcode/intel: Use appropriate type in microcode_sanity_check()
      x86/microcode/intel: Reuse find_matching_signature()
      platform/x86/intel/ifs: Remove memory allocation from load path
      platform/x86/intel/ifs: Remove image loading during init
      platform/x86/intel/ifs: Return a more appropriate error code
      platform/x86/intel/ifs: Remove unused selection
      x86/microcode: Drop struct ucode_cpu_info.valid
      ...
Comment 18 Stefan Trenker 2023-07-23 11:38:40 UTC
Ok ... now i have the full picture (at least i believe so).

Conclusion: I was chasing a ghost. Microcode early update works as designed and it works well.

Last week two things changed for my laptop with all this concerns.

At 1st i updated to gentoo-sources-6.4.4 (from 6.4.3) and journalctl lists:

Jul 19 22:46:54 bully kernel: microcode: updated early: 0xa4 -> 0xaa, date = 2022-12-28
Jul 19 22:46:54 bully kernel: Linux version 6.4.4-gentoo-x86_64 (root@bully) (x86_64-pc-linux-gnu-gcc (Gentoo 12.3.1_p20230526 p2) 12.3.1 20230526, GNU ld (Gentoo 2.40 p5) 2.40.0) #1 SMP PREEMPT_DYNAMIC Wed Jul 19 21:21:07 CEST 2023
Jul 19 22:46:54 bully kernel: microcode: Microcode Update Driver: v2.2.

Result of early microcode loading as expected.


2ndly i did a firmware update via fwupdmgr update. After this firmware update the microcode early update message disappeared. So i guess the new firmware already includes the latest Intel microcode update patches.


I still have to apologize for consuming your time.

------

In 6.1.x there is still a log message showing the running microcode version. This log message was eliminated with the microcode update commit for 6.2.x mentioned in my previous comment and caused my confusion. I messed this now missing log message up with the "updated early" message.
Comment 19 Mike Pagano gentoo-dev 2023-07-23 22:01:46 UTC
No worries at all.  I'm glad it worked out.