Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 558192 - sys-apps/microcode-data-20150121-r1: kernel panic with USE=initramfs
Summary: sys-apps/microcode-data-20150121-r1: kernel panic with USE=initramfs
Status: RESOLVED WORKSFORME
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo's Team for Core System packages
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-08-19 19:09 UTC by Balint SZENTE
Modified: 2015-09-19 11:37 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Balint SZENTE 2015-08-19 19:09:30 UTC
Following the instructions of bug 528712, comment 41 the generated initramfs causes kernel panic. The system is not bootable, cannot mount the rootfs. See bug 528712, comment 55 for explanation.

Please not that I'm using a Lenovo G510 laptop directly with UEFI stub loader. So, no GRUB and any other bootloader.

Please include the necessary /root/ and char device /dev/console in the /lib/firmware/microcode.cpio initramfs file in order to be directly usable for those users who does not use initramfs but were forced to initramfs because of deprecating of microcode_ctl.

Optionally add support for XZ compressing of this microcode.cpio file with a use flag for example to reduce the size of the image.

Reproducible: Always
Comment 1 SpanKY gentoo-dev 2015-08-23 18:11:19 UTC
i was able to boot a system just fine w/out any additional contents in the initramfs.  i don't think bugzilla is the right place for this support, so please try your luck in the forums / mailing lists.
Comment 2 Balint SZENTE 2015-08-24 10:40:08 UTC
(In reply to SpanKY from comment #1)
> i was able to boot a system just fine w/out any additional contents in the
> initramfs.  i don't think bugzilla is the right place for this support, so
> please try your luck in the forums / mailing lists.

I'm sorry, but this is not about "luck" or "support". It is about your instructions, that rendered my (and not just mine, but also for at least other user) system unbootable for some reason:

(4) set CONFIG_INITRAMFS_SOURCE to /lib/firmware/microcode.cpio

With this enabled, the kernel gets in panic. If I disable *just* this single option, the kernel boots fine. Let's not ignore this. And yes, if that cpio is incomplete for direct inclusion into the kernel without any other external initramfs passed to kernel, than this is certainly a *bug*.

I would be glad if we could figure this out, what is different on your system, why does it work for you, or accidentally what is missing from your instructions. There is clearly some *technical* issue here.

I'm aware that 99% of users are using GRUB and various kind of initrams, but let's not assume this for all users.

So please tell me, what information should I provide, how can I help you in this matter? Thank you.

I'm using sys-kernel/gentoo-sources-4.1.6 on x86_64.
Comment 3 SpanKY gentoo-dev 2015-08-24 16:21:15 UTC
(In reply to Balint SZENTE from comment #2)

the suggestions were just that -- suggestions.  getting you to build a kernel that works in your environment is out of scope.  if you want assistance in creating such a kernel, please use the existing user support forums.
Comment 4 SpanKY gentoo-dev 2015-08-24 16:21:15 UTC
(In reply to Balint SZENTE from comment #2)

the suggestions were just that -- suggestions.  getting you to build a kernel that works in your environment is out of scope.  if you want assistance in creating such a kernel, please use the existing user support forums.
Comment 5 Balint SZENTE 2015-08-25 12:22:58 UTC
I'm sorry but I don't accept this non-technical response. Please stop closing this issue with WORKSFORME by ignoring the given *technical* arguments. This is simply just not the right way to do it.

/lib/firmware/microcode.cpio is INCOMPLETE to be included in the kernel! DOES not work, because does not respect the directory structure necessary for early initramfs files, as the kernel requires. It is NOT USABLE as stand alone.

It does work for YOU (accidentally), because you must be using an external initramfs as well, which has the necessary /dev/console (and other) files.

I have tested on several system this, and all the system give kernel panic. On the other hand ALL systems boot properly if I include the following files in that cpio archive:

drwxr-xr-x   2 root     root            0 Aug 25 14:14 dev
crw-------   1 root     root       5,   1 Aug 25 14:14 dev/console
drwx------   2 root     root            0 Aug 25 14:14 root

So if this ebuild has this very handy "initramfs" USE flag, which is meant to generate a standalone initramfs to be included directly in kernel (because there is no other option without external initramfs), please include those missing files.

Gentoo is about choice, and not about forcing the user to certain use-case (in this case to use and external initramfs, which is not always possible).

I can help with the ebuild, if it is necessary, but please don't close this with WORKSFORME, because this issue is not about user error. Thank you.
Comment 6 SpanKY gentoo-dev 2015-08-25 17:37:37 UTC
(In reply to Balint SZENTE from comment #5)

you're sorely confused about how my system is configured.  the cpio is a fragment that people can integrate into their system.  if it isn't working for you, then that's a problem with your system.

if you want documentation, then write it -- in the wiki.  the ebuild is working as designed.
Comment 7 Doug Goldstein (RETIRED) gentoo-dev 2015-08-28 22:25:22 UTC
Mike: I'm going to open this issue up because I do believe there is a problem here. I'm not sure Balint's issue is related however but it could be that this fixes the problem.

I believe we need to have USE=initramfs depend on USE=split-ucode because a quick peek at the kernel code seems to show that it can't read microcode.dat. In fact now that the kernel does the reading of the data there are no more consumers of microcode.dat so I think it makes sense to drop split-ucode and always do that step but that's an aside.

I've pushed a branch with a new ebuild and you can check it out at: https://github.com/cardoe/gentoo/commit/9f094dcd2e635fe00d936b6a8538179aa9404ec3
Comment 8 SpanKY gentoo-dev 2015-08-30 07:56:35 UTC
(In reply to Doug Goldstein from comment #7)

it's not related to this issue.  start a new one please.
Comment 9 Balint SZENTE 2015-09-16 08:33:56 UTC
(In reply to Doug Goldstein from comment #7)
> Mike: I'm going to open this issue up because I do believe there is a
> problem here. I'm not sure Balint's issue is related however but it could be
> that this fixes the problem.

Doug: I have tested your ebuild, and indeed it is not related to my issue as SpanKY said. I have kernel panic with this -r2 ebuild as well, if I do not include the *necessary* /root/ and char device /dev/console files in cpio archive.

My issue is simple: /root/ and char device /dev/console must be included into microcode.cpio archive when it is solely embedded into kernel without any external initramfs, otherwise the kernel will not boot.

Nobody, except SpanKY, confirmed (s)he was able to boot a kernel with only the microcode.cpio embedded into kernel without any external initramfs. Unfortunately, Mike did not give any details how he made it.

One might say the ebuild *does not work* as designed, i.e. to be directly included into kernel without any manipulation or other initramfs. This was confirmed by several users and developers.

Mike, please forgive my perseverance, but I'm reopening this issue again. With all respect I kindly ask to revise the solution. There are two scenarios in my mind:

A. Include /root/ /dev/console in cpio archive, as the Linux kernel documentation for initramfs states. This way the microcode.cpio will work in all scenarios. Even if another initramfs has the same files, it will not be a problem (the last initramfs in the chain "wins").

B. If you *really* think my system (or better said everybody's system who tried and confirmed on forum the kernel panic) "has a problem", please try to give some details to figure this out. We have to find this out, because there are several users with this issue. I expect more reports when it will be stabilized.

I believe the problem is NOT my system, because the CONFIG_INITRAMFS_SOURCE=/lib/firmware/microcode.cpio was the ONLY change made in kernel config that caused the panic. Let's keep in mind we know exactly what causes the kernel panic and how it can be fixed.

Thank you for your patience and understanding.
Comment 10 Doug Goldstein (RETIRED) gentoo-dev 2015-09-16 17:39:51 UTC
(In reply to Balint SZENTE from comment #9)
> (In reply to Doug Goldstein from comment #7)
> > Mike: I'm going to open this issue up because I do believe there is a
> > problem here. I'm not sure Balint's issue is related however but it could be
> > that this fixes the problem.
> 
> Doug: I have tested your ebuild, and indeed it is not related to my issue as
> SpanKY said. I have kernel panic with this -r2 ebuild as well, if I do not
> include the *necessary* /root/ and char device /dev/console files in cpio
> archive.
> 
> My issue is simple: /root/ and char device /dev/console must be included
> into microcode.cpio archive when it is solely embedded into kernel without
> any external initramfs, otherwise the kernel will not boot.
> 
> Nobody, except SpanKY, confirmed (s)he was able to boot a kernel with only
> the microcode.cpio embedded into kernel without any external initramfs.
> Unfortunately, Mike did not give any details how he made it.
> 
> One might say the ebuild *does not work* as designed, i.e. to be directly
> included into kernel without any manipulation or other initramfs. This was
> confirmed by several users and developers.
> 
> Mike, please forgive my perseverance, but I'm reopening this issue again.
> With all respect I kindly ask to revise the solution. There are two
> scenarios in my mind:
> 
> A. Include /root/ /dev/console in cpio archive, as the Linux kernel
> documentation for initramfs states. This way the microcode.cpio will work in
> all scenarios. Even if another initramfs has the same files, it will not be
> a problem (the last initramfs in the chain "wins").
> 
> B. If you *really* think my system (or better said everybody's system who
> tried and confirmed on forum the kernel panic) "has a problem", please try
> to give some details to figure this out. We have to find this out, because
> there are several users with this issue. I expect more reports when it will
> be stabilized.
> 
> I believe the problem is NOT my system, because the
> CONFIG_INITRAMFS_SOURCE=/lib/firmware/microcode.cpio was the ONLY change
> made in kernel config that caused the panic. Let's keep in mind we know
> exactly what causes the kernel panic and how it can be fixed.
> 
> Thank you for your patience and understanding.

I meant to reply to you and forgot. The issue is that the cpio archive that's generated is not actually meant to be used as your only cpio archive. Its meant to be merged with an existing one. So in that regard Mike is correct that your configuration is not correct. You will need to create another initramfs to merge this with.
Comment 11 SpanKY gentoo-dev 2015-09-16 17:49:25 UTC
it would be easier to just embed the specific ucode files directly into your kernel config.  the AMD page has an example:
https://wiki.gentoo.org/wiki/AMD_microcode

but really we need to create a Microcode page that covers Intel & AMD and explains when to use each feature.
Comment 12 Balint SZENTE 2015-09-16 18:56:02 UTC
Thank you for the answers, I really appreciate it. I am glad that we have now a conclusion. Let me summarize it:

1. "CONFIG_INITRAMFS_SOURCE=/lib/firmware/microcode.cpio" is NOT working if it is the one and only initramfs embedded into kernel without any external initramfs, and *it was not meant to be used that way*.

2. /lib/firmware/microcode.cpio must be merged with another cpio, as I did after all, which contains the necessary files (/root/, /dev/console, etc.). This is how I made my system to boot.

3. On the other hand, the best way (for my setup) is SpanKY's recommendation: to embed directly the ucode file, so *no initramfs is necessary*. Thank you for this advise, it is really a good idea I never thought of.
Comment 13 Balint SZENTE 2015-09-19 11:20:48 UTC
SpanKY, I made some tests and investigation with the embedded firmware approach:

1. I does not work for Intel, so I started to search what is happening:

Discussion:
<http://comments.gmane.org/gmane.linux.kernel/1898819>
Commit:
<https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/arch/x86/kernel/cpu/microcode/intel_early.c?id=760d765b2bb662be177d4b5b271ced8debc803ac>

Conclusion: only 4.2.0 has this support, previous kernels support only initramfs based loading.

2. I upgraded to 4.2.0, but after selecting Gentoo kernel from the UEFI boot menu, I have instant freeze with black screen. Then I found this:

<https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/arch/x86/kernel/cpu/microcode/intel_early.c?id=ee38a90709084b1c91279cde8f783e90f85285a1>

So it seems, not only 32 bit is affected by this, but also x86_64. If I disable early microcode loading, the very same kernel boots fine.

So, apparently the embedded firmware approach does not work for Intel (yet). One must stick with embedded custom initramfs for the moment.
Comment 14 Balint SZENTE 2015-09-19 11:26:22 UTC
I made a typo, sorry:

"1. It does not work for Intel,"
Comment 15 Balint SZENTE 2015-09-19 11:37:50 UTC
Ok, furthermore:

<https://bugzilla.kernel.org/show_bug.cgi?id=85081>

"Early microcode loading ignores microcode provided in embedded initramfs"

And I can confirm this also. I have checked carefully my system, it has microcode revision 0x17 and it does not get updated to 0x1c.

So embedded custom initramfs is also not an option. This means shit-out-of-luck.

So the Gentoo Wiki page (<https://wiki.gentoo.org/wiki/Intel_microcode#Early_Microcode>) is OK, but perhaps a warning would be necessary, which explains an external initramfs is the ONLY working solution.