Bug 223489 – gentoo-sources-2.6.24-r8: Unable to handle kernel NULL pointer dereference (caused by serial console boot parameter?!)

• Product: Gentoo Linux
• Status: RESOLVED NEEDINFO

[Description Daniel 2008-05-24 22:40:28 UTC]

Hi Folks,

today I switched, from my x86 plattform to amd64.

It took me quite some time to get reproducable results, including a might be ugly/standard/known/unknown(?) workaround, regarding the following KERNEL PANIC during system boot. Hopefully you guys are able to find a rockstable solution which successfully can prevent this KERNEL PANIC, during system boot:


"Unable to handle kernel NULL pointer dereference at 0000000000000014 RIP"



The Kernel version "linux-2.6.24-gentoo-r8" 100% repeatedly and 100% reproducable crashes on my plattform, when using the following grub kernel parameter settings:

1.) Grub settings, boot messages using console=ttyS0 until the kernel crashes:
------------------------------------------------
root (hd0,0)
kernel /kernel-genkernel-x86_64-2.6.24-gentoo-r8 root=/dev/ram0 init=/linuxrc ramdisk=8192 real_root=/dev/md3 dodmraid udev splash=verbose,theme:emergence video=uvesafb:1024x768-32,mtrr:3,ywrap console=ttyS0 
initrd /initramfs-genkernel-x86_64-2.6.24-gentoo-r8


===============================================


On the other hand the same kernel-binary, grub, initrd (and so on) boots up smoothly using any of the following two grub kernel parameter settings. Both, serial-console logging as well as splash (using uvesafb and fbcondecor) work fine:


2.) Grub settings, boot messages via console=tty1 works fine:
-------------------------------------------------------------
title=Dr. Nick 2.6.24-r8 gentoo (tty0 Console)
root (hd0,0)
kernel /kernel-genkernel-x86_64-2.6.24-gentoo-r8 root=/dev/ram0 init=/linuxrc ramdisk=8192 real_root=/dev/md3 dodmraid udev splash=verbose,theme:emergence video=uvesafb:1024x768-32,mtrr:3,ywrap
initrd /initramfs-genkernel-x86_64-2.6.24-gentoo-r8


3.) Grub settings, boot messages via CONSOLE=/dev/ttyS0, works fine:
--------------------------------------------------------------------
root (hd0,0)
kernel /kernel-genkernel-x86_64-2.6.24-gentoo-r8 root=/dev/ram0 init=/linuxrc ramdisk=8192 real_root=/dev/md3 dodmraid udev splash=verbose,theme:emergence video=uvesafb:1024x768-32,mtrr:3,ywrap CONSOLE=/dev/ttyS0 
initrd /initramfs-genkernel-x86_64-2.6.24-gentoo-r8



Short conclusion:
Comparing the errornous setting 1.) above against the working setting 3.) the only difference is the "console=ttyS0" and "CONSOLE=/dev/ttyS0". Setting 2.) and 3.) should demonstrate that there is no problem regarding the uvesafb both work fine resulting in fbcondecor console on ttyN while 3.) also reports boot messages via serial console.

Reproducible: Always

Steps to Reproduce:
1.ctrl+alt+del or shutdown -r now ;-)

Actual Results:  
 * Waiting for uevents to be processed ...Unable to handle kernel NULL pointer dereference at 0000000000000014 RIP: 
 [<ffffffff80450c88>] uart_write_room+0xb/0x19
PGD 22eeeb067 PUD 22f78a067 PMD 0 
Oops: 0000 [1] SMP 
CPU 0 
Modules linked in: parport_pc parport iTCO_wdt i2c_i801 8250_pnp jfs reiserfs dm_mirror scsi_wait_scan sbp2 ohci1394 ieee1394 usbhid ohci_hcd uhci_hcd usb_storage libusual ehci_hcd usbcore
Pid: 2680, comm: bash Not tainted 2.6.24-gentoo-r8 #1
RIP: 0010:[<ffffffff80450c88>]  [<ffffffff80450c88>] uart_write_room+0xb/0x19
RSP: 0018:ffff81022eb2fe50  EFLAGS: 00010206
RAX: ffff81022e972cc0 RBX: 0000000000000027 RCX: ffff81022e9b31f8
RDX: 0000000000000000 RSI: 0000000000000246 RDI: ffff81022e9b3000
RBP: ffff81022e9b3000 R08: 0000000000000001 R09: ffffffff807713a0
R10: 2020202020202020 R11: 0000000000000296 R12: ffff81022e036c00
R13: 0000000000000027 R14: ffff81022eedb8c0 R15: ffff81022e036c00
FS:  00002b516cd66f40(0000) GS:ffffffff8070f000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000000000014 CR3: 000000022f771000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process bash (pid: 2680, threadinfo ffff81022eb2e000, task ffff81022ebcc840)
Stack:  ffffffff804000ee 0000000000000000 ffff81022ebcc840 ffffffff8022ca3d
 ffff81022e9b31f8 ffff81022e9b31f8 ffffffff8021f45b 0000000000000abe
 0000000000000027 0000000000000027 ffff81022e9b3000 0000000000000000
Call Trace:
 [<ffffffff804000ee>] write_chan+0x102/0x2ff
 [<ffffffff8022ca3d>] default_wake_function+0x0/0xe
 [<ffffffff8021f45b>] do_page_fault+0x33e/0x68f
 [<ffffffff803fdc4e>] tty_write+0x17a/0x210
 [<ffffffff803fffec>] write_chan+0x0/0x2ff
 [<ffffffff80281e68>] vfs_write+0xad/0x136
 [<ffffffff802823a5>] sys_write+0x45/0x6e
 [<ffffffff8020be2e>] system_call+0x7e/0x83


Code: 8b 42 14 2b 42 10 ff c8 25 ff 0f 00 00 c3 48 8b 87 48 02 00 
RIP  [<ffffffff80450c88>] uart_write_room+0xb/0x19
 RSP <ffff81022eb2fe50>
CR2: 0000000000000014
---[ end trace 69c7a960c8307cdd ]---

Expected Results:  
booooooooot until login-prompt ;-)

see attachments

Some common background informations regarding my system environment:

My hardware plattform is an IBASE Mainboard (Type: MB898) based on the Intel 965 Chipset running a Core2Duo 6600 CPU, equipped with 8GB RAM.

Attachments:
a) my serial-console output until the crash (including the trace regarding the PANIC)
b) lspci -vvv output (after booted with above setting 1 / 2)
c) lsmod output (after booted using above setting 1 / 2)
c) emerge --info
d) zcat /proc/config.gz

[Comment 1 Daniel 2008-05-24 22:42:29 UTC]

Created attachment 154199 [details]
Boot messages until crash

[Comment 2 Daniel 2008-05-24 22:43:00 UTC]

Created attachment 154201 [details]
emerge --info

[Comment 3 Daniel 2008-05-24 22:43:43 UTC]

Created attachment 154203 [details]
.config gentoo-sources-2.6.24-r8

[Comment 4 Daniel 2008-05-24 22:44:12 UTC]

Created attachment 154207 [details]
lspci -vvv

[Comment 5 Daniel 2008-05-24 22:44:41 UTC]

Created attachment 154209 [details]
lsmod

[Comment 6 Mike Pagano 2008-05-30 12:36:31 UTC]

Daniel,

Can you rebuild your kernel with CONFIG_SERIAL_8250_PNP=y and please retest.

[Comment 7 Gordon Malm (RETIRED) 2008-05-31 02:50:07 UTC]

Saw this thread awhile back, appears to be the same NULL ptr deref.  Looks like compiling CONFIG_SERIAL_8250_PNP=y as suggested in Comment #6 is the work-around for now.

http://marc.info/?t=120885070900001&r=1&w=2

[Comment 8 Daniel Drake (RETIRED) 2008-05-31 08:41:35 UTC]

we had an almost identical bug before as well
http://bugzilla.kernel.org/show_bug.cgi?id=8552

[Comment 9 Daniel 2008-05-31 09:49:58 UTC]

(In reply to comment #6)
> Daniel,
> 
> Can you rebuild your kernel with CONFIG_SERIAL_8250_PNP=y and please retest.
> 
Hi Mike,

in the moment I can't because this machine is located 250miles away from my current location and there's nobody available who could restart it after a possible oops.
Although MagicSysRq works via keyboard, via serial-console it does not work any more (right after *this* PANIC).

Nevertheless, of course I'll try your suggestion next time I'm standing next to this machine - approximately in a about one or two weeks I'll report the results. Sorry for the delay.

Cheers Daniel

[Comment 10 Daniel Drake (RETIRED) 2008-05-31 10:13:53 UTC]

I wouldn't worry about the debug option, the bug is identical the one before (since the patch that fixed it got reverted). We know why it happens and upstream developers have some idea of what needs to be done.

[Comment 11 Daniel 2008-05-31 10:38:06 UTC]

(In reply to comment #7)
> Saw this thread awhile back, appears to be the same NULL ptr deref.  Looks like
> compiling CONFIG_SERIAL_8250_PNP=y as suggested in Comment #6 is the
> work-around for now.
> 
> http://marc.info/?t=120885070900001&r=1&w=2
> 
Hi Gradon,

my workaround just was:
Instead of using "console=ttyS0", I use CONSOLE=/dev/ttyS0 and everthing is fine. Nothing else has been changed, same Kernel and same modules.

Cheers
Daniel

[Comment 12 Daniel Drake (RETIRED) 2008-06-29 16:27:12 UTC]

will track upstream bug

[Comment 13 Tom Wijsman (TomWij) (RETIRED) 2012-11-08 01:52:57 UTC]

Is this bug still present on recent 3.6 kernels?

Upstream has closed this as obsolete but Drake said the bug fix was reverted.