Kernel 2.6.37 oops - 2 servers, same oops code

kernel-2.6.37-oops.txt (text/plain), 9.27 KB, created by Valentin Avram on 2011-11-24 11:30:32 UTC

SERVER 1:

==> /var/log/everything/current <==
2011-11-24T12:16:37+02:00 SERVER1 auditd[12023]: Init complete, auditd 2.1.3 listening for events (startup state enable)

==> /var/log/audit/audit.log <==
type=CONFIG_CHANGE msg=audit(1322129841.362:36): auid=4294967295 ses=4294967295 op="remove rule" key="etc-directory" list=4 res=1
type=CONFIG_CHANGE msg=audit(1322129841.362:37): auid=4294967295 ses=4294967295 op="remove rule" key="sbin-directory" list=4 res=1
type=CONFIG_CHANGE msg=audit(1322129841.362:38): auid=4294967295 ses=4294967295 op="remove rule" key="bin-directory" list=4 res=1
type=CONFIG_CHANGE msg=audit(1322129841.362:39): auid=4294967295 ses=4294967295 op="remove rule" key="usr-sbin-directory" list=4 res=1
type=CONFIG_CHANGE msg=audit(1322129841.362:40): auid=4294967295 ses=4294967295 op="remove rule" key="usr-bin-directory" list=4 res=1
type=CONFIG_CHANGE msg=audit(1322129841.362:41): auid=4294967295 ses=4294967295 op="remove rule" key="skip-lib-rc" list=4 res=1
type=CONFIG_CHANGE msg=audit(1322129841.362:42): auid=4294967295 ses=4294967295 op="remove rule" key="lib-directory" list=4 res=1
type=CONFIG_CHANGE msg=audit(1322129841.362:43): auid=4294967295 ses=4294967295 op="remove rule" key="usr-lib-directory" list=4 res=1
type=CONFIG_CHANGE msg=audit(1322129841.362:44): auid=4294967295 ses=4294967295 op="remove rule" key="excluded-syscalls" list=4 res=1
type=CONFIG_CHANGE msg=audit(1322129841.362:45): audit_enabled=0 old=1 auid=4294967295 ses=4294967295 res=1
type=DAEMON_END msg=audit(1322129841.370:8462): auditd normal halt, sending auid=0 pid=12074 subj= res=success

==> /var/log/everything/current <==
2011-11-24T12:17:21+02:00 SERVER1 auditd[12023]: The audit daemon is exiting.
2011-11-24T12:17:21.845663+02:00 SERVER1 kernel: BUG: unable to handle kernel NULL pointer dereference at 00000004
2011-11-24T12:17:21.845673+02:00 SERVER1 kernel: IP: [<c10ebcb5>] fsnotify_mark_destroy+0x85/0x130
2011-11-24T12:17:21.845675+02:00 SERVER1 kernel: *pdpt = 0000000000000000 *pde = f000def8f000def8
2011-11-24T12:17:21.845678+02:00 SERVER1 kernel: Oops: 0002 [#1] SMP
2011-11-24T12:17:21.845680+02:00 SERVER1 kernel: last sysfs file: /sys/devices/pci0000:00/0000:00:1c.0/0000:03:00.0/host0/target0:2:0/0:2:0:0/block/sda/dev
2011-11-24T12:17:21.845682+02:00 SERVER1 kernel:
2011-11-24T12:17:21.845685+02:00 SERVER1 kernel: Pid: 638, comm: fsnotify_mark Not tainted 2.6.37-gentoo-r8392-drbd-version1 #1 Dell Inc. PowerEdge R610/0F0XJ6
2011-11-24T12:17:21.845688+02:00 SERVER1 kernel: EIP: 0060:[<c10ebcb5>] EFLAGS: 00010293 CPU: 3
2011-11-24T12:17:21.845690+02:00 SERVER1 kernel: EIP is at fsnotify_mark_destroy+0x85/0x130
2011-11-24T12:17:21.845692+02:00 SERVER1 kernel: EAX: f2602488 EBX: f2769fa8 ECX: 00000000 EDX: f26024c4
2011-11-24T12:17:21.845695+02:00 SERVER1 kernel: ESI: f27597a0 EDI: ffffffc4 EBP: c10e4ee0 ESP: f2769f90
2011-11-24T12:17:21.845696+02:00 SERVER1 kernel: DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
2011-11-24T12:17:21.845699+02:00 SERVER1 kernel: Process fsnotify_mark (pid: 638, ti=f2768000 task=f27597a0 task.ti=f2768000)
2011-11-24T12:17:21.845700+02:00 SERVER1 kernel: Stack:
2011-11-24T12:17:21.845702+02:00 SERVER1 kernel: f27597a0 00000000 f27597a0 c104fad0 f2769fa0 f2769fa0 f26024c4 f2602744
2011-11-24T12:17:21.845704+02:00 SERVER1 kernel: f2c47f68 00000000 c10ebc30 00000000 c104f744 00000000 00000000 00000000
2011-11-24T12:17:21.845706+02:00 SERVER1 kernel: 00000000 f2769fd4 f2769fd4 00000000 c104f6d0 f2c47f68 c1002fb6 00000000
2011-11-24T12:17:21.845708+02:00 SERVER1 kernel: Call Trace:
2011-11-24T12:17:21.845710+02:00 SERVER1 kernel: [<c104fad0>] ? autoremove_wake_function+0x0/0x40
2011-11-24T12:17:21.845713+02:00 SERVER1 kernel: [<c10ebc30>] ? fsnotify_mark_destroy+0x0/0x130
2011-11-24T12:17:21.845714+02:00 SERVER1 kernel: [<c104f744>] ? kthread+0x74/0x80
2011-11-24T12:17:21.845721+02:00 SERVER1 kernel: [<c104f6d0>] ? kthread+0x0/0x80
2011-11-24T12:17:21.845723+02:00 SERVER1 kernel: [<c1002fb6>] ? kernel_thread_helper+0x6/0x10
2011-11-24T12:17:21.845727+02:00 SERVER1 kernel: Code: c1 b8 10 51 87 c1 e8 cb 8c f6 ff 8b 54 24 18 8d 42 c4 39 da 8b 48 3c 74 32 8d 79 c4 eb 0a 90 8d b4 26 00 00 00 00 89 ef 8b 68 40 <89> 69 04 89 4d 00 89 50 3c 89 50 40 e8 fa fe ff ff 8b 4f 3c 8d
2011-11-24T12:17:21.845730+02:00 SERVER1 kernel: EIP: [<c10ebcb5>] fsnotify_mark_destroy+0x85/0x130 SS:ESP 0068:f2769f90
2011-11-24T12:17:21.845732+02:00 SERVER1 kernel: CR2: 0000000000000004
2011-11-24T12:17:21.845733+02:00 SERVER1 kernel: ---[ end trace d6aa0fd817b19900 ]---

------------------------------------------------------------------------------

SERVER 2:

2011-10-18T01:00:00+03:00 SERVER2 auditd[2037]: The audit daemon is exiting.
2011-10-18T01:00:00.452168+03:00 SERVER2 kernel: BUG: unable to handle kernel NULL pointer dereference at 00000004
2011-10-18T01:00:00.468924+03:00 SERVER2 kernel: IP: [<c10ebcb5>] fsnotify_mark_destroy+0x85/0x130
2011-10-18T01:00:00.468928+03:00 SERVER2 kernel: *pdpt = 0000000000000000 *pde = f000def8f000def8 
2011-10-18T01:00:00.468930+03:00 SERVER2 kernel: Oops: 0002 [#1] SMP 
2011-10-18T01:00:00.468932+03:00 SERVER2 kernel: last sysfs file: /sys/devices/pci0000:00/0000:00:1c.0/0000:03:00.0/host0/target0:2:0/0:2:0:0/block/sda/dev
2011-10-18T01:00:00.468933+03:00 SERVER2 kernel: 
2011-10-18T01:00:00.468935+03:00 SERVER2 kernel: Pid: 638, comm: fsnotify_mark Not tainted 2.6.37-gentoo-r8392-drbd-version1 #1 Dell Inc. PowerEdge R610/0F0XJ6
2011-10-18T01:00:00.468937+03:00 SERVER2 kernel: EIP: 0060:[<c10ebcb5>] EFLAGS: 00010297 CPU: 6
2011-10-18T01:00:00.468938+03:00 SERVER2 kernel: EIP is at fsnotify_mark_destroy+0x85/0x130
2011-10-18T01:00:00.468940+03:00 SERVER2 kernel: EAX: f1e06008 EBX: f275dfa8 ECX: 00000000 EDX: f1e06044
2011-10-18T01:00:00.468941+03:00 SERVER2 kernel: ESI: f2750bd0 EDI: ffffffc4 EBP: c1456bb0 ESP: f275df90
2011-10-18T01:00:00.468942+03:00 SERVER2 kernel: DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
2011-10-18T01:00:00.468944+03:00 SERVER2 kernel: Process fsnotify_mark (pid: 638, ti=f275c000 task=f2750bd0 task.ti=f275c000)
2011-10-18T01:00:00.468945+03:00 SERVER2 kernel: Stack:
2011-10-18T01:00:00.468946+03:00 SERVER2 kernel: f2750bd0 00000000 f2750bd0 c104fad0 f275dfa0 f275dfa0 f1e06044 f1e068c4
2011-10-18T01:00:00.468948+03:00 SERVER2 kernel: f2c47f68 00000000 c10ebc30 00000000 c104f744 00000000 00000000 00000000
2011-10-18T01:00:00.468949+03:00 SERVER2 kernel: 00000000 f275dfd4 f275dfd4 00000000 c104f6d0 f2c47f68 c1002fb6 00000000
2011-10-18T01:00:00.468950+03:00 SERVER2 kernel: Call Trace:
2011-10-18T01:00:00.468952+03:00 SERVER2 kernel: [<c104fad0>] ? autoremove_wake_function+0x0/0x40
2011-10-18T01:00:00.468953+03:00 SERVER2 kernel: [<c10ebc30>] ? fsnotify_mark_destroy+0x0/0x130
2011-10-18T01:00:00.468954+03:00 SERVER2 kernel: [<c104f744>] ? kthread+0x74/0x80
2011-10-18T01:00:00.468956+03:00 SERVER2 kernel: [<c104f6d0>] ? kthread+0x0/0x80
2011-10-18T01:00:00.468957+03:00 SERVER2 kernel: [<c1002fb6>] ? kernel_thread_helper+0x6/0x10
2011-10-18T01:00:00.468959+03:00 SERVER2 kernel: Code: c1 b8 10 51 87 c1 e8 cb 8c f6 ff 8b 54 24 18 8d 42 c4 39 da 8b 48 3c 74 32 8d 79 c4 eb 0a 90 8d b4 26 00 00 00 00 89 ef 8b 68 40 <89> 69 04 89 4d 00 89 50 3c 89 50 40 e8 fa fe ff ff 8b 4f 3c 8d
2011-10-18T01:00:00.468961+03:00 SERVER2 kernel: EIP: [<c10ebcb5>] fsnotify_mark_destroy+0x85/0x130 SS:ESP 0068:f275df90
2011-10-18T01:00:00.468961+03:00 SERVER2 kernel: CR2: 0000000000000004
2011-10-18T01:00:00.468962+03:00 SERVER2 kernel: ---[ end trace 6cf83470c1711e1c ]---
2011-10-18T01:00:08.710733+03:00 SERVER2 auditd: type=CONFIG_CHANGE msg=audit(1318888800.190:385): auid=4294967295 ses=4294967295 op="remove rule" key="etc-directory" list=4 res=1
2011-10-18T01:00:08.710753+03:00 SERVER2 auditd: type=CONFIG_CHANGE msg=audit(1318888800.190:386): auid=4294967295 ses=4294967295 op="remove rule" key="sbin-directory" list=4 res=1
2011-10-18T01:00:08.710759+03:00 SERVER2 auditd: type=CONFIG_CHANGE msg=audit(1318888800.190:387): auid=4294967295 ses=4294967295 op="remove rule" key="bin-directory" list=4 res=1
2011-10-18T01:00:08.710764+03:00 SERVER2 auditd: type=CONFIG_CHANGE msg=audit(1318888800.190:388): auid=4294967295 ses=4294967295 op="remove rule" key="usr-sbin-directory" list=4 res=1
2011-10-18T01:00:08.710769+03:00 SERVER2 auditd: type=CONFIG_CHANGE msg=audit(1318888800.190:389): auid=4294967295 ses=4294967295 op="remove rule" key="usr-bin-directory" list=4 res=1
2011-10-18T01:00:08.710773+03:00 SERVER2 auditd: type=CONFIG_CHANGE msg=audit(1318888800.190:390): auid=4294967295 ses=4294967295 op="remove rule" key="skip-lib-rc" list=4 res=1
2011-10-18T01:00:08.710780+03:00 SERVER2 auditd: type=CONFIG_CHANGE msg=audit(1318888800.190:391): auid=4294967295 ses=4294967295 op="remove rule" key="lib-directory" list=4 res=1
2011-10-18T01:00:08.710783+03:00 SERVER2 auditd: type=CONFIG_CHANGE msg=audit(1318888800.190:392): auid=4294967295 ses=4294967295 op="remove rule" key="usr-lib-directory" list=4 res=1
2011-10-18T01:00:08.710785+03:00 SERVER2 auditd: type=CONFIG_CHANGE msg=audit(1318888800.190:393): auid=4294967295 ses=4294967295 op="remove rule" key="excluded-syscalls" list=4 res=1
2011-10-18T01:00:08.710787+03:00 SERVER2 auditd: type=CONFIG_CHANGE msg=audit(1318888800.190:394): audit_enabled=0 old=1 auid=4294967295 ses=4294967295 res=1
2011-10-18T01:00:08.710790+03:00 SERVER2 auditd: type=DAEMON_END msg=audit(1318888800.221:5152): auditd normal halt, sending auid=0 pid=9237 subj= res=success