On high filesystem io, for example on an emerge over many packets, the kernel throw sometimes messages like: vmap allocation for size 1048576 failed: use vmalloc=<size> to increase size. increasing vmalloc on boot, for example vmalloc=256m didn't help, the filesystem crash again. This error comes up on >=sys-kernel/gentoo-sources-2.6.34. Reproducible: Sometimes Steps to Reproduce: 1. format and mount an xfs Partition (I use lvm under it): mkfs.xfs -L blabla -b size=1024 /dev/vg/blablabla ; mount -L blabla /var/tmp/portage 2. emerge -e system 3. on another shell: tail -f /var/log/messages and wait. Actual Results: vmap allocation for size 1048576 failed: use vmalloc=<size> to increase size. xfs_buf_get: failed to map pages BUG: unable to handle kernel NULL pointer dereference at 00000008 IP: [<f19234b3>] xfs_da_do_buf+0x4e9/0x672 [xfs] *pdpt = 0000000019ad2001 *pde = 0000000000000000 Oops: 0000 [#1] SMP last sysfs file: /sys/devices/system/cpu/cpu1/cpufreq/scaling_cur_freq Modules linked in: ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state ipt_REJECT iptable_mangle xt_tcpudp iptable_filter ipt_addrtype xt_DSCP xt_dscp xt_string xt_NFQUEUE xt_multiport xt_mark xt_hashlimit xt_conntrack xt_connmark nf_conntrack ip_tables x_tables bridge stp llc ipv6 rfcomm l2cap bluetooth snd_pcm_oss snd_mixer_oss snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device ext4 jbd2 tun kvm_intel kvm fan cpufreq_powersave cpufreq_conservative cpufreq_userspace cpufreq_ondemand acpi_cpufreq freq_table mperf snd_hda_codec_analog arc4 ecb i915 iwlagn drm_kms_helper iwlcore snd_hda_intel snd_hda_codec drm sierra i2c_algo_bit r852 sm_common nand nand_ids nand_ecc i2c_i801 usbserial mtd thinkpad_acpi led_class rtc_cmos nvram pcspkr e1000e rtc_core rtc_lib wacom mac80211 i2c_core intel_agp agpgart yenta_socket pcmcia_rsrc video snd_pcm processor tpm_tis backlight cfg80211 tpm tpm_bios snd_timer iTCO_wdt output iTCO_vendor_support joydev rfkill thermal snd snd_page_alloc battery thermal_sys ac button tg3 libphy e1000 fuse xfs exportfs nfs auth_rpcgss nfs_acl lockd sunrpc jfs raid10 dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd usbhid ohci_hcd ssb uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss 3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport dmx3191d sym53c8xx qlogicfas408 gdth advansys initio BusLogic arcmsr aic7xxx aic79xx scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_pcmcia pcmcia pcmcia_core Pid: 5585, comm: mv Not tainted 2.6.36-gentoo-r5 #1 6463WNW/6463WNW EIP: 0060:[<f19234b3>] EFLAGS: 00210246 CPU: 0 EIP is at xfs_da_do_buf+0x4e9/0x672 [xfs] EAX: 00000001 EBX: edd27400 ECX: 00014005 EDX: cfa3fcb8 ESI: 00000008 EDI: 00000000 EBP: cfa3fcc8 ESP: cfa3fc5c DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 Process mv (pid: 5585, ti=cfa3f000 task=e73d4d10 task.ti=cfa3f000) Stack: 003807ba 00000000 00000000 c056fe44 dff0345c dd751d98 000200da 00000001 <0> 00000000 00000000 ffffffff dff03440 0000001a cfa3fe54 cfa3fe58 cfa3fe5c <0> cfa3fe60 cfa3fcbc c0123195 fffff000 00000003 c241e700 00000001 00000000 Call Trace: [<c0123195>] ? kmap_atomic_prot+0xc8/0xe7 [<f1923698>] ? xfs_da_read_buf+0x18/0x1d [xfs] [<f1927f94>] ? xfs_dir2_block_addname+0x59/0x689 [xfs] [<f1927f94>] ? xfs_dir2_block_addname+0x59/0x689 [xfs] [<c028dfdc>] ? random32+0x17/0x19 [<f1910f17>] ? xfs_bmap_last_offset+0xe8/0xfc [xfs] [<f19261f7>] ? xfs_dir2_isblock+0x1a/0x9a [xfs] [<f1926da4>] ? xfs_dir_createname+0x106/0x139 [xfs] [<f194bd57>] ? xfs_rename+0x249/0x49a [xfs] [<c01bf993>] ? do_lookup+0x64/0xf4 [<f195ce1d>] ? xfs_vn_rename+0x58/0x62 [xfs] [<c01c02a8>] ? vfs_rename+0x20e/0x34c [<c01bfa56>] ? __lookup_hash+0x33/0x92 [<c01c1ccd>] ? sys_renameat+0x164/0x1c9 [<c01a4cd6>] ? remove_vma+0x46/0x4c [<c01a5a4f>] ? do_munmap+0x217/0x231 [<c01c1d44>] ? sys_rename+0x12/0x14 [<c0102758>] ? sysenter_do_call+0x12/0x28 Code: 55 b8 8b 45 b4 eb 13 31 ff 83 7d f0 00 74 12 8b 4d 18 8d 55 f0 b8 01 00 00 00 e8 ea f8 ff ff 89 c7 83 7d 14 01 0f 85 da 00 00 00 <8b> 57 08 89 55 b4 89 d1 8b 52 08 8b 01 0f c8 86 f2 0f b7 d2 81 EIP: [<f19234b3>] xfs_da_do_buf+0x4e9/0x672 [xfs] SS:ESP 0068:cfa3fc5c CR2: 0000000000000008 ---[ end trace b88f3c9551a0f88d ]--- vmap allocation for size 1048576 failed: use vmalloc=<size> to increase size. xfs_buf_get: failed to map pages Filesystem "dm-7": XFS internal error xfs_trans_cancel at line 1796 of file fs/xfs/xfs_trans.c. Caller 0xf1952a5f Pid: 16808, comm: plasma-desktop Tainted: G D 2.6.36-gentoo-r5 #1 Call Trace: [<f19309c8>] xfs_error_report+0x2c/0x2e [xfs] [<f194de06>] xfs_trans_cancel+0x54/0x155 [xfs] [<f1952a5f>] ? xfs_create+0x4d7/0x553 [xfs] [<f1952a5f>] xfs_create+0x4d7/0x553 [xfs] [<f195d1e3>] xfs_vn_mknod+0xca/0x156 [xfs] [<f195d288>] xfs_vn_create+0xa/0xc [xfs] [<c01c07bd>] vfs_create+0x85/0xcf [<c01c0e4e>] do_last+0x277/0x4f8 [<c01c25f1>] do_filp_open+0x197/0x458 [<c01b6ef9>] do_sys_open+0x48/0xc9 [<c01b6fbc>] sys_open+0x1e/0x26 [<c0102758>] sysenter_do_call+0x12/0x28 [<c03f0000>] ? x86_pmu_notifier+0x33/0x48 xfs_force_shutdown(dm-7,0x8) called from line 1797 of file fs/xfs/xfs_trans.c. Return address = 0xf194de1d XFS: Transforming an alert into a BUG. Filesystem "dm-7": Corruption of in-memory data detected. Shutting down filesystem: dm-7 ------------[ cut here ]------------ kernel BUG at fs/xfs/support/debug.c:101! invalid opcode: 0000 [#2] SMP last sysfs file: /sys/devices/system/cpu/cpu1/cpufreq/scaling_cur_freq Modules linked in: ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state ipt_REJECT iptable_mangle xt_tcpudp iptable_filter ipt_addrtype xt_DSCP xt_dscp xt_string xt_NFQUEUE xt_multiport xt_mark xt_hashlimit xt_conntrack xt_connmark nf_conntrack ip_tables x_tables bridge stp llc ipv6 rfcomm l2cap bluetooth snd_pcm_oss snd_mixer_oss snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device ext4 jbd2 tun kvm_intel kvm fan cpufreq_powersave cpufreq_conservative cpufreq_userspace cpufreq_ondemand acpi_cpufreq freq_table mperf snd_hda_codec_analog arc4 ecb i915 iwlagn drm_kms_helper iwlcore snd_hda_intel snd_hda_codec drm sierra i2c_algo_bit r852 sm_common nand nand_ids nand_ecc i2c_i801 usbserial mtd thinkpad_acpi led_class rtc_cmos nvram pcspkr e1000e rtc_core rtc_lib wacom mac80211 i2c_core intel_agp agpgart yenta_socket pcmcia_rsrc video snd_pcm processor tpm_tis backlight cfg80211 tpm tpm_bios snd_timer iTCO_wdt output iTCO_vendor_support joydev rfkill thermal snd snd_page_alloc battery thermal_sys ac button tg3 libphy e1000 fuse xfs exportfs nfs auth_rpcgss nfs_acl lockd sunrpc jfs raid10 dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd usbhid ohci_hcd ssb uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss 3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport dmx3191d sym53c8xx qlogicfas408 gdth advansys initio BusLogic arcmsr aic7xxx aic79xx scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_pcmcia pcmcia pcmcia_core Pid: 16808, comm: plasma-desktop Tainted: G D 2.6.36-gentoo-r5 #1 6463WNW/6463WNW EIP: 0060:[<f1960375>] EFLAGS: 00010246 CPU: 0 EIP is at xfs_fs_vcmn_err+0xa2/0xae [xfs] EAX: 00000282 EBX: 00000000 ECX: f19834e0 EDX: 00000282 ESI: edd20c00 EDI: f19788a3 EBP: ead22d74 ESP: ead22d64 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 Process plasma-desktop (pid: 16808, ti=ead22000 task=ec6e51c0 task.ti=ead22000) Stack: 00000282 00000000 00000008 00000000 ead22d84 f1930996 ead22d9c edd20c00 <0> ead22db0 f1954917 00000010 00000001 edd20c00 f19788a3 edac4580 00000000 <0> df5c6c58 00000000 0000000c ead22ddc f194de1d 00000705 f197784c 00000704 Call Trace: [<f1930996>] ? xfs_cmn_err+0x40/0x46 [xfs] [<f1954917>] ? xfs_do_force_shutdown+0x78/0xf9 [xfs] [<f194de1d>] ? xfs_trans_cancel+0x6b/0x155 [xfs] [<f1952a5f>] ? xfs_create+0x4d7/0x553 [xfs] [<f1952a5f>] ? xfs_create+0x4d7/0x553 [xfs] [<f195d1e3>] ? xfs_vn_mknod+0xca/0x156 [xfs] [<f195d288>] ? xfs_vn_create+0xa/0xc [xfs] [<c01c07bd>] ? vfs_create+0x85/0xcf [<c01c0e4e>] ? do_last+0x277/0x4f8 [<c01c25f1>] ? do_filp_open+0x197/0x458 [<c01b6ef9>] ? do_sys_open+0x48/0xc9 [<c01b6fbc>] ? sys_open+0x1e/0x26 [<c0102758>] ? sysenter_do_call+0x12/0x28 [<c03f0000>] ? x86_pmu_notifier+0x33/0x48 Code: 00 68 e4 34 98 f1 ff 34 9d c8 5c 96 f1 68 32 9e 97 f1 e8 94 3e a9 ce 83 c4 0c 8b 55 f0 b8 e0 34 98 f1 e8 2d 61 a9 ce 85 db 75 04 <0f> 0b eb fe 8d 65 f4 5b 5e 5f 5d c3 55 b8 e0 34 98 f1 89 e5 57 EIP: [<f1960375>] xfs_fs_vcmn_err+0xa2/0xae [xfs] SS:ESP 0068:ead22d64 ---[ end trace b88f3c9551a0f88e ]--- Filesystem "dm-7": xfs_log_force: error 5 returned. Filesystem "dm-7": xfs_log_force: error 5 returned. Expected Results: nothing of above The kernel dump is from my Lenovo T61 (32-bit gentoo) (Intel(R) Core(TM)2 Duo CPU T8300 @ 2.40GHz), but similar errors pop up on 32-bit Gentoo kvm guest systems with kernel gentoo-sources-2.6.36.
I can confirm this bug. Linux 2.6.36-gentoo-r5 #1 SMP Core(TM) i5 CPU 750 @ 2.67GHz 32 bit KVM guest w/ LVM on 64 bit Host (same OS than guest).
(In reply to comment #1) > I can confirm this bug. > Linux 2.6.36-gentoo-r5 #1 SMP Core(TM) i5 CPU 750 @ 2.67GHz > 32 bit KVM guest w/ LVM on 64 bit Host (same OS than guest). Also with a tainted kernel? All of you - please paste your `emerge --info' output in a comment, and attach your kernel .config as a file.
Created attachment 260230 [details] Output of emerge --info
> Also with a tainted kernel? What is a tainted kernel? Does this help: # grep -iF tainted /var/log/messages Jan 14 10:20:49 ittest01 kernel: Pid: 24964, comm: rsync Not tainted 2.6.36-gentoo-r5 #1 Jan 14 10:43:17 ittest01 kernel: Pid: 13242, comm: rsync Not tainted 2.6.36-gentoo-r5 #1 Jan 14 10:46:30 ittest01 kernel: Pid: 13756, comm: rm Not tainted 2.6.36-gentoo-r5 #1 /Bochs Jan 14 13:25:59 ittest01 kernel: Pid: 29339, comm: rsync Not tainted 2.6.36-gentoo-r5 #1 Jan 14 13:30:11 ittest01 kernel: Pid: 29927, comm: rm Not tainted 2.6.36-gentoo-r5 #1 /Bochs Jan 14 13:41:00 ittest01 kernel: Pid: 14211, comm: rsync Not tainted 2.6.36-gentoo-r5 #1 /Bochs Jan 17 10:42:20 ittest01 kernel: Pid: 24068, comm: tar Tainted: G D 2.6.36-gentoo-r5 #1 Jan 17 10:49:17 ittest01 kernel: Pid: 12861, comm: nmbd Tainted: G D 2.6.36-gentoo-r5 #1 /Bochs Jan 17 10:49:59 ittest01 kernel: Pid: 25034, comm: sshd Tainted: G D 2.6.36-gentoo-r5 #1 /Bochs Jan 17 14:55:08 ittest01 kernel: Pid: 12276, comm: modprobe Not tainted 2.6.36-gentoo-r5 #1 Jan 17 14:55:22 ittest01 kernel: Pid: 12556, comm: modprobe Not tainted 2.6.36-gentoo-r5 #1 Jan 17 14:55:23 ittest01 kernel: Pid: 12558, comm: modprobe Not tainted 2.6.36-gentoo-r5 #1 Jan 17 14:55:27 ittest01 kernel: Pid: 12622, comm: modprobe Not tainted 2.6.36-gentoo-r5 #1 Jan 17 14:55:28 ittest01 kernel: Pid: 12627, comm: modprobe Not tainted 2.6.36-gentoo-r5 #1 Jan 17 14:55:29 ittest01 kernel: Pid: 12631, comm: modprobe Not tainted 2.6.36-gentoo-r5 #1 Jan 17 14:55:32 ittest01 kernel: Pid: 12864, comm: modprobe Not tainted 2.6.36-gentoo-r5 #1 Jan 17 14:55:36 ittest01 kernel: Pid: 13188, comm: modprobe Not tainted 2.6.36-gentoo-r5 #1 Jan 17 14:56:13 ittest01 kernel: Pid: 13198, comm: modprobe Not tainted 2.6.36-gentoo-r5 #1 Jan 17 15:00:38 ittest01 kernel: Pid: 13582, comm: modprobe Not tainted 2.6.36-gentoo-r5 #1 Jan 18 18:21:19 ittest01 kernel: Pid: 4121, comm: rsync Not tainted 2.6.36-gentoo-r5 #1
Output of dmesg: vmap allocation for size 4194304 failed: use vmalloc=<size> to increase size. xfs_buf_get: failed to map pages vmap allocation for size 4194304 failed: use vmalloc=<size> to increase size. xfs_buf_get: failed to map pages vmap allocation for size 4194304 failed: use vmalloc=<size> to increase size. xfs_buf_get: failed to map pages vmap allocation for size 4194304 failed: use vmalloc=<size> to increase size. xfs_buf_get: failed to map pages vmap allocation for size 4194304 failed: use vmalloc=<size> to increase size. xfs_buf_get: failed to map pages Filesystem "dm-7": XFS internal error xfs_trans_cancel at line 1815 of file fs/xfs/xfs_trans.c. Caller 0xf91b2a1b Pid: 19917, comm: install Not tainted 2.6.37-gentoo #1 Call Trace: [<f9196bba>] xfs_error_report+0x2c/0x2e [xfs] [<f91aed2b>] xfs_trans_cancel+0x4b/0xc9 [xfs] [<f91b2a1b>] ? xfs_create+0x48c/0x508 [xfs] [<f91b2a1b>] xfs_create+0x48c/0x508 [xfs] [<f91bbbb6>] xfs_vn_mknod+0xc8/0x153 [xfs] [<f91bbc5a>] xfs_vn_create+0xa/0xc [xfs] [<c01bf6f4>] vfs_create+0x85/0xcf [<c01bfd9b>] do_last+0x28a/0x50b [<c01c1589>] do_filp_open+0x1a7/0x47b [<c01ae4c2>] ? free_pages_and_swap_cache+0x6a/0x7d [<c01b5d1a>] do_sys_open+0x44/0xbf [<c01b5dd7>] sys_open+0x1e/0x26 [<c0102818>] sysenter_do_call+0x12/0x28 xfs_force_shutdown(dm-7,0x8) called from line 1816 of file fs/xfs/xfs_trans.c. Return address = 0xf91aed41 Filesystem "dm-7": Corruption of in-memory data detected. Shutting down filesystem: dm-7 Please umount the filesystem, and rectify the problem(s) Filesystem "dm-7": xfs_log_force: error 5 returned. Filesystem "dm-7": xfs_log_force: error 5 returned. Filesystem "dm-7": xfs_log_force: error 5 returned. Filesystem "dm-7": xfs_log_force: error 5 returned. Filesystem "dm-7": xfs_log_force: error 5 returned. Filesystem "dm-7": xfs_log_force: error 5 returned. Filesystem "dm-7": xfs_log_force: error 5 returned.
(In reply to comment #5) The previous output was from a gentoo-sources-2.6.37 kernel.
Created attachment 260238 [details] kernelconfig 2.6.36-gentoo-r5 Lenovo T61 Core2
Created attachment 260240 [details] Lenovo T61 emerge --info
Created attachment 260242 [details] Kernel 2.6.36-r5 KVM-guest 32bit
Created attachment 260243 [details] Kernel config 2.6.37 KVM guest 32bit
I think three Kernel Configs are enough :-) I switched back to "2.6.32-gentoo-r24 #1 SMP" and use it on high disk io without any problems.
(In reply to comment #4) > What is a tainted kernel? Does this help: He means, Have you too many kernel modules loaded? If you all have many modules loaded, maybe you should pass a bigger vmalloc option/value in kernel's cmdline at startup. To see what is going on with vmalloc you can `grep -i vmalloc /proc/meminfo` and/or `cat /proc/modules` or `lsmod`, to see the size of modules etc. You can also try to debug this by checking the above results in previous and newer kernels. I'll resolve this as TEST-REQUEST. If you still think that this is a gentoo bug, please reopen.
(In reply to comment #12) > (In reply to comment #4) > > What is a tainted kernel? Does this help: > > He means, Have you too many kernel modules loaded? If you all have many modules > loaded, maybe you should pass a bigger vmalloc option/value in kernel's cmdline > at startup. # root@2.6.36-gentoo # lsmod Module Size Used by ipv6 182642 87 nfsd 195663 11 floppy 41011 0 tpm_tis 5165 0 tpm 7645 1 tpm_tis tpm_bios 3617 1 tpm i2c_piix4 6464 0 rtc_cmos 6774 0 rtc_core 9735 1 rtc_cmos i2c_core 12268 1 i2c_piix4 i6300esb 3415 0 thermal 9407 0 rtc_lib 1281 1 rtc_core processor 22584 0 pcspkr 1195 0 virtio_net 8164 0 button 3626 0 thermal_sys 8545 2 thermal,processor tg3 97162 0 libphy 12050 1 tg3 e1000 74940 0 fuse 47270 0 xfs 528542 15 exportfs 2574 2 nfsd,xfs nfs 203606 2 auth_rpcgss 26153 2 nfsd,nfs nfs_acl 1655 2 nfsd,nfs lockd 48813 2 nfsd,nfs sunrpc 133733 19 nfsd,nfs,auth_rpcgss,nfs_acl,lockd jfs 136033 0 raid10 17715 0 dm_snapshot 24676 0 dm_crypt 8623 0 dm_mirror 9949 0 dm_region_hash 5696 1 dm_mirror dm_log 6568 2 dm_mirror,dm_region_hash dm_mod 47900 51 dm_snapshot,dm_crypt,dm_mirror,dm_log scsi_wait_scan 459 0 sbp2 14947 0 ohci1394 21131 0 ieee1394 51276 2 sbp2,ohci1394 sl811_hcd 7262 0 usbhid 26037 0 ohci_hcd 17289 0 ssb 28994 1 ohci_hcd uhci_hcd 15652 0 usb_storage 30061 0 ehci_hcd 27642 0 usbcore 89458 7 sl811_hcd,usbhid,ohci_hcd,uhci_hcd,usb_storage,ehci_hcd aic94xx 56065 0 libsas 31650 1 aic94xx lpfc 343703 0 qla2xxx 255871 0 megaraid_sas 29063 0 megaraid_mbox 21317 0 megaraid_mm 5694 1 megaraid_mbox megaraid 31109 0 aacraid 51353 0 sx8 9701 0 DAC960 55852 0 cciss 80034 0 3w_9xxx 24098 0 3w_xxxx 18296 0 mptsas 39873 0 scsi_transport_sas 16745 3 aic94xx,libsas,mptsas mptfc 10473 0 scsi_transport_fc 28159 3 lpfc,qla2xxx,mptfc scsi_tgt 6866 1 scsi_transport_fc mptspi 11747 0 mptscsih 21091 3 mptsas,mptfc,mptspi mptbase 67805 4 mptsas,mptfc,mptspi,mptscsih atp870u 22285 0 dc395x 24912 0 qla1280 17128 0 imm 7058 0 parport 21355 1 imm dmx3191d 7621 0 sym53c8xx 56425 0 qlogicfas408 3517 0 gdth 70185 0 advansys 46330 0 initio 12889 0 BusLogic 17136 0 arcmsr 20528 0 aic7xxx 92894 0 aic79xx 97820 0 scsi_transport_spi 14768 5 mptspi,dmx3191d,sym53c8xx,aic7xxx,aic79xx sg 19462 0 pdc_adma 3942 0 sata_inic162x 5239 0 sata_mv 19454 0 ata_piix 17228 0 ahci 16545 0 libahci 13180 1 ahci sata_qstor 3889 0 sata_vsc 2846 0 sata_uli 2041 0 sata_sis 2650 0 sata_sx4 6586 0 sata_nv 15083 0 sata_via 5372 0 sata_svw 2918 0 sata_sil24 8813 0 sata_sil 5364 0 sata_promise 7177 0 pata_pcmcia 6933 0 pcmcia 25043 2 ssb,pata_pcmcia pcmcia_core 8237 1 pcmcia
(In reply to comment #12) root@2.6.36-gentoo-r5 # grep -i vmalloc /proc/meminfo VmallocTotal: 262144 kB VmallocUsed: 13776 kB VmallocChunk: 176648 kB root@2.6.36-gentoo-r5 # cat /boot/grub/grub.conf # snip title Gentoo Linux 2.6.36-r5 root (hd0,0) kernel /boot/kernel-genkernel-x86-2.6.36-gentoo-r5 root=/dev/ram0 real_root=/dev/vda3 doscsi elevator=deadline vmalloc=256M initrd /boot/initramfs-genkernel-x86-2.6.36-gentoo-r5
The error: Jan 21 16:44:16 ittest01 kernel: vmap allocation for size 2097152 failed: use vmalloc=<size> to increase size. Jan 21 16:44:16 ittest01 kernel: xfs_buf_get: failed to map pages Jan 21 16:44:16 ittest01 kernel: BUG: unable to handle kernel NULL pointer dereference at 00000008 Jan 21 16:44:16 ittest01 kernel: IP: [<f08b3691>] xfs_da_do_buf+0x53f/0x6a5 [xfs] Jan 21 16:44:16 ittest01 kernel: *pde = 00000000 Jan 21 16:44:16 ittest01 kernel: Oops: 0000 [#1] SMP Jan 21 16:44:16 ittest01 kernel: last sysfs file: /sys/devices/virtio-pci/virtio1/net/eth0/broadcast Jan 21 16:44:16 ittest01 kernel: Modules linked in: ipv6 nfsd floppy tpm_tis tpm tpm_bios i2c_piix4 rtc_cmos rtc_core i2c_core i6300esb thermal rtc_lib processor pcspkr virtio_net button thermal_sys tg3 libphy e1000 fuse xfs exportfs nfs auth_rpcgss nfs_acl lockd sunrpc jfs raid10 dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd usbhid ohci_hcd ssb uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss 3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport dmx3191d sym53c8xx qlogicfas408 gdth advansys initio BusLogic arcmsr aic7xxx aic79xx scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_pcmcia pcmcia pcmcia_core Jan 21 16:44:16 ittest01 kernel: Jan 21 16:44:16 ittest01 kernel: Pid: 15270, comm: configure Not tainted 2.6.36-gentoo-r5 #1 /Bochs Jan 21 16:44:16 ittest01 kernel: EIP: 0060:[<f08b3691>] EFLAGS: 00210246 CPU: 1 Jan 21 16:44:16 ittest01 kernel: EIP is at xfs_da_do_buf+0x53f/0x6a5 [xfs] Jan 21 16:44:16 ittest01 kernel: EAX: 00000001 EBX: ee389400 ECX: 00000008 EDX: da297cdc Jan 21 16:44:16 ittest01 kernel: ESI: 00000000 EDI: 00000000 EBP: 00000000 ESP: da297c88 Jan 21 16:44:16 ittest01 kernel: DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 Jan 21 16:44:16 ittest01 kernel: Process configure (pid: 15270, ti=da297000 task=edd7e550 task.ti=da297000) Jan 21 16:44:16 ittest01 kernel: Stack: Jan 21 16:44:16 ittest01 kernel: 0069207c 00000000 d6af9880 f08aee4c 00000000 ffffffff 00000001 d6af989c Jan 21 16:44:16 ittest01 kernel: <0> ee9dcc40 c69ccee0 00000000 00000000 00d24173 00000000 d0aaf580 c041fc2c Jan 21 16:44:16 ittest01 kernel: <0> f08c2075 0000000d 00000000 00000000 00000001 00000000 da297d10 ee389400 Jan 21 16:44:16 ittest01 kernel: Call Trace: Jan 21 16:44:16 ittest01 kernel: [<f08aee4c>] ? xfs_btree_setbuf+0x18/0x75 [xfs] Jan 21 16:44:16 ittest01 kernel: [<c041fc2c>] ? _raw_spin_lock+0x5/0x7 Jan 21 16:44:16 ittest01 kernel: [<f08c2075>] ? xfs_iget+0x513/0x55a [xfs] Jan 21 16:44:16 ittest01 kernel: [<f08b3854>] ? xfs_da_read_buf+0x19/0x1e [xfs] Jan 21 16:44:16 ittest01 kernel: [<f08b724f>] ? xfs_dir2_block_addname+0x57/0x6b3 [xfs] Jan 21 16:44:16 ittest01 kernel: [<f08b724f>] ? xfs_dir2_block_addname+0x57/0x6b3 [xfs] Jan 21 16:44:16 ittest01 kernel: [<f08d5aba>] ? xfs_trans_add_item+0x15/0x44 [xfs] Jan 21 16:44:16 ittest01 kernel: [<f08d7690>] ? xfs_trans_ijoin+0x1f/0x25 [xfs] Jan 21 16:44:16 ittest01 kernel: [<f08a43a9>] ? xfs_bmap_last_offset+0xc7/0xda [xfs] Jan 21 16:44:16 ittest01 kernel: [<f08b57cb>] ? xfs_dir2_isblock+0x14/0x5b [xfs] Jan 21 16:44:16 ittest01 kernel: [<f08b6282>] ? xfs_dir_createname+0xfb/0x131 [xfs] Jan 21 16:44:16 ittest01 kernel: [<f08d9a3c>] ? xfs_create+0x337/0x525 [xfs] Jan 21 16:44:16 ittest01 kernel: [<f08e2a81>] ? xfs_vn_mknod+0xc8/0x155 [xfs] Jan 21 16:44:16 ittest01 kernel: [<c01da7c2>] ? vfs_create+0x77/0xbe Jan 21 16:44:16 ittest01 kernel: [<c01dae31>] ? do_last+0x270/0x4e2 Jan 21 16:44:16 ittest01 kernel: [<c01dc5fd>] ? do_filp_open+0x1ae/0x4e1 Jan 21 16:44:16 ittest01 kernel: [<c01c821b>] ? free_pages_and_swap_cache+0x67/0x74 Jan 21 16:44:16 ittest01 kernel: [<c01d11ef>] ? do_sys_open+0x45/0xc8 Jan 21 16:44:16 ittest01 kernel: [<c01d12b6>] ? sys_open+0x1e/0x23 Jan 21 16:44:16 ittest01 kernel: [<c0127158>] ? sysenter_do_call+0x12/0x28 Jan 21 16:44:16 ittest01 kernel: Code: 10 eb 16 31 ed 83 7c 24 54 00 74 14 8b 4c 24 7c 8d 54 24 54 b8 01 00 00 00 e8 dc f9 ff ff 89 c5 83 7c 24 78 01 0f 85 d5 00 00 00 <8b> 55 08 89 54 24 10 89 d1 8b 52 08 8b 01 0f c8 86 f2 0f b7 d2 Jan 21 16:44:16 ittest01 kernel: EIP: [<f08b3691>] xfs_da_do_buf+0x53f/0x6a5 [xfs] SS:ESP 0068:da297c88 Jan 21 16:44:16 ittest01 kernel: CR2: 0000000000000008 Jan 21 16:44:16 ittest01 kernel: ---[ end trace 4053a7de60e49e7e ]--- Jan 21 16:44:16 ittest01 kernel: vmap allocation for size 2097152 failed: use vmalloc=<size> to increase size. Jan 21 16:44:16 ittest01 kernel: xfs_buf_get: failed to map pages Jan 21 16:44:16 ittest01 kernel: vmap allocation for size 2097152 failed: use vmalloc=<size> to increase size. Jan 21 16:44:16 ittest01 kernel: xfs_buf_get: failed to map pages Jan 21 16:44:26 ittest01 kernel: vmap allocation for size 2097152 failed: use vmalloc=<size> to increase size.
Behaviour of XFS filesystem after crash: root@2.6.36-gentoo-r5 # ll /var/tmp/ ls: cannot access /var/tmp/jboss: Cannot allocate memory ls: cannot access /var/tmp/pear: Cannot allocate memory ls: cannot access /var/tmp/pear-build-root: Cannot allocate memory total 12 drw-rw---- 4 root root 28 Nov 19 18:25 backup drwxrwxr-x 2 portage portage 6 Jan 21 16:43 binpkgs drwxr-xr-x 7 root root 4096 Jan 19 13:19 genkernel drwxrwxr-x 8 root www 99 Jan 12 10:09 git drwxrwx--- 23 root git 4096 Jan 14 14:18 git-repo ?????????? ? ? ? ? ? jboss drwxr-xr-x 7 root root 68 Dec 22 15:39 nginx ?????????? ? ? ? ? ? pear ?????????? ? ? ? ? ? pear-build-root drwxrwxr-x 4 portage portage 35 Jan 21 16:43 portage drwxr-xr-x 5 root root 56 Jan 14 12:58 portage-tmp drwxrwxr-x 4 tomcat tomcat 4096 Jan 21 16:14 tomcat-6
I'll reopen this.
Assigning to @kernel team. Guys, this may be an upstream bug, but better to leave the decision to you.
(In reply to comment #18) > Assigning to @kernel team. Guys, this may be an upstream bug, but better to https://bugzilla.kernel.org/show_bug.cgi?id=27492 > leave the decision to you. > You are right. It's a main kernel bug. I fetched the actual stable kernel 2.6.37 from kernel.org and it crashed also: vmap allocation for size 1048576 failed: use vmalloc=<size> to increase size. xfs_buf_get: failed to map pages vmap allocation for size 1048576 failed: use vmalloc=<size> to increase size. xfs_buf_get: failed to map pages vmap allocation for size 1048576 failed: use vmalloc=<size> to increase size. xfs_buf_get: failed to map pages vmap allocation for size 1048576 failed: use vmalloc=<size> to increase size. xfs_buf_get: failed to map pages BUG: unable to handle kernel NULL pointer dereference at 00000008 IP: [<f9187b72>] xfs_da_do_buf+0x4c8/0x61f [xfs] *pde = 00000000 Oops: 0000 [#1] SMP last sysfs file: /sys/devices/virtio-pci/virtio1/net/eth0/broadcast Modules linked in: ipv6 nfsd rtc_cmos tpm_tis rtc_core tpm tpm_bios rtc_lib i2c_piix4 virtio_net i6300esb i2c_core virtio_balloon processor pcspkr floppy thermal_sys button tg3 libphy e1000 fuse xfs exportfs nfs auth_rpcgss nfs_acl lockd sunrpc jfs raid10 dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod scsi_wait_scan sl811_hcd usbhid ohci_hcd ssb uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss 3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport dmx3191d sym53c8xx qlogicfas408 gdth advansys initio BusLogic arcmsr aic7xxx aic79xx scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_pcmcia pcmcia pcmcia_core Pid: 425, comm: i686-pc-linux-g Not tainted 2.6.37 #1 /Bochs EIP: 0060:[<f9187b72>] EFLAGS: 00210246 CPU: 1 EIP is at xfs_da_do_buf+0x4c8/0x61f [xfs] EAX: 00000001 EBX: f5947800 ECX: 00000008 EDX: c5631d64 ESI: 00000000 EDI: 00000000 EBP: c5631d74 ESP: c5631d0c DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 Process i686-pc-linux-g (pid: 425, ti=c5630000 task=c5248960 task.ti=c5630000) Stack: 007adb5f 00000000 c5631dc0 c55870dc f4c26220 00000000 00000001 00000000 00000000 ffffffff c55870c0 00000000 00000000 c5631d48 c011d903 c5631d5c c0414e4e c5631d5c 00200286 eb74ebe0 c5631d6c 00000001 00000000 c5631d9c Call Trace: [<c011d903>] ? default_spin_lock_flags+0x8/0xd [<c0414e4e>] ? _raw_spin_lock_irqsave+0x20/0x28 [<f9187d25>] ? xfs_da_read_buf+0x18/0x1d [xfs] [<f918af47>] ? xfs_dir2_block_lookup_int+0x34/0x178 [xfs] [<f918af47>] ? xfs_dir2_block_lookup_int+0x34/0x178 [xfs] [<f919fe7c>] ? xfs_log_release_iclog+0x11/0x3a [xfs] [<f918b3a0>] ? xfs_dir2_block_removename+0x3d/0x18a [xfs] [<f9179022>] ? xfs_bmap_last_offset+0xe4/0xfa [xfs] [<f9189c6b>] ? xfs_dir2_isblock+0x18/0x62 [xfs] [<f918a213>] ? xfs_dir_removename+0xb4/0xe6 [xfs] [<f91ad30a>] ? xfs_remove+0x1d9/0x306 [xfs] [<f91b6d1a>] ? xfs_vn_unlink+0x30/0x62 [xfs] [<c01bfdd2>] ? vfs_unlink+0x5f/0xbc [<c01c1c3b>] ? do_unlinkat+0x94/0xfe [<c01a8e06>] ? do_munmap+0x217/0x231 [<c01c1cb5>] ? sys_unlink+0x10/0x12 [<c0102858>] ? sysenter_do_call+0x12/0x28 Code: 55 b8 8b 45 b4 eb 13 31 ff 83 7d f0 00 74 12 8b 4d 18 8d 55 f0 b8 01 00 00 00 e8 4c fa ff ff 89 c7 83 7d 14 01 0f 85 c7 00 00 00 <8b> 57 08 89 55 b4 89 d1 8b 52 08 8b 01 0f c8 86 f2 0f b7 d2 81 EIP: [<f9187b72>] xfs_da_do_buf+0x4c8/0x61f [xfs] SS:ESP 0068:c5631d0c CR2: 0000000000000008 ---[ end trace 4aa7015e260c3faa ]--- vmap allocation for size 1048576 failed: use vmalloc=<size> to increase size. xfs_buf_get: failed to map pages vmap allocation for size 1048576 failed: use vmalloc=<size> to increase size. xfs_buf_get: failed to map pages
I see you posted this upstream. Well follow that bug and back port any fixes identified.
Reflected upstream status.