Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 351938

Summary: >=sys-kernel/gentoo-sources-2.6.34: xfs runs out of vmalloc memory (xfs_buf_get: failed to map pages)
Product: Gentoo Linux Reporter: Jochen Buchholz <bookwood>
Component: [OLD] Core systemAssignee: Gentoo Kernel Bug Wranglers and Kernel Maintainers <kernel>
Status: RESOLVED OBSOLETE    
Severity: normal CC: bookwood, pchrist, th.geist
Priority: High    
Version: 10.0   
Hardware: x86   
OS: Linux   
URL: https://bugzilla.kernel.org/show_bug.cgi?id=27492
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: Output of emerge --info
kernelconfig 2.6.36-gentoo-r5 Lenovo T61 Core2
Lenovo T61 emerge --info
Kernel 2.6.36-r5 KVM-guest 32bit
Kernel config 2.6.37 KVM guest 32bit

Description Jochen Buchholz 2011-01-17 20:54:06 UTC
On high filesystem io, for example on an emerge over many packets, the kernel throw sometimes messages like:
vmap allocation for size 1048576 failed: use vmalloc=<size> to increase size.
increasing vmalloc on boot, for example vmalloc=256m didn't help, the filesystem crash again.

This error comes up on  >=sys-kernel/gentoo-sources-2.6.34.

Reproducible: Sometimes

Steps to Reproduce:
1. format and mount an xfs Partition (I use lvm under it):
mkfs.xfs -L blabla -b size=1024 /dev/vg/blablabla ; mount -L blabla /var/tmp/portage
2. emerge -e system
3. on another shell:
tail -f /var/log/messages
 and wait.

Actual Results:  
vmap allocation for size 1048576 failed: use vmalloc=<size> to increase size.
xfs_buf_get: failed to map pages
BUG: unable to handle kernel NULL pointer dereference at 00000008
IP: [<f19234b3>] xfs_da_do_buf+0x4e9/0x672 [xfs]
*pdpt = 0000000019ad2001 *pde = 0000000000000000
Oops: 0000 [#1] SMP
last sysfs file: /sys/devices/system/cpu/cpu1/cpufreq/scaling_cur_freq
Modules linked in: ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state ipt_REJECT iptable_mangle xt_tcpudp iptable_filter ipt_addrtype xt_DSCP xt_dscp xt_string xt_NFQUEUE xt_multiport xt_mark xt_hashlimit xt_conntrack xt_connmark nf_conntrack ip_tables x_tables bridge stp llc ipv6 rfcomm l2cap bluetooth snd_pcm_oss snd_mixer_oss snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device ext4 jbd2 tun kvm_intel kvm fan cpufreq_powersave cpufreq_conservative cpufreq_userspace cpufreq_ondemand acpi_cpufreq freq_table mperf snd_hda_codec_analog arc4 ecb i915 iwlagn drm_kms_helper iwlcore snd_hda_intel snd_hda_codec drm sierra i2c_algo_bit r852 sm_common nand nand_ids nand_ecc i2c_i801 usbserial mtd thinkpad_acpi led_class rtc_cmos nvram pcspkr e1000e rtc_core rtc_lib wacom mac80211 i2c_core intel_agp agpgart yenta_socket pcmcia_rsrc video snd_pcm processor tpm_tis backlight cfg80211 tpm tpm_bios snd_timer iTCO_wdt output iTCO_vendor_support joydev rfkill thermal snd snd_page_alloc battery thermal_sys ac button tg3 libphy e1000 fuse xfs exportfs nfs auth_rpcgss nfs_acl lockd sunrpc jfs raid10 dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd usbhid ohci_hcd ssb uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss 3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport dmx3191d sym53c8xx qlogicfas408 gdth advansys initio BusLogic arcmsr aic7xxx aic79xx scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_pcmcia pcmcia pcmcia_core

Pid: 5585, comm: mv Not tainted 2.6.36-gentoo-r5 #1 6463WNW/6463WNW
EIP: 0060:[<f19234b3>] EFLAGS: 00210246 CPU: 0
EIP is at xfs_da_do_buf+0x4e9/0x672 [xfs]
EAX: 00000001 EBX: edd27400 ECX: 00014005 EDX: cfa3fcb8
ESI: 00000008 EDI: 00000000 EBP: cfa3fcc8 ESP: cfa3fc5c
 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process mv (pid: 5585, ti=cfa3f000 task=e73d4d10 task.ti=cfa3f000)
Stack:
 003807ba 00000000 00000000 c056fe44 dff0345c dd751d98 000200da 00000001
<0> 00000000 00000000 ffffffff dff03440 0000001a cfa3fe54 cfa3fe58 cfa3fe5c
<0> cfa3fe60 cfa3fcbc c0123195 fffff000 00000003 c241e700 00000001 00000000
Call Trace:
 [<c0123195>] ? kmap_atomic_prot+0xc8/0xe7
 [<f1923698>] ? xfs_da_read_buf+0x18/0x1d [xfs]
 [<f1927f94>] ? xfs_dir2_block_addname+0x59/0x689 [xfs]
 [<f1927f94>] ? xfs_dir2_block_addname+0x59/0x689 [xfs]
 [<c028dfdc>] ? random32+0x17/0x19
 [<f1910f17>] ? xfs_bmap_last_offset+0xe8/0xfc [xfs]
 [<f19261f7>] ? xfs_dir2_isblock+0x1a/0x9a [xfs]
 [<f1926da4>] ? xfs_dir_createname+0x106/0x139 [xfs]
 [<f194bd57>] ? xfs_rename+0x249/0x49a [xfs]
 [<c01bf993>] ? do_lookup+0x64/0xf4
 [<f195ce1d>] ? xfs_vn_rename+0x58/0x62 [xfs]
 [<c01c02a8>] ? vfs_rename+0x20e/0x34c
 [<c01bfa56>] ? __lookup_hash+0x33/0x92
 [<c01c1ccd>] ? sys_renameat+0x164/0x1c9
 [<c01a4cd6>] ? remove_vma+0x46/0x4c
 [<c01a5a4f>] ? do_munmap+0x217/0x231
 [<c01c1d44>] ? sys_rename+0x12/0x14
 [<c0102758>] ? sysenter_do_call+0x12/0x28
Code: 55 b8 8b 45 b4 eb 13 31 ff 83 7d f0 00 74 12 8b 4d 18 8d 55 f0 b8 01 00 00 00 e8 ea f8 ff ff 89 c7 83 7d 14 01 0f 85 da 00 00 00 <8b> 57 08 89 55 b4 89 d1 8b 52 08 8b 01 0f c8 86 f2 0f b7 d2 81
EIP: [<f19234b3>] xfs_da_do_buf+0x4e9/0x672 [xfs] SS:ESP 0068:cfa3fc5c
CR2: 0000000000000008
---[ end trace b88f3c9551a0f88d ]---
vmap allocation for size 1048576 failed: use vmalloc=<size> to increase size.
xfs_buf_get: failed to map pages
Filesystem "dm-7": XFS internal error xfs_trans_cancel at line 1796 of file fs/xfs/xfs_trans.c.  Caller 0xf1952a5f

Pid: 16808, comm: plasma-desktop Tainted: G      D     2.6.36-gentoo-r5 #1
Call Trace:
 [<f19309c8>] xfs_error_report+0x2c/0x2e [xfs]
 [<f194de06>] xfs_trans_cancel+0x54/0x155 [xfs]
 [<f1952a5f>] ? xfs_create+0x4d7/0x553 [xfs]
 [<f1952a5f>] xfs_create+0x4d7/0x553 [xfs]
 [<f195d1e3>] xfs_vn_mknod+0xca/0x156 [xfs]
 [<f195d288>] xfs_vn_create+0xa/0xc [xfs]
 [<c01c07bd>] vfs_create+0x85/0xcf
 [<c01c0e4e>] do_last+0x277/0x4f8
 [<c01c25f1>] do_filp_open+0x197/0x458
 [<c01b6ef9>] do_sys_open+0x48/0xc9
 [<c01b6fbc>] sys_open+0x1e/0x26
 [<c0102758>] sysenter_do_call+0x12/0x28
 [<c03f0000>] ? x86_pmu_notifier+0x33/0x48
xfs_force_shutdown(dm-7,0x8) called from line 1797 of file fs/xfs/xfs_trans.c.  Return address = 0xf194de1d
XFS: Transforming an alert into a BUG.
Filesystem "dm-7": Corruption of in-memory data detected.  Shutting down filesystem: dm-7
------------[ cut here ]------------
kernel BUG at fs/xfs/support/debug.c:101!
invalid opcode: 0000 [#2] SMP
last sysfs file: /sys/devices/system/cpu/cpu1/cpufreq/scaling_cur_freq
Modules linked in: ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state ipt_REJECT iptable_mangle xt_tcpudp iptable_filter ipt_addrtype xt_DSCP xt_dscp xt_string xt_NFQUEUE xt_multiport xt_mark xt_hashlimit xt_conntrack xt_connmark nf_conntrack ip_tables x_tables bridge stp llc ipv6 rfcomm l2cap bluetooth snd_pcm_oss snd_mixer_oss snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device ext4 jbd2 tun kvm_intel kvm fan cpufreq_powersave cpufreq_conservative cpufreq_userspace cpufreq_ondemand acpi_cpufreq freq_table mperf snd_hda_codec_analog arc4 ecb i915 iwlagn drm_kms_helper iwlcore snd_hda_intel snd_hda_codec drm sierra i2c_algo_bit r852 sm_common nand nand_ids nand_ecc i2c_i801 usbserial mtd thinkpad_acpi led_class rtc_cmos nvram pcspkr e1000e rtc_core rtc_lib wacom mac80211 i2c_core intel_agp agpgart yenta_socket pcmcia_rsrc video snd_pcm processor tpm_tis backlight cfg80211 tpm tpm_bios snd_timer iTCO_wdt output iTCO_vendor_support joydev rfkill thermal snd snd_page_alloc battery thermal_sys ac button tg3 libphy e1000 fuse xfs exportfs nfs auth_rpcgss nfs_acl lockd sunrpc jfs raid10 dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd usbhid ohci_hcd ssb uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss 3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport dmx3191d sym53c8xx qlogicfas408 gdth advansys initio BusLogic arcmsr aic7xxx aic79xx scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_pcmcia pcmcia pcmcia_core

Pid: 16808, comm: plasma-desktop Tainted: G      D     2.6.36-gentoo-r5 #1 6463WNW/6463WNW
EIP: 0060:[<f1960375>] EFLAGS: 00010246 CPU: 0
EIP is at xfs_fs_vcmn_err+0xa2/0xae [xfs]
EAX: 00000282 EBX: 00000000 ECX: f19834e0 EDX: 00000282
ESI: edd20c00 EDI: f19788a3 EBP: ead22d74 ESP: ead22d64
 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process plasma-desktop (pid: 16808, ti=ead22000 task=ec6e51c0 task.ti=ead22000)
Stack:
 00000282 00000000 00000008 00000000 ead22d84 f1930996 ead22d9c edd20c00
<0> ead22db0 f1954917 00000010 00000001 edd20c00 f19788a3 edac4580 00000000
<0> df5c6c58 00000000 0000000c ead22ddc f194de1d 00000705 f197784c 00000704
Call Trace:
 [<f1930996>] ? xfs_cmn_err+0x40/0x46 [xfs]
 [<f1954917>] ? xfs_do_force_shutdown+0x78/0xf9 [xfs]
 [<f194de1d>] ? xfs_trans_cancel+0x6b/0x155 [xfs]
 [<f1952a5f>] ? xfs_create+0x4d7/0x553 [xfs]
 [<f1952a5f>] ? xfs_create+0x4d7/0x553 [xfs]
 [<f195d1e3>] ? xfs_vn_mknod+0xca/0x156 [xfs]
 [<f195d288>] ? xfs_vn_create+0xa/0xc [xfs]
 [<c01c07bd>] ? vfs_create+0x85/0xcf
 [<c01c0e4e>] ? do_last+0x277/0x4f8
 [<c01c25f1>] ? do_filp_open+0x197/0x458
 [<c01b6ef9>] ? do_sys_open+0x48/0xc9
 [<c01b6fbc>] ? sys_open+0x1e/0x26
 [<c0102758>] ? sysenter_do_call+0x12/0x28
 [<c03f0000>] ? x86_pmu_notifier+0x33/0x48
Code: 00 68 e4 34 98 f1 ff 34 9d c8 5c 96 f1 68 32 9e 97 f1 e8 94 3e a9 ce 83 c4 0c 8b 55 f0 b8 e0 34 98 f1 e8 2d 61 a9 ce 85 db 75 04 <0f> 0b eb fe 8d 65 f4 5b 5e 5f 5d c3 55 b8 e0 34 98 f1 89 e5 57
EIP: [<f1960375>] xfs_fs_vcmn_err+0xa2/0xae [xfs] SS:ESP 0068:ead22d64
---[ end trace b88f3c9551a0f88e ]---
Filesystem "dm-7": xfs_log_force: error 5 returned.
Filesystem "dm-7": xfs_log_force: error 5 returned. 

Expected Results:  
nothing of above

The kernel dump is from my Lenovo T61 (32-bit gentoo) (Intel(R) Core(TM)2 Duo CPU     T8300  @ 2.40GHz), but similar errors pop up on 32-bit Gentoo kvm guest systems with kernel gentoo-sources-2.6.36.
Comment 1 Tobias Hoffmann 2011-01-18 08:54:40 UTC
I can confirm this bug.
Linux 2.6.36-gentoo-r5 #1 SMP Core(TM) i5 CPU 750 @ 2.67GHz 
32 bit KVM guest w/ LVM on 64 bit Host (same OS than guest).
Comment 2 Jeroen Roovers (RETIRED) gentoo-dev 2011-01-18 20:04:52 UTC
(In reply to comment #1)
> I can confirm this bug.
> Linux 2.6.36-gentoo-r5 #1 SMP Core(TM) i5 CPU 750 @ 2.67GHz 
> 32 bit KVM guest w/ LVM on 64 bit Host (same OS than guest).

Also with a tainted kernel?

All of you - please paste your `emerge --info' output in a comment, and attach your kernel .config as a file.
Comment 3 Tobias Hoffmann 2011-01-19 08:13:27 UTC
Created attachment 260230 [details]
Output of emerge --info
Comment 4 Tobias Hoffmann 2011-01-19 08:14:16 UTC
> Also with a tainted kernel?

What is a tainted kernel? Does this help:

 # grep -iF tainted /var/log/messages
Jan 14 10:20:49 ittest01 kernel: Pid: 24964, comm: rsync Not tainted 2.6.36-gentoo-r5 #1
Jan 14 10:43:17 ittest01 kernel: Pid: 13242, comm: rsync Not tainted 2.6.36-gentoo-r5 #1
Jan 14 10:46:30 ittest01 kernel: Pid: 13756, comm: rm Not tainted 2.6.36-gentoo-r5 #1 /Bochs
Jan 14 13:25:59 ittest01 kernel: Pid: 29339, comm: rsync Not tainted 2.6.36-gentoo-r5 #1
Jan 14 13:30:11 ittest01 kernel: Pid: 29927, comm: rm Not tainted 2.6.36-gentoo-r5 #1 /Bochs
Jan 14 13:41:00 ittest01 kernel: Pid: 14211, comm: rsync Not tainted 2.6.36-gentoo-r5 #1 /Bochs
Jan 17 10:42:20 ittest01 kernel: Pid: 24068, comm: tar Tainted: G      D     2.6.36-gentoo-r5 #1
Jan 17 10:49:17 ittest01 kernel: Pid: 12861, comm: nmbd Tainted: G      D     2.6.36-gentoo-r5 #1 /Bochs
Jan 17 10:49:59 ittest01 kernel: Pid: 25034, comm: sshd Tainted: G      D     2.6.36-gentoo-r5 #1 /Bochs
Jan 17 14:55:08 ittest01 kernel: Pid: 12276, comm: modprobe Not tainted 2.6.36-gentoo-r5 #1
Jan 17 14:55:22 ittest01 kernel: Pid: 12556, comm: modprobe Not tainted 2.6.36-gentoo-r5 #1
Jan 17 14:55:23 ittest01 kernel: Pid: 12558, comm: modprobe Not tainted 2.6.36-gentoo-r5 #1
Jan 17 14:55:27 ittest01 kernel: Pid: 12622, comm: modprobe Not tainted 2.6.36-gentoo-r5 #1
Jan 17 14:55:28 ittest01 kernel: Pid: 12627, comm: modprobe Not tainted 2.6.36-gentoo-r5 #1
Jan 17 14:55:29 ittest01 kernel: Pid: 12631, comm: modprobe Not tainted 2.6.36-gentoo-r5 #1
Jan 17 14:55:32 ittest01 kernel: Pid: 12864, comm: modprobe Not tainted 2.6.36-gentoo-r5 #1
Jan 17 14:55:36 ittest01 kernel: Pid: 13188, comm: modprobe Not tainted 2.6.36-gentoo-r5 #1
Jan 17 14:56:13 ittest01 kernel: Pid: 13198, comm: modprobe Not tainted 2.6.36-gentoo-r5 #1
Jan 17 15:00:38 ittest01 kernel: Pid: 13582, comm: modprobe Not tainted 2.6.36-gentoo-r5 #1
Jan 18 18:21:19 ittest01 kernel: Pid: 4121, comm: rsync Not tainted 2.6.36-gentoo-r5 #1
Comment 5 Tobias Hoffmann 2011-01-19 13:00:33 UTC
Output of dmesg:

vmap allocation for size 4194304 failed: use vmalloc=<size> to increase size.
xfs_buf_get: failed to map pages
vmap allocation for size 4194304 failed: use vmalloc=<size> to increase size.
xfs_buf_get: failed to map pages
vmap allocation for size 4194304 failed: use vmalloc=<size> to increase size.
xfs_buf_get: failed to map pages
vmap allocation for size 4194304 failed: use vmalloc=<size> to increase size.
xfs_buf_get: failed to map pages
vmap allocation for size 4194304 failed: use vmalloc=<size> to increase size.
xfs_buf_get: failed to map pages
Filesystem "dm-7": XFS internal error xfs_trans_cancel at line 1815 of file fs/xfs/xfs_trans.c.  Caller 0xf91b2a1b

Pid: 19917, comm: install Not tainted 2.6.37-gentoo #1
Call Trace:
 [<f9196bba>] xfs_error_report+0x2c/0x2e [xfs]
 [<f91aed2b>] xfs_trans_cancel+0x4b/0xc9 [xfs]
 [<f91b2a1b>] ? xfs_create+0x48c/0x508 [xfs]
 [<f91b2a1b>] xfs_create+0x48c/0x508 [xfs]
 [<f91bbbb6>] xfs_vn_mknod+0xc8/0x153 [xfs]
 [<f91bbc5a>] xfs_vn_create+0xa/0xc [xfs]
 [<c01bf6f4>] vfs_create+0x85/0xcf
 [<c01bfd9b>] do_last+0x28a/0x50b
 [<c01c1589>] do_filp_open+0x1a7/0x47b
 [<c01ae4c2>] ? free_pages_and_swap_cache+0x6a/0x7d
 [<c01b5d1a>] do_sys_open+0x44/0xbf
 [<c01b5dd7>] sys_open+0x1e/0x26
 [<c0102818>] sysenter_do_call+0x12/0x28
xfs_force_shutdown(dm-7,0x8) called from line 1816 of file fs/xfs/xfs_trans.c.  Return address = 0xf91aed41
Filesystem "dm-7": Corruption of in-memory data detected.  Shutting down filesystem: dm-7
Please umount the filesystem, and rectify the problem(s)
Filesystem "dm-7": xfs_log_force: error 5 returned.
Filesystem "dm-7": xfs_log_force: error 5 returned.
Filesystem "dm-7": xfs_log_force: error 5 returned.
Filesystem "dm-7": xfs_log_force: error 5 returned.
Filesystem "dm-7": xfs_log_force: error 5 returned.
Filesystem "dm-7": xfs_log_force: error 5 returned.
Filesystem "dm-7": xfs_log_force: error 5 returned.
Comment 6 Tobias Hoffmann 2011-01-19 13:01:18 UTC
(In reply to comment #5)
The previous output was from a gentoo-sources-2.6.37 kernel.
Comment 7 Jochen Buchholz 2011-01-19 13:10:11 UTC
Created attachment 260238 [details]
kernelconfig 2.6.36-gentoo-r5 Lenovo T61 Core2
Comment 8 Jochen Buchholz 2011-01-19 13:13:25 UTC
Created attachment 260240 [details]
Lenovo T61 emerge --info
Comment 9 Tobias Hoffmann 2011-01-19 13:23:04 UTC
Created attachment 260242 [details]
Kernel 2.6.36-r5 KVM-guest 32bit
Comment 10 Tobias Hoffmann 2011-01-19 13:24:08 UTC
Created attachment 260243 [details]
Kernel config 2.6.37 KVM guest 32bit
Comment 11 Jochen Buchholz 2011-01-20 13:09:37 UTC
I think three Kernel Configs are enough :-) I switched back to "2.6.32-gentoo-r24 #1 SMP" and use it on high disk io without any problems.
Comment 12 Panagiotis Christopoulos (RETIRED) gentoo-dev 2011-01-20 14:30:56 UTC
(In reply to comment #4)
 > What is a tainted kernel? Does this help:

He means, Have you too many kernel modules loaded? If you all have many modules loaded, maybe you should pass a bigger vmalloc option/value in kernel's cmdline at startup. To see what is going on with vmalloc you can `grep -i vmalloc /proc/meminfo` and/or `cat /proc/modules` or `lsmod`, to see the size of modules etc. You can also try to debug this by checking the above results in previous and newer kernels. I'll resolve this as TEST-REQUEST. If you still think that this is a gentoo bug, please reopen. 
Comment 13 Tobias Hoffmann 2011-01-21 16:25:38 UTC
(In reply to comment #12)
> (In reply to comment #4)
>  > What is a tainted kernel? Does this help:
> 
> He means, Have you too many kernel modules loaded? If you all have many modules
> loaded, maybe you should pass a bigger vmalloc option/value in kernel's cmdline
> at startup. 
# root@2.6.36-gentoo # lsmod
Module                  Size  Used by
ipv6                  182642  87
nfsd                  195663  11
floppy                 41011  0
tpm_tis                 5165  0
tpm                     7645  1 tpm_tis
tpm_bios                3617  1 tpm
i2c_piix4               6464  0
rtc_cmos                6774  0
rtc_core                9735  1 rtc_cmos
i2c_core               12268  1 i2c_piix4
i6300esb                3415  0
thermal                 9407  0
rtc_lib                 1281  1 rtc_core
processor              22584  0
pcspkr                  1195  0
virtio_net              8164  0
button                  3626  0
thermal_sys             8545  2 thermal,processor
tg3                    97162  0
libphy                 12050  1 tg3
e1000                  74940  0
fuse                   47270  0
xfs                   528542  15
exportfs                2574  2 nfsd,xfs
nfs                   203606  2
auth_rpcgss            26153  2 nfsd,nfs
nfs_acl                 1655  2 nfsd,nfs
lockd                  48813  2 nfsd,nfs
sunrpc                133733  19 nfsd,nfs,auth_rpcgss,nfs_acl,lockd
jfs                   136033  0
raid10                 17715  0
dm_snapshot            24676  0
dm_crypt                8623  0
dm_mirror               9949  0
dm_region_hash          5696  1 dm_mirror
dm_log                  6568  2 dm_mirror,dm_region_hash
dm_mod                 47900  51 dm_snapshot,dm_crypt,dm_mirror,dm_log
scsi_wait_scan           459  0
sbp2                   14947  0
ohci1394               21131  0
ieee1394               51276  2 sbp2,ohci1394
sl811_hcd               7262  0
usbhid                 26037  0
ohci_hcd               17289  0
ssb                    28994  1 ohci_hcd
uhci_hcd               15652  0
usb_storage            30061  0
ehci_hcd               27642  0
usbcore                89458  7 sl811_hcd,usbhid,ohci_hcd,uhci_hcd,usb_storage,ehci_hcd
aic94xx                56065  0
libsas                 31650  1 aic94xx
lpfc                  343703  0
qla2xxx               255871  0
megaraid_sas           29063  0
megaraid_mbox          21317  0
megaraid_mm             5694  1 megaraid_mbox
megaraid               31109  0
aacraid                51353  0
sx8                     9701  0
DAC960                 55852  0
cciss                  80034  0
3w_9xxx                24098  0
3w_xxxx                18296  0
mptsas                 39873  0
scsi_transport_sas     16745  3 aic94xx,libsas,mptsas
mptfc                  10473  0
scsi_transport_fc      28159  3 lpfc,qla2xxx,mptfc
scsi_tgt                6866  1 scsi_transport_fc
mptspi                 11747  0
mptscsih               21091  3 mptsas,mptfc,mptspi
mptbase                67805  4 mptsas,mptfc,mptspi,mptscsih
atp870u                22285  0
dc395x                 24912  0
qla1280                17128  0
imm                     7058  0
parport                21355  1 imm
dmx3191d                7621  0
sym53c8xx              56425  0
qlogicfas408            3517  0
gdth                   70185  0
advansys               46330  0
initio                 12889  0
BusLogic               17136  0
arcmsr                 20528  0
aic7xxx                92894  0
aic79xx                97820  0
scsi_transport_spi     14768  5 mptspi,dmx3191d,sym53c8xx,aic7xxx,aic79xx
sg                     19462  0
pdc_adma                3942  0
sata_inic162x           5239  0
sata_mv                19454  0
ata_piix               17228  0
ahci                   16545  0
libahci                13180  1 ahci
sata_qstor              3889  0
sata_vsc                2846  0
sata_uli                2041  0
sata_sis                2650  0
sata_sx4                6586  0
sata_nv                15083  0
sata_via                5372  0
sata_svw                2918  0
sata_sil24              8813  0
sata_sil                5364  0
sata_promise            7177  0
pata_pcmcia             6933  0
pcmcia                 25043  2 ssb,pata_pcmcia
pcmcia_core             8237  1 pcmcia
Comment 14 Tobias Hoffmann 2011-01-21 16:28:27 UTC
(In reply to comment #12)

root@2.6.36-gentoo-r5 # grep -i vmalloc /proc/meminfo
VmallocTotal:     262144 kB
VmallocUsed:       13776 kB
VmallocChunk:     176648 kB

root@2.6.36-gentoo-r5 # cat /boot/grub/grub.conf
# snip
title Gentoo Linux 2.6.36-r5
root (hd0,0)
kernel /boot/kernel-genkernel-x86-2.6.36-gentoo-r5 root=/dev/ram0 real_root=/dev/vda3 doscsi elevator=deadline vmalloc=256M
initrd /boot/initramfs-genkernel-x86-2.6.36-gentoo-r5
Comment 15 Tobias Hoffmann 2011-01-21 16:28:49 UTC
The error:

Jan 21 16:44:16 ittest01 kernel: vmap allocation for size 2097152 failed: use vmalloc=<size> to increase size.
Jan 21 16:44:16 ittest01 kernel: xfs_buf_get: failed to map pages
Jan 21 16:44:16 ittest01 kernel: BUG: unable to handle kernel NULL pointer dereference at 00000008
Jan 21 16:44:16 ittest01 kernel: IP: [<f08b3691>] xfs_da_do_buf+0x53f/0x6a5 [xfs]
Jan 21 16:44:16 ittest01 kernel: *pde = 00000000
Jan 21 16:44:16 ittest01 kernel: Oops: 0000 [#1] SMP
Jan 21 16:44:16 ittest01 kernel: last sysfs file: /sys/devices/virtio-pci/virtio1/net/eth0/broadcast
Jan 21 16:44:16 ittest01 kernel: Modules linked in: ipv6 nfsd floppy tpm_tis tpm tpm_bios i2c_piix4 rtc_cmos rtc_core i2c_core i6300esb thermal rtc_lib processor pcspkr virtio_net button thermal_sys tg3 libphy e1000 fuse xfs exportfs nfs auth_rpcgss nfs_acl lockd sunrpc jfs raid10 dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd usbhid ohci_hcd ssb uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss 3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport dmx3191d sym53c8xx qlogicfas408 gdth advansys initio BusLogic arcmsr aic7xxx aic79xx scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_pcmcia pcmcia pcmcia_core
Jan 21 16:44:16 ittest01 kernel:
Jan 21 16:44:16 ittest01 kernel: Pid: 15270, comm: configure Not tainted 2.6.36-gentoo-r5 #1 /Bochs
Jan 21 16:44:16 ittest01 kernel: EIP: 0060:[<f08b3691>] EFLAGS: 00210246 CPU: 1
Jan 21 16:44:16 ittest01 kernel: EIP is at xfs_da_do_buf+0x53f/0x6a5 [xfs]
Jan 21 16:44:16 ittest01 kernel: EAX: 00000001 EBX: ee389400 ECX: 00000008 EDX: da297cdc
Jan 21 16:44:16 ittest01 kernel: ESI: 00000000 EDI: 00000000 EBP: 00000000 ESP: da297c88
Jan 21 16:44:16 ittest01 kernel: DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Jan 21 16:44:16 ittest01 kernel: Process configure (pid: 15270, ti=da297000 task=edd7e550 task.ti=da297000)
Jan 21 16:44:16 ittest01 kernel: Stack:
Jan 21 16:44:16 ittest01 kernel: 0069207c 00000000 d6af9880 f08aee4c 00000000 ffffffff 00000001 d6af989c
Jan 21 16:44:16 ittest01 kernel: <0> ee9dcc40 c69ccee0 00000000 00000000 00d24173 00000000 d0aaf580 c041fc2c
Jan 21 16:44:16 ittest01 kernel: <0> f08c2075 0000000d 00000000 00000000 00000001 00000000 da297d10 ee389400
Jan 21 16:44:16 ittest01 kernel: Call Trace:
Jan 21 16:44:16 ittest01 kernel: [<f08aee4c>] ? xfs_btree_setbuf+0x18/0x75 [xfs]
Jan 21 16:44:16 ittest01 kernel: [<c041fc2c>] ? _raw_spin_lock+0x5/0x7
Jan 21 16:44:16 ittest01 kernel: [<f08c2075>] ? xfs_iget+0x513/0x55a [xfs]
Jan 21 16:44:16 ittest01 kernel: [<f08b3854>] ? xfs_da_read_buf+0x19/0x1e [xfs]
Jan 21 16:44:16 ittest01 kernel: [<f08b724f>] ? xfs_dir2_block_addname+0x57/0x6b3 [xfs]
Jan 21 16:44:16 ittest01 kernel: [<f08b724f>] ? xfs_dir2_block_addname+0x57/0x6b3 [xfs]
Jan 21 16:44:16 ittest01 kernel: [<f08d5aba>] ? xfs_trans_add_item+0x15/0x44 [xfs]
Jan 21 16:44:16 ittest01 kernel: [<f08d7690>] ? xfs_trans_ijoin+0x1f/0x25 [xfs]
Jan 21 16:44:16 ittest01 kernel: [<f08a43a9>] ? xfs_bmap_last_offset+0xc7/0xda [xfs]
Jan 21 16:44:16 ittest01 kernel: [<f08b57cb>] ? xfs_dir2_isblock+0x14/0x5b [xfs]
Jan 21 16:44:16 ittest01 kernel: [<f08b6282>] ? xfs_dir_createname+0xfb/0x131 [xfs]
Jan 21 16:44:16 ittest01 kernel: [<f08d9a3c>] ? xfs_create+0x337/0x525 [xfs]
Jan 21 16:44:16 ittest01 kernel: [<f08e2a81>] ? xfs_vn_mknod+0xc8/0x155 [xfs]
Jan 21 16:44:16 ittest01 kernel: [<c01da7c2>] ? vfs_create+0x77/0xbe
Jan 21 16:44:16 ittest01 kernel: [<c01dae31>] ? do_last+0x270/0x4e2
Jan 21 16:44:16 ittest01 kernel: [<c01dc5fd>] ? do_filp_open+0x1ae/0x4e1
Jan 21 16:44:16 ittest01 kernel: [<c01c821b>] ? free_pages_and_swap_cache+0x67/0x74
Jan 21 16:44:16 ittest01 kernel: [<c01d11ef>] ? do_sys_open+0x45/0xc8
Jan 21 16:44:16 ittest01 kernel: [<c01d12b6>] ? sys_open+0x1e/0x23
Jan 21 16:44:16 ittest01 kernel: [<c0127158>] ? sysenter_do_call+0x12/0x28
Jan 21 16:44:16 ittest01 kernel: Code: 10 eb 16 31 ed 83 7c 24 54 00 74 14 8b 4c 24 7c 8d 54 24 54 b8 01 00 00 00 e8 dc f9 ff ff 89 c5 83 7c 24 78 01 0f 85 d5 00 00 00 <8b> 55 08 89 54 24 10 89 d1 8b 52 08 8b 01 0f c8 86 f2 0f b7 d2
Jan 21 16:44:16 ittest01 kernel: EIP: [<f08b3691>] xfs_da_do_buf+0x53f/0x6a5 [xfs] SS:ESP 0068:da297c88
Jan 21 16:44:16 ittest01 kernel: CR2: 0000000000000008
Jan 21 16:44:16 ittest01 kernel: ---[ end trace 4053a7de60e49e7e ]---
Jan 21 16:44:16 ittest01 kernel: vmap allocation for size 2097152 failed: use vmalloc=<size> to increase size.
Jan 21 16:44:16 ittest01 kernel: xfs_buf_get: failed to map pages
Jan 21 16:44:16 ittest01 kernel: vmap allocation for size 2097152 failed: use vmalloc=<size> to increase size.
Jan 21 16:44:16 ittest01 kernel: xfs_buf_get: failed to map pages
Jan 21 16:44:26 ittest01 kernel: vmap allocation for size 2097152 failed: use vmalloc=<size> to increase size.
Comment 16 Tobias Hoffmann 2011-01-21 16:30:20 UTC
Behaviour of XFS filesystem after crash:

root@2.6.36-gentoo-r5 # ll /var/tmp/
ls: cannot access /var/tmp/jboss: Cannot allocate memory
ls: cannot access /var/tmp/pear: Cannot allocate memory
ls: cannot access /var/tmp/pear-build-root: Cannot allocate memory
total 12
drw-rw----  4 root    root      28 Nov 19 18:25 backup
drwxrwxr-x  2 portage portage    6 Jan 21 16:43 binpkgs
drwxr-xr-x  7 root    root    4096 Jan 19 13:19 genkernel
drwxrwxr-x  8 root    www       99 Jan 12 10:09 git
drwxrwx--- 23 root    git     4096 Jan 14 14:18 git-repo
??????????  ? ?       ?          ?            ? jboss
drwxr-xr-x  7 root    root      68 Dec 22 15:39 nginx
??????????  ? ?       ?          ?            ? pear
??????????  ? ?       ?          ?            ? pear-build-root
drwxrwxr-x  4 portage portage   35 Jan 21 16:43 portage
drwxr-xr-x  5 root    root      56 Jan 14 12:58 portage-tmp
drwxrwxr-x  4 tomcat  tomcat  4096 Jan 21 16:14 tomcat-6
Comment 17 Panagiotis Christopoulos (RETIRED) gentoo-dev 2011-01-21 17:00:14 UTC
I'll reopen this.
Comment 18 Panagiotis Christopoulos (RETIRED) gentoo-dev 2011-01-21 17:01:46 UTC
Assigning to @kernel team. Guys, this may be an upstream bug, but better to leave the decision to you.
Comment 19 Jochen Buchholz 2011-01-24 15:08:49 UTC
(In reply to comment #18)
> Assigning to @kernel team. Guys, this may be an upstream bug, but better to
https://bugzilla.kernel.org/show_bug.cgi?id=27492
> leave the decision to you.
> 
You are right. It's a main kernel bug. I fetched the actual stable kernel 2.6.37 from kernel.org and it crashed also:

vmap allocation for size 1048576 failed: use vmalloc=<size> to increase size.
xfs_buf_get: failed to map pages
vmap allocation for size 1048576 failed: use vmalloc=<size> to increase size.
xfs_buf_get: failed to map pages
vmap allocation for size 1048576 failed: use vmalloc=<size> to increase size.
xfs_buf_get: failed to map pages
vmap allocation for size 1048576 failed: use vmalloc=<size> to increase size.
xfs_buf_get: failed to map pages
BUG: unable to handle kernel NULL pointer dereference at 00000008
IP: [<f9187b72>] xfs_da_do_buf+0x4c8/0x61f [xfs]
*pde = 00000000 
Oops: 0000 [#1] SMP 
last sysfs file: /sys/devices/virtio-pci/virtio1/net/eth0/broadcast
Modules linked in: ipv6 nfsd rtc_cmos tpm_tis rtc_core tpm tpm_bios rtc_lib i2c_piix4 virtio_net i6300esb i2c_core virtio_balloon processor pcspkr floppy thermal_sys button tg3 libphy e1000 fuse xfs exportfs nfs auth_rpcgss nfs_acl lockd sunrpc jfs raid10 dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod scsi_wait_scan sl811_hcd usbhid ohci_hcd ssb uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss 3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport dmx3191d sym53c8xx qlogicfas408 gdth advansys initio BusLogic arcmsr aic7xxx aic79xx scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_pcmcia pcmcia pcmcia_core

Pid: 425, comm: i686-pc-linux-g Not tainted 2.6.37 #1 /Bochs
EIP: 0060:[<f9187b72>] EFLAGS: 00210246 CPU: 1
EIP is at xfs_da_do_buf+0x4c8/0x61f [xfs]
EAX: 00000001 EBX: f5947800 ECX: 00000008 EDX: c5631d64
ESI: 00000000 EDI: 00000000 EBP: c5631d74 ESP: c5631d0c
 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process i686-pc-linux-g (pid: 425, ti=c5630000 task=c5248960 task.ti=c5630000)
Stack:
 007adb5f 00000000 c5631dc0 c55870dc f4c26220 00000000 00000001 00000000
 00000000 ffffffff c55870c0 00000000 00000000 c5631d48 c011d903 c5631d5c
 c0414e4e c5631d5c 00200286 eb74ebe0 c5631d6c 00000001 00000000 c5631d9c
Call Trace:
 [<c011d903>] ? default_spin_lock_flags+0x8/0xd
 [<c0414e4e>] ? _raw_spin_lock_irqsave+0x20/0x28
 [<f9187d25>] ? xfs_da_read_buf+0x18/0x1d [xfs]
 [<f918af47>] ? xfs_dir2_block_lookup_int+0x34/0x178 [xfs]
 [<f918af47>] ? xfs_dir2_block_lookup_int+0x34/0x178 [xfs]
 [<f919fe7c>] ? xfs_log_release_iclog+0x11/0x3a [xfs]
 [<f918b3a0>] ? xfs_dir2_block_removename+0x3d/0x18a [xfs]
 [<f9179022>] ? xfs_bmap_last_offset+0xe4/0xfa [xfs]
 [<f9189c6b>] ? xfs_dir2_isblock+0x18/0x62 [xfs]
 [<f918a213>] ? xfs_dir_removename+0xb4/0xe6 [xfs]
 [<f91ad30a>] ? xfs_remove+0x1d9/0x306 [xfs]
 [<f91b6d1a>] ? xfs_vn_unlink+0x30/0x62 [xfs]
 [<c01bfdd2>] ? vfs_unlink+0x5f/0xbc
 [<c01c1c3b>] ? do_unlinkat+0x94/0xfe
 [<c01a8e06>] ? do_munmap+0x217/0x231
 [<c01c1cb5>] ? sys_unlink+0x10/0x12
 [<c0102858>] ? sysenter_do_call+0x12/0x28
Code: 55 b8 8b 45 b4 eb 13 31 ff 83 7d f0 00 74 12 8b 4d 18 8d 55 f0 b8 01 00 00 00 e8 4c fa ff ff 89 c7 83 7d 14 01 0f 85 c7 00 00 00 <8b> 57 08 89 55 b4 89 d1 8b 52 08 8b 01 0f c8 86 f2 0f b7 d2 81 
EIP: [<f9187b72>] xfs_da_do_buf+0x4c8/0x61f [xfs] SS:ESP 0068:c5631d0c
CR2: 0000000000000008
---[ end trace 4aa7015e260c3faa ]---
vmap allocation for size 1048576 failed: use vmalloc=<size> to increase size.
xfs_buf_get: failed to map pages
vmap allocation for size 1048576 failed: use vmalloc=<size> to increase size.
xfs_buf_get: failed to map pages
Comment 20 Mike Pagano gentoo-dev 2011-02-01 18:54:08 UTC
I see you posted this upstream. Well follow that bug and back port any fixes identified.
Comment 21 Tom Wijsman (TomWij) (RETIRED) gentoo-dev 2013-12-11 14:08:08 UTC
Reflected upstream status.