Bug 417409 – sys-kernel/hardened-sources-3.2.11 - beam: page allocation failure: order:1, mode:0x20 or swapper: page allocation failure: order:1, mode:0x20

• Product: Gentoo Linux
• Status: RESOLVED OBSOLETE

[Description cyberbat 2012-05-24 18:50:24 UTC]

Created attachment 312981 [details]
.config

I have VPS base on XEN guest. I've update to hardened-sources 3.2.11. Error in kernel log began after update:

May 24 18:11:37 gsom kernel: beam: page allocation failure: order:1, mode:0x20
May 24 18:11:37 gsom kernel: Pid: 4061, comm: beam Not tainted 3.2.11-hardened #1
May 24 18:11:37 gsom kernel: Call Trace:
May 24 18:11:37 gsom kernel: [<c1084e5e>] ? warn_alloc_failed+0xae/0xf0
May 24 18:11:37 gsom kernel: [<c108778b>] ? __alloc_pages_nodemask+0x3eb/0x620
May 24 18:11:37 gsom kernel: [<c10aab40>] ? cache_alloc_refill+0x2b0/0x480
May 24 18:11:37 gsom kernel: [<c10aadbb>] ? __kmalloc+0xab/0xe0
May 24 18:11:37 gsom kernel: [<c120ee38>] ? pskb_expand_head+0x158/0x2b0
May 24 18:11:37 gsom kernel: [<c12958aa>] ? ipt_do_table+0x1ca/0x370
May 24 18:11:37 gsom kernel: [<c120fa31>] ? __pskb_pull_tail+0x241/0x350
May 24 18:11:37 gsom kernel: [<c12191f2>] ? dev_hard_start_xmit+0x262/0x530
May 24 18:11:37 gsom kernel: [<c125c880>] ? ip_fragment+0x790/0x790
May 24 18:11:37 gsom kernel: [<c123482c>] ? nf_iterate+0x6c/0xa0
May 24 18:11:37 gsom kernel: [<c125c880>] ? ip_fragment+0x790/0x790
May 24 18:11:37 gsom kernel: [<c122f23b>] ? sch_direct_xmit+0x7b/0x130
May 24 18:11:37 gsom kernel: [<c125c880>] ? ip_fragment+0x790/0x790
May 24 18:11:37 gsom kernel: [<c12196ca>] ? dev_queue_xmit+0xfa/0x370
May 24 18:11:37 gsom kernel: [<c125c9a1>] ? ip_finish_output+0x121/0x2c0
May 24 18:11:37 gsom kernel: [<c125bb5a>] ? ip_local_out+0x1a/0x20
May 24 18:11:37 gsom kernel: [<c1271006>] ? tcp_transmit_skb+0x376/0x830
May 24 18:11:37 gsom kernel: [<c1271744>] ? tcp_write_xmit+0x1d4/0x9c0
May 24 18:11:37 gsom kernel: [<c1271fbb>] ? __tcp_push_pending_frames+0x3b/0xb0
May 24 18:11:37 gsom kernel: [<c126401c>] ? tcp_sendmsg+0x92c/0xf00
May 24 18:11:37 gsom kernel: [<c1284ef9>] ? inet_sendmsg+0x39/0x80
May 24 18:11:37 gsom kernel: [<c1135896>] ? security_socket_sendmsg+0x26/0x40
May 24 18:11:37 gsom kernel: [<c1206bed>] ? sock_aio_write+0x15d/0x180
May 24 18:11:37 gsom kernel: [<c1206a90>] ? sock_aio_dtor+0x10/0x10
May 24 18:11:37 gsom kernel: [<c10aed86>] ? do_sync_readv_writev+0xc6/0x110
May 24 18:11:37 gsom kernel: [<c12bec3e>] ? error_code+0x5e/0x70
May 24 18:11:37 gsom kernel: [<c10ae29c>] ? rw_verify_area+0x6c/0x130
May 24 18:11:37 gsom kernel: [<c10af1d8>] ? do_readv_writev+0xb8/0x1b0
May 24 18:11:37 gsom kernel: [<c1206a90>] ? sock_aio_dtor+0x10/0x10
May 24 18:11:37 gsom kernel: [<c10af399>] ? vfs_writev+0x59/0x70
May 24 18:11:37 gsom kernel: [<c10af491>] ? sys_writev+0x41/0xa0
May 24 18:11:37 gsom kernel: [<c12be894>] ? syscall_call+0x7/0xb
May 24 18:11:37 gsom kernel: [<c1021c60>] ? vmalloc_sync_all+0x1c0/0x1c0
May 24 18:11:37 gsom kernel: [<c12be8ba>] ? restore_all_pax+0xc/0xc
May 24 18:11:37 gsom kernel: [<c102007b>] ? native_flush_tlb_global+0x2b/0x30
May 24 18:11:37 gsom kernel: [<c1021c60>] ? vmalloc_sync_all+0x1c0/0x1c0
May 24 18:11:37 gsom kernel: Mem-Info:
May 24 18:11:37 gsom kernel: DMA per-cpu:
May 24 18:11:37 gsom kernel: CPU    0: hi:    0, btch:   1 usd:   0
May 24 18:11:37 gsom kernel: Normal per-cpu:
May 24 18:11:37 gsom kernel: CPU    0: hi:   90, btch:  15 usd:  72
May 24 18:11:37 gsom kernel: active_anon:9614 inactive_anon:9596 isolated_anon:0
May 24 18:11:37 gsom kernel: active_file:4368 inactive_file:35370 isolated_file:0
May 24 18:11:37 gsom kernel: unevictable:0 dirty:12 writeback:0 unstable:0
May 24 18:11:37 gsom kernel: free:986 slab_reclaimable:1758 slab_unreclaimable:713
May 24 18:11:37 gsom kernel: mapped:3919 shmem:71 pagetables:355 bounce:0
May 24 18:11:37 gsom kernel: DMA free:1212kB min:120kB low:148kB high:180kB active_anon:6476kB inactive_anon:6496kB active_file:0kB inactive_file:12kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15808kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:76kB slab_reclaimable:900kB slab_unreclaimable:172kB kernel_stack:392kB pagetables:128kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:6 all_unreclaimable? no
May 24 18:11:37 gsom kernel: lowmem_reserve[]: 0 246 246 246
May 24 18:11:37 gsom kernel: Normal free:2732kB min:1944kB low:2428kB high:2916kB active_anon:31980kB inactive_anon:31888kB active_file:17472kB inactive_file:141468kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:251968kB mlocked:0kB dirty:48kB writeback:0kB mapped:15676kB shmem:208kB slab_reclaimable:6132kB slab_unreclaimable:2680kB kernel_stack:512kB pagetables:1292kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
May 24 18:11:37 gsom kernel: lowmem_reserve[]: 0 0 0 0
May 24 18:11:37 gsom kernel: DMA: 295*4kB 4*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1212kB
May 24 18:11:37 gsom kernel: Normal: 671*4kB 6*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2732kB
May 24 18:11:37 gsom kernel: 39836 total pagecache pages
May 24 18:11:37 gsom kernel: 27 pages in swap cache
May 24 18:11:37 gsom kernel: Swap cache stats: add 436, delete 409, find 6/9
May 24 18:11:37 gsom kernel: Free swap  = 522636kB
May 24 18:11:37 gsom kernel: Total swap = 524284kB
May 24 18:11:37 gsom kernel: 67568 pages RAM
May 24 18:11:37 gsom kernel: 0 pages HighMem
May 24 18:11:37 gsom kernel: 4040 pages reserved
May 24 18:11:37 gsom kernel: 37326 pages shared
May 24 18:11:37 gsom kernel: 33938 pages non-shared

and other one:
May 24 18:11:42 gsom kernel: swapper: page allocation failure: order:1, mode:0x20
May 24 18:11:42 gsom kernel: Pid: 0, comm: swapper Not tainted 3.2.11-hardened #1
May 24 18:11:42 gsom kernel: Call Trace:
May 24 18:11:42 gsom kernel: [<c1084e5e>] ? warn_alloc_failed+0xae/0xf0
May 24 18:11:42 gsom kernel: [<c108778b>] ? __alloc_pages_nodemask+0x3eb/0x620
May 24 18:11:42 gsom kernel: [<c10aab40>] ? cache_alloc_refill+0x2b0/0x480
May 24 18:11:42 gsom kernel: [<c10aadbb>] ? __kmalloc+0xab/0xe0
May 24 18:11:42 gsom kernel: [<c120ee38>] ? pskb_expand_head+0x158/0x2b0
May 24 18:11:42 gsom kernel: [<c12958aa>] ? ipt_do_table+0x1ca/0x370
May 24 18:11:42 gsom kernel: [<c120fa31>] ? __pskb_pull_tail+0x241/0x350
May 24 18:11:42 gsom kernel: [<c12191f2>] ? dev_hard_start_xmit+0x262/0x530
May 24 18:11:42 gsom kernel: [<c125c880>] ? ip_fragment+0x790/0x790
May 24 18:11:42 gsom kernel: [<c123482c>] ? nf_iterate+0x6c/0xa0
May 24 18:11:42 gsom kernel: [<c125c880>] ? ip_fragment+0x790/0x790
May 24 18:11:42 gsom kernel: [<c122f23b>] ? sch_direct_xmit+0x7b/0x130
May 24 18:11:42 gsom kernel: [<c125c880>] ? ip_fragment+0x790/0x790
May 24 18:11:42 gsom kernel: [<c12196ca>] ? dev_queue_xmit+0xfa/0x370
May 24 18:11:42 gsom kernel: [<c125c9a1>] ? ip_finish_output+0x121/0x2c0
May 24 18:11:42 gsom kernel: [<c125bb5a>] ? ip_local_out+0x1a/0x20
May 24 18:11:42 gsom kernel: [<c1271006>] ? tcp_transmit_skb+0x376/0x830
May 24 18:11:42 gsom kernel: [<c1271744>] ? tcp_write_xmit+0x1d4/0x9c0
May 24 18:11:42 gsom kernel: [<c1271fbb>] ? __tcp_push_pending_frames+0x3b/0xb0
May 24 18:11:42 gsom kernel: [<c126ef22>] ? tcp_rcv_established+0x3b2/0x600
May 24 18:11:42 gsom kernel: [<c12757d5>] ? tcp_v4_do_rcv+0xb5/0x1d0
May 24 18:11:42 gsom kernel: [<c11359fe>] ? security_sock_rcv_skb+0x1e/0x30
May 24 18:11:42 gsom kernel: [<c1277d71>] ? tcp_v4_rcv+0x721/0x800
May 24 18:11:42 gsom kernel: [<c12348be>] ? nf_hook_slow+0x5e/0x110
May 24 18:11:42 gsom kernel: [<c1257ec8>] ? ip_local_deliver_finish+0x88/0x220
May 24 18:11:42 gsom kernel: [<c12581c2>] ? ip_local_deliver+0x52/0x90
May 24 18:11:42 gsom kernel: [<c1257c4f>] ? ip_rcv_finish+0xff/0x2f0
May 24 18:11:42 gsom kernel: [<c1216d4d>] ? __netif_receive_skb+0x1cd/0x380
May 24 18:11:42 gsom kernel: [<c10532b0>] ? getnstimeofday+0x40/0xe0
May 24 18:11:42 gsom kernel: [<c11fb05a>] ? xennet_poll+0x9da/0xb90
May 24 18:11:42 gsom kernel: [<c1071dbb>] ? handle_irq_event+0x1b/0x30
May 24 18:11:42 gsom kernel: [<c1073f0a>] ? handle_edge_irq+0x4a/0xc0
May 24 18:11:42 gsom kernel: [<c121a9ff>] ? net_rx_action+0x6f/0x110
May 24 18:11:42 gsom kernel: [<c1035d2b>] ? __do_softirq+0x6b/0xf0
May 24 18:11:42 gsom kernel: [<c1035cc0>] ? local_bh_enable_ip+0x80/0x80
May 24 18:11:42 gsom kernel: [<c1009f3f>] ? do_softirq+0x6f/0xa0
May 24 18:11:42 gsom kernel: <IRQ>  [<c1035f36>] ? irq_exit+0x66/0x90
May 24 18:11:42 gsom kernel: [<c11bdce7>] ? xen_evtchn_do_upcall+0x27/0x40
May 24 18:11:42 gsom kernel: [<c12bf597>] ? xen_do_upcall+0x7/0x10
May 24 18:11:42 gsom kernel: [<c10013a7>] ? hypercall_page+0x3a7/0x1000
May 24 18:11:42 gsom kernel: [<c1005dea>] ? xen_safe_halt+0x1a/0x30
May 24 18:11:42 gsom kernel: [<c100f337>] ? default_idle+0x27/0x60
May 24 18:11:42 gsom kernel: [<c1007586>] ? cpu_idle+0x36/0x50
May 24 18:11:42 gsom kernel: [<c1399655>] ? 0xc1399654
May 24 18:11:42 gsom kernel: [<c139917c>] ? 0xc139917b
May 24 18:11:42 gsom kernel: [<c139a5ca>] ? 0xc139a5c9
May 24 18:11:42 gsom kernel: Mem-Info:
May 24 18:11:42 gsom kernel: DMA per-cpu:
May 24 18:11:42 gsom kernel: CPU    0: hi:    0, btch:   1 usd:   0
May 24 18:11:42 gsom kernel: Normal per-cpu:
May 24 18:11:42 gsom kernel: CPU    0: hi:   90, btch:  15 usd:  79
May 24 18:11:42 gsom kernel: active_anon:11539 inactive_anon:11445 isolated_anon:0
May 24 18:11:42 gsom kernel: active_file:4433 inactive_file:31477 isolated_file:0
May 24 18:11:42 gsom kernel: unevictable:0 dirty:42 writeback:0 unstable:0
May 24 18:11:42 gsom kernel: free:918 slab_reclaimable:1713 slab_unreclaimable:760
May 24 18:11:42 gsom kernel: mapped:4200 shmem:70 pagetables:385 bounce:0
May 24 18:11:42 gsom kernel: DMA free:1204kB min:120kB low:148kB high:180kB active_anon:6428kB inactive_anon:6488kB active_file:0kB inactive_file:4kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15808kB mlocked:0kB dirty:4kB writeback:0kB mapped:0kB shmem:72kB slab_reclaimable:880kB slab_unreclaimable:208kB kernel_stack:416kB pagetables:128kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
May 24 18:11:42 gsom kernel: lowmem_reserve[]: 0 246 246 246
May 24 18:11:42 gsom kernel: Normal free:2468kB min:1944kB low:2428kB high:2916kB active_anon:39728kB inactive_anon:39292kB active_file:17732kB inactive_file:125904kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:251968kB mlocked:0kB dirty:164kB writeback:0kB mapped:16800kB shmem:208kB slab_reclaimable:5972kB slab_unreclaimable:2832kB kernel_stack:592kB pagetables:1412kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
May 24 18:11:42 gsom kernel: lowmem_reserve[]: 0 0 0 0
May 24 18:11:42 gsom kernel: DMA: 301*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1204kB
May 24 18:11:42 gsom kernel: Normal: 607*4kB 1*8kB 0*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2468kB
May 24 18:11:42 gsom kernel: 36028 total pagecache pages
May 24 18:11:42 gsom kernel: 48 pages in swap cache
May 24 18:11:42 gsom kernel: Swap cache stats: add 1137, delete 1089, find 14/21
May 24 18:11:42 gsom kernel: Free swap  = 519964kB
May 24 18:11:42 gsom kernel: Total swap = 524284kB
May 24 18:11:42 gsom kernel: 67568 pages RAM
May 24 18:11:42 gsom kernel: 0 pages HighMem
May 24 18:11:42 gsom kernel: 4040 pages reserved
May 24 18:11:42 gsom kernel: 33656 pages shared
May 24 18:11:42 gsom kernel: 38192 pages non-shared

merge --info
Portage 2.1.10.49 (hardened/linux/x86, gcc-4.5.3, glibc-2.14.1-r3, 3.2.11-hardened i686)
=================================================================
System uname: Linux-3.2.11-hardened-i686-Intel-R-_Core-TM-2_Quad_CPU_Q9550_@_2.83GHz-with-gentoo-2.1
Timestamp of tree: Thu, 24 May 2012 09:15:01 +0000
app-shells/bash:          4.2_p20
dev-lang/python:          2.7.3-r1
sys-apps/baselayout:      2.1-r1
sys-apps/openrc:          0.9.8.4
sys-apps/sandbox:         2.5
sys-devel/binutils:       2.21.1-r1
sys-devel/gcc:            4.5.3-r2
sys-devel/gcc-config:     1.5-r2
sys-devel/libtool:        2.4-r1
sys-devel/make:           3.82-r1
sys-kernel/linux-headers: 3.1 (virtual/os-headers)
sys-libs/glibc:           2.14.1-r3
Repositories: gentoo x-overlay
ACCEPT_KEYWORDS="x86"
ACCEPT_LICENSE="* -@EULA"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=i686 -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -march=i686 -pipe"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--autounmask=n"
FEATURES="assume-digests binpkg-logs distlocks ebuild-locks fixlafiles news nodoc noinfo noman parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
FFLAGS=""
GENTOO_MIRRORS="ftp://gentoo.bloodhost.ru/"
LANG="ru_RU.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="ru en"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_EXTRA_OPTS="--exclude-from=/etc/portage/exclude-from-sync --delete-excluded"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/var/lib/overlay"
SYNC="rsync://gentoo.bloodhost.ru/gentoo-portage"
USE="acl bzip2 cli cracklib crypt cxx dri gnutls hardened iconv icu idn modules mudflap mysql mysqli ncurses nls nptl openmp pam pax_kernel pcre pic pppd readline session ssl symlink threads tls unicode urandom vhosts x86 xml zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="ru en" PHP_TARGETS="php5-3" PYTHON_TARGETS="python3_2 python2_7" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="apm ark chips cirrus cyrix dummy fbdev glint i128 i740 intel mach64 mga neomagic nsc nv r128 radeon rendition s3 s3virge savage siliconmotion sis sisusb tdfx tga trident tseng v4l vesa via vmware nouveau" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, MAKEOPTS, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, USE_PYTHON

[Comment 1 cyberbat 2012-05-24 18:54:41 UTC]

Created attachment 312983 [details]
full dmesg

[Comment 2 Anthony Basile 2012-05-25 16:44:32 UTC]

This is probably pax related.  Can you turn off pax and see if the problem goes away.  I'm cc-ing upstream too.

[Comment 3 PaX Team 2012-05-25 17:56:07 UTC]

(In reply to comment #2)
> This is probably pax related.  Can you turn off pax and see if the problem
> goes away.  I'm cc-ing upstream too.

i don't see how PaX is related to this OOM ;). just give your VM enough memory.

[Comment 4 cyberbat 2012-05-26 08:19:53 UTC]

Created attachment 313091 [details]
.config diff

I'm not sure that it's because of memory amount. Before upgrade everything was ok. I have played with highmem optuions in kernel (turn highmem off, change processor type, and so on see .config diff. I have 512MB on my VPS) and errors gone away. But I'm not sure what option made it.

[Comment 5 PaX Team 2012-05-27 19:29:19 UTC]

(In reply to comment #4)
> I'm not sure that it's because of memory amount.

well, strictly speaking it wasn't 'not enough memory' in general, but 'no big enough contiguous free memory'. thing is, i still think the memory allocated to your VM was on the edge and you were just lucky before to not hit this fragmentation problem (what was your previous kernel version btw?).

[Comment 6 cyberbat 2012-05-28 10:16:10 UTC]

I have never such problem. There was no grow of load on VDS and according to htop I have near 50% of RAM (128 of 256) and 100% of swap (512) free. I've used hardened-sources 3.2.2-r1 before.
I have to repeat that after making changes described in comment 4 I have no error at all.

[Comment 7 Anthony Basile 2013-04-13 22:42:14 UTC]

3.2.11 has been off the tree for a while.  If this is a problem with any kernel on the tree, please reopen.