Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 447042

Summary: =sys-fs/udev-197-r8 with Linux 3.7.x - systemd-udevd causes page allocation failures at bootup
Product: Gentoo Linux Reporter: ernsteiswuerfel <erhard_f>
Component: [OLD] Core systemAssignee: udev maintainers <udev-bugs>
Status: VERIFIED WORKSFORME    
Severity: normal CC: god, gregkh
Priority: Normal    
Version: unspecified   
Hardware: AMD64   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: logfiles

Description ernsteiswuerfel archtester 2012-12-12 21:33:07 UTC
Watching a dmesg output I realized I got several allocation failures at bootup lately...

First I thought this has been caused by some kernel update or config issue - but this doesn't seem to be the case. Tried out gentoo-sources-3.5.7, gentoo-sources-3.6.10, vanilla-sources-3.6.9, vanilla-sources-3.6.10, vanilla-sources-3.7.0  I get these faults every time. Also on my other machine (which has a more conservative setup with stable gcc and udev, ...).

I suspect udev but that's only a guess based on my dmesg output:
[...]
vmalloc: allocation failure: 0 bytes
systemd-udevd: page allocation failure: order:0, mode:0xd2
Pid: 1252, comm: systemd-udevd Not tainted 3.6.10-gentoo #3
Call Trace:
 [<ffffffff8109b46f>] ? 0xffffffff8109b46f
 [<ffffffff810c2221>] ? 0xffffffff810c2221
 [<ffffffff81311cf7>] ? 0xffffffff81311cf7
vmalloc: allocation failure: 0 bytes
systemd-udevd: page allocation failure: order:0, mode:0xd2
Pid: 1253, comm: systemd-udevd Not tainted 3.6.10-gentoo #3
[...]

For full logs (dmesg, messages, kernel config, rc.log) see the attatchment.

Apart from these page allocation issues both of my machines are running. I have not encoutered any specifig error or crash yet. So I really got no clue what's it about??

Reproducible: Always




Portage 2.1.11.31 (default/linux/amd64/10.0/desktop/gnome, gcc-4.6.3, glibc-2.15-r3, 3.6.10-gentoo x86_64)
=================================================================
System uname: Linux-3.6.10-gentoo-x86_64-AMD_FX-tm-8350_Eight-Core_Processor-with-gentoo-2.1
Timestamp of tree: Tue, 11 Dec 2012 22:00:01 +0000
ld GNU ld (GNU Binutils) 2.22
distcc 3.1 x86_64-pc-linux-gnu [disabled]
app-shells/bash:          4.2_p37
dev-java/java-config:     2.1.11-r3
dev-lang/python:          2.7.3-r2, 3.2.3
dev-util/cmake:           2.8.9
dev-util/pkgconfig:       0.27.1
sys-apps/baselayout:      2.1-r1
sys-apps/openrc:          0.11.8
sys-apps/sandbox:         2.5
sys-devel/autoconf:       2.13, 2.68
sys-devel/automake:       1.9.6-r3, 1.11.6
sys-devel/binutils:       2.22-r1
sys-devel/gcc:            4.6.3
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4-r1
sys-devel/make:           3.82-r3
sys-kernel/linux-headers: 3.6 (virtual/os-headers)
sys-libs/glibc:           2.15-r3
Repositories: gentoo seden x-portage
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-pipe -Os -mtune=native -march=native -fstack-protector -fpredictive-commoning -fgcse-after-reload -ftree-vectorize -fivopts -floop-interchange -floop-strip-mine -floop-block -ftree-loop-im"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-pipe -Os -mtune=native -march=native -fstack-protector -fpredictive-commoning -fgcse-after-reload -ftree-vectorize -fivopts -floop-interchange -floop-strip-mine -floop-block -ftree-loop-im -fvisibility-inlines-hidden"
DISTDIR="/usr/portage/distfiles"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="ftp://mirror.switch.ch/mirror/gentoo/ http://mirror.switch.ch/ftp/mirror/gentoo/ "
LANG="de_DE.utf8"
LDFLAGS="-Wl,-O1 -Wl,-z,now -Wl,-z,relro -Wl,--sort-common -Wl,--as-needed"
MAKEOPTS="-j9"
PKGDIR="/usr/portage/packages"
PORTAGE_COMPRESS="xz"
PORTAGE_COMPRESS_FLAGS="-6"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/var/lib/layman/seden /usr/local/portage"
SYNC="rsync://yea/gentoo-portage"
USE="X a52 aac acl acpi alsa amd64 avx bash-completion berkdb bluetooth bluray bzip2 cairo cdda cdr cli colord consolekit cracklib crypt cups custom-cflags custom-optimization cxx dbus dirac djvu dri dts dvd dvdr eds emboss enca encode evo exif firefox flac fortran gdbm gif gnome gnome-keyring gnome-online-accounts gnutls gstreamer gtk iconv icu ipv6 ithreads java5 java6 jbig jpeg jpeg2k lcms libnotify libsamplerate live lzma mad matroska mmap mmx mmxext mng modules mp3 mp4 mpeg mudflap multilib nautilus ncurses nls no-old-linux nptl nptlonly ogg ogm opengl openmp pam pango pcre pcsc-lite pdf png policykit ppds pppd pulseaudio readline rle schroedinger sdl session smp socialweb spell sse sse2 sse3 sse4 sse4_1 sse4a ssl ssse3 startup-notification svg system-sqlite tcpd theora threads tiff truetype udev udisks unicode upower usb vorbis vpx webp wmf x264 xcb xml xv xvid zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="de en" PHP_TARGETS="php5-3" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_2" RUBY_TARGETS="ruby18 ruby19" SANE_BACKENDS="plustek" USERLAND="GNU" VIDEO_CARDS="modesetting radeon r600" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHO
Comment 1 ernsteiswuerfel archtester 2012-12-12 21:34:09 UTC
Created attachment 332170 [details]
logfiles
Comment 2 ernsteiswuerfel archtester 2012-12-27 20:10:59 UTC
Ok, finally I got some more time to test this out. I noticed that a standard kernel built with genkernel does not show these page allocation failures. So I systematically changed the kernel .config until I got my custom kernel right...

It turned out that it was a single option causing these 'systemd-udevd: page allocation failure': KALLSYMS [=y]

General setup  --->
Configure standard kernel features (expert users)  --->
Load all symbols for debugging/ksymoops [=y]

Setting it to [=n] -> pagefaults, setting it to [=y] -> no pagefaults.

I did not know that KALLSYMS is important for udev at all. As previously said I can't tell when I got these errors exactly, but I've been using this kernel config since 3.0, updating it with make oldconfig. And I am sure at that time I did not get these errors.

Seems the problem is caused by some kernel <-> udev interaction. Any suspicious udev-activity lately, concerning KALLSYMS?
Comment 3 William Hubbs gentoo-dev 2013-01-21 20:58:24 UTC
No, there is nothing going on in this area that I know of.
Can you please upgrade to 197-r3 and report back whether this is still
happening?
Comment 4 ernsteiswuerfel archtester 2013-01-24 20:05:48 UTC
Upgraded to 197-r3 and kernel 3.7.4 and the issue still persists in the same way.
Comment 5 Samuli Suominen (RETIRED) gentoo-dev 2013-01-29 16:38:37 UTC
If your kernel has CONFIG_EXPERT=y then you also need CONFIG_KALLSYMS=y
Or use CONFIG_EXPERT=n

$ zgrep EXPERT /proc/config.gz
$ grep EXPERT /usr/src/linux/.config

Let us know if this was the case. Thanks.
Comment 6 Samuli Suominen (RETIRED) gentoo-dev 2013-01-31 10:47:20 UTC
Closing TEST-REQUEST then pending on reply to last Comment #5
Comment 7 ernsteiswuerfel archtester 2013-02-02 18:26:44 UTC
$ grep EXPERT /usr/src/linux/.config
CONFIG_EXPERT=y
$ grep KALLSYMS /usr/src/linux/.config
# CONFIG_KALLSYMS is not set
Comment 8 Samuli Suominen (RETIRED) gentoo-dev 2013-02-02 18:46:06 UTC
(In reply to comment #7)
> $ grep EXPERT /usr/src/linux/.config
> CONFIG_EXPERT=y
> $ grep KALLSYMS /usr/src/linux/.config
> # CONFIG_KALLSYMS is not set

looks like unworking combo...
Comment 9 ernsteiswuerfel archtester 2013-02-02 19:53:42 UTC
At least on my machine(s) it certainly is. However as said in Comment #5 this was not always the case.
Comment 10 Samuli Suominen (RETIRED) gentoo-dev 2013-02-03 06:33:12 UTC
Check added to -r6 and 9999
Comment 11 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2013-02-15 02:26:21 UTC
I'm reopening this, as there is NOTHING in the systemd-197 source that reads kallsyms. I've CC'd gregkh as maybe he has some insight into it.
Comment 12 Denis M. (Phr33d0m) 2013-02-15 02:48:36 UTC
I have had EXPERT=y and KALLSYMS=n (using the same config) for over a year with different kernel versions (using 3.8-rc7 now) and never seen such thing in my dmesg.

I've also just checked 2 other systems of mine (with EXPERT=y & KALLSYMS=n) with gentoo-sources-3.7.8 and 3.8-rc7 respectively and none of them shows this.

As we discussed on IRC, (and per what Robin says in comment 11) I'd agree to drop this check as OP might be the only one having this issue for some reason.
Comment 13 Kenny Klatt 2013-02-16 04:33:22 UTC
I think, I do not know, I may be having the same issue.  I get a kernel panic just after the boot process switches to a frame buffer.  I normally backup to a 16G USB thumbdrive.  When I remove the thumbdrive, boot process is fine.  When I leave it in, kernel crashes as it can not seem to find the root partition. No entries in the message log. Process is repeatable on a six-core AMD every time a boot is attempted, and sometimes on an Intel I7.  When I remove all thumb drives on boot, which I have not done in the past,no boot issues.

When searching the config file
# CONFIG_EXPERT is not set
CONFIG_KALLSYMS=y
CONFIG_KALLSYMS_ALL=y
Comment 14 Kenny Klatt 2013-02-24 16:53:29 UTC
Rebuild using 3.7.9 release.  Kernel panics with a USB drive in a port during boot, remove the drives boot is fine.  Bit more info.

panic
arch/x86/kernel/smpc 123
update_process_times +0x56/0x62()
warn /slowpath/common
Comment 15 ernsteiswuerfel archtester 2013-03-09 12:50:26 UTC
Transfered my kernel .config from 3.7.9 to 3.8.2 via oldconfig and recompiled. Now the page allocation failures are gone on this machine, despite
# CONFIG_KALLSYMS is not set
Comment 16 Samuli Suominen (RETIRED) gentoo-dev 2013-03-09 12:59:00 UTC
good, and with 3.7 marked EOL at kernel.org, I think it's safe to close this one now