Watching a dmesg output I realized I got several allocation failures at bootup lately... First I thought this has been caused by some kernel update or config issue - but this doesn't seem to be the case. Tried out gentoo-sources-3.5.7, gentoo-sources-3.6.10, vanilla-sources-3.6.9, vanilla-sources-3.6.10, vanilla-sources-3.7.0 I get these faults every time. Also on my other machine (which has a more conservative setup with stable gcc and udev, ...). I suspect udev but that's only a guess based on my dmesg output: [...] vmalloc: allocation failure: 0 bytes systemd-udevd: page allocation failure: order:0, mode:0xd2 Pid: 1252, comm: systemd-udevd Not tainted 3.6.10-gentoo #3 Call Trace: [<ffffffff8109b46f>] ? 0xffffffff8109b46f [<ffffffff810c2221>] ? 0xffffffff810c2221 [<ffffffff81311cf7>] ? 0xffffffff81311cf7 vmalloc: allocation failure: 0 bytes systemd-udevd: page allocation failure: order:0, mode:0xd2 Pid: 1253, comm: systemd-udevd Not tainted 3.6.10-gentoo #3 [...] For full logs (dmesg, messages, kernel config, rc.log) see the attatchment. Apart from these page allocation issues both of my machines are running. I have not encoutered any specifig error or crash yet. So I really got no clue what's it about?? Reproducible: Always Portage 2.1.11.31 (default/linux/amd64/10.0/desktop/gnome, gcc-4.6.3, glibc-2.15-r3, 3.6.10-gentoo x86_64) ================================================================= System uname: Linux-3.6.10-gentoo-x86_64-AMD_FX-tm-8350_Eight-Core_Processor-with-gentoo-2.1 Timestamp of tree: Tue, 11 Dec 2012 22:00:01 +0000 ld GNU ld (GNU Binutils) 2.22 distcc 3.1 x86_64-pc-linux-gnu [disabled] app-shells/bash: 4.2_p37 dev-java/java-config: 2.1.11-r3 dev-lang/python: 2.7.3-r2, 3.2.3 dev-util/cmake: 2.8.9 dev-util/pkgconfig: 0.27.1 sys-apps/baselayout: 2.1-r1 sys-apps/openrc: 0.11.8 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.68 sys-devel/automake: 1.9.6-r3, 1.11.6 sys-devel/binutils: 2.22-r1 sys-devel/gcc: 4.6.3 sys-devel/gcc-config: 1.7.3 sys-devel/libtool: 2.4-r1 sys-devel/make: 3.82-r3 sys-kernel/linux-headers: 3.6 (virtual/os-headers) sys-libs/glibc: 2.15-r3 Repositories: gentoo seden x-portage ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-pipe -Os -mtune=native -march=native -fstack-protector -fpredictive-commoning -fgcse-after-reload -ftree-vectorize -fivopts -floop-interchange -floop-strip-mine -floop-block -ftree-loop-im" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-pipe -Os -mtune=native -march=native -fstack-protector -fpredictive-commoning -fgcse-after-reload -ftree-vectorize -fivopts -floop-interchange -floop-strip-mine -floop-block -ftree-loop-im -fvisibility-inlines-hidden" DISTDIR="/usr/portage/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="ftp://mirror.switch.ch/mirror/gentoo/ http://mirror.switch.ch/ftp/mirror/gentoo/ " LANG="de_DE.utf8" LDFLAGS="-Wl,-O1 -Wl,-z,now -Wl,-z,relro -Wl,--sort-common -Wl,--as-needed" MAKEOPTS="-j9" PKGDIR="/usr/portage/packages" PORTAGE_COMPRESS="xz" PORTAGE_COMPRESS_FLAGS="-6" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/var/lib/layman/seden /usr/local/portage" SYNC="rsync://yea/gentoo-portage" USE="X a52 aac acl acpi alsa amd64 avx bash-completion berkdb bluetooth bluray bzip2 cairo cdda cdr cli colord consolekit cracklib crypt cups custom-cflags custom-optimization cxx dbus dirac djvu dri dts dvd dvdr eds emboss enca encode evo exif firefox flac fortran gdbm gif gnome gnome-keyring gnome-online-accounts gnutls gstreamer gtk iconv icu ipv6 ithreads java5 java6 jbig jpeg jpeg2k lcms libnotify libsamplerate live lzma mad matroska mmap mmx mmxext mng modules mp3 mp4 mpeg mudflap multilib nautilus ncurses nls no-old-linux nptl nptlonly ogg ogm opengl openmp pam pango pcre pcsc-lite pdf png policykit ppds pppd pulseaudio readline rle schroedinger sdl session smp socialweb spell sse sse2 sse3 sse4 sse4_1 sse4a ssl ssse3 startup-notification svg system-sqlite tcpd theora threads tiff truetype udev udisks unicode upower usb vorbis vpx webp wmf x264 xcb xml xv xvid zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="de en" PHP_TARGETS="php5-3" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_2" RUBY_TARGETS="ruby18 ruby19" SANE_BACKENDS="plustek" USERLAND="GNU" VIDEO_CARDS="modesetting radeon r600" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHO
Created attachment 332170 [details] logfiles
Ok, finally I got some more time to test this out. I noticed that a standard kernel built with genkernel does not show these page allocation failures. So I systematically changed the kernel .config until I got my custom kernel right... It turned out that it was a single option causing these 'systemd-udevd: page allocation failure': KALLSYMS [=y] General setup ---> Configure standard kernel features (expert users) ---> Load all symbols for debugging/ksymoops [=y] Setting it to [=n] -> pagefaults, setting it to [=y] -> no pagefaults. I did not know that KALLSYMS is important for udev at all. As previously said I can't tell when I got these errors exactly, but I've been using this kernel config since 3.0, updating it with make oldconfig. And I am sure at that time I did not get these errors. Seems the problem is caused by some kernel <-> udev interaction. Any suspicious udev-activity lately, concerning KALLSYMS?
No, there is nothing going on in this area that I know of. Can you please upgrade to 197-r3 and report back whether this is still happening?
Upgraded to 197-r3 and kernel 3.7.4 and the issue still persists in the same way.
If your kernel has CONFIG_EXPERT=y then you also need CONFIG_KALLSYMS=y Or use CONFIG_EXPERT=n $ zgrep EXPERT /proc/config.gz $ grep EXPERT /usr/src/linux/.config Let us know if this was the case. Thanks.
Closing TEST-REQUEST then pending on reply to last Comment #5
$ grep EXPERT /usr/src/linux/.config CONFIG_EXPERT=y $ grep KALLSYMS /usr/src/linux/.config # CONFIG_KALLSYMS is not set
(In reply to comment #7) > $ grep EXPERT /usr/src/linux/.config > CONFIG_EXPERT=y > $ grep KALLSYMS /usr/src/linux/.config > # CONFIG_KALLSYMS is not set looks like unworking combo...
At least on my machine(s) it certainly is. However as said in Comment #5 this was not always the case.
Check added to -r6 and 9999
I'm reopening this, as there is NOTHING in the systemd-197 source that reads kallsyms. I've CC'd gregkh as maybe he has some insight into it.
I have had EXPERT=y and KALLSYMS=n (using the same config) for over a year with different kernel versions (using 3.8-rc7 now) and never seen such thing in my dmesg. I've also just checked 2 other systems of mine (with EXPERT=y & KALLSYMS=n) with gentoo-sources-3.7.8 and 3.8-rc7 respectively and none of them shows this. As we discussed on IRC, (and per what Robin says in comment 11) I'd agree to drop this check as OP might be the only one having this issue for some reason.
I think, I do not know, I may be having the same issue. I get a kernel panic just after the boot process switches to a frame buffer. I normally backup to a 16G USB thumbdrive. When I remove the thumbdrive, boot process is fine. When I leave it in, kernel crashes as it can not seem to find the root partition. No entries in the message log. Process is repeatable on a six-core AMD every time a boot is attempted, and sometimes on an Intel I7. When I remove all thumb drives on boot, which I have not done in the past,no boot issues. When searching the config file # CONFIG_EXPERT is not set CONFIG_KALLSYMS=y CONFIG_KALLSYMS_ALL=y
Rebuild using 3.7.9 release. Kernel panics with a USB drive in a port during boot, remove the drives boot is fine. Bit more info. panic arch/x86/kernel/smpc 123 update_process_times +0x56/0x62() warn /slowpath/common
Transfered my kernel .config from 3.7.9 to 3.8.2 via oldconfig and recompiled. Now the page allocation failures are gone on this machine, despite # CONFIG_KALLSYMS is not set
good, and with 3.7 marked EOL at kernel.org, I think it's safe to close this one now
