I'm trying to generate initramfs using dracut (wanted to test plymouth), but I'm getting hangs or BUGs during boot with dracut generated initramfs. I have tried dracut-026-r1 as well as git version and a range of kernels from 3.7.10-r1-gentoo, trough vanilla 3.8.4 and up to 3.9.0-rc4 from git (all kernels compiled with genkernel). All seems to have the same issue. I'm able to boot ~1 of 3-5 boots, the rest of the time it hangs in various places. I'm not sure if it is kernel issue, dracut or my configuration, but in the rare cases where the system survives the boot it has BUGs in syslog about unhandled NULLs. Reproducible: Always Steps to Reproduce: 1. USE=debug emerge -1 dracut 2. dracut -f -a "debug" -o "plymouth" foobar.img 3.9.0-rc4 3. mv foobar.img /boot/initramfs-dracut-x86_64-3.9.0-rc4 4. Set initrd to use initramfs-dracut-x86_64-3.9.0-rc4 during boot Actual Results: System lock-up or BUGs during boot. The same kernel boots fine with genkernel generated initramfs. Expected Results: Boots every time without errors. Portage 2.1.11.58 (default/linux/amd64/13.0/desktop/gnome, gcc-4.7.2, glibc-2.16.0, 3.9.0-rc4 x86_64) ================================================================= System Settings ================================================================= System uname: Linux-3.9.0-rc4-x86_64-Intel-R-_Core-TM-_i7-2630QM_CPU_@_2.00GHz-with-gentoo-2.2 KiB Mem: 8077892 total, 1051148 free KiB Swap: 2047996 total, 2047996 free Timestamp of tree: Sun, 24 Mar 2013 07:45:01 +0000 ld GNU ld (GNU Binutils) 2.23.1 app-shells/bash: 4.2_p45 dev-java/java-config: 2.1.12-r1 dev-lang/python: 2.6.8-r1, 2.7.3-r3, 3.1.5-r1, 3.2.3-r2 dev-util/cmake: 2.8.10.2-r1 dev-util/pkgconfig: 0.28 sys-apps/baselayout: 2.2 sys-apps/openrc: 0.11.8 sys-apps/sandbox: 2.6-r1 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.9.6-r3, 1.11.6, 1.12.6, 1.13.1 sys-devel/binutils: 2.23.1 sys-devel/gcc: 4.5.4, 4.6.3, 4.7.2-r1 sys-devel/gcc-config: 1.8 sys-devel/libtool: 2.4.2 sys-devel/make: 3.82-r4 sys-kernel/linux-headers: 3.8 (virtual/os-headers) sys-libs/glibc: 2.16.0 Repositories: gentoo x11 science gamerlay bumblebee slyfox-gentoo dmol tryton local ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -mtune=native -O2 -pipe -ggdb" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /var/lib/hsqldb" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=native -mtune=native -O2 -pipe -ggdb" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="" FCFLAGS="-march=native -mtune=native -O2 -pipe -ggdb" FEATURES="assume-digests binpkg-logs compress-build-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch parallel-install protect-owned sandbox sfperms splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync" FFLAGS="-march=native -mtune=native -O2 -pipe -ggdb" GENTOO_MIRRORS="http://gentoo.tups.lv/source/ " LANG="lv_LV.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j5" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/var/lib/layman/x11 /var/lib/layman/science /var/lib/layman/gamerlay /var/lib/layman/bumblebee /var/lib/layman/slyfox /var/lib/layman/dmol /var/lib/layman/tryton /usr/local/portage" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="X a52 aac acl acpi alsa amd64 avx bash-completion berkdb bluetooth branding bzip2 cairo cdda cdio cdr cjk cleartype cli colord consolekit cracklib crypt cups cxx dbus dirac djvu dri dts dvd dvdr eds emboss encode evo exif fam ffmpeg fftw firefox flac fontconfig fortran gdbm gif gnome gnome-keyring gnome-online-accounts gphoto2 gpm gsm gstreamer gtk gtk3 iconv idn ipv6 jpeg kate lcms ldap libcaca libnotify live mad matroska mmx mng modules mp3 mp4 mpeg mtp mudflap multilib musepack nautilus ncurses networkmanager nls nptl ogg openexr opengl openmp opus pam pango pcre pdf png policykit ppds pulseaudio qt3support qt4 raw readline schroedinger sdl session smp socialweb speex spell sse sse2 sse41 sse4_1 ssl ssse3 startup-notification svg system-sqlite tcpd theora tiff truetype udev udisks unicode upower usb v4l v4l2 vaapi vorbis vpx wmf wxwidgets x264 xcb xetex xml xmp xpm xv xvid xvmc zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DRACUT_MODULES="caps plymouth syslog" ELIBC="glibc" FOO2ZJS_DEVICES="hp1018" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="lv en" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-3" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_2" RUBY_TARGETS="ruby18 ruby19" USERLAND="GNU" VIDEO_CARDS="dummy fbdev i965 intel nouveau nvidia vesa" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON ================================================================= Package Settings ================================================================= sys-kernel/dracut-9999 was built with the following: USE="debug (multilib) -device-mapper -net -optimization (-selinux)" ABI_X86="64" DRACUT_MODULES="caps plymouth syslog -biosdevname -bootchart -btrfs -cifs -crypt -crypt-gpg -crypt-loop -dmraid -dmsquash-live -gensplash -iscsi -livenet -lvm -mdraid -multipath -nbd -nfs -ssh-client -systemd" sys-fs/udev-198-r5 was built with the following: USE="acl gudev hwdb introspection keymap kmod (multilib) openrc static-libs -doc (-selinux)" ABI_X86="64"
Created attachment 343070 [details] kernel-3.9.0-rc4.config
Created attachment 343072 [details] dracut.conf
Created attachment 343074 [details] dracut.log
Created attachment 343076 [details] initramfs-dracut-x86_64-3.9.0-rc4.syslog
> I'm not sure if it is kernel issue I'm not sure if it is unreliable hardware, because of the random nature of your success with booting. You can negate both - hardware - kernel .config issues by trying to use genkernel with the exact same .config. Because hard bug search only pays the effort, if this shows reliable booting ...
I was using the exact same .config with genkernel, and as I said in the initial report it works reliably with genkernel-generated iniramfs. The issues are only with dracut-generated initramfs. I think it is a race somewhere, that is the reason why I'm able to boot sometimes.
I have talked with Dracut lead developer: 13:04 < haraldh> kernel bug 13:04 < haraldh> triggered by udevadm in the initramfs 13:05 < haraldh> nothing todo with dracut I am reassigning to bug-wranglers again and CC'ing udev team and kernel team to decide.
Could you provide us the BUG information, especially the call trace? A similar bug regarding waking the futex can be found at http://thread.gmane.org/gmane.linux.kernel/1377368/focus=1397725.
All the information is already in the attached syslog. Have you looked at it?
I have similar problem, but I am able to boot. I've tried gentoo-source-3.7.9, gentoo-source-3.7.10, gentoo-source-3.10.r1 and gentoo-source-3.8.6. Every time I get "BUG: unable to handle kernel NULL pointer dereference at (null)". Additionally with the 3.8.6 kernel I update from dracut-23 to 27 and 26. Nothing changed, but dracut-27 couldn't handle my encrypted disk. I'll add my /var/log/dmesg and emergre --info output. It's about this section: [ 0.443940] BUG: unable to handle kernel NULL pointer dereference at (null) [ 0.444216] IP: [<ffffffff81091059>] futex_wake+0x79/0x130 [ 0.444323] PGD 40db34067 PUD 40dabd067 PMD 0 [ 0.444509] Oops: 0000 [#2] PREEMPT SMP [ 0.444694] Modules linked in: [ 0.444794] CPU 1 [ 0.444839] Pid: 453, comm: udevadm Tainted: G D 3.8.6-gentoo #1 LENOVO 23252DG/23252DG [ 0.444966] RIP: 0010:[<ffffffff81091059>] [<ffffffff81091059>] futex_wake+0x79/0x130 [ 0.445076] RSP: 0018:ffff88040c499d78 EFLAGS: 00010282 [ 0.445136] RAX: 0000000000000000 RBX: 00000000ffffffff RCX: 000000007a29f47f [ 0.445199] RDX: ffffffff817df9f0 RSI: 000000004f8accff RDI: 0000000000000001 [ 0.445262] RBP: ffff88040c499dd8 R08: 00000000e6f79060 R09: 0000000000000001 [ 0.445325] R10: 0000000000000001 R11: 00007fff9b7782dc R12: 0000000000000001 [ 0.445387] R13: 0000000000000000 R14: 000000000000004a R15: ffffffff817df9f8 [ 0.445451] FS: 00007f1698018780(0000) GS:ffff88041e280000(0000) knlGS:0000000000000000 [ 0.445517] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 0.445579] CR2: 0000000000000000 CR3: 000000040db28000 CR4: 00000000001407e0 [ 0.445642] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 0.445706] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 0.445769] Process udevadm (pid: 453, threadinfo ffff88040c498000, task ffff88040d4800c0) [ 0.445835] Stack: [ 0.445890] ffff88040c499da8 ffffffff817df9f0 ffff88040c499e40 00007fff9b778000 [ 0.446119] ffff88040d849b00 00000000000002dc 0000000000000000 0000000000000000 [ 0.447565] 00007fff9b7782dc 0000000000000001 000000000000004a 0000000000000000 [ 0.447796] Call Trace: [ 0.447856] [<ffffffff81092c38>] do_futex+0x318/0xbc0 [ 0.447919] [<ffffffff8102b63c>] ? __do_page_fault+0x1bc/0x440 [ 0.447986] [<ffffffff81132711>] ? mntput+0x21/0x30 [ 0.448047] [<ffffffff81116188>] ? __fput+0x168/0x220 [ 0.448107] [<ffffffff8109361d>] sys_futex+0x13d/0x190 [ 0.448167] [<ffffffff8102b8e9>] ? do_page_fault+0x9/0x10 [ 0.448227] [<ffffffff81477992>] system_call_fastpath+0x16/0x1b [ 0.448286] Code: c5 0f 85 b5 00 00 00 48 8d 7d b8 e8 92 f0 ff ff 48 89 c7 48 89 45 a8 e8 e6 5a 3e 00 48 8b 55 a8 48 8b 42 08 49 89 d7 49 83 c7 08 <4c> 8b 30 48 8d 78 e8 49 83 ee 18 49 39 c7 75 1e eb 65 0f 1f 44 [ 0.451133] RIP [<ffffffff81091059>] futex_wake+0x79/0x130 [ 0.451238] RSP <ffff88040c499d78> [ 0.451297] CR2: 0000000000000000 [ 0.451373] ---[ end trace e85b95ce93221c34 ]--- [ 0.451375] note: udevadm[453] exited with preempt_count 1 it appears 5 time
Created attachment 344750 [details] My dmesg log The dmesg log booting of gentoo-sources-3.8.6 with an initramfs generated by dracut-26
Created attachment 344752 [details] emerge --info
This happens in match_futex where one of the parameters given to it is null. This call is made from kernel/futex.c:2154, where I assume this->key to be the problematic parameter as the other parameter is set properly at the start of that function. This apparently comes from a list of waiters. One of the waiters to be problematic, not sure how to know which one. From a blame we can tell this function hasn't been adjusted, I assume the problem therefore to be elsewhere in the kernel. A hard to debug problem without first gaining more knowledge about this code. I'd suggest you to report this bug upstream at http://bugzilla.kernel.org as they know the code better...
When you do, please leave a link to the upstream bug here.
I have the same problem. https://bugzilla.kernel.org/show_bug.cgi?id=65661
*** Bug 492458 has been marked as a duplicate of this bug. ***
The problem can be resolved by installing systemd (and so removing (e)udev).
I still see the bug with vanilla 3.18.3 and I'm not happy with the Manuel solution, i.e. installing systemd. I've filled another kernel bug, already before I found this one (the Manuel's is marked as resolved.) https://bugzilla.kernel.org/show_bug.cgi?id=82111 The solution can be to make the initramfs by dracut in a chroot with systemd. Then the oopses are gone and there is no need to have systemd installed system-wide. Anyway I think kernel should simply not oops on udevadm, should it?
