I have my whole Gentoo Linux installed on btrfs + subvolume, with a separate /boot of ext4. I split the btrfs partition into several subvolumes for /home, /usr, /var and etc. After I upgrade sys-kernel/dracut version from 034-r4 to 036, I re-generated a new initramfs for my kernel (vanilla-sources-3.13.3). When I rebooted Gentoo, I am dropped into dracut shell. I checked journalctl and found there was an error of failed to mount /sysroot/usr. However, I checked /sysroot/usr and found it was mounted successfully. I searched the Internet and found a similiar bug report at: https://bugzilla.redhat.com/show_bug.cgi?id=890577. My situation is a bit different, I had no /etc/fstab when I was dropped into dracut shell. Anyway, sys-kernel/dracut-034-r4 works fine for me, so I have to mask >=sys-kernel/dracut-036 at the moment. I think you can repeat this issue easily with separate /usr subvolume on btrfs filesystem. If you need further information, please let me know.
And also, line 557 of /usr/bin/dracut shall be: if [ "${1%%=*}" == "++include" ]; then But not: if [ ${1%%=*} == "++include" ]; then I think this is an upstream issue.
Please attach rdsosreport.txt to this bug report. How to obtain it: append "rd.debug" to the kernel cmdline and after you dropped to emergency shell, save the file /run/initramfs/rdsosreport.txt on persistent storage.
Created attachment 370750 [details] rdsosreport.txt
Here I attach the rdsosreport.txt I obtained with rd.debug enabled. And also, emerge --info for sys-kernel/dracut: Portage 2.2.8-r1 (default/linux/amd64/13.0, gcc-4.8.2, glibc-2.18-r1, 3.13.3 x86_64) ================================================================= System Settings ================================================================= System uname: Linux-3.13.3-x86_64-Intel-R-_Core-TM-_i7-2630QM_CPU_@_2.00GHz-with-gentoo-2.2 KiB Mem: 12209444 total, 10453492 free KiB Swap: 12582908 total, 12582908 free Timestamp of tree: Tue, 18 Feb 2014 13:15:01 +0000 ld GNU ld (GNU Binutils) 2.24 ccache version 3.1.9 [enabled] app-shells/bash: 4.2_p45-r1 dev-lang/python: 2.7.6, 3.3.4 dev-util/ccache: 3.1.9-r3 dev-util/cmake: 2.8.12.2 dev-util/pkgconfig: 0.28 sys-apps/baselayout: 2.2 sys-apps/openrc: 0.12.4 sys-apps/sandbox: 2.6-r1 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.11.6, 1.14.1 sys-devel/binutils: 2.24-r2 sys-devel/gcc: 4.8.2 sys-devel/gcc-config: 1.8 sys-devel/libtool: 2.4.2 sys-devel/make: 4.0-r1 sys-kernel/linux-headers: 3.13 (virtual/os-headers) sys-libs/glibc: 2.18-r1 Repositories: gentoo gentoo-zh steam-overlay ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="* -@EULA AdobeFlash-11.x google-chrome Oracle-BCLA-JavaSE TeamViewer" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -mtune=native -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=native -mtune=native -O2 -pipe" DISTDIR="/usr/portage/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs ccache config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://mirrors.ustc.edu.cn/gentoo/" LANG="zh_CN.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j9" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/var/lib/layman/gentoo-zh /var/lib/layman/steam" SYNC="rsync://mirrors.ustc.edu.cn/gentoo-portage" USE="X Xaw3d a52 aac aalib acl acpi alsa amd64 apng avcodec avformat avx bash-completion berkdb bindist bluetooth bluray branding bzip2 cairo cdda cddb cli connection-sharing cracklib crypt cups cxx dbus dconf directx dri dts dv dvb eds encode exif extra ffmpeg firefox flac fontconfig fortran games gbm gconf gdbm gdu gif glamor gnome gnome-keyring gnome-online-accounts gpm gsettings gstreamer gtk gtk3 gudev hwdb iconv icu imagemagick infinality introspection ipv6 jadetex jpeg keymap kpathsea lame latex libass libkms libv4l lirc m17n-lib media-library minizip mmx modemmanager modules motif mp3 mp4 mpeg multilib nautilus ncurses networkmanager nls nptl ogg opengl openmp pam pango pcre pdf png policykit pulseaudio python readline sdl sdl-image sendto session skins sound speex sqlite sse sse2 ssl svg symlink systemd taglib tcpd theora threads tiff tk toolkit-scroll-bars truetype twolame udisks unicode upcall upnp usb uxa v4l vaapi vala vdpau vorbis vpx wavpack win32codecs winbind x264 xcb xft xpm xv xvid xvmc zlib" ABI_X86="64 32" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DRACUT_MODULES="btrfs" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse synaptics evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="zh zh_CN en en_US" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" RUBY_TARGETS="ruby21" USERLAND="GNU" VIDEO_CARDS="intel nouveau nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON ================================================================= Package Settings ================================================================= sys-kernel/dracut-036-r1 was built with the following: USE="(multilib) systemd -debug (-selinux)" ABI_X86="64"
> DRACUT_MODULES="btrfs" systemd module is not included in initramfs generated with dracut-0.34, right? :) [ 3.762845] Gentoo systemd[1]: Mounting /sysroot/usr... [ 3.798087] Gentoo systemd[1]: sysroot-usr.mount: Directory /sysroot/usr to mount over is not empty, mounting anyway. [ 3.971534] Gentoo kernel: btrfs: device fsid f0cc0adb-a9d2-4711-8d10-b8952882997c devid 1 transid 14051 /dev/sdc4 [ 3.801486] Gentoo systemd[1]: Started Reload Configuration from the Real Root. [ 3.801760] Gentoo systemd[1]: Started dracut mount hook. [ 3.810453] Gentoo mount[442]: mount: /dev/sdc4 is already mounted or /sysroot/usr busy [ 3.810689] Gentoo mount[442]: /dev/sdc4 is already mounted on /sysroot [ 3.810885] Gentoo systemd[1]: sysroot-usr.mount mount process exited, code=exited status=32 [ 3.811132] Gentoo systemd[1]: Failed to mount /sysroot/usr. sysroot-usr.mount is a unit generated by systemd-fstab-generator. So I suspect this is a systemd bug.
Please do the following (with dracut-036): 1. Attach your fstab. 2. Attach /run/systemd/generator/sysroot-usr.mount file. You can access it from emergency shell. 3. Generate initramfs with '--omit "systemd"' option and confirm that system boots fine.
Created attachment 370850 [details] fstab
Created attachment 370852 [details] sysroot-usr.mount
Indeed, with dracut-034, there is no systemd enabled. Anyway, the '--omit "systemd"' option for dracut-036 seems work fine. I think this should be the correct workaround.
If systemd module is not included in initramfs, then /usr is mounted by usrmount module. This module appends "ro" or "rw" to mount options for /usr depending on corresponding options from kernel cmdline. This is critical for btrfs subvolumes. See the patch for btrfs [1] (seems not applied by upstream yet). Whith systemd module fstab-generator is ignoring rw and ro options when generating sysroot-usr.mount unit: (In reply to wood_elf from comment #8) > Created attachment 370852 [details] > sysroot-usr.mount Options=subvolid=260,defaults,compress=lzo,space_cache,autodefrag (In reply to wood_elf from comment #3) > Created attachment 370750 [details] > rdsosreport.txt /sbin/rdsosreport@12(): cat /proc/mounts ... /dev/sdc4 /sysroot btrfs ro,relatime,space_cache 0 0 ... BOOT_IMAGE=/vmlinuz-3.13.3 root=UUID=f0cc0adb-a9d2-4711-8d10-b8952882997c ro rootflags=subvol=gentoo quiet init=/usr/lib/systemd/systemd modprobe.blacklist=nvidia rd.debug [1] http://comments.gmane.org/gmane.comp.file-systems.btrfs/30176
OK, I would try this patch and report the feedback ASAP after I made some tests. However, if the dracut option '--omit "systemd"' is harmless, it would be a considerable solution too. Anyway, if that btrfs patch works fine, it would be the preference for me.
Bad news, that btrfs patch does not help solve my problem. I am still dropped into dracut shell. And there is also good news, I tried to append 'rw' option to kernel cmdline, and I found my gentoo boots fine now. So, if either 'rw' option to the cmdline or '--omit "systemd"' option to dracut is harmless, I think we can close this report. Thank you for your help, Alexander Tsoy.
I think we should leave this bug open until this limitation will be fixed by btrfs upstream or workarounded by systemd upstream. %)
(In reply to Alexander Tsoy from comment #13) > I think we should leave this bug open until this limitation will be fixed by > btrfs upstream or workarounded by systemd upstream. %) OK, and I may take the 'rw' kernel cmdline option temporarily.
(In reply to wood_elf from comment #14) > (In reply to Alexander Tsoy from comment #13) > > I think we should leave this bug open until this limitation will be fixed by > > btrfs upstream or workarounded by systemd upstream. %) > > OK, and I may take the 'rw' kernel cmdline option temporarily. This is just a workaround. So please reopen the bug (I don't have permissions to do so :( ).
I was having the same problem using a BTRFS rootfs with a RAID-1 configuration. For me the problem turned out to be that the device mapper or one of the related frameworks was stealing the block devices. I added the following modules to my omit list since I am never going to use them: lvm multipath dmraid dm dmsquash-live mdraid I am not sure which one works but I am able to boot now with the systemd module enabled.
(In reply to Jordan Yelloz from comment #16) > I was having the same problem using a BTRFS rootfs with a RAID-1 > configuration. > For me the problem turned out to be that the device mapper or one of the > related frameworks was stealing the block devices. You actually have a completely different problem. Probably a stale metadata left on your disks.
Reassigning to Mike who's taking care about btrfs. @systemd: See comments #c10 and #c12.
Ok, so after reading through the comments and reproducing the issue myself, I think I'm just going to sit on this until the issue with ro/rw on btrfs subvolumes is resolved in the kernel.
JFYI: patch [1] is included in 3.15. [1] http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=0723a0473fb48a1c93b113a28665b64ce5faf35a
(In reply to Alexander Tsoy from comment #20) > JFYI: patch [1] is included in 3.15. Thanks. Closing this.
