Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 484040

Summary: sys-kernel/genkernel-next-24-r1 with sys-apps/systemd-206-r3 - !! Block device /dev/md1 is not a valid root device
Product: Gentoo Linux Reporter: Juergen Rose <rose>
Component: [OLD] Core systemAssignee: Fabio Erculiani (RETIRED) <lxnay>
Status: RESOLVED INVALID    
Severity: normal CC: systemd
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: .config of linux-3.11.0-gentoo
genkernel.conf
grub.cfg

Description Juergen Rose 2013-09-06 19:50:51 UTC
If I try to boot my system, it fails with:
...
>> Initializing root device ...
!! Block device /dev/md1 is not a valid root device

or 

>> Initializing root device ...
!! Unable to resolve root: UUID=af,,,,,
Comment 1 Mike Gilbert gentoo-dev 2013-09-06 23:05:05 UTC
What are you using for your initramfs? This is probably happening before systemd is even called.
Comment 2 Pacho Ramos gentoo-dev 2013-09-07 09:07:17 UTC
Remember that you will also need genkernel-next and it being used with "--udev" option
Comment 3 Juergen Rose 2013-09-08 19:32:53 UTC
The kernel and initial ramdisk was generated with genkernel-next-24 with gentoo-sources-3.10.5-gentoo-r1 and gentoo-sources-3.11.0 with the command:
genkernel --oldconfig --udev all .
In /etc/default/grub I have
GRUB_CMDLINE_LINUX_DEFAULT="dolvm domdadm root=/dev/md1 real_init=/usr/lib/systemd rootfstype=ext4
or
GRUB_CMDLINE_LINUX_DEFAULT="dolvm domdadm real_init=/usr/lib/systemd rootfstype=ext4 .

In the first case I get:
!! Block device /dev/md1 is not a valid root device
in the latter:
>> Scanning for and activating Volume Groups
  No volume groups found
>> Initializing root device...
[    3.013...] usb-storage 2-1.2:1.0: USB Mass Storage device detected
[    3.014...] scsi6 : usb-storage 2-1.2:1.0
[    3.088...] usb 2-1.3: new hight-speed USB device number 4 using ehci-pci
!! Unable to resolve root: UUID=af6fe4ae-3392-4cbb-98ee-8a5d91efbc29
!! Please file a bug report with this message
[    3.101...] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100
...
[    3.1234...] drm_kms_helper: panic occured, switching back to text console


the UUID=af6fe4ae-3392-4cbb-98ee-8a5d91efbc29 is the correct UUID of /dev/md1.



BTW., if I reinstalling the kernel and the initial ramdisk with 'genkernel --oldconfig --udev all', I get the following messages:

* LVM: Adding support (copying binaries from system)...
*         >> Appending mdadm cpio data...
*         >> Appending modules cpio data...
*         >> Appending blkid cpio data...
*         >> Appending ld_so_conf cpio data...
* ld.so.conf: adding /etc/ld.so.conf{.d/*,}...
cpio: lib64 not created: newer or same age version exists
cpio: usr/lib64 not created: newer or same age version exists
cpio: lib64 not created: newer or same age version exists
cpio: lib64/ld-linux-x86-64.so.2 not created: newer or same age version exists
cpio: lib64/libpthread.so.0 not created: newer or same age version exists
cpio: lib64/libc.so.6 not created: newer or same age version exists
cpio: lib64/librt.so.1 not created: newer or same age version exists
cpio: lib64 not created: newer or same age version exists
cpio: lib64/ld-linux-x86-64.so.2 not created: newer or same age version exists
cpio: lib64/libpthread.so.0 not created: newer or same age version exists
cpio: lib64/libc.so.6 not created: newer or same age version exists
cpio: lib64 not created: newer or same age version exists
cpio: lib64/libblkid.so.1 not created: newer or same age version exists
cpio: lib64/ld-linux-x86-64.so.2 not created: newer or same age version exists
cpio: lib64/libuuid.so.1 not created: newer or same age version exists
cpio: lib64/libc.so.6 not created: newer or same age version exists
*         >> Finalizing cpio...
*         >> Compressing cpio data (.xz)...
* 
* Kernel compiled successfully!

Are these cpio messages normal or is something wrong?
Comment 4 Juergen Rose 2013-09-08 19:33:52 UTC
Created attachment 358314 [details]
.config of linux-3.11.0-gentoo
Comment 5 Juergen Rose 2013-09-08 19:34:29 UTC
Created attachment 358316 [details]
genkernel.conf
Comment 6 Juergen Rose 2013-09-08 19:34:56 UTC
Created attachment 358318 [details]
grub.cfg
Comment 7 Mike Gilbert gentoo-dev 2013-09-08 19:53:10 UTC
So your /usr volume is separate from your root filesystem?

This is a genkernel-next bug, not systemd.
Comment 8 Juergen Rose 2013-09-09 08:22:09 UTC
(In reply to Pacho Ramos from comment #2)
> Remember that you will also need genkernel-next and it being used with
> "--udev" option

(In reply to Mike Gilbert from comment #7)
> So your /usr volume is separate from your root filesystem?
> 
> This is a genkernel-next bug, not systemd.

There was a separate /usr volume in the past. After some days with making backups, repartioning the harddisk, recreating soft arrays, reformating the the new volumes and restoring these volumes / and /usr is now together at /dev/md1.
I.e,, /usr is now not a separate volume.
Comment 9 Mike Gilbert gentoo-dev 2013-09-09 14:27:14 UTC
Did you also include the --mdadm option when running genkernel? I think you'll need that to get the md devices detected.
Comment 10 Juergen Rose 2013-09-10 05:25:59 UTC
(In reply to Mike Gilbert from comment #9)
> Did you also include the --mdadm option when running genkernel? I think
> you'll need that to get the md devices detected.

To have MDADM=yes in /etc/genkernel.conf is not enough?
Comment 11 Fabio Erculiani (RETIRED) gentoo-dev 2013-09-10 06:02:04 UTC
(In reply to Juergen Rose from comment #10)
> (In reply to Mike Gilbert from comment #9)
> > Did you also include the --mdadm option when running genkernel? I think
> > you'll need that to get the md devices detected.
> 
> To have MDADM=yes in /etc/genkernel.conf is not enough?

No.
Comment 12 Mike Gilbert gentoo-dev 2013-09-10 15:11:03 UTC
(In reply to Fabio Erculiani from comment #11)
> > To have MDADM=yes in /etc/genkernel.conf is not enough?
> 
> No.

Really? That seems to work just fine here.
Comment 13 Juergen Rose 2013-09-11 16:56:47 UTC
(In reply to Juergen Rose from comment #10)
> (In reply to Mike Gilbert from comment #9)
> > Did you also include the --mdadm option when running genkernel? I think
> > you'll need that to get the md devices detected.
> 
> To have MDADM=yes in /etc/genkernel.conf is not enough?

Even if I run genkernel with --mdadm, I get the same error:

Unable to resolve root: UUID=af6fe...
Comment 14 Juergen Rose 2013-09-12 09:22:27 UTC
This system was working for more than a year without problems, There has been three soft array drives, first, very small - 128MB, /dev/md0 for /boot, second small - 2GB, /dev/md1 for / and third /dev/md2 with more than 900 GB for a physical volume containing logical volumes for /usr, /home, /var etc. After the appearance of systemd and the removal of sys-fs/udev from the portage tree I had to merge the / and the /usr partition. So I have now a larger /dev/md1 with 128G and a smaller /dev/md2. But I can not boot the system, because the the root partion is not recognized.

If I boot from systemrescuecd, I do not have any problems to mount the root partions on /mnt/gentoo and the other partitions at the corresponding mount points:

root@sysresccd /root % cat /proc/mdstat                  
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
md2 : active raid1 sda6[0] sdb6[1]
      825599568 blocks super 1.2 [2/2] [UU]
md1 : active raid1 sda3[0] sdb3[1]
      134217664 blocks [2/2] [UU]      
md0 : active raid1 sda1[0] sdb1[1]
      131008 blocks [2/2] [UU]
unused devices: <none>

root@sysresccd /root % pvscan
  PV /dev/md2   VG vg0   lvm2 [787.35 GiB / 0    free]
  Total: 1 [787.35 GiB] / in use: 1 [787.35 GiB] / in no VG: 0 [0   ]

root@sysresccd /root % df -T  | grep /mnt/gentoo   
/dev/md1                ext4     132111780  36712184  88688716  30% /mnt/gentoo
/dev/mapper/vg0-var     ext3       4128448   1561912   2356824  40% /mnt/gentoo/var
/dev/mapper/vg0-vtmp    reiserfs  33553404  11574736  21978668  35% /mnt/gentoo/var/tmp
/dev/mapper/vg0-portage ext2       4142882    404861   3528306  11% /mnt/gentoo/usr/portage_leopard
/dev/mapper/vg0-opt     ext3       8256952   1954368   5883156  25% /mnt/gentoo/opt
/dev/mapper/vg0-tmp     reiserfs  16776700     32840  16743860   1% /mnt/gentoo/tmp
/dev/mapper/vg0-src     reiserfs  33553404   6034900  27518504  18% /mnt/gentoo/usr/src_leopard
/dev/mapper/vg0-home    ext4     322022508 194640984 111023740  64% /mnt/gentoo/home_leopard
/dev/mapper/vg0-ftp     xfs      134152192  89076532  45075660  67% /mnt/gentoo/home_leopard/ftp
/dev/mapper/vg0-data    ext4     226397604   8985260 205912008   5% /mnt/gentoo/data_leopard
/dev/md0                ext2        126867    113373      6944  95% /mnt/gentoo/boot

I can chroot to /mnt/gentoo and install new kernels and initial ramdisks, now with genkernel-next-24-r1 and linux-3.11.0-gentoo:

root@sysresccd /root % chroot /mnt/gentoo /bin/bash

root@sysresccd:/(1)# qlist -Iv genkernel
sys-kernel/genkernel-next-24-r1

root@sysresccd:/(2)# ll /dev/disk/by-uuid/ | grep md1
lrwxrwxrwx 1 root root  9 Sep 12 09:40 af6fe4ae-3392-4cbb-98ee-8a5d91efbc29 -> ../../md1
root@sysresccd:/(3)# ll /usr/src/linux
lrwxrwxrwx 1 root root 19 Sep  8 20:15 /usr/src/linux -> linux-3.11.0-gentoo/

root@sysresccd:/(13)# grep -e  "SYSTEMD\|RAID\|TMPFS" /usr/src/linux-3.11.0-gentoo/.config
CONFIG_GENTOO_LINUX_INIT_SYSTEMD=y
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
# CONFIG_RAID_ATTRS is not set
# CONFIG_MD_RAID0 is not set
CONFIG_MD_RAID1=y
CONFIG_MD_RAID10=m
CONFIG_MD_RAID456=y
# CONFIG_DM_RAID is not set
CONFIG_TMPFS=y
CONFIG_TMPFS_POSIX_ACL=y
CONFIG_TMPFS_XATTR=y
# CONFIG_ASYNC_RAID6_TEST is not set
CONFIG_ASYNC_RAID6_RECOV=y
CONFIG_RAID6_PQ=y

root@sysresccd:/(15)# mdadm --detail /dev/md1
/dev/md1:
        Version : 0.90
  Creation Time : Tue Sep  3 19:30:53 2013
     Raid Level : raid1
     Array Size : 134217664 (128.00 GiB 137.44 GB)
  Used Dev Size : 134217664 (128.00 GiB 137.44 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 1
    Persistence : Superblock is persistent

    Update Time : Thu Sep 12 10:55:02 2013
          State : clean 
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

           UUID : 1aa5f934:544de9ca:c44c77eb:7ee19756 (local to host sysresccd)
         Events : 0.18

    Number   Major   Minor   RaidDevice State
       0       8        3        0      active sync   /dev/sda3
       1       8       19        1      active sync   /dev/sdb3


The last kernel was generated yesterday evening with "genkernel --oldconfig --udev --mdadm all":

root@sysresccd:/(19)# ll -t /boot/ | head -n 7
total 99960
drwxr-xr-x 6 root root    1024 Sep 11 18:44 grub2/
-rw-r--r-- 1 root root 4046832 Sep 11 18:43 initramfs-genkernel-x86_64-3.11.0-gentoo
-rw-r--r-- 1 root root 4892864 Sep 11 18:42 kernel-genkernel-x86_64-3.11.0-gentoo
-rw-r--r-- 1 root root 2259851 Sep 11 18:42 System.map-genkernel-x86_64-3.11.0-gentoo
-rw-r--r-- 1 root root 4119728 Sep  8 19:43 initramfs-genkernel-x86_64-3.10.5-gentoo-r1
-rw-r--r-- 1 root root 4900880 Sep  8 19:42 kernel-genkernel-x86_64-3.10.5-gentoo-r1


root@sysresccd:/(20)# grep 3.11 /boot/grub2/grub.cfg 
        echo    'Loading Linux x86_64-3.11.0-gentoo ...'
        linux   /kernel-genkernel-x86_64-3.11.0-gentoo root=UUID=af6fe4ae-3392-4cbb-98ee-8a5d91efbc29 ro  dolvm domdadm real_init=/usr/lib/systemd/systemd rootfstype=ext4
        initrd  /initramfs-genkernel-x86_64-3.11.0-gentoo
        menuentry 'Gentoo GNU/Linux, with Linux x86_64-3.11.0-gentoo' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-x86_64-3.11.0-gentoo-advanced-af6fe4ae-3392-4cbb-98ee-8a5d91efbc29' {
                echo    'Loading Linux x86_64-3.11.0-gentoo ...'
                linux   /kernel-genkernel-x86_64-3.11.0-gentoo root=UUID=af6fe4ae-3392-4cbb-98ee-8a5d91efbc29 ro  dolvm domdadm real_init=/usr/lib/systemd/systemd rootfstype=ext4
                initrd  /initramfs-genkernel-x86_64-3.11.0-gentoo
        menuentry 'Gentoo GNU/Linux, with Linux x86_64-3.11.0-gentoo (recovery mode)' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-x86_64-3.11.0-gentoo-recovery-af6fe4ae-3392-4cbb-98ee-8a5d91efbc29' {
                echo    'Loading Linux x86_64-3.11.0-gentoo ...'
                linux   /kernel-genkernel-x86_64-3.11.0-gentoo root=UUID=af6fe4ae-3392-4cbb-98ee-8a5d91efbc29 ro single 
                initrd  /initramfs-genkernel-x86_64-3.11.0-gentoo


If I try to boot from hardisk it fails with:

>> Loading modules
   :: Loading from pata: ...
...
   :: Loading from sata: ...
   :: Loading from scsi: ...
...
   :: Loading from usb: ...
...
   :: Loading from dmraid: dm-mirror dm-crypt
   :: Loading from mdadm: dm-snapshot dm-mirror dm-crypt [    2.64...] md: raid10 personality regeistered for level 10 
raid10
   :: Loading from fs: ...
...
   :: Loading from net:
...
   :: Loading from lvm: dm-snapshot dm-mirror dm -crypt dm-log dm-region-hash dm-zero
[    2.76....]  md0: unknow partition table
>> Scanning for and activatiing Volume Groups
  No volume groups found
>> Initilizing root device...
[    2.94....] Switching to clocksource tsc
[    3.010...] usb-storage 2-1.2:1.0: USB Mass Storage device detected
[    3,011...] scsi6 : usb-storage 2-1.2:1.0
!! Unable to resolve root: UUID-af6fe4ae-3392-4cbb-98ee-8a5d91efbc29
!! Please file a but report with this message
[    3.090...] Kernel panic - not syncing: Attempting to kill init! exitcode=0x00000100
[    3.090...]
[    3.092...] CPU 1 PID: 1 Comm: init Not tainted 3.11.0-gentoo #4
...
[    3.110...] drm_kms_helper: panic occured, switching back to text console


I can not boot this system from hd for longer than one week. Any hint is apprecieated.
Comment 15 Juergen Rose 2013-09-13 14:38:46 UTC
Removing the MDADM, MDADM_CONFIG and DISKLABEL entries from /etc/genkernel.conf and deleting root=/dev/md1 part in /etc/default/grub seems to solve the problem.
My soft arrays changed now there names, I have now md125, md126 and md127:

root@leopard:/root(16)# cat /proc/mdstat 
Personalities : [raid1] [raid6] [raid5] [raid4] [raid10] 
md125 : active raid1 sda1[0] sdb1[1]
      131008 blocks [2/2] [UU]
      
md126 : active raid1 sda3[0] sdb3[1]
      134217664 blocks [2/2] [UU]
      
md127 : active raid1 sda6[0] sdb6[1]
      825599568 blocks super 1.2 [2/2] [UU]
Comment 16 Fabio Erculiani (RETIRED) gentoo-dev 2013-10-14 15:26:13 UTC
Not related to genkernel-next itself, closing.