When sys-fs/mdadm (3.3.1-r2 is called by /etc/init.d/mdraid at early boot time, if the kernel has alredy assembled the RAID-1 array mdadm returns an empty error string. This results in a red asterisk appearing alone on the line following "Starting up RAID devices..." Reproducible: Always My amd64 system (not ~) has /boot on /dev/sda1, / on /dev/md5 and everything else in lvm2 volumes on /dev/md7. /dev/md5 is built on /dev/sda5 and /dev/sdb5; /dev/md7 is built on /dev/sda7 and /dev/sdb7. I set metadata 0.9 when creating /dev/md5 because it's the root device so the kernel needs to be able to assemble it. I thought I'd set metadata 1.x in /dev/md7 but apparently I was mistaken and set 0.9. The postinst message from sys-fs/mdadm is ambiguous about whether I need mdraid in my boot run-level, and the installation docs at the time said I definitely do, so I did "rc-update add mdraid boot". Now, at boot time, the kernel detects /dev/md5 and /dev/md7 and assembles them, so by the time /etc/init.d/mdraid is called the devices are already active. The script fails to assemble them again, so returns an error - but the message is empty. I see two problems: 1. The docs should distinguish between the two metadata types. 2. /etc/init.d/mdraid should act sensibly in the nothing-to-do case. It would be good also to have a clearer description somewhere of how mdraid and lvm work, including how to use the tools. As far as I can see, at present you have to know most of the answer you're looking for before setting out to find it.
Created attachment 383812 [details] emerge --info
wieneke ~ # mdadm -As mdadm: No arrays found in config file or automatically wieneke ~ # echo $? 1
Peter, could you also attach: 1) /proc/mdstat (to show array configuration) 2) /etc/mdadm.conf 3) Output of mdadm -As and subsequent value of $? when run directly The latter would be to prove that it prints no error message, yet returns a non-zero exit status, leading to an eerror line with no actual message (the red asterisk in isolation that you report). This makes for extremely poor user-facing behaviour. I would propose this as a workaround: ebegin "Starting up RAID devices" output=$(mdadm -As 2>&1) eend $(( $? && ${#output} )) "${output}" Granted, that is a kludge. But, if mdadm can't return a meaningful exit status to indicate that it had no work to do, or even be relied upon to print something to stderr when otherwise seeing fit to return a non-zero exit status, I would say that it is the only viable option.
(In reply to Kerin Millar in comment #3) > Peter, could you also attach: > > 1) /proc/mdstat (to show array configuration) # cat /proc/mdstat Personalities : [raid1] md5 : active raid1 sdb5[1] sda5[0] 20971456 blocks [2/2] [UU] md7 : active raid1 sdb7[1] sda7[0] 524287936 blocks [2/2] [UU] md9 : active raid1 sdb9[1] sda9[0] 395845248 blocks [2/2] [UU] Md9 is unused; I created it to hold another system which was never instlled. > 2) /etc/mdadm.conf /etc/mdadm.conf has nothing but comments - it is as installed. > 3) Output of mdadm -As and subsequent value of $? when run directly mdadm -As returns value 1 when run from the command line. It did so during boot as well, when I had inserted eerror "$?" immediately after the call to mdadm -As in /etc/init.d/mdraid.
(In reply to Peter Humphrey from comment #4) > mdadm -As returns value 1 when run from the command line. It did so during > boot as well, when I had inserted eerror "$?" immediately after the call to > mdadm -As in /etc/init.d/mdraid. If you adjust the mdraid script per my proposal above, does that prevent the anomalous behaviour from occurring?
Any update on this, Peter?
(In reply to Kerin Millar from comment #6) > Any update on this, Peter? Oh, sorry Kerin. Substituting your eerror line in /etc/init.d/mdraid causes this error during startup: "* mdadm: No arrays found in config file or automatically". But the arrays are assembled automatically anyway. During shutdown I get "Cannot get exclusive access to /dev/md5 ..." Neither of these happens if I just remove mdraid from the boot phase. I hope I'm not going to be asked to re-create md5 with 1.x metadata and test that. I could do so but it seems excessive.
the same problem appears because mdadm is called from udev rules so well before the /etc/init.d/mdadm service is run
Yes, mdadm initscript is useless on systems with udev novadays since all arrays gets incrementally assembled by udev.
I wouldn't say the mdraid init script is useless when udev has already assembled your arrays. There is one important thing that it does -- its dependency declaration causes the fsck init script to not start until mdraid is done. So if udev hasn't assembled your array yet, then mdraid should do it, and if you have your md device in fstab configured to run fsck, then it should be ready by the time the fsck init script runs. There's a problem with this however. If your mdadm.conf refers to devices via symlinks in /dev/disk/, then those symlinks may not have been setup by the time mdraid runs, which means mdadm will still fail in that case. And the way to fix it is to make mdraid depend on udev-settle by adding rc_need="udev-settle" to /etc/conf.d/mdraid. I really wish the documentation and recommendations mentioned this, or that mdraid depended on udev-settle by default (maybe there's a good reason it doesn't, I don't know). I understand why fsck doesn't depend on udev-settle by default, because generally you can add entries to /etc/fstab that don't directly reference the /dev/disk/... symlinks (because those symlinks are all based on device properties that you can reference directly in fstab). But md devices are a weird case where you can end up with fstab entries that indirectly depend on /dev/disk/... symlinks. Not really sure what the best fix is here. Anyway, this init script really should be fixed to more gracefully handle the array having already been assembled, no matter who or what did that assembly. However there's the slight problem that mdadm prints nothing when it fails due to no arrays requiring assembly, and also nothing when it fails due to not all component devices being available, and the exit statuses for assemble mode are undocumented in the manpage (plus I think it exits with 2 in both cases but I haven't confirmed that). So if there is no way to tell these two cases apart, then the mdraid script shouldn't just ignore the exit status but should instead explicitly check if all possible arrays are assembled and in such case skip calling `mdadm -As`. Though really this seems like an mdadm bug -- if there is simply nothing to do, it seems reasonable for it to exit with status 0 (or at least with a separate and documented status code). (BTW mdraid also shuts down your array when the init script stops, though I'm not sure how important that is.)
Others have noticed this behavior of `mdadm -As` failing when arrays are already assembled: https://bbs.archlinux.org/viewtopic.php?id=90415
Here's an upstream bug report for mdadm: https://github.com/md-raid-utilities/mdadm/issues/144
I presume you've both (Michael, Sam) noticed that this bug was last updated in 2014 before you chimed in. And that it's an mdadm bug, and comment 10 refers a few times to mdraid, which is something entirely different, though that might just be a typo. (And the Arch bug in comment 11 is 2010.) Necromancy, or just upstream never read their bugzilla? (I'm just commenting in passing, while looking to see if a problem I currently have has been reported.)
My mentions of mdraid were not in error. The issue lies with the mdadm call made from the init script /etc/init.d/mdraid, which sets up arrays. All my mentions of "mdraid" are in reference to that init script. (There is another init script, /etc/init.d/mdadm, which runs mdadm in monitor mode in order to send alerts when arrays fail, etc.)
