Trival patch to baselayout-1.8.6.13-r1 to create device nodes for those crazy people having an udev-only setup and root on lvm2. Please apply. Reproducible: Always Steps to Reproduce: 1. 2. 3.
Created attachment 28728 [details, diff] Patch to /sbin/rc
Created attachment 29651 [details, diff] Patch to /sbin/rc for baselayout-1.8.11
How nice, this conflicts with my patch in bug 48629 :( If that patch goes in (mine) then I have another one which will move all the udev stuff out of the /sbin/rc script entirely, so baselayout will not have to be changed for future udev things. Let's wait for that to happen to get this patch in, ok?
They wont conflict. Simply merge them together. Add: ls /sys/block/dm-*/dev 1>/dev/null 2>&1 ACTIVEDMDEVS=$? # mknodes if conditions are right if [ $ACTIVEDMDEVS -eq 0 -a -x /sbin/vgscan ]; then /sbin/vgscan --mknodes --ignorelockingfailure fi somewhere in populate_udev() Thats the same as creating the other problematic nodes per hand.
A month later and LVM still no worky with udev? I just built a 2.6.5-r1 box with udev and LVM, but I can't get it to boot even after adding that snippet to my /sbin/rc. Getting "...already mounted or is busy" errors. I have /usr, /var/, and a couple other directories mounted on logical volumes managed by LVM. So I guess I'm rebuilding my kernel with devfs in it now?
My apologies, I haven't been able to do anything significant on baselayout in the past couple of weeks, so everything is lagging at the moment. I'm hoping to get some time this week to handle this.
Root on lvm2 works fine for me. Make a intrd for lvm2, (I put my initrd here for your pleasure (This one should work with recent 2.6 kernels, you must have initrd, lvm, ramfs, yourrootfs support compiled in, disable "automaticly mount dev on boot" !) http://www.fkurth.de/~felix/initrd-lvm2-2.6.5.gz My grub entry looks: title Gentoo-2.6.6 root (hd0,0) kernel /kernel-2.6.6 root=/dev/ram0 lvm2root=/dev/vg/root vga=794 gentoo=nodevfs initrd /initrd-lvm2-2.6.5.gz My fstab entry /dev/vg/root / reiserfs noatime,notail 0 1
If I understand this solution correctly the dev nodes have to be created two times, once during initrd processing and once during execution of /sbin/rc. And the same must be done not only for lvm but for evms too. Wouldn't it be a more general (and more elegant) solution to just start udev in the initrd and make the ramdisk with /dev persistent. That way the lvm or evms devices have only to be created once during the initrd processing and the devices would be still there later when /sbin/rc and the initrd scripts get executed. Unfortunately I have no patch to realize this. But AFAIK Thomas Woerner from Red Hat is working on this for Fedora. Maybe if he has finished that it wouldn't hurt to look at his scripts. (Just something to keep in mind)
14:49 <@gregkh> agriffis: don't add it to baselayout. I'm still waiting for a real fix for this before adding it to the udev stuff. Reassigning to gregkh
I found a very easy solution in the Gentoo forums (it's somewhere burried in that super large udev thread). Because device-mapper itself is sysfs aware, why not just let udev create the missing devices? No need to change the initrd scripts. To do this put the following rule in /etc/udev/rules.d/: KERNEL="dm-[0-9]*", PROGRAM="/sbin/evms_activate", NAME="" This is for evms. For lvm it should be KERNEL="dm-[0-9]*", PROGRAM="/sbin/vgscan --mknodes --ignorelockingfailure", NAME="" Because of a dm rule in 50-udev.rules the above rule must be put into a rules file before 50-udev.rules, i.e. 20-evms.rules or 20-lvm.rules. I use the above evms rule with my Gentoo system and the default initrd from the evms project. Works like a charm. I also have the lvm rule on my Debian system together with an initrd from Debian's mkinitrd. Works too. Hmm ... maybe a nice solution would be if the evms and lvm2 ebuilds would just install these udev rules automatically under /etc/udev/rules.d/ ? An other possible solution would be to call a small script instead of evms_activate or vgscan directly which checks if evms or lvm are installed and then calls the appropriate command.
The idea is good, but it dont work for me. When using udev without the device tarball, the first thing populate_udev() in /sbin/rc does is a call to /sbin/udevstart. On this time, /dev is absolutly empty. When the rules KERNEL="dm-[0-9]*", PROGRAM="/sbin/vgscan --mknodes --ignorelockingfailure", NAME="" are called before 50-udev.rules, it wont work because there are no device nodes at all to seach for vg's on. Calling it later (removing the dm-* rule from 50-udev.rules first) wont work at this place too for mee. I simply called a script within the rule KERNEL="dm-[0-9]*", PROGRAM="/sbin/lvm_startup", NAME="%k", SYMLINK="%c" with the following contents: #!/bin/sh /sbin/vgscan -v --mknodes --ignorelockingfailure 1>&2 /sbin/vgchange -a y 1>&2 ls /dev/ 1>&2 The output of vgscan is: No volumegroups found. The ls output is simply dm-0 dm-1, nothing more ! (after calling it twice) So i understand that /sbin/vgscan is unable to find any nodes ! But what goes wrong here ? Calling /sbin/vgscan -v --mknodes --ignorelockingfailure 1>&2 directly AFTER udevstart (in /sbin/rc) works fine... So this looks like a bug in udevstart ?
Do you have "sysfs_scan = 1" in your /etc/lvm/lvm.conf? With that lvm recurs through the sysfs directories to find devices (it does this before the regex and device type filters). My devices section in lvm.conf looks like devices { dir = "/dev" scan = [ "/dev" ] filter = [ "r|/dev/cdrom|" ] cache = "/etc/lvm/.cache" write_cache_state = 1 sysfs_scan = 1 md_component_detection = 0 } Also do you have a recent lvm2 version? I think the sysfs support wasn't available in version 2.00.08. But it should work 2.00.15.
no, i upgraded lvm to 2.00.15, added sysfs_scan = 1 it dont work. One of the errors is: No device/mapper found. But my /etc/udev/rules.d/11-local.rules is KERNEL="device-mapper", NAME="mapper/control" KERNEL="dm-[0-9]*", PROGRAM="/sbin/lvm_startup", NAME="%k", SYMLINK="%c" At execution time of this rule, lvm_startup outuputs me only those dm-1 dm-0 devices so no more. Is udevstart putting every rule in a sandbox and moves the nodes later? I didnt understand the work of this programm. Is there a way to see in wich order udev will execute rules ? Is there a way to make udev apply changed rules without restarting my computer ? Is there a way to get more debug output ? How can i get the output when syslog isnt started, because my root fs is read-only at this point ? Lots of questions, and no doc available -:(
I tracked down the problem. udevstart creates device nodes in the wrong order !!! It parses sysfs recursively, looking for entrys, if an entry was found the node gets created if a rule is avaialable. But the creation of lvm nodes depends on the existance of /dev/mapper/control and /dev/hda* on other systems even more. So what we need is an ORDER/DENDENCY rule for udevstart/udev to solve this.
So the quickest hack is to call /sbin/udevstart twice in /sbin/rc ugly.
ooh, nice catch. I never thought about order dependancies for udevstart. Care to make a list of rules that it should follow so that I can fix it up?
The first idea i have is to create the nodes in major/minor order. Starting with 0. But i dont know wether this works for every cases, it schould work for lvm/evms and every dev should know this order. what about a DEPEND="hda*" rule ? Instead of having a big 50-udev.rules file, i suggest to make every app (alsa, lvm, evms, ...) install a rule/perm file. Those files schould have a lower number, so in the meantime, we can have fallback-rules in 50-udev.rules ?
Does lvm/evms really need hd? nodes at all? It should be enough for evms/lvm to scan /sys for block devices. I just made a small test and configured lvm to not scan /dev for devices. It still found all volumes. Thus I think they depend only on /dev/mapper/control, at least if they are configured to scan sysfs too.
>Does lvm/evms really need hd? nodes at all? Yes, they are needed. Having sysfs_scan=1 and /dev/mapper/control avialable is not enough. >I just made a small test and configured lvm to not scan /dev for devices. No, it doesnt work for me. The description in /etc/lvm/lvm.conf for sysfs_scan is: # If sysfs is mounted (2.6 kernels) restrict device scanning to # the block devices it believes are valid. # 1 enables; 0 disables. In wich way does this mean that vgscan is scanning sysfs itself for nodes ? But you may right, strace vgscan 2>&1 | grep /sys open("/sys/block", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 3 open("/sys/block/sr2", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 4 open("/sys/block/sr2/queue", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 5 open("/sys/block/sr2/queue/iosched", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 6 and so on. But it seems to depend on more then only /dev/mapper/control Did you try it on an empty /dev ? Or do you have nodes in there ? Are you sure vgscan does not use the hda node ?
You're right, I still had nodes under /dev left. With an empty /dev LVM doesn't find any volumes. Looking at the ltrace it apparently scans /dev after /sysfs even when it's configured to not look under /dev for devices).
Ok, so we are left up to make udevstart create nodes in the right order..., or doing it by hand...
Ah, right now udevstart is scanning the /sys/class tree before scanning the /sys/block tree. If I reverse these, lvm might have a chance. Let me go make a new udev release to change this and let's see if this fixes the issue for you.
A short test with a changed udevstart (reversed order of class/block) doesnt help that much. The /dev/mapper/control node gets created earlier. dm-* and hda* are both block-devs, and dm-* is executed earlier, this causes the error.
I digged a little bit through the code of udevstart/udev. Is there a reason not to use libsysfs in udevstart ? There is a function get_major_minor(...) in udevstart.c Why is this usefull function not part of libsysfs ? It could be very usefull when sorting udev_ecec requests... What do you think about the major/minor order ? Or is this nonsense ?
No, we can't trigger on major number (think of lots of scsi devices or other block devices above the dm- nodes. Anyway, so you are saying that the dm* block devices should be scanned last? If so, I'll go change udevstart to do that. Any other ordering issues you can think of?
I think, put the dm-* and on the end will fix this. But creating in major order wont be that bad, since the dm-* have a major of 254 wich will be behind any other node.
Not true. We can have majors and minors bigger than 256 now with the 2.6 kernel :) I'll go work on udevstart. Should get to it tomorrow.
Ok, I've reworked udevstart. The dm* nodes should be created last now. Please test the new 027 release and let me know if it solves the issue for you or not.
This works now (as of version 029). Thanks. Maybe we should think about a dev policy for packages, like lvm, evms,nvidia-kernel, etc, to install a default udev rule for there devices ? Fill bugs against them ?
Yes. please file bugs against those packages, that is what is needed. I'm marking this one done.
