Summary: | sys-fs/udev[extras] and other packages with probers in /lib/udev broken with separate /usr | ||
---|---|---|---|
Product: | Gentoo Linux | Reporter: | Xake <kanelxake> |
Component: | New packages | Assignee: | udev maintainers <udev-bugs> |
Status: | RESOLVED UPSTREAM | ||
Severity: | normal | CC: | bug, eric.chatellier, joost_gentoo_bugzilla, Martin.vGagern, moonlapse81, netbox253, non7top, pacho, rdalek1967, vyperin |
Priority: | Normal | ||
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Bug Depends on: | |||
Bug Blocks: | 384375 |
Description
Xake
2011-04-20 12:30:48 UTC
Ok, this just got a bit more fun. In short: udevd does not look for exitcodes for probers, and thinks a failing prober is a prober failing to match the device instead of plainly failing. Long version: (this is for mtp-probe) udev/udev-rules.c:udev_rules_apply_to_event you will find the following code: if (util_run_program(event->udev, program, envp, result, sizeof(result), NULL, NULL, false) < 0) { if (cur->key.op != OP_NOMATCH) goto nomatch; } else { Look how it handles all exitcodes the same? So a prober usually exits with 0 (rule matched) or 1 (rule not matched), however this dos not check for other exitcodes, so 127 (which is the exitcode for missing shared and alike) marks plainly as "nomatch" instead of "failed" as they should, and because of that these rules does not re-run in udev-postmount. This also affects rules which have RUN+="/usr/ as they all will also fail at early bootup when /usr is not yet available. media-sound/alsa-utils-1.0.24.2-r1 (/lib/udev/rules.d/90-alsa-restore.rules) net-wireless/bluez-4.94-r1 (/lib/udev/rules.d/97-bluetooth.rules) Running separate /usr without mounting it from initramfs on top of / prior to 'init' has been broken for long time and far as I'm concerned it's an unsupported configuration. Both uncompressed usb.ids and pci.ids databases are used from /usr/share by udev as well, not just the helpers. http://www.freedesktop.org/wiki/Software/systemd/separate-usr-is-broken mostly applies to openrc just as same. *** Bug 376723 has been marked as a duplicate of this bug. *** *** Bug 376725 has been marked as a duplicate of this bug. *** Then it is definitely time to rewrite some documentation. at least the following documents are talking about a separate /usr without mentioning possible problems that may appear: http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?part=1&chap=4#doc_chap2 http://www.gentoo.org/doc/en/gentoo-x86+raid+lvm2-quickinstall.xml http://www.gentoo.org/doc/en/lvm2.xml#doc_chap2 So maybe we should build our own version of the systemd page linked here, link it into at least the above mentioned official gentoo documentation, and have the udev init script check whether /usr is empty, and if it is print a warning regarding this issue? Or may be udev rules should be fixed? Those in question were created as broken from the beginning. I don't see much sense when udev tries to bring up bluetoothd even before localmount. Moreover we have a /etc/init.d/bluetooth which is how it shopuld be controled. Also there is alsa udev rule which tries to restore alsa settings from /var/lib/alsa, but /var is also unavailable before localmount and it will never be as we have now a stupid workaround with /run, so /var is not a mandatory. Needless to say that /etc/init.d/bluetooth and /etc/init.d/alsasound can be disabled at all for default runlevels and not intended to be ever run, and udev trying to launch them is wrong. (In reply to comment #6) > Then it is definitely time to rewrite some documentation. > > at least the following documents are talking about a separate /usr without > mentioning possible problems that may appear: > > http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?part=1&chap=4#doc_chap2 > http://www.gentoo.org/doc/en/gentoo-x86+raid+lvm2-quickinstall.xml > http://www.gentoo.org/doc/en/lvm2.xml#doc_chap2 Let's hold off on changing documentation yet; there is a possible issue in the udev-postmount script I am having Samulee test for us. We are not using a valid command line option on the udevadm trigger command, and the fix might be as simple as changing that to the correct option. (In reply to comment #7) > Or may be udev rules should be fixed? Those in question were created as broken > from the beginning. I don't see much sense when udev tries to bring up > bluetoothd even before localmount. Moreover we have a /etc/init.d/bluetooth > which is how it shopuld be controled. Also there is alsa udev rule which tries > to restore alsa settings from /var/lib/alsa, but /var is also unavailable > before localmount and it will never be as we have now a stupid workaround with > /run, so /var is not a mandatory. > Needless to say that /etc/init.d/bluetooth and /etc/init.d/alsasound can be > disabled at all for default runlevels and not intended to be ever run, and udev > trying to launch them is wrong. Yes, this is a separate issue that I am also aware of and I will be looking into it as well; that should be a separate bug. (In reply to comment #7) > Or may be udev rules should be fixed? Those in question were created as broken > from the beginning. I don't see much sense when udev tries to bring up > bluetoothd even before localmount. > Moreover we have a /etc/init.d/bluetooth > which is how it shopuld be controled. That bluetooth udev rule you think is broken (97-bluetooth.rules) is directly provided by upstream and is how upstream think bluetooth should be handled, we shouldn't even have a bluetooth init.d file, but we still need it due a different bug. (In reply to comment #8) > Yes, this is a separate issue that I am also aware of and I will be looking > into it as well; that should be a separate bug. Please check 376723 and 376725, as you say it's possible those are not duplicate. (In reply to comment #9) > That bluetooth udev rule you think is broken (97-bluetooth.rules) is directly > provided by upstream and is how upstream think bluetooth should be handled, we > shouldn't even have a bluetooth init.d file, but we still need it due a > different bug. Then there is something wrong with udev if it tries to start bluetooth before other system vital services.Also I don't think upstream uproach is the best way to do it. Bluetoothd is a service and should be handled as a service That rule is simply launching /usr/sbin/bluetoothd when a bluetooth adapter is connected. As your bluetooth adapter is always there (like mine), it's started at boot time. Maybe this could be solved if that rules needing localmount would be started by udev-postmount, but I don't know how all this udev stuff work :S And no, it shouldn't be started as a service => the idea is to launch /usr/sbin/bluetoothd only when the adapter is connected, once it's disconnected, it exits. That way it's only running when needed All, based on the information in comment #1, I have emailed upstream to enquire about marking these events as failed instead of nomatch. That will make it possible for udev-postmount to pick them up. All, I heard back from upstream on this, and they have given us two options: We can use the approach suggested in comment #3, which would require everyone who has a separate /usr to have an initramfs and mount /usr there before / is mounted. The other method, which I like the sound of better, is to make sure that /usr is mounted before udev is started. That would be easy enough to do by adjusting the sysinit/boot runlevels in openrc and having udev "need localmount". All comments and thoughts are welcome. And what if /usr resides on the device which is suppoosed to be created by udev? (In reply to comment #14) > And what if /usr resides on the device which is suppoosed to be created by > udev? There is CONFIG_DEVTMPFS in the kernel; I have an email off to upstream right now asking them if that is a preferred setting for systems using udev. If it is, that will allow the kernel itself to make the device at bootup, then we will mount devtmpfs on /dev, then when udev starts it will take over managing the devices in /dev. (In reply to comment #12) > All, > > based on the information in comment #1, I have emailed upstream to > enquire about marking these events as failed instead of nomatch. That > will make it possible for udev-postmount to pick them up. This sounds to me like the best option as it would still allow other people to have a different /usr partition and, at least for bluez case, it should still work as expected :-/ (In reply to comment #15) > (In reply to comment #14) > > And what if /usr resides on the device which is suppoosed to be created by > > udev? > > There is CONFIG_DEVTMPFS in the kernel; I have an email off to upstream right > now asking them if that is a preferred setting for systems using udev. > > If it is, that will allow the kernel itself to make the device at bootup, then > we will mount devtmpfs on /dev, then when udev starts it will take over > managing the devices in /dev. Did you get a reply back from upstream on the CONFIG_DEVTMPFS question? Out of curiosity, is there a possibility to tell "udev" to only create the /dev-tree and run the commands only after it has been told to? That way udev can start like it does now, but queues all the scripts. And then runs the scripts when localmount has run. (In reply to bug #384375 comment #1) > Separate /usr is not supported without initramfs that mounts it before init. Argh! Thanks for the info, but I'm definitely not going to add initramfs to my boot sequence. Both grub and the kernel are perfectly able to access the root partition by themselves, which would be the single reason I'll accept for having an initramfs. From there on, the system should boot from live disk partitions, and if it doesn't, I'll consider it broken and attempt to fix it, not work around it using initramfs. udev running custom commands that early feels very much like a mistake. Who added the "before checkfs fsck" line to the udev init script, and for what reason? So udev can load the modules required to access other partitions? Note that the current requirement violates FHS: http://www.pathname.com/fhs/pub/fhs-2.3.html#THEROOTFILESYSTEM "To boot a system, enough must be present on the root partition to mount other filesystems. This includes utilities, configuration, boot loader information, and other essential start-up data. /usr, /opt, and /var are designed such that they may be located on other partitions or filesystems." OK, so what possible solutions do we have? 1. The initramfs approach you and upstream currently suggest. 2. Fix rules, as comment #7 and my own bug #384375 comment #0 suggests. One could decide the rule to use based on the STARTUP variable, and use another init script to run any command that requires access to /usr, /var or whatever. 3. Write a new /etc/init.d/earlymount script, which depends on lvm, runs before udev, and will mount /usr and /var early on. Probably should add an earlyfsck as well. 4. Configure rc_coldplug=NO, and see what stuff breaks due to unloaded modules. 5. Add a new setting, e.g. rc_coldplug=LATE, to control when to do coldplugging. (In reply to comment #18) > udev running custom commands that early feels very much like a mistake. what about bluetooth input devices (keyboard, mouse) ? bluez is in /usr (bug 376725) having keyboard early as possible seems necessary. (In reply to comment #19) > (In reply to comment #18) > > udev running custom commands that early feels very much like a mistake. > > what about bluetooth input devices (keyboard, mouse) ? bluez is in /usr (bug > 376725) > having keyboard early as possible seems necessary. What is the earliest possible use for keyboard interaction? Usually fsck uses -p even in the case of an error, thus it won't ask for user intervention. So early keyboard seems reasonable, but there is little point in having a keyboard before someone is willing to listen to it. I currently perceive the scenario like this: "coldplugging is important, because we want the bluetooth keyboard available. So let's put it before lvm, fsck and localmount. But /usr is even more important, so let's put it before udev again, into initramfs, including lvm and so on." One cannot have both in front at the same time. So either there is a reason to always have /usr first. Then we either want localmount started before udev, or option 3 from above. Or there is a reason to always have udev coldplugging first. Then we want option 2 to make it work. Or, and this I consider most likely, there are scenarios for each. Then we either want to allow users to configure their order (option 5) or to split mounting into two scripts (option 3 again). Configurable order of startup scripts should take care of most scenarios. Users with a single partition and bluetooth keyboard would start udev before localmount, while those with kernelk-space-only hardware but separate /usr partition would have localmount first. A single switch somewhere could add the appropriate dependency. Having two scripts would probably be more flexible. Users could mount /usr (do we generally want /var as well, or do we fix the alsa rules nevertheless?) read-only in earlymount. Then they could start udev, and get their bluetooth keyboard. Then they could do fsck even in interactive mode, answering questions using their keyboard. After fsck succeeded, /usr would be re-mounted read/write, and other file systems could be mounted using coldplugged devices. The read-only mounting would avoid a second early fsck script, but there no longer would be a single correct time to start lvm, as it might be needed for early-mounted /usr, but also might require scanning udev-coldplugged devices. Current dependencies on my system: udev: before fsck fsck: use modules localmount: need fsck, after lvm modules lvm: before fsck, after modules So apparently there is nothing to prevent lvm being started before udev even now. So nothing would be lost by putting lvm before earlymount, specifying the deps of earlymount like this: earlymount: after lvm modules, before fsck udev Would you be willing to pursue this earlymount approach? Do you need a patch? Should this be installed by udef or rather by openrc as localmount is? This is solved by the requirement for initramfs for separate /usr. Closing. Heh, yeah I saw the somewhat misleading news post. Thanks you all for pushing this solution/workaround/whatever you want to call it. And hope you do not get too discouraged by all crazy comments from people not really understanding the problem. *** Bug 403493 has been marked as a duplicate of this bug. *** Bug 407959 is for updating the documentation in case anyone is looking. I've got another fun breakage case that came up for a user's system recently. lvm links against libudev. With libudev in /usr on a seperate partition, if /usr is an LVM LV, there is _no_ way to get it mounted from / unless you have a static LVM, or mounted it during the initramfs environment. This to me sounds more like libudev belonging in /lib instead of /usr/lib If it's essential for bootstrapping or mounting, IMHO it doesn't belong in /usr to begin with |