Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 364235 - sys-fs/udev[extras] and other packages with probers in /lib/udev broken with separate /usr
Summary: sys-fs/udev[extras] and other packages with probers in /lib/udev broken with ...
Status: RESOLVED UPSTREAM
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All Linux
: Normal normal with 1 vote (vote)
Assignee: udev maintainers
URL:
Whiteboard:
Keywords:
: 376723 376725 403493 (view as bug list)
Depends on:
Blocks: 384375
  Show dependency tree
 
Reported: 2011-04-20 12:30 UTC by Xake
Modified: 2013-10-11 22:29 UTC (History)
10 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Xake 2011-04-20 12:30:48 UTC
Applications associated with udev (udev-acl) and udisks (udisks-part-id, udisks-probe-ata-smart) and libmtp and maybe even more needs libraries stores in /usr/ib, namely libglib (and libatasmart, and libmtp...).
I workaround the SMART-thing by installing libatasmart to /lib, but after realizing that stuff broke because of libglib not being in /lib I start to ask myself if this is the correct way, and maybe some of these rules should depend on localmount.
I also wounder why these is not picked up by udev-postmount where it with udevadm should trigger "failed"?

How to reproduce:
1. Install a Gentoo system with separate /usr
2. Install sys-fs/udev with USE="extras"
3. enable debug in /etc/conf.d/udev
4. reboot

Expected result: everything is fine

Actual result: 
# grep "cannot open" /dev/.udev/udev.log
1303305497.776978 [2557] util_run_program: '/lib64/udev/udev-acl' (stderr) '/lib64/udev/udev-acl: error while loading shared libraries: libglib-2.0.so.0: cannot open shared object file: No such file or directory'
1303305497.777242 [2546] util_run_program: '/lib64/udev/udev-acl' (stderr) '/lib64/udev/udev-acl: error while loading shared libraries: libglib-2.0.so.0: cannot open shared object file: No such file or directory'
1

(lots of those errors, at least one per disk/partition)

Also "grep udev-acl /dev/.udev/udev.log" seems to suggest that no instance of udev-acl ever worked, which makes me suspect udev-postmount does not rerun these.


Portage 2.2.0_alpha30 (hardened/linux/amd64, gcc-4.5.2, glibc-2.12.2-r0, 2.6.38-hardened x86_64)
=================================================================
                        System Settings
=================================================================
System uname: Linux-2.6.38-hardened-x86_64-Intel-R-_Core-TM-_i7_CPU_920_@_2.67GHz-with-gentoo-2.0.2
Timestamp of tree: Wed, 20 Apr 2011 09:30:01 +0000
distcc 3.1 x86_64-pc-linux-gnu [disabled]
ccache version 3.1.4 [disabled]
app-shells/bash:     4.2_p8
dev-java/java-config: 2.1.11-r3
dev-lang/python:     2.7.1-r1, 3.1.3-r1
dev-util/ccache:     3.1.4
dev-util/cmake:      2.8.4
sys-apps/baselayout: 2.0.2
sys-apps/openrc:     0.8.2
sys-apps/sandbox:    2.5
sys-devel/autoconf:  2.13, 2.68
sys-devel/automake:  1.9.6-r3, 1.10.3, 1.11.1-r1
sys-devel/binutils:  2.21
sys-devel/gcc:       4.5.2, 4.6.0
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.4-r1
sys-devel/make:      3.82
sys-kernel/linux-headers: 2.6.38
virtual/os-headers:  0
Repositories: gentoo gamerlay-stable x11 xake-overlay Mine
Installed sets: @system
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O2 -pipe -ggdb -mtune=native"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /usr/share/maven-bin-3.0/conf /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-march=native -O2 -pipe -ggdb -mtune=native"
DISTDIR="/var/portage/distfiles"
FEATURES="assume-digests binpkg-logs buildpkg distlocks fixlafiles fixpackages metadata-transfer news parallel-fetch preserve-libs protect-owned sandbox sfperms splitdebug strict test unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox"
FFLAGS=""
GENTOO_MIRRORS="ftp://ftp.sunet.se/pub/os/Linux/distributions/gentoo"
LANG="en_US.utf-8"
LC_ALL="en_US.utf-8"
LDFLAGS="-Wl,--as-needed -Wl,-O1 -Wl,--sort-common -Wl,--hash-style=gnu"
LINGUAS="sv en"
MAKEOPTS="-j16 -l15"
PKGDIR="/var/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/var/portage"
PORTDIR_OVERLAY="/var/overlays/layman/gamerlay /var/overlays/layman/x11 /var/overlays/layman/xake-overlay /var/overlays/mine"
SYNC="rsync://liten.csbnet.se/gentoo-portage"
USE="X a52 aac accessibility acl acpi alsa amd64 amr amrnb amrwb applet archive asyncns autoipd avahi bash-completion bluetooth branding btrfs bzip2 cairo caps ccache cdaudio cdda cdr cleartype cli clutter connection-sharing consolekit coverart cracklib crypt cups cxx dbus device-mapper devicekit devkit dhcpcd digitalradio djvu dri dts dvd dvdr dvi eds enca encode eselect evo exif fat fbcondecor ffmpeg fftw flac fluidsynth fontconfig fuse gdbm gdm gdu geoip gif gimp glib gmp gnome gnome-keyring gphoto2 gpm grammar graphite gsf gsm gstreamer gtk gtk3 gudev hardened hires-icons hpn ical iconv iconvacl icq icu id3tag idn ieee1394 iptc ipv6 jabber jack java6 jingle jpeg jpeg2k justify kate kvm lcms libffi libnotify libsamplerate lm_sensors logrotate lvm lzma mad maps math matroska md mdadm midi mms mmx mmxext mng moonlight mp2 mp3 mpeg mpfr mpi msn mtp mudflap multilib musepack musicbrainz natspec nautilus ncurses network-cron networkmanager nfs nls nntp nptl nptlonly ntfs ntp nut offensive ogg openal opencore-amr opengl openmp openntpd ots pam pango parted pcre pdf perl pidgin playlist png policykit pppd pulseaudio python qt3support quicktime quvi raw readline realtime rrdcgi rtmp samba schroedinger seed sensord session smp sms speex spell sse sse2 sse3 ssl ssse3 startup-notification subversion svg sysfs test tex theora thesaurus threads tiff totem truetype udev unicode upnp urandom usb userlocales v4l2 vaapi vhook videos vim-syntax vorbis webkit wmf x264 xattr xcb xcomposite xinerama xml xmp xmpp xorg xrandr xscreensaver xulrunner xv xvid xvmc zeroconf zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DRACUT_MODULES="lvm mdraid plymouth syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="sv en" NETBEANS_MODULES="cnd profiler dlight harness ide java websvccommon apisupport nb" PHP_TARGETS="php5-3" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="nouveau" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

=================================================================
                        Package Settings
=================================================================

sys-fs/udev-167-r1 was built with the following:
USE="extras (multilib) test (-selinux)"
Comment 1 Xake 2011-04-23 11:12:53 UTC
Ok, this just got a bit more fun.

In short:
udevd does not look for exitcodes for probers, and thinks a failing prober is a prober failing to match the device instead of plainly failing.

Long version:
(this is for mtp-probe)
udev/udev-rules.c:udev_rules_apply_to_event you will find the following code:

if (util_run_program(event->udev, program, envp, result, sizeof(result), NULL, NULL, false) < 0) {
   if (cur->key.op != OP_NOMATCH)
   goto nomatch;
} else {

Look how it handles all exitcodes the same? So a prober usually exits with 0 (rule matched) or 1 (rule not matched), however this dos not check for other exitcodes, so 127 (which is the exitcode for missing shared and alike) marks plainly as "nomatch" instead of "failed" as they should, and because of that these rules does not re-run in udev-postmount.
Comment 2 Vladimir Berezhnoy 2011-07-28 12:38:35 UTC
This also affects rules which have RUN+="/usr/ as they all will also fail at early bootup when /usr is not yet available.

media-sound/alsa-utils-1.0.24.2-r1 (/lib/udev/rules.d/90-alsa-restore.rules)
net-wireless/bluez-4.94-r1 (/lib/udev/rules.d/97-bluetooth.rules)
Comment 3 Samuli Suominen gentoo-dev 2011-07-30 07:24:09 UTC
Running separate /usr without mounting it from initramfs on top of / prior to 'init' has been broken for long time and far as I'm concerned it's an unsupported configuration.

Both uncompressed usb.ids and pci.ids databases are used from /usr/share by udev as well, not just the helpers.

http://www.freedesktop.org/wiki/Software/systemd/separate-usr-is-broken mostly applies to openrc just as same.
Comment 4 Samuli Suominen gentoo-dev 2011-07-30 07:25:55 UTC
*** Bug 376723 has been marked as a duplicate of this bug. ***
Comment 5 Samuli Suominen gentoo-dev 2011-07-30 07:26:00 UTC
*** Bug 376725 has been marked as a duplicate of this bug. ***
Comment 6 Xake 2011-07-30 09:06:53 UTC
Then it is definitely time to rewrite some documentation.

at least the following documents are talking about a separate /usr without mentioning possible problems that may appear:

http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?part=1&chap=4#doc_chap2
http://www.gentoo.org/doc/en/gentoo-x86+raid+lvm2-quickinstall.xml
http://www.gentoo.org/doc/en/lvm2.xml#doc_chap2


So maybe we should build our own version of the systemd page linked here, link it into at least the above mentioned official gentoo documentation, and have the udev init script check whether /usr is empty, and if it is print a warning regarding this issue?
Comment 7 Vladimir Berezhnoy 2011-07-30 09:23:49 UTC
Or may be udev rules should be fixed? Those in question were created as broken from the beginning. I don't see much sense when udev tries to bring up bluetoothd even before localmount. Moreover we have a /etc/init.d/bluetooth which is how it shopuld be controled. Also there is alsa udev rule which tries to restore alsa settings from /var/lib/alsa, but /var is also unavailable before localmount and it will never be as we have now a stupid workaround with /run, so /var is not a mandatory.
Needless to say that /etc/init.d/bluetooth and /etc/init.d/alsasound can be disabled at all for default runlevels and not intended to be ever run, and udev trying to launch them is wrong.
Comment 8 William Hubbs gentoo-dev 2011-07-30 09:34:29 UTC
(In reply to comment #6)
> Then it is definitely time to rewrite some documentation.
> 
> at least the following documents are talking about a separate /usr without
> mentioning possible problems that may appear:
> 
> http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?part=1&chap=4#doc_chap2
> http://www.gentoo.org/doc/en/gentoo-x86+raid+lvm2-quickinstall.xml
> http://www.gentoo.org/doc/en/lvm2.xml#doc_chap2


Let's hold off on changing documentation yet; there is a possible issue in the udev-postmount script I am having Samulee test for us. We are not using a valid  command line option on the udevadm trigger command, and the fix might be as simple as changing that to the correct option.

(In reply to comment #7)
> Or may be udev rules should be fixed? Those in question were created as broken
> from the beginning. I don't see much sense when udev tries to bring up
> bluetoothd even before localmount. Moreover we have a /etc/init.d/bluetooth
> which is how it shopuld be controled. Also there is alsa udev rule which tries
> to restore alsa settings from /var/lib/alsa, but /var is also unavailable
> before localmount and it will never be as we have now a stupid workaround with
> /run, so /var is not a mandatory.
> Needless to say that /etc/init.d/bluetooth and /etc/init.d/alsasound can be
> disabled at all for default runlevels and not intended to be ever run, and udev
> trying to launch them is wrong.

Yes, this is a separate issue that I am also aware of and I will be looking into it as well; that should be a separate bug.
Comment 9 Pacho Ramos gentoo-dev 2011-07-30 10:27:52 UTC
(In reply to comment #7)
> Or may be udev rules should be fixed? Those in question were created as broken
> from the beginning. I don't see much sense when udev tries to bring up
> bluetoothd even before localmount.
> Moreover we have a /etc/init.d/bluetooth
> which is how it shopuld be controled. 

That bluetooth udev rule you think is broken (97-bluetooth.rules) is directly provided by upstream and is how upstream think bluetooth should be handled, we shouldn't even have a bluetooth init.d file, but we still need it due a different bug.
Comment 10 Vladimir Berezhnoy 2011-07-30 11:03:40 UTC
(In reply to comment #8)
> Yes, this is a separate issue that I am also aware of and I will be looking
> into it as well; that should be a separate bug.
Please check 376723 and 376725, as you say it's possible those are not duplicate.


(In reply to comment #9)
> That bluetooth udev rule you think is broken (97-bluetooth.rules) is directly
> provided by upstream and is how upstream think bluetooth should be handled, we
> shouldn't even have a bluetooth init.d file, but we still need it due a
> different bug.
Then there is something wrong with udev if it tries to start bluetooth before other system vital services.Also I don't think upstream uproach is the best way to do it. Bluetoothd is a service and should be handled as a service
Comment 11 Pacho Ramos gentoo-dev 2011-07-30 11:08:41 UTC
That rule is simply launching /usr/sbin/bluetoothd when a bluetooth adapter is connected. As your bluetooth adapter is always there (like mine), it's started at boot time. Maybe this could be solved if that rules needing localmount would be started by udev-postmount, but I don't know how all this udev stuff work :S

And no, it shouldn't be started as a service => the idea is to launch /usr/sbin/bluetoothd only when the adapter is connected, once it's disconnected, it exits. That way it's only running when needed
Comment 12 William Hubbs gentoo-dev 2011-07-30 20:43:05 UTC
All,

based on the information in comment #1, I have emailed upstream to
enquire about marking these events as failed instead of nomatch. That
will make it possible for udev-postmount to pick them up.
Comment 13 William Hubbs gentoo-dev 2011-07-30 23:13:59 UTC
All,

I heard back from upstream on this, and they have given us two options:

We can use the approach suggested  in comment #3, which would require everyone who has a separate /usr to have an initramfs and mount /usr there before / is mounted.

The other method, which I like the sound of better, is to make sure that /usr is mounted before udev is started. That would be easy enough to do by adjusting the sysinit/boot runlevels in openrc and having udev "need localmount".

All comments and thoughts are welcome.
Comment 14 Vladimir Berezhnoy 2011-07-31 03:45:18 UTC
And what if /usr resides on the device which is suppoosed to be created by udev?
Comment 15 William Hubbs gentoo-dev 2011-07-31 17:39:36 UTC
(In reply to comment #14)
> And what if /usr resides on the device which is suppoosed to be created by
> udev?

There is CONFIG_DEVTMPFS in the kernel; I have an email off to upstream right now asking them if that is a preferred setting for systems using udev.

If it is, that will allow the kernel itself to make the device at bootup, then we will mount devtmpfs on /dev, then when udev starts it will take over managing the devices in /dev.
Comment 16 Pacho Ramos gentoo-dev 2011-08-01 11:20:51 UTC
(In reply to comment #12)
> All,
> 
> based on the information in comment #1, I have emailed upstream to
> enquire about marking these events as failed instead of nomatch. That
> will make it possible for udev-postmount to pick them up.

This sounds to me like the best option as it would still allow other people to have a different /usr partition and, at least for bluez case, it should still work as expected :-/
Comment 17 J. Roeleveld 2011-09-15 12:51:00 UTC
(In reply to comment #15)
> (In reply to comment #14)
> > And what if /usr resides on the device which is suppoosed to be created by
> > udev?
> 
> There is CONFIG_DEVTMPFS in the kernel; I have an email off to upstream right
> now asking them if that is a preferred setting for systems using udev.
> 
> If it is, that will allow the kernel itself to make the device at bootup, then
> we will mount devtmpfs on /dev, then when udev starts it will take over
> managing the devices in /dev.

Did you get a reply back from upstream on the CONFIG_DEVTMPFS question?

Out of curiosity, is there a possibility to tell "udev" to only create the /dev-tree and run the commands only after it has been told to?
That way udev can start like it does now, but queues all the scripts.
And then runs the scripts when localmount has run.
Comment 18 Martin von Gagern 2011-09-25 08:22:59 UTC
(In reply to bug #384375 comment #1)
> Separate /usr is not supported without initramfs that mounts it before init.

Argh! Thanks for the info, but I'm definitely not going to add initramfs to my boot sequence. Both grub and the kernel are perfectly able to access the root partition by themselves, which would be the single reason I'll accept for having an initramfs. From there on, the system should boot from live disk partitions, and if it doesn't, I'll consider it broken and attempt to fix it, not work around it using initramfs.

udev running custom commands that early feels very much like a mistake. Who added the "before checkfs fsck" line to the udev init script, and for what reason? So udev can load the modules required to access other partitions?

Note that the current requirement violates FHS:
http://www.pathname.com/fhs/pub/fhs-2.3.html#THEROOTFILESYSTEM
"To boot a system, enough must be present on the root partition to mount other filesystems. This includes utilities, configuration, boot loader information, and other essential start-up data. /usr, /opt, and /var are designed such that they may be located on other partitions or filesystems."

OK, so what possible solutions do we have?
1. The initramfs approach you and upstream currently suggest.
2. Fix rules, as comment #7 and my own bug #384375 comment #0 suggests.
   One could decide the rule to use based on the STARTUP variable, and use
   another init script to run any command that requires access to /usr, /var
   or whatever.
3. Write a new /etc/init.d/earlymount script, which depends on lvm,
   runs before udev, and will mount /usr and /var early on.
   Probably should add an earlyfsck as well.
4. Configure rc_coldplug=NO, and see what stuff breaks due to unloaded modules.
5. Add a new setting, e.g. rc_coldplug=LATE, to control when to do coldplugging.
Comment 19 Samuli Suominen gentoo-dev 2011-09-25 10:20:24 UTC
(In reply to comment #18)
> udev running custom commands that early feels very much like a mistake.

what about bluetooth input devices (keyboard, mouse) ? bluez is in /usr (bug 376725)
having keyboard early as possible seems necessary.
Comment 20 Martin von Gagern 2011-09-25 11:57:00 UTC
(In reply to comment #19)
> (In reply to comment #18)
> > udev running custom commands that early feels very much like a mistake.
> 
> what about bluetooth input devices (keyboard, mouse) ? bluez is in /usr (bug
> 376725)
> having keyboard early as possible seems necessary.

What is the earliest possible use for keyboard interaction? Usually fsck uses -p even in the case of an error, thus it won't ask for user intervention. So early keyboard seems reasonable, but there is little point in having a keyboard before someone is willing to listen to it.

I currently perceive the scenario like this: "coldplugging is important, because we want the bluetooth keyboard available. So let's put it before lvm, fsck and localmount. But /usr is even more important, so let's put it before udev again, into initramfs, including lvm and so on." One cannot have both in front at the same time.

So either there is a reason to always have /usr first. Then we either want localmount started before udev, or option 3 from above. Or there is a reason to always have udev coldplugging first. Then we want option 2 to make it work. Or, and this I consider most likely, there are scenarios for each. Then we either want to allow users to configure their order (option 5) or to split mounting into two scripts (option 3 again).

Configurable order of startup scripts should take care of most scenarios. Users with a single partition and bluetooth keyboard would start udev before localmount, while those with kernelk-space-only hardware but separate /usr partition would have localmount first. A single switch somewhere could add the appropriate dependency.

Having two scripts would probably be more flexible. Users could mount /usr (do we generally want /var as well, or do we fix the alsa rules nevertheless?) read-only in earlymount. Then they could start udev, and get their bluetooth keyboard. Then they could do fsck even in interactive mode, answering questions using their keyboard. After fsck succeeded, /usr would be re-mounted read/write, and other file systems could be mounted using coldplugged devices. The read-only mounting would avoid a second early fsck script, but there no longer would be a single correct time to start lvm, as it might be needed for early-mounted /usr, but also might require scanning udev-coldplugged devices.

Current dependencies on my system:
udev:       before fsck
fsck:       use modules
localmount: need fsck, after lvm modules
lvm:        before fsck, after modules

So apparently there is nothing to prevent lvm being started before udev even now. So nothing would be lost by putting lvm before earlymount, specifying the deps of earlymount like this:

earlymount: after lvm modules, before fsck udev

Would you be willing to pursue this earlymount approach? Do you need a patch? Should this be installed by udef or rather by openrc as localmount is?
Comment 21 Samuli Suominen gentoo-dev 2012-03-20 23:01:40 UTC
This is solved by the requirement for initramfs for separate /usr. Closing.
Comment 22 Xake 2012-03-21 00:12:33 UTC
Heh, yeah I saw the somewhat misleading news post.

Thanks you all for pushing this solution/workaround/whatever you want to call it. And hope you do not get too discouraged by all crazy comments from people not really understanding the problem.
Comment 23 Samuli Suominen gentoo-dev 2012-03-21 00:17:54 UTC
*** Bug 403493 has been marked as a duplicate of this bug. ***
Comment 24 Samuli Suominen gentoo-dev 2012-03-21 00:19:22 UTC
Bug 407959 is for updating the documentation in case anyone is looking.
Comment 25 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2012-03-22 21:41:06 UTC
I've got another fun breakage case that came up for a user's system recently.

lvm links against libudev.

With libudev in /usr on a seperate partition, if /usr is an LVM LV, there is _no_ way to get it mounted from / unless you have a static LVM, or mounted it during the initramfs environment.
Comment 26 Raymond Jennings 2013-10-11 22:29:55 UTC
This to me sounds more like libudev belonging in /lib instead of /usr/lib

If it's essential for bootstrapping or mounting, IMHO it doesn't belong in /usr to begin with