| Summary: | sys-apps/systemd and sys-fs/udev with sys-fs/multipath-tools-0.5.0-r1 - systemd-udevd: Unable to flock(/dev/sda), skipping event handling: Resource temporarily unavailable | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Alexander E. Patrakov <patrakov> |
| Component: | [OLD] Core system | Assignee: | Gentoo's Team for Core System packages <base-system> |
| Status: | RESOLVED UPSTREAM | ||
| Severity: | normal | CC: | systemd, udev-bugs |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| URL: | http://lists.freedesktop.org/archives/systemd-devel/2014-October/024572.html | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
I assume this is also a problem for sys-fs/udev. I have tried to ping upstream about this, unsuccessfully. Fedora applies some 100 patches over inactive multipath-tools upstream package: http://pkgs.fedoraproject.org/cgit/device-mapper-multipath.git/tree/ So, I'm not exectly suprised our package is screwed in multiple ways. Fixed in tree: https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=e78d65c9e9ffa54a6bece600c8d09881361c4889 I have looked through the commit log of the upstream project, and the code that calls flock(), and found nothing that would explain the "fixed" status. flock() is still called on block devices, and I found nothing that would tell udev to retry. Could you please explain what are you basing your "this is fixed" claim on? P.S. I have not tested the new version either, so it may indeed be fixed. Upstream has included many of those fedora patches, so I presumed it was imported there. I think the bug more relies with dracut in this case. If it is starting multipathd, it should be using the multipath devices (/dev/mapper/...) and not the locked child devices (/dev/sda) Closing this as RESOLVED/UPSTREAM since it was never a Gentoo-specific bug in the first place. |
(this is an upstream bug, the point here is to warn gentoo users) multipathd, if it exists in the filesystem, gets included into initramfs by dracut. Unfortunately, its disk-flocking logic interferes with udev, sometimes leading to the boot process never completing. Reproducible: Sometimes Steps to Reproduce: 1. Install Gentoo on a typical desktop machine with a single SATA SSD, onto /dev/sda2. 2. Install multipath-tools, even though they are not needed on this machine. 3. Install dracut, generate an initramfs using it. 4. Alter the root=... argument in the boot loader configuration, so that it says /dev/disk/by-id/...-part2. 5. Reboot the machine several times. Actual Results: Sometimes the boot completes successfully, but in rare cases it doesn't, and dracut drops into an emergency shell. In that shell, it is obvious that /dev/sda2 exists, but /dev/disk/by-id/...-part2 symlink doesn't. Expected Results: The boot should succeed always. I know that multipathd is guilty because of the following arguments. 1. When I add rd.udev.log-priority=debug, I get this in the log during failed boots: [ 3.026515] localhost systemd-udevd[319]: Unable to flock(/dev/sda), skipping event handling: Resource temporarily unavailable [ 3.027224] localhost systemd-udevd[333]: Unable to flock(/dev/sda), skipping event handling: Resource temporarily unavailable Event handling in this case means calling various helper programs to figure out the id and actually creating symlinks based on the results. 2. To figure out which process interferes with systemd-udevd event handling, I wrote a small preloadable shared library that syslogs all attempts to call flock() on any block device. It turned out that the only other process who does this in my initramfs (besides systemd-udevd itself) is multipathd. Below is the source code of this library, please compile as follows: gcc -shared -fPIC -D_GNU_SOURCE -ldl -o liblockcomplain.so liblockcomplain.c and then create etc/ld.so.preload file in your initramfs with "liblockcomplain.so" line in it, and of course include the library into the initramfs. #include <sys/file.h> #include <sys/types.h> #include <sys/stat.h> #include <unistd.h> #include <stdio.h> #include <err.h> #include <syslog.h> #include <dlfcn.h> static int (*orig_flock)(int fd, int operation) = NULL; int flock(int fd, int operation) { struct stat st; int fs = fstat(fd, &st); if (fs) { goto finish; } if (!(st.st_mode & S_IFBLK)) { goto finish; } syslog(LOG_ERR, "Attempt to flock /dev/block/%d:%d would possibly interfere with udev", major(st.st_rdev), minor(st.st_rdev)); finish: return orig_flock(fd, operation); } void __attribute__ ((constructor)) flock_police_init(void) { orig_flock = dlsym(RTLD_NEXT, "flock"); } And here is emerge --info so that nobody complains: Portage 2.2.14 (python 2.7.8-final-0, default/linux/amd64/13.0/desktop/kde/systemd, gcc-4.8.3, glibc-2.19-r1, 3.17.1-gentoo-r1 x86_64) ================================================================= System uname: Linux-3.17.1-gentoo-r1-x86_64-Intel-R-_Core-TM-_i7-4770S_CPU_@_3.10GHz-with-gentoo-2.2 KiB Mem: 16361436 total, 13395596 free KiB Swap: 17237312 total, 17237312 free Timestamp of tree: Sun, 26 Oct 2014 00:45:01 +0000 ld GNU ld (GNU Binutils) 2.24 app-shells/bash: 4.3_p30 dev-java/java-config: 2.2.0 dev-lang/perl: 5.20.1-r2 dev-lang/python: 2.7.8, 3.4.2 dev-util/cmake: 3.0.2 dev-util/pkgconfig: 0.28-r2 sys-apps/baselayout: 2.2 sys-apps/openrc: 0.13.1 sys-apps/sandbox: 2.6-r1 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.11.6, 1.14.1 sys-devel/binutils: 2.24-r3 sys-devel/gcc: 4.8.3 sys-devel/gcc-config: 1.8 sys-devel/libtool: 2.4.2-r1 sys-devel/make: 4.1-r1 sys-kernel/linux-headers: 3.17 (virtual/os-headers) sys-libs/glibc: 2.19-r1 Repositories: gentoo gentoo-overlay-tox local ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/themes/oxygen-gtk/gtk-2.0 /usr/share/themes/oxygen-gtk/gtk-3.0" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-O2 -pipe" DISTDIR="/usr/portage/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs buildpkg config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync webrsync-gpg xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://mirror.yandex.ru/gentoo-distfiles" LANG="ru_RU.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j3" PKGDIR="/usr/portage/packages/kde" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/usr/portage/packages" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/var/lib/layman/tox-overlay /usr/local/portage" SYNC="rsync://mirror.yandex.ru/gentoo-portage" USE="X a52 aac acl acpi alsa amd64 amr avahi bash-completion berkdb bluetooth bluray bzip2 cairo cdda cddb cdr cjk cli cracklib crypt cryptsetup cups cxx dbus declarative djvu dri dts dvd dvdr emboss encode exif ffmpeg firefox flac fontconfig fortran gdbm gif glamor gphoto2 gtk gtk3 iconv icu idn introspection ipv6 jack jbig jpeg jpeg2k kde kipi lcms ldap libnotify lzma mad matroska mmx mmxext mng modules mp3 mp4 mpeg multilib musepack ncurses networkmanager nls nptl ogg opengl openmp opus pam pango pcre pdf phonon plasma png policykit postscript ppds pulseaudio qt3support qt4 quvi readline samba scanner sdl session speex spell sse sse2 sse3 ssl ssse3 startup-notification svg systemd tcpd theora tiff truetype udev udisks unicode upnp upower usb v4l v4l2 vaapi vdpau vim-syntax vorbis vp8 wavpack webkit webm webp wxwidgets x264 x265 xattr xcb xcomposite xface xinerama xml xscreensaver xv xvid zlib" ABI_X86="32 64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DRACUT_MODULES="lvm mdraid crypt plymouth systemd" DVB_CARDS="tda10046lifeview" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_4" RUBY_TARGETS="ruby21" USERLAND="GNU" VIDEO_CARDS="intel nouveau radeon" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" USE_PYTHON="2.7 3.4" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS