The tools referred to in the 40-multipath.rules file and (uncommented) 50-udev.rules (devmap_name, kpartx) don't work until multipath(d) userspace tool is run. multipathd runs after localmount, so any evms volumes on a dm multipath device cannot be loaded from fstab. the evms_activate call while udev-start is finalizing only activates volumes from non-multipath devices such as sd*. If you are seeing messages in evms about "incorrect kernel mappings" immediately following boot, this may be the cause. Reproducible: Always Steps to Reproduce: 1. Install and configure multipath-tools and evms 2. If you remove multipathd from runlevel default, it's less confusing (opt). 3. Create evms volumes on your multipath devices, add them to fstab 4. Add "sd*" to "exclude" directive in sysfs_devices section of /etc/evms.conf (assuming your multipath constituent devices are sd's) 5. Reboot. Actual Results: Even though evms_activate has been called at least twice (udev-start.sh & evms-start.sh), no volume devices appear in /dev/evms. If you removed multipathd from runlevel default, "multipath -l" shows no path info. No dm dirs exist in /sys/block. The volumes did not mount. Expected Results: If the multipath dm nodes were available to evms, the volumes would appear in /dev/evms without evms accessing sd*. Firing off an /sbin/multipath before the evms_activate in udev-start.sh is my suggested fix, like so: --- /lib/rcscripts/addons/udev-start.sh 2007-03-08 11:46:00.000000000 -0800 +++ /lib/rcscripts/addons/udev-start.sh 2007-03-08 11:22:05.000000000 -0800 @@ -139,6 +139,7 @@ # Create nodes that udev can't ebegin "Finalizing udev configuration" [[ -x /sbin/dmsetup ]] && /sbin/dmsetup mknodes &>/dev/null + [[ -x /sbin/multipath ]] && /sbin/multipath -v0 &>/dev/null [[ -x /sbin/lvm ]] && \ /sbin/lvm vgscan -P --mknodes --ignorelockingfailure &>/dev/null # Running evms_activate on a LiveCD causes lots of headaches I tried using dmtab to create the multipath devices prior to evms initialization, but that didn't help (the nodes were created, they just didn't work). Another symptom of this problem under a normal configuration is that when you run evmsn immediately following a reboot, you are confronted with messages telling you something to the effect of "/dev/evms/volume has an incorrect kernel mapping, it will be remapped on the next commit." evms, multipath-tools, udev versions: sys-fs/evms-2.5.5-r5 USE="ncurses -debug -gtk -nls" sys-fs/multipath-tools-0.4.7-r1 sys-fs/udev-104-r11 USE="(-selinux)" emerge --info output: Portage 2.1.2.2 (default-linux/amd64/2006.1, gcc-4.1.1, glibc-2.5-r0, 2.6.16.37-lsit x86_64) ================================================================= System uname: 2.6.16.37-lsit x86_64 Dual-Core AMD Opteron(tm) Processor 2218 Gentoo Base System release 1.12.9 Timestamp of tree: Thu, 08 Mar 2007 16:50:01 +0000 dev-lang/python: 2.4.3-r4 dev-python/pycrypto: 2.0.1-r5 sys-apps/sandbox: 1.2.17 sys-devel/autoconf: 2.13, 2.61 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10 sys-devel/binutils: 2.16.1-r3 sys-devel/gcc-config: 1.3.14 sys-devel/libtool: 1.5.22 virtual/os-headers: 2.6.17-r2 ACCEPT_KEYWORDS="amd64" AUTOCLEAN="yes" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=opteron -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /var/bind" CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/revdep-rebuild /etc/terminfo" CXXFLAGS="-march=opteron -O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig distlocks metadata-transfer sandbox sfperms userpriv usersandbox" GENTOO_MIRRORS="ftp://ftp.ucsb.edu/pub/mirrors/linux/gentoo" MAKEOPTS="-j5" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --filter=H_**/files/digest-*" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.ucsb.edu/gentoo-portage" USE="amd64 berkdb bitmap-fonts cli cracklib dri fortran gd iconv imap isdnlog libg++ logrotate mailwrapper midi mmx2 mysql ncurses nptl nptlonly pam pcre perl pic postfix ppds pppd python readline reflection sasl session snmp spl ssl syslog-ng truetype-fonts type1-fonts unicode xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="apm ark ati chips cirrus cyrix dummy fbdev glint i128 i810 mga neomagic nv rendition s3 s3virge savage siliconmotion sis sisusb tdfx tga trident tseng v4l vesa vga via vmware voodoo" Unset: CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS, PORTAGE_RSYNC_EXTRA_OPTS
I think udev-start only calls other programs to create nodes in /dev for devices that are already started, and udev cannot create the nodes itself. But /sbin/multipath sets up the devices itself. I think perhaps multipath-tools should add its own rcscript addon multipath-start.sh for that purpose.
Btw. I think we should remove all calls to volume managers from udev-start.sh Only problem: checkroot that calls "fsck /". Possible solution: look what device fsck will access and check its existance. If it is missing, look for major/minor of root-directory - and either mknod it or link to appropriate /dev/mapper/xyz.
Please add an addon /lib/rcscript/addons/multipath-start.sh to contain /sbin/multipath call. evms_activate will be removed from udev-start.sh soon.
Created attachment 113913 [details] a proposed multipath-start.sh script This is modelled after the evms-start.sh script ... we might want a better way to keep multipath from running on systems where the user has only installed it to get the devmap_name utility. It should also be noted that 'multipath' needs to be added to RC_VOLUME_ORDER in /etc/conf.d/rc (before evms) ... perhaps this could be done via a multipath-tools USE flag? that would also solve the case where people only want the package for devmap_name.
It is ok to add multipath unconditionally, as you do not need to emerge multipath-tools to get devmap_name as # /sbin/dmsetup -j %M -m %m --noopencount --noheadings -c -o name info does the same (%M/%m being major/minor number) and is used by default udev-rules now. And perhaps I can convince maintainer to install devmap_name with sys-fs/device-mapper.
For convenience: If this script gets added, one also should add an init-script, like the one from Bug #175983, to make it work with baselayout2.
Ok, I'm going to fix this in the morning when I wake up. It's now critically needed for some Gentoo infra boxes with SAN storage, as the SAN isn't coming up on boot now that we have multipath.
This is implemented in multipath-tools-0.4.8 now.
