* stable system (amd64) * USE="cman clvmd" * lvm.conf: locking_type = 2 * DLM up and running properly * non-clustered VG on local disk (sda) * clustered VG on local DRBD device (sdb) Example command: lvcreate -n testlv -L 1G vg_kvm (It does not matter if you try a clustered or nonclustered VG here, result is the same) Current result: /dev/vg_kvm/testlv: not found: device not cleared Aborting. Failed to wipe start of new LV. Workaround is to add "-Z n" to prevent lvcreate from wiping the beginning of the LV. To me this seems to be some sort of timing problem because the missing device file will be there right after creating the LV when using "-Z n". I already tried to set/unset the variuous udev relevant settings in lvm.conf and I also tried to use lvm2 with USE=-udev but the only thing I could find so far was setting locking_type back to "1" which is not suitable for clustered LVM of course. Here is part of the output of the same lvcreate command with -vvvv: [....] #metadata/pv_map.c:55 Allowing allocation on /dev/drbd0 start PE 0 length 238408 #metadata/lv_manip.c:2084 Trying allocation using contiguous policy. #metadata/lv_manip.c:1695 Still need 256 total extents: #metadata/lv_manip.c:1698 1 (1 data/0 parity) parallel areas of 256 extents each #metadata/lv_manip.c:1700 0 mirror logs of 0 extents each #metadata/lv_manip.c:1388 Considering allocation area 0 as /dev/drbd0 start PE 0 length 256 leaving 238152. #metadata/lv_manip.c:1166 Allocating parallel area 0 on /dev/drbd0 start PE 0 length 256. #metadata/pv_manip.c:339 /dev/drbd0 0: 0 256: testlv(0:0) #metadata/pv_manip.c:339 /dev/drbd0 1: 256 238152: NULL(0:0) #cluster_locking.c:504 Locking VG P_vg_kvm PW (VG|CACHE) (0x104) #mm/memlock.c:394 Unlock: Memlock counters: locked:0 critical:0 daemon:0 suspended:0 #device/dev-io.c:577 Closed /dev/drbd0 #device/dev-io.c:524 Opened /dev/drbd0 RW O_DIRECT #device/dev-io.c:137 /dev/drbd0: block size is 4096 bytes #format_text/format-text.c:632 Writing vg_kvm metadata to /dev/drbd0 at 108032 len 1039 #format_text/format-text.c:737 Pre-Committing vg_kvm metadata (82) to /dev/drbd0 header at 4096 #format_text/format-text.c:737 Committing vg_kvm metadata (82) to /dev/drbd0 header at 4096 #cluster_locking.c:504 Locking VG P_vg_kvm PW (VG|HOLD|CACHE) (0x124) #cluster_locking.c:430 Requesting backup of VG metadata for vg_kvm #format_text/archiver.c:411 Creating volume group backup "/etc/lvm/backup/vg_kvm" (seqno 82). #format_text/format-text.c:934 Writing vg_kvm metadata to /etc/lvm/backup/.lvm_yankee_12385_56476654 #format_text/format-text.c:978 Committing vg_kvm metadata (82) #format_text/format-text.c:979 Renaming /etc/lvm/backup/vg_kvm.tmp to /etc/lvm/backup/vg_kvm #cluster_locking.c:504 Locking LV VVDu8CZwKr0SV26ZGCfXqiD0J5SVfUrU9uzUvP5LdF6gAU932E2BDTFP9VswtdUs CR (LV|NONBLOCK|CLUSTER) (0x99) #mm/memlock.c:394 Unlock: Memlock counters: locked:0 critical:0 daemon:0 suspended:0 #activate/fs.c:489 Syncing device names #metadata/lv_manip.c:4071 Clearing start of logical volume "testlv" #device/dev-cache.c:600 /dev/vg_kvm/testlv: stat failed: No such file or directory #metadata/lv_manip.c:4074 /dev/vg_kvm/testlv: not found: device not cleared #metadata/lv_manip.c:4538 Aborting. Failed to wipe start of new LV. [....] emerge --info Portage 2.1.11.55 (hardened/linux/amd64, gcc-4.6.3, glibc-2.15-r3, 3.8.4-hardened-r1 x86_64) ================================================================= System uname: Linux-3.8.4-hardened-r1-x86_64-Intel-R-_Xeon-R-_CPU_E5520_@_2.27GHz-with-gentoo-2.1 KiB Mem: 49526644 total, 48853748 free KiB Swap: 8388604 total, 8388604 free Timestamp of tree: Thu, 28 Mar 2013 08:45:01 +0000 ld GNU ld (GNU Binutils) 2.22 app-shells/bash: 4.2_p37 dev-lang/python: 2.7.3-r3, 3.2.3-r2 dev-util/cmake: 2.8.9 dev-util/pkgconfig: 0.28 sys-apps/baselayout: 2.1-r1 sys-apps/openrc: 0.11.8 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.11.6 sys-devel/binutils: 2.22-r1 sys-devel/gcc: 4.6.3 sys-devel/gcc-config: 1.7.3 sys-devel/libtool: 2.4-r1 sys-devel/make: 3.82-r4 sys-kernel/linux-headers: 3.7 (virtual/os-headers) sys-libs/glibc: 2.15-r3 Repositories: gentoo local-root ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-march=native -O2 -pipe" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--keep-going --with-bdeps=y" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://de-mirror.org/gentoo/ http://mirror.opteamax.de/gentoo/ ftp://sunsite.informatik.rwth-aachen.de/pub/Linux/gentoo" LANG="C" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j8" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/root/local-overlay" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="7zip acl amd64 bash-completion berkdb bzip2 caps cli cracklib crypt cxx device-mapper dri gdbm gpm hardened iconv idn ipv6 justify mmx modules mudflap multilib ncurses nls nptl openmp openssl pacemaker pam pax_kernel pcre readline session sse sse2 ssl tcpd threads unicode urandom vim-syntax xattr zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="de" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-3" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_2" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby18 ruby19" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga nouveau nv r128 radeon savage sis tdfx trident vesa via vmware dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
Hi, I'm also experiencing this problem. I only set USE="readline thin udev" and use the default locking_type=1 in the lvm.conf. Also there are more error reports in the funtoo wiki http://forums.funtoo.org/viewtopic.php?id=1206 Also here the '-Z n' argument did help to work around the error. I tried to further nail down the problem, but without much success. I don't think it has anything to do with cman or cluster lvm. What I found is the following message when running the `lvcreate -L15G -n images vg_pool` command in the very verbose mode: [...] Loading vg_pool-images table (254:8) Adding target to (254:8): 0 31457280 linear 8:18 41945088 dm table (254:8) OF [16384] (*1) dm reload (254:8) NF [16384] (*1) Table size changed from 0 to 31457280 for vg_pool-images (254:8). Resuming vg_pool-images (254:8) Udev cookie 0xd4d9f36 (semid 884736) created Udev cookie 0xd4d9f36 (semid 884736) incremented to 1 Udev cookie 0xd4d9f36 (semid 884736) incremented to 2 Udev cookie 0xd4d9f36 (semid 884736) assigned to RESUME task(5) with flags DISABLE_LIBRARY_FALLBACK (0x20) dm resume (254:8) NF [16384] (*1) Uevent not generated! Calling udev_complete internally to avoid process lock-up. [...] Clearing start of logical volume "images" /dev/vg_pool/images: stat failed: No such file or directory /dev/vg_pool/images: not found: device not cleared [...] Device mapper activates the new logical volume device, but udev doesn't get a message to create the device node. Further I saw strange messages like the following. However, I couldn't find out, if this has anything to do with this problem: [...] #mm/memlock.c:174 mlock area unreadable 7f623f52d000-7f623f72c000 ---p 00018000 fe:00 44790 /lib64/libpthread-2.16.so : Skipping. #mm/memlock.c:174 mlock area unreadable 7f623f783000-7f623f982000 ---p 00051000 fe:00 9207 /lib64/libncurses.so.5.9 : Skipping. #mm/memlock.c:174 mlock area unreadable 7f623f98f000-7f623fb8e000 ---p 00007000 fe:00 44780 /lib64/librt-2.16.so : Skipping. #mm/memlock.c:174 mlock area unreadable 7f623fd32000-7f623ff32000 ---p 001a2000 fe:00 44789 /lib64/libc-2.16.so : Skipping. #mm/memlock.c:174 mlock area unreadable 7f623ff79000-7f6240179000 ---p 0003d000 fe:00 8983 /lib64/libreadline.so.6.2 : Skipping. #mm/memlock.c:174 mlock area unreadable 7f62401bd000-7f62403bc000 ---p 0003a000 fe:00 10333 /lib64/libdevmapper.so.1.02 : Skipping. #mm/memlock.c:174 mlock area unreadable 7f62403c7000-7f62405c6000 ---p 00006000 fe:00 44145 /lib64/libdevmapper-event.so.1.02 : Skipping. #mm/memlock.c:174 mlock area unreadable 7f62405cb000-7f62407ca000 ---p 00003000 fe:00 44740 /lib64/libdl-2.16.so : Skipping. #mm/memlock.c:174 mlock area unreadable 7f62407dd000-7f62409dd000 ---p 00011000 fe:04 749373 /usr/lib64/libudev.so.1.4.0 : Skipping. [...] Here my setup: * lvm2: 2.02.103 * systemd: 208-r2 # emerge --info Portage 2.2.7 (default/linux/amd64/13.0, gcc-4.7.3, glibc-2.16.0, 3.12.3-gentoo x86_64) ================================================================= System uname: Linux-3.12.3-gentoo-x86_64-AMD_Phenom-tm-_II_X6_1075T_Processor-with-gentoo-2.2 KiB Mem: 16440012 total, 1274632 free KiB Swap: 4194300 total, 4194300 free Timestamp of tree: Wed, 11 Dec 2013 02:30:01 +0000 ld GNU ld (GNU Binutils) 2.23.1 ccache version 3.1.9 [disabled] app-shells/bash: 4.2_p45 dev-java/java-config: 2.2.0 dev-lang/python: 2.6.8-r3, 2.7.5-r3, 3.2.5-r3, 3.3.2-r2 dev-util/ccache: 3.1.9 dev-util/cmake: 2.8.11.2 dev-util/pkgconfig: 0.28 sys-apps/baselayout: 2.2 sys-apps/openrc: 0.12.4 sys-apps/sandbox: 2.6-r1 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.6, 1.12.6, 1.13.4 sys-devel/binutils: 2.23.1 sys-devel/gcc: 4.7.3-r1 sys-devel/gcc-config: 1.7.3 sys-devel/libtool: 2.4.2 sys-devel/make: 3.82-r4 sys-kernel/linux-headers: 3.12 (virtual/os-headers) sys-libs/glibc: 2.16.0 Repositories: gentoo x11 sunrise virtualization steam-overlay ubelix docker linuxmonk ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /var/lib/hsqldb /var/spool/munin-async/.ssh" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.5/ext-active/ /etc/php/c gi-php5.5/ext-active/ /etc/php/cli-php5.5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=native -O2 -pipe" DISTDIR="/mnt/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fakeroot fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://mirror.switch.ch/ftp/mirror/gentoo/ http://de-mirror.org/distro/gentoo/" LANG="en_US" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j7" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/var/lib/layman/x11 /var/lib/layman/sunrise /var/lib/layman/virtualization /var/lib/layman/steam /var/lib/layman/ubelix /var/lib/layman/docker /var/lib/layman/linuxmonk" SYNC="rsync://centos6/gentoo-portage" USE="3dnow 3dnowext X X509 a52 aac aalib acl acpi aio aisleriot alsa amd64 amr apng archive aspell async audio audit avahi bash-completion berkdb bluetooth bonjour bookmarks branding brasero btrfs bzip2 cairo caps ccache cdaudio cdda cddb cdio cdr cdrkit cjk cli clutter colord console corefonts cracklib crypt css cups curl cxx daap dbus dc1394 dconf deskbar device-mapper dga dhcp dia discouraged disk-partition diskio divx dmi dmraid dri drm dv dvd dvdr dvi dvipdfm eds emerald enca encode evo exchange exif expat fam fasttrack fat fbcon ffmpeg flac flickr fontconfig fortran ftp fts fts3 fuse gaim gallium gbm gcj gconf gd gdbm gdu gedit geoip gif gimp gimpprint git glib glitz glsa glut gmp gnome gnome-keyring gnome-print gnuplot gnutella gnutls gphoto2 graphics graphviz grilo gs gsf gstreamer gtk gtk3 gtkhtml gudev guile h323 hddtemp hfs hwdb ical iconv icq icu id3tag idn ieee1394 imagemagick imap inherit-graph inkjar introspection ios ipod iptc ipv6 irc ithreads jabber jpeg kate kerberos keymap keyring kpathsea kqemu kvm lame lastfm latex lcms libass libnotify libsamplerate libsexy live llvm-gcc lm_sensors logrotate lvm lzma lzo magic map matroska mdadm mdnsresponder-compat midi mjpeg mmap mmx mmxext modules mono mp2 mp3 mpeg msn mtp mudflap multilib musepack musicbrainz nautilus ncurses network networking networkmanager nfs nfsidmap nls nntp nolvm1 nolvmstatic nptl nsplugin nss ntfs ntfsprogs nvidia ocamlopt ofx ogg opengl openmp openvg osmesa pam parted pcntl pcre pdf pdfimport playlist plotutils png podcast policykit postscript ppds prediction print pstricks pulseaudio python3 qemu qt3support quicktime rar rdesktop readline resolvconf rtc rtmp rtsp sasl schroedinger science scrobbler sdl secure-delete sendto session shout sip smi smp socialweb soundtouch soup sourceview spell sse sse2 ssh ssl stream subtitles subversion svg symlink systemd taglib tagwriting tcpd telepathy tex4ht tga theora threads threadsonly tiff tk toolkit-scroll-bars totem tracker truetype udev unicode unique upnp urandom usb usbredir v4l2 vala valgrind vcd vdisk vdpau video vim-syntax virtualbox vlm vnc vorbis vpx vte wav webgl webkit wma wmf x264 xa xanim xattr xcb xcomposite xetex xfs xinerama xml xmldoclet xmp xulrunner xv xvid xvmc youtube zeitgeist zeroconf zlib" ABI_X86="64" ALSA_CARDS="hda-intel" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DRACUT_MODULES="caps lvm syslog systemd" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-3" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_2 python3_3" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby19" SANE_BACKENDS="plustek" USERLAND="GNU" VIDEO_CARDS="nouveau vesa" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
I finally found the problem in my setup: # zcat /proc/config.gz | grep UEVENT CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug" CONFIG_DM_UEVENT=y After setting the CONFIG_UEVENT_HELPER_PATH="" lvcreate works again as expected. Cheers, Reto
Many thanks for the update. I will try that ASAP
This does not fix my initial problem. :-/
Hit this bug myself, my workaround: changing locking_type to 3 and building lvm2 with USE="static"
The most recently stabilized lvm2 made some changes for cman and clvmd. Can you please retest?
