Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 684882 - >=sys-fs/lvm2-2.02.184 - systemd[1]: Timed out waiting for device /dev/mapper/gentoo-swap.
Summary: >=sys-fs/lvm2-2.02.184 - systemd[1]: Timed out waiting for device /dev/mapper...
Status: UNCONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo's Team for Core System packages
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-05-01 14:01 UTC by Mariusz Ceier
Modified: 2020-12-27 23:04 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mariusz Ceier 2019-05-01 14:01:27 UTC
>=sys-fs/lvm2-2.02.184 causes timeout during boot when mounting/activating swap, making the boot 1m 30s longer

it doesn't happen with =sys-fs/lvm2-2.02.183


Reproducible: Always

Steps to Reproduce:
1. emerge '>=sys-fs/lvm2-2.02.184'
2. reboot


Actual Results:  
maj 01 15:21:25 systemd[1]: dev-mapper-gentoo\x2dswap.device: Job dev-mapper-gentoo\x2dswap.device/start timed out.
maj 01 15:21:25 systemd[1]: Timed out waiting for device /dev/mapper/gentoo-swap.

boot delayed by 1m 30s because of timeout

Expected Results:  
swap should get activated and there shouldn't be any timeout

I'm using systemd and at first I thought that was systemd issue, but downgrading systemd didn't help. Then I tried downgrading lvm2, since some time ago I had problem with it - it stopped working with genkernel-next and I had to move to genkernel. Downgrading lvm2 fixes the issue for me.

/etc/fstab:

/dev/mapper/gentoo-home	/	ext4	auto,rw 0 1
/dev/mapper/gentoo-swap none            swap    sw              0       0


emerge --info sys-fs/lvm2 :

Portage 2.3.66 (python 3.6.8-final-0, default/linux/amd64/17.0/systemd, gcc-8.3.0, glibc-2.29-r2, 5.1.0-rc7- x86_64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-5.1.0-rc7-x86_64-Intel-R-_Core-TM-_i5-6600_CPU_@_3.30GHz-with-gentoo-2.6
KiB Mem:    32893148 total,  30453520 free
KiB Swap:          0 total,         0 free
Timestamp of repository gentoo: Wed, 01 May 2019 06:30:01 +0000
Head commit of repository gentoo: f5f89add9c4f095797e5030e4bfd5a312dae3f39
sh bash 5.0_p7
ld GNU ld (Gentoo 2.31.1 p7) 2.31.1
app-shells/bash:          5.0_p7::gentoo
dev-java/java-config:     2.2.0-r4::gentoo
dev-lang/perl:            5.28.2::gentoo
dev-lang/python:          2.7.16::gentoo, 3.6.8::gentoo
dev-util/cmake:           3.14.3::gentoo
dev-util/pkgconfig:       0.29.2::gentoo
sys-apps/baselayout:      2.6-r1::gentoo
sys-apps/sandbox:         2.17::gentoo
sys-devel/autoconf:       2.13-r1::gentoo, 2.69-r4::gentoo
sys-devel/automake:       1.16.1-r1::gentoo
sys-devel/binutils:       2.31.1-r6::gentoo, 2.32-r1::gentoo
sys-devel/gcc:            8.2.0-r6::gentoo, 8.3.0-r1::gentoo
sys-devel/gcc-config:     2.0::gentoo
sys-devel/libtool:        2.4.6-r5::gentoo
sys-devel/make:           4.2.1-r4::gentoo
sys-kernel/linux-headers: 5.0-r1::gentoo (virtual/os-headers)
sys-libs/glibc:           2.29-r2::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://192.168.0.9/gentoo-portage
    priority: -1000
    sync-rsync-verify-metamanifest: yes
    sync-rsync-verify-max-age: 24
    sync-rsync-verify-jobs: 1
    sync-rsync-extra-opts: 

haskell
    location: /var/lib/layman/haskell
    masters: gentoo
    priority: 50

rust
    location: /var/lib/layman/rust
    masters: gentoo
    priority: 50

steam-overlay
    location: /var/lib/layman/steam-overlay
    masters: gentoo
    priority: 50

x11
    location: /var/lib/layman/x11
    masters: gentoo
    priority: 50

local
    location: /usr/local/portage
    masters: gentoo
    priority: 80

ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O3 -pipe -g -ggdb -Wvla"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/easy-rsa /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php7.2/ext-active/ /etc/php/apache2-php7.3/ext-active/ /etc/php/cgi-php7.2/ext-active/ /etc/php/cgi-php7.3/ext-active/ /etc/php/cli-php7.2/ext-active/ /etc/php/cli-php7.3/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-march=native -O3 -pipe -g -ggdb -Wvla"
DISTDIR="/usr/portage/distfiles"
ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fail-clean fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned sandbox sfperms split-log splitdebug strict strict-keepdir unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="pl_PL.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="pl en"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="X acl adns alsa amd64 audio avx avx2 berkdb bluetooth bzip2 caps clang cli clutter crypt cups cxx dbus dri dri3 drm egl emacs ffmpeg fortran fpm fuse gadu gbm gdbm git glade gles glib gnutls gold gstreamer hardened iconv icu idn inotify ipv6 jack jpeg libdrm libtirpc llvm mmx mono mtp multilib multitarget ncurses net45 nls nptl opengl openmp openrc pam pcre perl pgf pkcs pkcs11 pkcs12 png policykit pulseaudio python readline samba sasl seccomp socks socks4 socks5 sound sqlite sqlite3 sse sse2 sse3 sse4 sse4_1 sse4_2 ssl ssse3 startup-notification syslog system-sqlite systemd tcpd theora threads truetype udev unicode v4l2 vaapi valgrind vdpau video vim vpx wayland wifi x265 xattr xcb xft zlib" ABI_X86="64 32" ALSA_CARDS="ca0132" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 fma3 mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" CURL_SSL="gnutls" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" L10N="pl en" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LLVM_TARGETS="AMDGPU BPF NVPTX AArch64 ARM RISCV WebAssembly" NETBEANS_MODULES="apisupport cnd groovy gsf harness ide identity j2ee java mobility nb php profiler soa visualweb webcommon websvccommon xml" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-2" POSTGRES_TARGETS="postgres9_5 postgres10" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" RUBY_TARGETS="ruby25" USERLAND="GNU" VIDEO_CARDS="amdgpu radeonsi" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

=================================================================
                        Package Settings
=================================================================

sys-fs/lvm2-2.02.183::gentoo was built with the following:
USE="readline systemd thin udev -device-mapper-only -lvm2create_initrd -sanlock (-selinux) (-static) (-static-libs)" ABI_X86="(64)"
Comment 1 Mariusz Ceier 2019-05-05 20:03:20 UTC
`systemctl mask lvm2-monitor.service` fixes it for me, thanks to helpful viewer from random YT livestream.
Comment 2 Mariusz Ceier 2019-09-11 14:02:54 UTC
That workaround doesn't work anymore... ffs, how hard is to do "swapon -a"
Comment 3 vax456 2020-06-29 15:31:39 UTC
I'm experiencing the same problem, except it started with >=sys-fs/lvm2-2.02.187.

I noticed lvmetad is disabled on 2.02.187 and later.  I've also confirmed when you're on 2.02.184, and you set use_lvmetad = 0, you get the same problem as 2.02.187.  

So, disabling lvmetad seems to be a part of the problem, though I don't think this is the root of the problem.
Comment 4 Mariusz Ceier 2020-06-29 15:54:13 UTC
(In reply to vax456 from comment #3)
> I'm experiencing the same problem, except it started with
> >=sys-fs/lvm2-2.02.187.
> 
> I noticed lvmetad is disabled on 2.02.187 and later.  I've also confirmed
> when you're on 2.02.184, and you set use_lvmetad = 0, you get the same
> problem as 2.02.187.  
> 
> So, disabling lvmetad seems to be a part of the problem, though I don't
> think this is the root of the problem.

In /etc/lvm/lvm.conf I have use_lvmetad=1 and I don't remember ever changing it.

The bug is still present with sys-fs/lvm2-2.02.187-r2 and as a "workaround" I just removed swap from /etc/stab, since I have 32GB of ram.

My partitions look like this (part of lsblk output):
sdb               8:16   0 119,2G  0 disk
└─sdb4            8:20   0   118G  0 part 
  ├─gentoo-swap 253:0    0  15,8G  0 lvm  
  ├─gentoo-home 253:1    0   1,9T  0 lvm  /
  └─gentoo-root 253:2    0    50G  0 lvm  

I have no idea what's the root of the problem (except if only systemd were executing "swapon -a" instead of reimplementing everything itself there wouldn't be a problem).
Comment 5 Mattias Winther 2020-07-19 21:19:32 UTC
I am having the same problem, the machine adds my root which is on the same (LUKS) LVM vg as the swap file, but after mounting everything else, it waits for 1m30s for the swap to activate before timing out and continuing. The 'funny' thing is that there is no trace of that attempted mount in the genkernel-boot.log or in dmesg, so I don't know what's going on. Am using 2.02.187-r2. As soon as I am done booting, I can just do a swapon /dev/vg_sys/swap and everything works just fine.
Comment 6 Adrian 2020-07-24 23:42:42 UTC
Same problem for me. Would really like to know WTF is causing this, and how to avoid it (besides not using swap or not using fstab to enable swap automatically)
Comment 7 Darragh Bailey 2020-08-08 11:54:14 UTC
As I'm using a slightly more split up layout this is causing me more trouble:

Aug 08 12:01:48 speedie systemd[1]: dev-ssdforce-portage.device: Job dev-ssdforce-portage.device/start timed out.
Aug 08 12:01:48 speedie systemd[1]: Timed out waiting for device /dev/ssdforce/portage.
Aug 08 12:01:48 speedie systemd[1]: Dependency failed for File System Check on /dev/ssdforce/portage.
Aug 08 12:01:48 speedie systemd[1]: Dependency failed for /var/tmp/portage.
Aug 08 12:01:48 speedie systemd[1]: Dependency failed for Local File Systems.
Aug 08 12:01:48 speedie systemd[1]: local-fs.target: Job local-fs.target/start failed with result 'dependency'.
Aug 08 12:01:48 speedie systemd[1]: var-tmp-portage.mount: Job var-tmp-portage.mount/start failed with result 'dependency'.
Aug 08 12:01:48 speedie systemd[1]: systemd-fsck@dev-ssdforce-portage.service: Job systemd-fsck@dev-ssdforce-portage.service/start failed w>
Aug 08 12:01:48 speedie systemd[1]: dev-ssdforce-portage.device: Job dev-ssdforce-portage.device/start failed with result 'timeout'.
Aug 08 12:01:48 speedie systemd[1]: dev-hddgen-media.device: Job dev-hddgen-media.device/start timed out.
Aug 08 12:01:48 speedie systemd[1]: Timed out waiting for device /dev/hddgen/media.
Aug 08 12:01:48 speedie systemd[1]: Dependency failed for /srv/media.
Aug 08 12:01:48 speedie systemd[1]: srv-media.mount: Job srv-media.mount/start failed with result 'dependency'.
Aug 08 12:01:48 speedie systemd[1]: Dependency failed for File System Check on /dev/hddgen/media.
Aug 08 12:01:48 speedie systemd[1]: systemd-fsck@dev-hddgen-media.service: Job systemd-fsck@dev-hddgen-media.service/start failed with resu>
Aug 08 12:01:48 speedie systemd[1]: dev-hddgen-media.device: Job dev-hddgen-media.device/start failed with result 'timeout'.
Aug 08 12:01:48 speedie systemd[1]: dev-hddgen-var.device: Job dev-hddgen-var.device/start timed out.
Aug 08 12:01:48 speedie systemd[1]: Timed out waiting for device /dev/hddgen/var.
Aug 08 12:01:48 speedie systemd[1]: Dependency failed for File System Check on /dev/hddgen/var.
Aug 08 12:01:48 speedie systemd[1]: Dependency failed for /var.
Aug 08 12:01:48 speedie systemd[1]: Dependency failed for Update UTMP about System Runlevel Changes.
Aug 08 12:01:48 speedie systemd[1]: systemd-update-utmp-runlevel.service: Job systemd-update-utmp-runlevel.service/start failed with result>
Aug 08 12:01:48 speedie systemd[1]: Dependency failed for Network Time Synchronization.
Aug 08 12:01:48 speedie systemd[1]: systemd-timesyncd.service: Job systemd-timesyncd.service/start failed with result 'dependency'.
Aug 08 12:01:48 speedie systemd[1]: Dependency failed for Flush Journal to Persistent Storage.
Aug 08 12:01:48 speedie systemd[1]: systemd-journal-flush.service: Job systemd-journal-flush.service/start failed with result 'dependency'.
Aug 08 12:01:48 speedie systemd[1]: Dependency failed for /var/tmp.
Aug 08 12:01:48 speedie systemd[1]: var-tmp.mount: Job var-tmp.mount/start failed with result 'dependency'.
Aug 08 12:01:48 speedie systemd[1]: Dependency failed for Update UTMP about System Boot/Shutdown.
Aug 08 12:01:48 speedie systemd[1]: systemd-update-utmp.service: Job systemd-update-utmp.service/start failed with result 'dependency'.
Aug 08 12:01:48 speedie systemd[1]: Dependency failed for Load/Save Random Seed.
Aug 08 12:01:48 speedie systemd[1]: systemd-random-seed.service: Job systemd-random-seed.service/start failed with result 'dependency'.
Aug 08 12:01:48 speedie systemd[1]: var.mount: Job var.mount/start failed with result 'dependency'.
Aug 08 12:01:48 speedie systemd[1]: systemd-fsck@dev-hddgen-var.service: Job systemd-fsck@dev-hddgen-var.service/start failed with result '>
Aug 08 12:01:48 speedie systemd[1]: dev-hddgen-var.device: Job dev-hddgen-var.device/start failed with result 'timeout'.
Aug 08 12:01:48 speedie systemd[1]: dev-hddgen-vartmp.device: Job dev-hddgen-vartmp.device/start timed out.
Aug 08 12:01:48 speedie systemd[1]: Timed out waiting for device /dev/hddgen/vartmp.
Aug 08 12:01:48 speedie systemd[1]: Dependency failed for File System Check on /dev/hddgen/vartmp.
Aug 08 12:01:48 speedie systemd[1]: systemd-fsck@dev-hddgen-vartmp.service: Job systemd-fsck@dev-hddgen-vartmp.service/start failed with re>
Aug 08 12:01:48 speedie systemd[1]: dev-hddgen-vartmp.device: Job dev-hddgen-vartmp.device/start failed with result 'timeout'.
Aug 08 12:01:48 speedie systemd[1]: dev-hddgen-usrsrc.device: Job dev-hddgen-usrsrc.device/start timed out.
Aug 08 12:01:48 speedie systemd[1]: Timed out waiting for device /dev/hddgen/usrsrc.
Aug 08 12:01:48 speedie systemd[1]: Dependency failed for File System Check on /dev/hddgen/usrsrc.
Aug 08 12:01:48 speedie systemd[1]: Dependency failed for /usr/src.
Aug 08 12:01:48 speedie systemd[1]: usr-src.mount: Job usr-src.mount/start failed with result 'dependency'.
Aug 08 12:01:48 speedie systemd[1]: systemd-fsck@dev-hddgen-usrsrc.service: Job systemd-fsck@dev-hddgen-usrsrc.service/start failed with re>
Aug 08 12:01:48 speedie systemd[1]: dev-hddgen-usrsrc.device: Job dev-hddgen-usrsrc.device/start failed with result 'timeout'.
Aug 08 12:01:48 speedie systemd[1]: dev-ssdforce-opt.device: Job dev-ssdforce-opt.device/start timed out.
Aug 08 12:01:48 speedie systemd[1]: Timed out waiting for device /dev/ssdforce/opt.
Aug 08 12:01:48 speedie systemd[1]: Dependency failed for File System Check on /dev/ssdforce/opt.
Aug 08 12:01:48 speedie systemd[1]: Dependency failed for /opt.
Aug 08 12:01:48 speedie systemd[1]: opt.mount: Job opt.mount/start failed with result 'dependency'.
Aug 08 12:01:48 speedie systemd[1]: systemd-fsck@dev-ssdforce-opt.service: Job systemd-fsck@dev-ssdforce-opt.service/start failed with resu>
Aug 08 12:01:48 speedie systemd[1]: dev-ssdforce-opt.device: Job dev-ssdforce-opt.device/start failed with result 'timeout'.
Aug 08 12:01:48 speedie systemd[1]: dev-hddgen-portage.device: Job dev-hddgen-portage.device/start timed out.
Aug 08 12:01:48 speedie systemd[1]: Timed out waiting for device /dev/hddgen/portage.
Aug 08 12:01:48 speedie systemd[1]: Dependency failed for File System Check on /dev/hddgen/portage.
Aug 08 12:01:48 speedie systemd[1]: Dependency failed for /usr/portage.

attempting to mount the paths manually succeed but the filesystem doesn't actually appear.

Instead I have to run 
pvscan
vgscan --mknodes
vgchange --sysinit -a ly

After which point I can now see files appear when I mount the filesystems though the names generated by mapper point to the wrong devices and I have to work out the correct filesystem to mount onto the various paths in order for the systemd jobs to all succeed subsequently.

Looking through the journal log output subsequently I see the following messages appear 
Aug 08 12:10:23 speedie systemd-udevd[4412]: dm-0: Conflicting device node '/dev/mapper/ssdforce-opt' found, link to '/dev/dm-0' will not be created.
Aug 08 12:10:23 speedie systemd-udevd[4412]: dm-1: Conflicting device node '/dev/mapper/ssdforce-portage' found, link to '/dev/dm-1' will not be created.
Aug 08 12:10:24 speedie systemd-udevd[4432]: dm-7: Conflicting device node '/dev/mapper/hddgen-usrsrc' found, link to '/dev/dm-7' will not be created.
Aug 08 12:10:24 speedie systemd-udevd[4413]: dm-8: Conflicting device node '/dev/mapper/hddgen-media' found, link to '/dev/dm-8' will not be created.
Aug 08 12:10:24 speedie systemd-udevd[4412]: dm-2: Conflicting device node '/dev/mapper/hddgen-portage' found, link to '/dev/dm-2' will not be created.
Aug 08 12:10:24 speedie systemd-udevd[4415]: dm-3: Conflicting device node '/dev/mapper/hddgen-var' found, link to '/dev/dm-3' will not be created.
Aug 08 12:10:24 speedie systemd-udevd[4428]: dm-4: Conflicting device node '/dev/mapper/hddgen-vartmp' found, link to '/dev/dm-4' will not be created.

I suspect these are the reason that I end up with the wrong devices getting mounted when I attempt to use the default names, unclear currently whether that is due to me running one of the commands above or they were generated incorrectly during the boot up.

Downgrading to a local ebuild of =sys-fs/lvm2-2.02.184-r5 that I copied out of the gentoo repository history is sufficient to have me booting normally again.

I'm guessing the timeout that we're seeing is due to /dev/mapper/* not pointing to the correct device node and consequently the expected device/filesystem it should reference not being found.

What I don't know is why? Is it a bug with the newer versions of lvm2 or some stale configuration/files that is not expected by newer lvm2 causing the behaviour.
Comment 8 Joakim Tjernlund 2020-10-12 08:58:36 UTC
Just started to test out systemd at work and we got this far too.
There is something odd with crypto/lvm2. Without crypto(only lvm2) is fine