Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 821268 - sys-fs/cryptsetup: udev cookie waiting for zero, shutdown stuck.
Summary: sys-fs/cryptsetup: udev cookie waiting for zero, shutdown stuck.
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Gentoo's Team for Core System packages
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-11-02 04:07 UTC by David Duchesne
Modified: 2021-11-05 22:51 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
shutdown failure (4 encrypted drives) (file_821268.txt,4.67 KB, text/plain)
2021-11-02 04:08 UTC, David Duchesne
Details
shutdown success (1 encrypted drive) (file_821268.txt,4.53 KB, text/plain)
2021-11-02 04:09 UTC, David Duchesne
Details
shutdown success (4 encrypted drives) with lvmetad disabled (file_821268.txt,16.95 KB, text/plain)
2021-11-04 01:34 UTC, David Duchesne
Details

Note You need to log in before you can comment on or make changes to this bug.
Description David Duchesne 2021-11-02 04:07:32 UTC
Full discussion here : https://forums.gentoo.org/viewtopic-t-1116760.html
Quoting the relevant part to get to the point :

---
On top of my encrypted root device i'm having 2 more encrypted partitions which are mounted on boot via dmcrypt.

On shutdown/reboot these partitions won't be unmounted because dmcrypt is waiting for a udev cookie to be zero which is not going to happen 99% of the time.
If i'm changing content of /etc/conf.d/dmcrypt or shutdown right after boot (without login) sometimes the notebook will shutdown/reboot proper.

[...]

Still haven't found the reason for this behaviour, but removing lvm leads to an error while closing encrypted volumes ("still in use").
Modify stop-function of /etc/init.d/dmcrypt to manually deactivate a volumegroup: 

lvchange <volume_group> -an

Here is my current /etc/init.d/dmcrypt which solved all issues. 

[...]
stop() {
   local line header
   
   ### closing volume groups
   lvchange avg -an
   lvchange xvg -an
[...]
----

Having the same issue, I successfully reproduced the issue in a Gentoo VM with qemu. (A bit easier to capture all the output)
The Gentoo VM is fully updated, recent kernel; needed conf (/etc/conf.d/dmcrypt, fstab) and services (dmcrypt and lvm added to boot runlevel) all configured and setup.
Please see the two logs.

In the first log, 4 encrypted drives are used (LUKS/LVM) and during shutdown, it hangs on "udev cookie waiting for zero" and nothing happens. I waited more than 20 minutes and it is completely stuck.

In the second log, only 1 encrypted drive is used (LUKS/LVM) and shutdown completes successfully.

I tried with systemd (using arch & debian) with the exact same setup in an other VM and shutdown completed in both cases (with one or with 4 encrypted drives) so it looks like some kind of race conditon with dmcrypt and openrc *when* several encrypted LUKS/LVM drives are used.
The suggested solution (adding "lvchange the_lv -an" in /etc/init.d/dmcrypt) works but it would better to understand and fix the root cause I think.

Reproducible: Always
Comment 1 David Duchesne 2021-11-02 04:08:20 UTC
Created attachment 747966 [details]
shutdown failure (4 encrypted drives)
Comment 2 David Duchesne 2021-11-02 04:09:01 UTC
Created attachment 747969 [details]
shutdown success (1 encrypted drive)
Comment 3 David Duchesne 2021-11-04 01:34:21 UTC
Created attachment 748374 [details]
shutdown success (4 encrypted drives) with lvmetad disabled
Comment 4 David Duchesne 2021-11-04 01:47:30 UTC
(In reply to David Duchesne from comment #3)
> Created attachment 748374 [details]
> shutdown success (4 encrypted drives) with lvmetad disabled

I noticed that if I disable lvmetad in /etc/lvm/lvm.conf

use_lvmetad = 0

Then everything is good and shutdown works with one, two or the four encrypted drives.

But as soon as I enable lvmetad (it is the default when lvm package is installed) then the issues are back with "udev cookie waiting for zero". (While shutdown was somehow stuck on that message, I was able to see this process running "lvm pvscan --background --cache --activate ay --major 253" so it looks like related to lvmetad although I'm not an expert at all.

Initially, I opened this bug thinking it was sys-fs/cryptsetup causing the issue, but it looks like more sys-fs/lvm2 and lvmetad component (when enabled) in a scenario (like I use) where multiple encrypted drives, LVM.. are used.
Comment 5 David Duchesne 2021-11-04 02:14:12 UTC
After some more trial and error, I found I needed to add lvmetad to runlevel shutdown, no need to disable lvmetad at all from /etc/lvm/lvm.conf

rc-update add lvmetad shutdown

No more udev cookie waiting for zero message.
The cookie is correctly destroyed for each drive and shutdown completes without delay (really fast) :

# Udev cookie 0xd4d9f61 (semid 23) decremented to 1
# Udev cookie 0xd4d9f61 (semid 23) waiting for zero
# Udev cookie 0xd4d9f61 (semid 23) destroyed

Marking as "worksforme" and closing.
Comment 6 Lars Wendler (Polynomial-C) (RETIRED) gentoo-dev 2021-11-04 17:19:26 UTC
Just a heads up here; once we unmask(/stabilize) sys-fs/lvm2-2.03*, the lvmetad will be gone anyway.
Comment 7 David Duchesne 2021-11-04 23:45:51 UTC
(In reply to Lars Wendler (Polynomial-C) from comment #6)
> Just a heads up here; once we unmask(/stabilize) sys-fs/lvm2-2.03*, the
> lvmetad will be gone anyway.

Thanks Lars. How funny, I just found out that myself an hour ago. I noticed lvmetad is indeed gone in currently masked version. I installed it (in a vm to try) and it solved the issue completely !
Cheers
Comment 8 Larry the Git Cow gentoo-dev 2021-11-05 22:51:29 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ddf4794a0f5cb2a8ecbff62d6a1f85a848d16249

commit ddf4794a0f5cb2a8ecbff62d6a1f85a848d16249
Author:     David Seifert <soap@gentoo.org>
AuthorDate: 2021-11-05 22:51:20 +0000
Commit:     David Seifert <soap@gentoo.org>
CommitDate: 2021-11-05 22:51:20 +0000

    profiles: unmask >=sys-fs/lvm2-2.03
    
    Closes: https://bugs.gentoo.org/821268
    Acked-by: Robin H. Johnson <robbat2@gentoo.org>
    Signed-off-by: David Seifert <soap@gentoo.org>

 profiles/package.mask | 5 -----
 1 file changed, 5 deletions(-)