| Summary: | sys-apps/systemd-220 - breaks previously working Dracut-built initramfs setup (LUKS encrypted disk + LVM) | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Andrew Udvare <audvare> |
| Component: | [OLD] Core system | Assignee: | Gentoo systemd Team <systemd> |
| Status: | RESOLVED OBSOLETE | ||
| Severity: | major | CC: | aidecoe, alexander, ben.c.schubert, chutzpah, leho, prometheanfire |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| See Also: |
https://bugs.gentoo.org/show_bug.cgi?id=551688 https://bugs.gentoo.org/show_bug.cgi?id=554698 |
||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: |
Log from emergency console.
systemd-9999 build log Dracut debug log |
||
|
Description
Andrew Udvare
2015-06-09 07:48:58 UTC
Created attachment 404808 [details]
Log from emergency console.
Please give full versions with revisions, not just "220" and "219". It matters because some encryption-related bugs were present in 220 and fixed in 220-r1, for example. Also please attach the output of "emerge --info systemd". (In reply to Alexandre Rostovtsev from comment #2) > Also please attach the output of "emerge --info systemd". Ah, never mind, you already did that. Sorry for failing to read :/ Copying dracut maintainers. (In reply to Andrew Udvare from comment #0) If you're up for some testing, please give systemd-9999 a try. If that works, at least we know that the problem has been resolved upstream and we can look for a patch to backport. (In reply to Mike Gilbert from comment #5) > (In reply to Andrew Udvare from comment #0) > > If you're up for some testing, please give systemd-9999 a try. If that > works, at least we know that the problem has been resolved upstream and we > can look for a patch to backport. Will do in a little bit and report. Created attachment 404856 [details]
systemd-9999 build log
I was unable to build systemd-9999 and so I am attaching the log. Is it broken at commit 3a3701f03360f9a292dcab31fda1f6e0e0629c64? Is there something else I can try?
I have made some updates to the ebuild which should resolve that build failure. + 10 Jun 2015; Mike Gilbert <floppym@gentoo.org> systemd-9999.ebuild: + Drop references to gudev, gtk-doc, and gobject-introspection. (In reply to Mike Gilbert from comment #8) > I have made some updates to the ebuild which should resolve that build > failure. > > + 10 Jun 2015; Mike Gilbert <floppym@gentoo.org> systemd-9999.ebuild: > + Drop references to gudev, gtk-doc, and gobject-introspection. How can I get that newer ebuild? https://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/sys-apps/systemd/ doesn't show your commit (yet) (In reply to Andrew Udvare from comment #9) > How can I get that newer ebuild? > https://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/sys-apps/systemd/ > doesn't show your commit (yet) In general, wait an hour and run emerge --sync. It takes rsync some time to catch up to CVS. It should be updated now. If I am updated, I am now getting this issue:
# emerge -av =sys-apps/systemd-9999
These are the packages that would be merged, in order:
Calculating dependencies... done!
[ebuild U *] sys-apps/systemd-9999:0/2::gentoo [219_p112:0/2::gentoo] USE="acl cryptsetup idn kmod lz4 pam (policykit) seccomp ssl -apparmor -audit -curl -elfutils -gcrypt -gnuefi% -http -importd (-kdbus) -lzma -nat -python -qrcode (-selinux) -sysv-utils -terminal {-test} -vanilla -xkb (-doc%) (-gudev%*) (-introspection%*)" ABI_X86="32 (64) (-x32)" PYTHON_SINGLE_TARGET="python2_7 -python3_3 -python3_4" PYTHON_TARGETS="python2_7 python3_4 -python3_3" 0 KiB
Total: 1 package (1 upgrade), Size of downloads: 0 KiB
!!! Multiple package instances within a single package slot have been pulled
!!! into the dependency graph, resulting in a slot conflict:
sys-apps/systemd:0
(sys-apps/systemd-9999:0/2::gentoo, ebuild scheduled for merge) pulled in by
=sys-apps/systemd-9999 (Argument)
(sys-apps/systemd-219_p112:0/2::gentoo, installed) pulled in by
>=sys-apps/systemd-212-r5:0/2[abi_x86_32(-)?,abi_x86_64(-)?,abi_x86_x32(-)?,abi_mips_n32(-)?,abi_mips_n64(-)?,abi_mips_o32(-)?,abi_ppc_32(-)?,abi_ppc_64(-)?,abi_s390_32(-)?,abi_s390_64(-)?,gudev,introspection?] required by (virtual/libgudev-215-r1:0/0::gentoo, installed)
^^^^^ ^^^^^^^^^^^^^^
I have masked 220 and unmasked with **.
# equery d libgudev
* These packages depend on libgudev:
gnome-base/gvfs-1.22.3 (virtual/libgudev)
media-gfx/gimp-2.8.14 (udev ? virtual/libgudev)
sys-fs/udisks-1.0.5-r1 (virtual/libgudev)
sys-fs/udisks-2.1.4 (virtual/libgudev)
sys-power/upower-pm-utils-0.9.23-r2 (kernel_linux ? virtual/libgudev)
You need to upgrade virtual/libgudev to 215-r2 first. (In reply to Michał Górny from comment #12) > You need to upgrade virtual/libgudev to 215-r2 first. I did this, but I am still getting an error about the introspection USE flag. I cannot see what is pulling this other than a lot of packages having set +introspection. I even built virtual/libgudev with -introspection but that did not help. None of my configurations mentions adding this USE flag. # emerge -av =sys-apps/systemd-9999 These are the packages that would be merged, in order: Calculating dependencies ... done! [ebuild U *] sys-apps/systemd-9999:0/2::gentoo [219_p112:0/2::gentoo] USE="acl cryptsetup idn kmod lz4 pam (policykit) seccomp ssl -apparmor -audit -curl -elfutils -gcrypt -gnuefi% -http -importd (-kdbus) -lzma -nat -python -qrcode (-selinux) -sysv-utils -terminal {-test} -vanilla -xkb (-doc%) (-gudev%*) (-introspection%*)" ABI_X86="32 (64) (-x32)" PYTHON_SINGLE_TARGET="python2_7 -python3_3 -python3_4" PYTHON_TARGETS="python2_7 python3_4 -python3_3" 0 KiB Total: 1 package (1 upgrade), Size of downloads: 0 KiB !!! Multiple package instances within a single package slot have been pulled !!! into the dependency graph, resulting in a slot conflict: sys-apps/systemd:0 (sys-apps/systemd-9999:0/2::gentoo, ebuild scheduled for merge) pulled in by =sys-apps/systemd-9999 (Argument) (sys-apps/systemd-219_p112:0/2::gentoo, installed) pulled in by >=sys-apps/systemd-212-r5:0/2[abi_x86_32(-)?,abi_x86_64(-)?,abi_x86_x32(-)?,abi_mips_n32(-)?,abi_mips_n64(-)?,abi_mips_o32(-)?,abi_ppc_32(-)?,abi_ppc_64(-)?,abi_s390_32(-)?,abi_s390_64(-)?,gudev(-),introspection?] required by (virtual/libgudev-215-r2:0/0::gentoo, installed) ^^^^^^^^^^^^^^ When I use @world, I get this:
WARNING: One or more updates/rebuilds have been skipped due to a dependency conflict:
sys-apps/systemd:0
(sys-apps/systemd-9999:0/2::gentoo, ebuild scheduled for merge) conflicts with
>=sys-apps/systemd-212-r5:0/2[abi_x86_64(-),gudev(-)] required by (virtual/libgudev-215-r2:0/0::gentoo, installed)
(In reply to Michał Górny from comment #12) > You need to upgrade virtual/libgudev to 215-r2 first. I am still stuck unable to build systemd-9999 with that fix among other changes in configuration. Sorry not build. I meant to say that Portage will not let it be emerged due to some conflict I have not figured out how to resolve. As of right now, the only way I can get it to work is to explicitly install libgudev. emerge --oneshot sys-apps/systemd dev-libs/libgudev virtual/libgudev (In reply to Mike Gilbert from comment #17) > As of right now, the only way I can get it to work is to explicitly install > libgudev. > > emerge --oneshot sys-apps/systemd dev-libs/libgudev virtual/libgudev Thanks. That worked. But I had to set USE="-introspection". Unfortunately, booting up from systemd-9999 did not work. Git commit is 4061eed4d91226f1f75e79350c5e1bdd4529d578 . Also note that this is a new version of Dracut 041-r3 but I do not think that is to blame because it was broken with the old version. Created attachment 404984 [details]
Dracut debug log
Thinking to do a bit of git bisect'ing. What is the commit hash for systemd-219_p112? (In reply to Andrew Udvare from comment #20) > Thinking to do a bit of git bisect'ing. What is the commit hash for > systemd-219_p112? It's 85a6fabdd3e43cfab0fc6359e9f2a9e368d4a3ed from the v219-stable branch -- not master. You would probably be better off bisecting from the v219 tag to the v220 tag. (In reply to Andrew Udvare from comment #0) > add_fstab+="/usr/src/initrd-fstab" mount-sys.sh from fstab-sys module is a pre-pivot hook that gets executed *after* root is mounted. So I have no idea how this setup worked for you. Please generate working initramfs with an older systemd version, add "rd.break rd.debug systemd.log_level=debug" to the kernel cmdline and attach resulting rdsosreport.txt. (In reply to Alexander Tsoy from comment #22) > (In reply to Andrew Udvare from comment #0) > > add_fstab+="/usr/src/initrd-fstab" > > mount-sys.sh from fstab-sys module is a pre-pivot hook that gets executed > *after* root is mounted. So I have no idea how this setup worked for you. > > Please generate working initramfs with an older systemd version, add > "rd.break rd.debug systemd.log_level=debug" to the kernel cmdline and attach > resulting rdsosreport.txt. That only applies if not using systemd. From mount-sys.sh: # systemd will mount and run fsck from /etc/fstab and we don't want to # run into a race condition. if [ -z "$DRACUT_SYSTEMD" ]; then [ -f /etc/fstab ] && fstab_mount /etc/fstab fi (In reply to Andrew Udvare from comment #23) You're right. systemd-fstab-generator parses /etc/fstab even inside initrd. (In reply to Andrew Udvare from comment #0) > For this /usr/src/initrd-fstab file: > UUID="XXXX-XXXX" /mnt/somewhere vfat noatime 1 2 What if you change sixth field to "0"? After commit 4dda4e637e4c17a14db6cd265f36f5e8a5050367 all fsck units are ordered after systemd-fsck-root.service which was not generated inside initrd in <systemd-220. (In reply to Alexander Tsoy from comment #25) > (In reply to Andrew Udvare from comment #0) > > For this /usr/src/initrd-fstab file: > > UUID="XXXX-XXXX" /mnt/somewhere vfat noatime 1 2 > > What if you change sixth field to "0"? After commit > 4dda4e637e4c17a14db6cd265f36f5e8a5050367 all fsck units are ordered after > systemd-fsck-root.service which was not generated inside initrd in > <systemd-220. Will try this and report. Also I will post the SOS log of the working initrd breaking before switch-root. This bug may be related: https://bugs.freedesktop.org/show_bug.cgi?id=90913 I have not had time to try bisecting but I will soon. This is what it looks like (if anyone thinks I should start at another commit please comment): Bisecting: 358 revisions left to test after this (roughly 9 steps) [42f1ab5009eed71f0d4f83681b8fdbed8664fca3] man: add sd_event_{run,wait,prepare,dispatch,loop} Bisecting: 179 revisions left to test after this (roughly 8 steps) [9e4ded3064e9a683e004ff8f6a8ce53ac20b79d7] build-sys: generate CLEANFILES from EXTRA_DIST Bisecting: 89 revisions left to test after this (roughly 7 steps) [858a109f4a336be6c258ae615d31651347b9b67b] machined: fix check if host directory could be opened Bisecting: 44 revisions left to test after this (roughly 6 steps) [0377e373d1b4973effe14ca19e21f0c10740085d] systemd-sysv-generator test: Adjust to dropped runlevelN.target mapping Bisecting: 21 revisions left to test after this (roughly 5 steps) [19e887e709c31ee4366ec44a770d3963cd48cb86] systemd-fsck: always connect to systemd-fsckd Bisecting: 10 revisions left to test after this (roughly 4 steps) [0370612e0522191f929e3feb7d4937fff3d421e2] machined: make "machinectl copy-to" and "machinectl copy-from" server side operations Bisecting: 5 revisions left to test after this (roughly 3 steps) [0974a682d155a5874123ba7de9c1e314c6681e0f] bootctl: add sd-boot support Bisecting: 2 revisions left to test after this (roughly 1 step) [32c3d7144cf9a5c8c03761d7f198142ca0f5f7b8] journal-remote: fix client_cert memory leak Bisecting: 0 revisions left to test after this (roughly 0 steps) [9c3cf9693ac5c0a332ba376f99e6adea28b1bb0d] journal-remote: fix certificate status memory leak (In reply to Andrew Udvare from comment #27) > This bug may be related: https://bugs.freedesktop.org/show_bug.cgi?id=90913 No, it's a different bug. > I have not had time to try bisecting but I will soon. This is what it looks > like (if anyone thinks I should start at another commit please comment): I still believe that commit 4dda4e637e4c17a14db6cd265f36f5e8a5050367 is the culprit. Have you already tried to disable fsck for partition containing the key file? > I still believe that commit 4dda4e637e4c17a14db6cd265f36f5e8a5050367 is the
> culprit. Have you already tried to disable fsck for partition containing the
> key file?
That is in the fstab I am adding correct?
Currently mine is like so:
UUID="XXXX-XXXX" /mnt/somewhere vfat noatime 1 2
(In reply to Andrew Udvare from comment #29) > > I still believe that commit 4dda4e637e4c17a14db6cd265f36f5e8a5050367 is the > > culprit. Have you already tried to disable fsck for partition containing the > > key file? > > That is in the fstab I am adding correct? > > Currently mine is like so: > UUID="XXXX-XXXX" /mnt/somewhere vfat noatime 1 2 Yes. If sixth fild != 0 then systemd-fstab-generator generates two units: - systemd-fsck@dev-disk-by\x2duuid-XXXX\x2dXXXX.service with dependency After=systemd-fsck-root.service - mnt-somewhere.mount with dependency After=systemd-fsck@dev-disk-by\x2duuid-XXXX\x2dXXXX.service <systemd-220 doesn't generate systemd-fsck-root.service, so the above two units can be started before fsck on the root device. systemd-220 generates systemd-fsck-root.service, so fsck and mount units for /mnt/somewhere are ordered after fsck unit for the root device. The latter depends on the luks device which in turn needs a key file from /mnt/somewhere. %) (In reply to Alexander Tsoy from comment #30) > Yes. If sixth fild != 0 then systemd-fstab-generator generates two units: > - systemd-fsck@dev-disk-by\x2duuid-XXXX\x2dXXXX.service with dependency > After=systemd-fsck-root.service > - mnt-somewhere.mount with dependency > After=systemd-fsck@dev-disk-by\x2duuid-XXXX\x2dXXXX.service > > <systemd-220 doesn't generate systemd-fsck-root.service, so the above two > units can be started before fsck on the root device. > systemd-220 generates systemd-fsck-root.service, so fsck and mount units for > /mnt/somewhere are ordered after fsck unit for the root device. The latter > depends on the luks device which in turn needs a key file from > /mnt/somewhere. %) I think this fixed it. I set the sixth field to 0 and booted fine. No bisecting or more debugging with 9999 needed. I extracted the fstab file, after successful boot, to confirm that the sixth field was 0 in in the initramfs image and it is. So it appears if you have a service that depends on mounting a device before while in initramfs, you must not allow file system checks on that device. This is a bit unfortunate IMHO. The only way around I presume is to manually modify the initramfs image after and set the dependencies in the services affected. (In reply to Andrew Udvare from comment #31) > So it appears if you have a service that depends on mounting a device before > while in initramfs, you must not allow file system checks on that device. > This is a bit unfortunate IMHO. I would call it a bug, not an intended behavior. So if I understand correctly we need following changes: 1. If partition is mounted outside /sysroot in initramfs, then corresponding fsck service should not have After=systemd-fsck-root.service 2. That partition should be unmounted before switching to real root (Conflicts=initrd-switch-root.target? bug 554698). (In reply to Alexander Tsoy from comment #33) > So if I understand correctly we need following changes: > 1. If partition is mounted outside /sysroot in initramfs, then corresponding > fsck service should not have After=systemd-fsck-root.service > 2. That partition should be unmounted before switching to real root > (Conflicts=initrd-switch-root.target? bug 554698). #2 is the main thing (safe unmount prior to switch root). As for #1, it should be possible and safe IMHO to fsck within the chroot prior to mounting (removing my prior need for the entry in real root /etc/fstab). This does not seem to work, and as with bug 554698, real root will not allow this either. I have no idea if this is just vfat related. (In reply to Alexander Tsoy from comment #33) > So if I understand correctly we need following changes: > 1. If partition is mounted outside /sysroot in initramfs, then corresponding > fsck service should not have After=systemd-fsck-root.service > 2. That partition should be unmounted before switching to real root > (Conflicts=initrd-switch-root.target? bug 554698). #2 is the main thing (safe unmount prior to switch root). As for #1, it should be possible and safe IMHO to fsck within the chroot prior to mounting (removing my prior need for the entry in real root /etc/fstab). This does not seem to work, and as with bug 554698, real root will not allow this either. I have no idea if this is just vfat related. What is the status of this with systemd-225? I have not tried reverting back to the old setting without nofail in fstab (in the initramfs fstab). Basically, machine boots, finds USB partition with key, mounts it, decrypts, then the root switches without an unmount before. Yes this leaves the partition in dirty state. |
