Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 527894 - sys-apps/systemd-217 missing run directories cause start failures for some packages
Summary: sys-apps/systemd-217 missing run directories cause start failures for some pa...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: Normal major
Assignee: Gentoo systemd Team
URL: http://lists.freedesktop.org/archives...
Whiteboard:
Keywords:
: 528224 (view as bug list)
Depends on:
Blocks:
 
Reported: 2014-11-01 20:07 UTC by Harris Landgarten
Modified: 2014-11-05 22:10 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Harris Landgarten 2014-11-01 20:07:24 UTC
After upgrading systemd to 217 from 216 on the following boot the following 3 services failed to start:

clamd

redis

postgresql-9.3

In all cases the cause was missing run directories. After creating the missing run directories manually and fixing permissions all the services now start
Comment 1 Harris Landgarten 2014-11-02 00:15:31 UTC
After looking further it seems this is a more serious issue with ordering issues causing system unit to be deleted. The issue is known but when I made the changed to the journal-flush.service file reported to be the problem my system hung after reaching the local-filesystems.target and would not boot.

I suggest this version of systemd but blocked until a proper fix is out.
Comment 2 Stefan G. Weichinger 2014-11-02 13:51:20 UTC
I confirm this behavior.

v217 introduced some cycle problem which breaks starting the tmpfiles-setup ... that results in non-starting services (in my case mysqld.service).

->

Nov 02 13:33:55 goto.oops.intern systemd[1]: Found dependency on basic.target/start
Nov 02 13:33:55 goto.oops.intern systemd[1]: Breaking ordering cycle by deleting job systemd-tmpfiles-setup.service/start
Nov 02 13:33:55 goto.oops.intern systemd[1]: Job systemd-tmpfiles-setup.service/start deleted to break ordering cycle starting with basic.target/start

Downgrading to v216 helped but for sure it should be checked what's wrong here.

Maybe some upstream issue (haven't seen anything on the systemd-devel-ml yet), maybe something in the units gentoo provides.
Comment 3 Stefan G. Weichinger 2014-11-02 13:58:27 UTC
request sent to systemd-devel-ml
Comment 4 Harris Landgarten 2014-11-02 14:08:11 UTC
Take a look at this:

https://lists.fedoraproject.org/pipermail/devel/2014-October/203866.html


I tried Lennard's patch but that broke booting completely
Comment 5 Pacho Ramos gentoo-dev 2014-11-02 15:01:49 UTC
(In reply to Harris Landgarten from comment #4)
> Take a look at this:
> 
> https://lists.fedoraproject.org/pipermail/devel/2014-October/203866.html
> 
> 
> I tried Lennard's patch but that broke booting completely

I cannot access the patch pointed there. I found a similar bug report in redhat bugzilla that pointed to http://cgit.freedesktop.org/systemd/systemd/commit/?id=919699ec301ea507edce4a619141ed22e789ac0d as solution :/
Comment 6 Stefan G. Weichinger 2014-11-02 15:03:44 UTC
(In reply to Harris Landgarten from comment #4)
> Take a look at this:
> 
> https://lists.fedoraproject.org/pipermail/devel/2014-October/203866.html
> 
> 
> I tried Lennard's patch but that broke booting completely

For me it worked on my thinkpad, I created /etc/systemd/system/systemd-journal-flush.service with the mentioned changes and rebooted, no more circle created.

I applied it to a VM as well but there it seems to have broken something, it has not come up again so far ....
Comment 7 Stefan G. Weichinger 2014-11-02 15:05:16 UTC
(In reply to Pacho Ramos from comment #5)
> (In reply to Harris Landgarten from comment #4)
> > Take a look at this:
> > 
> > https://lists.fedoraproject.org/pipermail/devel/2014-October/203866.html
> > 
> > 
> > I tried Lennard's patch but that broke booting completely
> 
> I cannot access the patch pointed there. I found a similar bug report in
> redhat bugzilla that pointed to
> http://cgit.freedesktop.org/systemd/systemd/commit/
> ?id=919699ec301ea507edce4a619141ed22e789ac0d as solution :/

Will look up your link as soon as I have my VM booted again ....

What I did on the thinkpad:

--- /usr/lib/systemd/system/systemd-journal-flush.service       2014-10-30 13:31:32.763858167 +0100
+++ /etc/systemd/system/systemd-journal-flush.service   2014-11-02 15:50:27.502053371 +0100
@@ -10,8 +10,9 @@
 Documentation=man:systemd-journald.service(8) man:journald.conf(5)
 DefaultDependencies=no
 Requires=systemd-journald.service
-After=systemd-journald.service local-fs.target remote-fs.target
+After=systemd-journald.service
 Before=systemd-user-sessions.service systemd-tmpfiles-setup.service
+RequiresMountsFor=/var/log/journal
Comment 8 Stefan G. Weichinger 2014-11-02 15:12:49 UTC
OK, VM is up again, just some fsck-related delay.

tmpdirs are there now with v217, so it seems to work for me.
Comment 9 Canek Peláez Valdés 2014-11-02 18:51:49 UTC
(In reply to Stefan G. Weichinger from comment #7)
[...]
> What I did on the thinkpad:
> 
> --- /usr/lib/systemd/system/systemd-journal-flush.service       2014-10-30
> 13:31:32.763858167 +0100
> +++ /etc/systemd/system/systemd-journal-flush.service   2014-11-02
> 15:50:27.502053371 +0100
> @@ -10,8 +10,9 @@
>  Documentation=man:systemd-journald.service(8) man:journald.conf(5)
>  DefaultDependencies=no
>  Requires=systemd-journald.service
> -After=systemd-journald.service local-fs.target remote-fs.target
> +After=systemd-journald.service
>  Before=systemd-user-sessions.service systemd-tmpfiles-setup.service
> +RequiresMountsFor=/var/log/journal

Slightly off-topic, but if you want to override only one line of a system unit file, it's easier to use drop-ins. Just create the directory:

/etc/systemd/system/systemd-journal-flush.service.d

and inside it put the file break-cycle.conf, with the following contents:

[Unit]
After=systemd-journald.service
RequiresMountsFor=/var/log/journal
Comment 10 Stefan G. Weichinger 2014-11-02 18:55:59 UTC
(In reply to Canek Peláez Valdés from comment #9)
> (In reply to Stefan G. Weichinger from comment #7)
> [...]
> Slightly off-topic, but if you want to override only one line of a system
> unit file, it's easier to use drop-ins. Just create the directory:
> 
> /etc/systemd/system/systemd-journal-flush.service.d
> 
> and inside it put the file break-cycle.conf, with the following contents:
> 
> [Unit]
> After=systemd-journald.service
> RequiresMountsFor=/var/log/journal

Nice to learn about this different approach, thanks @Canek
Comment 11 Stefan G. Weichinger 2014-11-02 19:04:11 UTC
Redid the change to my VM as Canek suggested ... but with this setup the cycle problem is back!

Checking for typos ... but can't find some so far.
Comment 12 Canek Peláez Valdés 2014-11-02 22:47:04 UTC
(In reply to Stefan G. Weichinger from comment #11)
> Redid the change to my VM as Canek suggested ... but with this setup the
> cycle problem is back!
> 
> Checking for typos ... but can't find some so far.

It's possible that systemd treats After= like it does with ExecStart=, as if they're lists, and then your drop-in just adds systemd-journald.service to the After= list.

If that's the case, just change the drop-in to:

[Unit]
After=
After=systemd-journald.service
RequiresMountsFor=/var/log/journal

The first After= will clear the list, the second one sets systemd-journald.service as the only member on the list, and then you will get the desired change.
Comment 13 Stefan G. Weichinger 2014-11-02 23:26:20 UTC
(In reply to Canek Peláez Valdés from comment #12)

> It's possible that systemd treats After= like it does with ExecStart=, as if
> they're lists, and then your drop-in just adds systemd-journald.service to
> the After= list.
> 
> If that's the case, just change the drop-in to:
> 
> [Unit]
> After=
> After=systemd-journald.service
> RequiresMountsFor=/var/log/journal
> 
> The first After= will clear the list, the second one sets
> systemd-journald.service as the only member on the list, and then you will
> get the desired change.

Sorry, didn't work either.

But in the context of this bug report it's really kind of off-topic.
It should and will be fixed upstream anyway.

Although it would be interesting to get this drop-in working correctly, sure.
Comment 14 Stefan G. Weichinger 2014-11-05 11:55:46 UTC
Will the corrected servicefile go into portage or do you plan to wait for upstream releasing a new version?
Comment 15 Mike Gilbert gentoo-dev 2014-11-05 20:49:03 UTC
*** Bug 528224 has been marked as a duplicate of this bug. ***
Comment 16 Mike Gilbert gentoo-dev 2014-11-05 20:51:44 UTC
I'll work on a patchset for the bugfixes that have been applied since 217.
Comment 17 Stefan G. Weichinger 2014-11-05 21:31:31 UTC
(In reply to Mike Gilbert from comment #16)
> I'll work on a patchset for the bugfixes that have been applied since 217.

nice. v217 is affected by 

https://bugs.gentoo.org/show_bug.cgi?id=527832

as well ( and I am ... )

thanks!
Comment 18 Mike Gilbert gentoo-dev 2014-11-05 22:10:27 UTC
+*systemd-217-r2 (05 Nov 2014)
+
+  05 Nov 2014; Mike Gilbert <floppym@gentoo.org> +systemd-217-r2.ebuild:
+  Backport several patches. Resolves bugs 527832, 527894.