Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 910233 - sys-apps/systemd-utils[tmpfiles] munges files on live systems
Summary: sys-apps/systemd-utils[tmpfiles] munges files on live systems
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo systemd Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-07-12 06:42 UTC by Logan Perkins
Modified: 2024-03-28 21:12 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Logan Perkins 2023-07-12 06:42:03 UTC
systemd-tmpfiles's configs from upstream are badly configured.  Specifically /usr/lib/tmpfiles.d/tmp.conf is configured to wipe files older than 10d from /tmp.  This includes files which are still open, read or write.  

As of fall 2022, gentoo now runs `systemd-tmpfiles --clean` 5 minutes after midnight daily, rather than only at system boot.  Combined with the 10d timeout in `tmp.conf`, this means live files, including `.X[0-9]-lock`, temporary opened downloads, and application temporary data are silently purged when the system has been up for more than 10 days.

For reference, debian replaces 
`q /tmp/ 1777 root root 10d`
with
`q /tmp/ 1777 root root -`
While this fixes the immediate problem, it completely disables cleaning /tmp via systemd.  It is likely sufficient to use
`q! /tmp/ 1777 root root 10d`
which will only allow cleaning `/tmp` during boot.  (as a side note, the reason `X0-lock` gets munged is because the `r!` entry in x11.conf, which intuitively would protect it except at boot gets suppressed when not at boot due to the `!`).

Please note that fixing systemd-tmpfiles so it respects opened files, while worthy of its own bug report, would *not* resolve this issue.  systemd-tmpfiles decides what files to unlink at the leaf, rather than directory, level, so programs which have a collection of volatile files, but only a subset of them open (such as long-running CFD simulations), will still suffer data loss if files are periodically purged.

Bottom line is that deleting files, especially from a running system, needs to be an opt-in rather than opt-out setting.
Comment 1 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-07-12 06:47:10 UTC
(In reply to Logan Perkins from comment #0)
> As of fall 2022, gentoo now runs `systemd-tmpfiles --clean` 5 minutes after
> midnight daily, rather than only at system boot.  

Yes, announced in a news item at https://www.gentoo.org/support/news-items/2022-11-21-tmpfiles-clean.html.

> [...]
> Please note that fixing systemd-tmpfiles so it respects opened files, while
> worthy of its own bug report, would *not* resolve this issue. 

Please make one.

> Bottom line is that deleting files, especially from a running system, needs
> to be an opt-in rather than opt-out setting.

On the other hand, you then end up with junk in directories where applications rely on tmpfiles to clean them up. The news item makes clear how to opt-out. We could add it to the handbook too as an optional step.
Comment 2 Logan Perkins 2023-07-12 07:22:46 UTC
(In reply to Sam James from comment #1)
> (In reply to Logan Perkins from comment #0)
> > As of fall 2022, gentoo now runs `systemd-tmpfiles --clean` 5 minutes after
> > midnight daily, rather than only at system boot.  
> 
> Yes, announced in a news item at
> https://www.gentoo.org/support/news-items/2022-11-21-tmpfiles-clean.html.
> 

Removing the timer added per that news item is not a suitable solution, as there are other targets than /tmp (such as `/var/cache/man`, which is cleaned after 1w) which are skipped too.  

> > [...]
> > Please note that fixing systemd-tmpfiles so it respects opened files, while
> > worthy of its own bug report, would *not* resolve this issue. 
> 
> Please make one.

Already in progress, along with a couple other related ones.

> 
> > Bottom line is that deleting files, especially from a running system, needs
> > to be an opt-in rather than opt-out setting.
> 
> On the other hand, you then end up with junk in directories where
> applications rely on tmpfiles to clean them up. The news item makes clear
> how to opt-out. We could add it to the handbook too as an optional step.

Negative.  Applications rely on their files staying put *at least* until program exit, and generally until system shutdown.  If an application relies on systemd-tmpfiles cleaning up its cruft between runs (or during runs), then *that application* is majorly broken, and will fail if started twice in a single day (since systemd-tmpfiles only runs once a day).  

Both `bootmisc` and `systemd-tmpfiles` clean /tmp on boot, so disk space loss and accumulated cruft are already limited.  

Adding a section to the handbook would be the bare minimum.  New installs do not get the last 20 years of news items, so new users need to be warned of the unintuitive and terrible defaults so they can be fixed.  Especially given that bootmisc still exists and is now essentially ignored.  

And to reiterate, the suggested fix ought not to be opting out of the cron.daily script, but marking /tmp as sacrosanct at runtime (though mentioning the ability to completely opt out of the cron.daily, or emerging with USE=-tmpfiles could get their own mentions).

Anyway, I'm working on the related issues presently.
Comment 3 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-07-12 07:35:21 UTC
> > > Bottom line is that deleting files, especially from a running system, needs
> > > to be an opt-in rather than opt-out setting.
> > 
> > On the other hand, you then end up with junk in directories where
> > applications rely on tmpfiles to clean them up. The news item makes clear
> > how to opt-out. We could add it to the handbook too as an optional step.
> 
> Negative.  Applications rely on their files staying put *at least* until
> program exit, and generally until system shutdown.  If an application relies
> on systemd-tmpfiles cleaning up its cruft between runs (or during runs),
> then *that application* is majorly broken, and will fail if started twice in
> a single day (since systemd-tmpfiles only runs once a day).  
> 

Talking past each other slightly here. I'm not disagreeing with you entirely, I'm just pointing out why clean in general probably shouldn't be exclusively run at boot. An application may well maintain a large cache which it wants flushed every so often.

That is not me saying an application *requires* it removed, or that it's okay to yank stuff while an application is using it.
Comment 4 Larry the Git Cow gentoo-dev 2024-03-28 21:12:27 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=dafe17091a2bfa128ee7f706d63e76cc42e4c58f

commit dafe17091a2bfa128ee7f706d63e76cc42e4c58f
Author:     Mike Gilbert <floppym@gentoo.org>
AuthorDate: 2024-03-28 21:09:36 +0000
Commit:     Mike Gilbert <floppym@gentoo.org>
CommitDate: 2024-03-28 21:12:06 +0000

    sys-apps/systemd-utils: disable auto-cleanup of /tmp and /var/tmp
    
    This can go awry when people have non-standard mount options for these
    paths.
    
    Closes: https://bugs.gentoo.org/910233
    Bug: https://bugs.gentoo.org/916623
    Closes: https://bugs.gentoo.org/917777
    Signed-off-by: Mike Gilbert <floppym@gentoo.org>

 sys-apps/systemd-utils/files/tmp.conf                                 | 2 ++
 .../{systemd-utils-254.10.ebuild => systemd-utils-254.10-r1.ebuild}   | 4 ++--
 .../{systemd-utils-254.8.ebuild => systemd-utils-254.8-r1.ebuild}     | 4 ++--
 sys-apps/systemd-utils/systemd-utils-255.4.ebuild                     | 4 ++--
 4 files changed, 8 insertions(+), 6 deletions(-)