Since I merged systemd-248 (actually since 248-rc4), the logrotate service has failed to run. It was running fine with earlier systemds (247.2-rX, etc) and still runs fine if I run the exact same command manually (from root of course). The error as reported by systemctl status logrotate.service is a namespacing error: May 02 00:00:16 h2 systemd[1]: Starting logrotate.service... May 02 00:00:16 h2 systemd[20094]: logrotate.service: Failed to set up mount namespacing: /run/systemd/unit-root/.: Invalid argument May 02 00:00:16 h2 systemd[20094]: logrotate.service: Failed at step NAMESPACE spawning /usr/bin/logrotate: Invalid argument May 02 00:00:16 h2 systemd[1]: logrotate.service: Main process exited, code=exited, status=226/NAMESPACE May 02 00:00:16 h2 systemd[1]: logrotate.service: Failed with result 'exit-code'. May 02 00:00:16 h2 systemd[1]: Failed to start logrotate.service. The service file is modified slightly from the default to put the state file in /var/log (which obviously must be writable by logrotate or it can't do its job), because here the normal location in /var/lib/misc is mounted read-only by default (except when I'm doing system updates): # JED ExecStart=/usr/bin/logrotate /etc/logrotate.conf ExecStart=/usr/bin/logrotate --state /var/log/logrotate.state /etc/logrotate.conf (JED are my initials, used to tag my changes to make them easy to find.) Because logrotate runs fine with the same logrotate config when run manually I suspect it's something to do with the hardening options in the service file. But they were working fine with systemd-247* so it's something that changed with systemd-248. However, unlike most systemd-upgrade related issues I've run into on ~arch, looking in the systemd NEWS file (as I routinely do for any systemd update) doesn't yield any obvious (at least to me) culprits.
Confirming that it is the hardening options. Based on a google (using ddg) I tried commenting all the Private* and Protected* options and it ran! Now to figure out which ones...
The problem option is: ProtectSystem=full
So the systemd.exec manpage says exit code 226/EXIT_NAMESPACE is triggered by failure to setup mount, UTS or IPC namespacing. FWIW the kernel has all six namespaces (UTS/TIME/IPC/USER/PID/NET) on, tho that leaves me wondering about mount, which unlike the others doesn't seem to be listed at least by that name in either .config or the TUI. And why would 247 work but 248 not, with nothing related that I can see in the NEWS file? Could it possibly be related to phasing out the old v1 control-groups? FWIW I don't screw with them as I don't know enough about them to be comfortable doing so, so everything there is at defaults. But that's the only reasonable candidate I see in the NEWS file and I've looked.
I am unable to reproduce this failure.
Please provide emerge --info so that we have it on record.
Created attachment 705771 [details] emerge --info logrotate
Anything interesting in dmesg?
(In reply to Mike Gilbert from comment #4) > I am unable to reproduce this failure. A couple unusual bits about my system the above got me thinking about... Might be relevant: * Reverse usr-merge so /usr -> . * /var/log -> /lg (with /lg a mountpoint for a dedicated filesystem just for the logs, logrotate's olddir directive pointing to a subdir on the same /lg filesystem). Might one of those be triggering a systemd 248 ProtectSystem corner-case? (FWIW I tried turning it down to true, didn't help. Neither did listing even / or all /-subdirs and symlinks in a tested ReadWritePaths entry, but I think that's because it never gets to the ReadWritePaths once the ProtectSystem loads as I believe it's the loading itself that triggers the mount namespace errors.) Likely irrelevant: * The (non-virtual) filesystems are all btrfs. * The systemd journal is virtual-only (all tmpfs, no on-ssd, thus avoiding some issues with systemd journals on btrfs), but I do have it routed to syslog-ng for (text-only) logging that's rotated by a systemd boot-time service so logrotate doesn't touch the main syslog-ng system messages log either. (Logrotate's purely for other logs, including my portage logs.)
(In reply to Mike Gilbert from comment #7) > Anything interesting in dmesg? Running logrotate.service (with the ProtectSystem entry uncommented again for the test) doesn't add anything to dmesg at all. However, naturally after editing the service file systemd says to run systemctl daemon-reload. That does spit out a few systemd-gpt-auto-generator and systemd-fstab-generator lines, but the latter have been there since I switched to systemd years ago and the former has been there since the systemd feature was added; nothing new with systemd 248 which triggered the problem. But here's a couple of them just in case (/bk is my backups mountpoint, multiple backups are rotated, each with its own label and a LABEL= entry in fstab, so despite being listed for the same mountpoint, they're unique with the LABEL= lines, and FWIW I don't use the gpt-auto-generator feature at all)... systemd-gpt-auto-generator[325311]: Failed to determine block device of /usr file system: Too many levels of symbolic links systemd-fstab-generator[325309]: Failed to create unit file /run/systemd/generator/bk.mount, as it already exists. Duplicate entry in /etc/fstab?
(Oh and of course the backups are all noauto, only manually mounted with still for-now-at-least non-systemd mount.)
We don't support he "reverse usr merge".
