| Summary: | sys-devel/systemd-233-r1: all services with PrivateTmp=yes fail to start: Failed at step NAMESPACE spawning <path_to_service_executable>: No such file or directory | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Maciej Mrozowski <reavertm> |
| Component: | Current packages | Assignee: | Gentoo systemd Team <systemd> |
| Status: | RESOLVED INVALID | ||
| Severity: | normal | ||
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: |
emerge --info
systemd log strace -p1 -s 32768.log strace - last failing lines.txt |
||
Created attachment 471554 [details]
systemd log
systemd installed with the following flags:
maciek@liwardyna ~ $ emerge -1Op systemd
These are the packages that would be merged, in order:
[ebuild R ] sys-apps/systemd-233-r1:0/2::gentoo USE="curl gcrypt kmod lz4 lzma pam policykit seccomp ssl -acl (-apparmor) -audit -build -cryptsetup -doc -elfutils -gnuefi -http -idn -importd -nat -qrcode (-selinux) -sysv-utils {-test} -vanilla -xkb" ABI_X86="(64) -32 (-x32)" 0 KiB
Using systemd kernel Gentoo "profile" (so kernel namespaces are enabled).
Also attaching systemd log.
system-hostnamed fails in identical way - 'no such file or directory' when executed as service but /usr/lib/systemd/systemd-hostnamed works when executed in ex from shell. NetworkManager-wait-online just failed due to unsatisfied service dependency during startup, when manually restarted once network is up, it's no longer marked as failed so not relevant here. Please try commenting-out the various ProtectXXX settings in /usr/lib/systemd/system/systemd-timesyncd.service. Indeed it's not related to symlink traversal, but.. PrivateTmp=yes My systemd /tmp setup is non-standard - I use zramfs for /tmp and /var/tmp being symlink to /tmp) /dev/zram1 on /tmp type ext4 (rw,nodev,noatime,block_validity,discard,delalloc,user_xattr) Activated by zram-init .service by: ExecStart=/usr/sbin/zram-init -d1 -s1 -alzo -text4 -obarrier=0,noatime,nodev -m1777 7168 /tmp For this to work, I had to (?) redirect systemd's own tmp.mount to /dev/null in /etc/systemd as it would interfere. Original tmp.mount mounts with: Options=mode=1777,strictatime,nosuid,nodev Passing nosuid to zram /tmp mount doesn't solve the problem while strictatime doesn't seem to work (perhaps not supported by zramfs, at least not shown by mount as applied mount option). So the way systemd handles PrivateTmp=yes seems to by tied to how /tmp was mounted in first place. Which sounds silly a bit since the option was supposed to provide private tmp for the service from scratch. I tried to "reproduce" the private mount problem and bind this zramfs /tmp to some other mount point and make that other mount point private, as shown in example: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Storage_Administration_Guide/sect-Using_the_mount_Command-Mounting-Bind.html but private mount works well for this zramfs /tmp. You could enable debug logging to see exactly what system call is failing. You can do that by setting LogLevel=debug in /etc/systemd/system.conf, or by setting systemd.log_level=debug on the kernel command line. There are more traces due to service file discovery but sadly nothing more (verbose) from service runner. Created attachment 477572 [details]
strace -p1 -s 32768.log
Attached strace of systemd process while restarting systemd-timesyncd service.
No attempt there to even execute /usr/lib/systemd/systemd-timesyncd process..
I have a feeling I'll need to gdb-the-hell-out-of that crap...
Created attachment 477574 [details]
strace - last failing lines.txt
Narrowed down strace to I think relevant part.
I mentioned that my systemd /tmp setup is non-standard - I use zramfs for /tmp and /var/tmp being symlink to /tmp). This is what breaks PrivateTmp=yes. systemd tries to create mount point in "privatized" directory for its services and while filesystem name of this mount point does not overlap: maciek@liwardyna ~ $ ls -la /tmp/ /var/tmp /tmp/: (...) drwx------ 3 root root 4096 lip 8 21:39 systemd-private-a3855fb39e12431fab04fbd7b9462e5d-systemd-resolved.service-KnXOBs drwx------ 3 root root 4096 lip 8 21:39 systemd-private-a3855fb39e12431fab04fbd7b9462e5d-systemd-timesyncd.service-y175F4 (...) /var/tmp: (...) drwx------ 3 root root 4096 lip 8 21:39 systemd-private-a3855fb39e12431fab04fbd7b9462e5d-systemd-resolved.service-taeNzE drwx------ 3 root root 4096 lip 8 21:39 systemd-private-a3855fb39e12431fab04fbd7b9462e5d-systemd-timesyncd.service-cC0QDg (...) making /var/tmp not a symlink to /tmp solves the problem. I'll mark bug as invalid as I'm not certain such symlink is supported use case. |
Created attachment 471552 [details] emerge --info After systemd 233-r1 stabilization, certain services fail to start: maciek@liwardyna ~ $ sudo systemctl --failed UNIT LOAD ACTIVE SUB DESCRIPTION ● NetworkManager-wait-online.service loaded failed failed Network Manager Wait Online ● systemd-hostnamed.service loaded failed failed Hostname Service ● systemd-timesyncd.service loaded failed failed Network Time Synchronization timesyncd fails with: maj 03 06:56:32 liwardyna systemd[29321]: systemd-timesyncd.service: Failed at step NAMESPACE spawning /usr/lib/systemd/systemd-timesyncd: No such file or directory but clearly: maciek@liwardyna ~ $ file /usr/lib/systemd/systemd-timesyncd /usr/lib/systemd/systemd-timesyncd: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, stripped There is also: maj 03 06:56:32 liwardyna systemd[1]: systemd-timesyncd.service: Main process exited, code=exited, status=226/NAMESPACE maj 03 06:56:32 liwardyna systemd[1]: Failed to start Network Time Synchronization. But neither my /home, /usr or root is a symlink. Perhaps upstream correction for detecting symlinks actually broke following symlinks by systemd itself? Because our /usr/lib is a symlink to /usr/lib64.