Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 946466 - app-admin/syslog-ng-4.8.1 with <dev-libs/ivykis-0.43.2 ignores SIGTERM from systemd, causes 90 second delay on system shutdown
Summary: app-admin/syslog-ng-4.8.1 with <dev-libs/ivykis-0.43.2 ignores SIGTERM from s...
Status: UNCONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal major
Assignee: Tomáš Mózes
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-12-14 13:21 UTC by Paul Osmialowski
Modified: 2025-01-23 20:36 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge --info output, greater than copy-paste limit (emerge-info.txt,23.38 KB, text/plain)
2024-12-14 13:21 UTC, Paul Osmialowski
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Paul Osmialowski 2024-12-14 13:21:14 UTC
I'm observing it since one of the most recent systemd updates. Whenever I'm powering down or rebooting the systemd-controlled desktop machine, it always stucks for 1min. 30sec. on this:

A stop job is running for System Logger Daemon "default" instance (1min 30s)

Reproducible: Always

Steps to Reproduce:
1. Install gentoo linux on a desktop PC and chose to use systemd
2. Start it and use as usual
3. Try to reboot it or power it down
Actual Results:  
It takes annoying 1min 30sec extra time to complete the shutdown.

Expected Results:  
Immediate shutdown as in openrc-controlled systems.

Relevant packages installed:

[ebuild   R    ] sys-apps/systemd-256.7:0/2::gentoo  USE="acl audit bpf cryptsetup curl dns-over-tls elfutils gcrypt gnutls http idn iptables kmod lz4 lzma openssl pam pcre pkcs11 (policykit) qrcode seccomp sysv-utils xkb zstd -apparmor -boot -cgroup-hybrid -fido2 -homed -importd -kernel-install -pwquality -resolvconf -secureboot (-selinux) (-split-usr) -test -tpm -ukify -vanilla" ABI_X86="32 (64) (-x32)" PYTHON_SINGLE_TARGET="python3_12 -python3_10 -python3_11 -python3_13" 0 KiB
[ebuild   R    ] app-admin/syslog-ng-4.8.1::gentoo  USE="caps dbi http json python snmp systemd -amqp -geoip2 -grpc -kafka -mongodb -mqtt -pacct -redis -smtp -spoof-source -tcpd -test" PYTHON_SINGLE_TARGET="python3_12 -python3_10 -python3_11 -python3_13" 0 KiB
Comment 1 Paul Osmialowski 2024-12-14 13:21:52 UTC
Created attachment 914014 [details]
emerge --info output, greater than copy-paste limit
Comment 2 Paul Osmialowski 2024-12-14 13:28:36 UTC
As the 'logger' USE flag is set globally, I'm pasting the installed packages affected by it:

[ebuild   R    ] app-crypt/qca-2.3.9:2::gentoo  USE="botan gcrypt gpg logger nss pkcs11 qt5 qt6 sasl softstore ssl -debug -doc -examples -test" 0 KiB
[ebuild   R    ] sys-apps/util-linux-2.40.2::gentoo  USE="audit caps cramfs cryptsetup fdformat hardlink logger magic ncurses nls pam python readline slang systemd tty-helpers udev (unicode) verify-sig -build -kill (-rtas) (-selinux) -static-libs -su -suid -test -uuidd" ABI_X86="32 (64) (-x32)" PYTHON_TARGETS="python3_12 -python3_10 -python3_11 -python3_13" 0 KiB
[ebuild   R    ] sys-apps/systemd-256.7:0/2::gentoo  USE="acl audit bpf cryptsetup curl dns-over-tls elfutils gcrypt gnutls http idn iptables kmod lz4 lzma openssl pam pcre pkcs11 (policykit) qrcode seccomp sysv-utils xkb zstd -apparmor -boot -cgroup-hybrid -fido2 -homed -importd -kernel-install -pwquality -resolvconf -secureboot (-selinux) (-split-usr) -test -tpm -ukify -vanilla" ABI_X86="32 (64) (-x32)" PYTHON_SINGLE_TARGET="python3_12 -python3_10 -python3_11 -python3_13" 0 KiB
Comment 3 Mike Gilbert gentoo-dev 2024-12-14 21:49:49 UTC
"System Logger Daemon" sounds like a syslog daemon.

Do you have a syslog daemon installed/enabled? That's not a typical thing for systemd.
Comment 4 Mike Gilbert gentoo-dev 2024-12-14 21:52:57 UTC
If unsure, please run this:

grep -RlF "System Logger Daemon" /lib/systemd
Comment 5 Mike Gilbert gentoo-dev 2024-12-14 21:57:08 UTC
Comment on attachment 914014 [details]
emerge --info output, greater than copy-paste limit

As an aside, you should really make use of package.use instead of declaring all your USE flags globally.
Comment 6 Paul Osmialowski 2024-12-14 22:36:06 UTC
$ grep -RlF "System Logger Daemon" /lib/systemd
/lib/systemd/system/syslog-ng@.service

Should different logger be used on systemd machines?
Comment 7 Mike Gilbert gentoo-dev 2024-12-15 00:52:45 UTC
Most people do not run a syslog daemon at all. The journal suffices.
Comment 8 Mike Gilbert gentoo-dev 2024-12-15 01:00:44 UTC
What version of app-admin/syslog-ng is installed?
Comment 9 Mike Gilbert gentoo-dev 2024-12-15 01:36:20 UTC
I can reproduce the issue with app-admin/syslog-ng-4.8.1-r1.

The daemon simply doesn't respond to the SIGTERM that systemd sends it when you stop the service.
Comment 10 Paul Osmialowski 2024-12-15 09:33:10 UTC
(In reply to Mike Gilbert from comment #8)
> What version of app-admin/syslog-ng is installed?

It's app-admin/syslog-ng-4.8.1::gentoo, but apparenty 4.8.1-r1 is also affected.
Comment 11 Paul Osmialowski 2024-12-15 09:34:55 UTC
(In reply to Mike Gilbert from comment #7)
> Most people do not run a syslog daemon at all. The journal suffices.

Does it work properly with things like sshd server and fail2ban?
Comment 12 Mike Gilbert gentoo-dev 2024-12-15 14:47:23 UTC
(In reply to Paul Osmialowski from comment #11)
> (In reply to Mike Gilbert from comment #7)
> > Most people do not run a syslog daemon at all. The journal suffices.
> 
> Does it work properly with things like sshd server and fail2ban?

Yes.
Comment 13 Mike Gilbert gentoo-dev 2024-12-15 20:50:03 UTC
The daemon seems to be stuck in a loop calling epoll_wait() without actually doing anything.

I think this issue should probably be reported upstream.

strace output when I send SIGTERM:

epoll_wait(3, 0x7ffe72e84040, 8, -1)    = -1 EINTR (Interrupted system call)
--- SIGTERM {si_signo=SIGTERM, si_code=SI_USER, si_pid=6073, si_uid=0} ---
getpid()                                = 6098
write(7, "\1\0\0\0\0\0\0\0", 8)         = 8
rt_sigreturn({mask=[]})                 = -1 EINTR (Interrupted system call)
epoll_wait(3, [{events=EPOLLIN, data={u32=2471511704, u64=140718485030552}}], 8, -1) = 1
read(7, "\1\0\0\0\0\0\0\0", 8)          = 8
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
epoll_wait(3, 


strace output when I run "syslog-ng-ctl stop":

epoll_wait(3, [{events=EPOLLIN, data={u32=514385152, u64=93884204508416}}], 8, -1) = 1
accept(113, {sa_family=AF_UNIX}, [1024 => 2]) = 117
fcntl(117, F_GETFD)                     = 0
fcntl(117, F_SETFD, FD_CLOEXEC)         = 0
fcntl(117, F_GETFL)                     = 0x2 (flags O_RDWR)
fcntl(117, F_SETFL, O_RDWR|O_NONBLOCK)  = 0
setsockopt(117, SOL_SOCKET, SO_OOBINLINE, [1], 4) = 0
epoll_ctl(3, EPOLL_CTL_ADD, 117, {events=EPOLLIN, data={u32=514892784, u64=93884205016048}}) = 0
epoll_wait(3, [{events=EPOLLIN, data={u32=514892784, u64=93884205016048}}], 9, -1) = 1
read(117, "STOP\n", 128)                = 5
epoll_ctl(3, EPOLL_CTL_MOD, 117, {events=EPOLLOUT, data={u32=514892784, u64=93884205016048}}) = 0
epoll_wait(3, [{events=EPOLLOUT, data={u32=514892784, u64=93884205016048}}], 9, -1) = 1
write(117, "OK Shutdown initiated\n.\n", 24) = 24
epoll_ctl(3, EPOLL_CTL_MOD, 117, {events=EPOLLIN, data={u32=514892784, u64=93884205016048}}) = 0
epoll_wait(3, [{events=EPOLLIN|EPOLLHUP, data={u32=514892784, u64=93884205016048}}], 9, -1) = 1
read(117, "", 128)                      = 0
epoll_ctl(3, EPOLL_CTL_DEL, 117, 0x7ffe72e8404c) = 0
close(117)                              = 0
epoll_wait(3,
Comment 14 Balazs Scheidler 2025-01-23 16:58:42 UTC
Original syslog-ng author here (and now working on the axosyslog fork). 

This problem is caused by missing this fix from ivykis 0.43.2

As I can't post URLs as I just registered my gentoo bugzilla account, the fix is
the #34 PR in the ivykis repository, which is on github.

The commit id is: ffccf089fcd0e928405214cf8b21934cfcc38300

Please upgrade ivykis to 0.43.2 and this should resolve itself. 0.43.2 only contained that single fix over 0.43.1
Comment 15 Mike Gilbert gentoo-dev 2025-01-23 17:06:58 UTC
Thanks for that hint!
Comment 16 Paul Osmialowski 2025-01-23 20:08:07 UTC
Yes, I have dev-libs/ivykis-0.43.1 installed on that machine. But sadly, I also got rid of the syslog-ng from it, as the situation was too annoying (fighting with other bug I needed to reboot it frequently). I assume, this bug report qualifies for closure.
Comment 17 Ed Santiago 2025-01-23 20:36:17 UTC
(In reply to Balazs Scheidler from comment #14)

> Please upgrade ivykis to 0.43.2 and this should resolve itself. 0.43.2 only
> contained that single fix over 0.43.1

Confirmed, and my /var/log is now rapidly filling up with almost two months of missed log entries. Thank you!