Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 542306 - app-admin/rsyslog: Block >=sys-apps/util-linux-2.26 due to new RFC5424 default format
Summary: app-admin/rsyslog: Block >=sys-apps/util-linux-2.26 due to new RFC5424 defaul...
Status: RESOLVED OBSOLETE
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Server (show other bugs)
Hardware: All Linux
: Normal critical (vote)
Assignee: Ultrabug
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-03-05 23:21 UTC by Thomas Deutschmann (RETIRED)
Modified: 2015-03-15 21:01 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Deutschmann (RETIRED) gentoo-dev 2015-03-05 23:21:41 UTC
tl;dr
=====
Please add a blocker for >=sys-apps/util-linux-2.26 to every app-admin/rsyslog ebuild (I think a revbump is required to propagate this block), because rsyslog's imuxsock isn't compatible with RFC5424 which was added in util-linux-2.26 and is also logger's new default format in v2.26.
If we won't block util-linux-2.26 we cannot guarantee reliable logging through sockets (but upstream also confirmed that remote logging is affected).


Since app-admin/rsyslog-8.7.0 we have test suite support for rsyslog in Gentoo.
This uncovered a hugh problem with sys-apps/util-linux-2.26 which should affect many syslogger (because I only care about sys-apps/rsyslog I won't fill bugs against other syslog implementation but I will CC other maintainers so they will be notified and can check):

util-linux got RFC5424 support in Juli 2014 [1], which became also the new default log format.
The first released version including these changes is util-linux-2.26.

RFC5424 isn't new (from 2009) but because it did not materialize in time so at least for rsyslog this change was unexpected [2]: rsyslog's imuxsock doesn’t support RFC5424 yet.

While investigating the problem with rsyslog and util-linux authors multiple other problems related to util-linux RFC5424 implementation and regressions in v2.26 were found [3].

Until rsyslog get full RFC5424 support and util-linux RFC5424 support is working as expected, upstream does not recommend to run rsyslog with util-linux using RFC5424 per default [4].


See also:
=========
[1] https://github.com/karelzak/util-linux/commit/4de2e8a03859aaab2c25dc98f33409cd28de6acc

[2] http://thread.gmane.org/gmane.comp.sysutils.rsyslog/16531/focus=16534

[3] https://github.com/karelzak/util-linux/issues/created_by/rgerhards

[4] http://thread.gmane.org/gmane.comp.sysutils.rsyslog/16531/focus=16567


Reproducible: Always




Setting severity to critical because using rsyslog with util-linux-2.26 could cause data loss (invalid log data).
Comment 1 Thomas Deutschmann (RETIRED) gentoo-dev 2015-03-05 23:24:29 UTC
CC'ing Michael Sterrett (app-admin/syslog-ng maintainer)
Comment 2 Thomas Deutschmann (RETIRED) gentoo-dev 2015-03-05 23:31:43 UTC
CC'ing systemd team so they can verify if journald is also affected (seems like the current systemd version in tree is lacking of RFC5424 support)
Comment 3 Mike Gilbert gentoo-dev 2015-03-05 23:39:42 UTC
(In reply to Thomas D. from comment #2)
> CC'ing systemd team so they can verify if journald is also affected (seems
> like the current systemd version in tree is lacking of RFC5424 support)

systemd-journald only supports ingestion of local syslog messages (/dev/log), so I don't see how a change to a format that is only used when sending messages over the network would be relevant.
Comment 4 Thomas Deutschmann (RETIRED) gentoo-dev 2015-03-05 23:44:58 UTC
Every application using "logger" to log will log in new RFC5424 format.
So if systemd is listening in /dev/log, journald will receive syslog messages in RFC5424 format. If systemd don't know the format, you will log at least garbage...
Comment 5 Mike Gilbert gentoo-dev 2015-03-05 23:48:56 UTC
(In reply to Thomas D. from comment #4)
> Every application using "logger" to log will log in new RFC5424 format.
> So if systemd is listening in /dev/log, journald will receive syslog
> messages in RFC5424 format. If systemd don't know the format, you will log
> at least garbage...

According to the manual page, logger just calls syslog(3) by default.

Running "logger foobar" works just fine here with util-linux-2.26; I see "logger[28971]: foobar" in my journal.

Can you provide a simple way to reproduce this issue?
Comment 6 Thomas Deutschmann (RETIRED) gentoo-dev 2015-03-06 12:50:31 UTC
Your test is valid BUT:

1) See the following acknowledged bugs in util-linux-2.26:

- https://github.com/karelzak/util-linux/issues/163

- https://github.com/karelzak/util-linux/issues/161

- https://github.com/karelzak/util-linux/issues/159

- https://github.com/karelzak/util-linux/issues/157


2) Your syslog configuration is important: If you are only interested in the message itself you will probably don't hit any problems. But if you are interested in all the other information/details you will notice missing/invalid data. I was told that the best thing to test across different implementations is to use a JSON-based output.

If for example systemd can handle RFC5424 format, all fields should be populated with the right values. If not, you will see empty/missing fields or empty/invalid values.
Comment 7 Mike Gilbert gentoo-dev 2015-03-06 16:17:57 UTC
(In reply to Thomas D. from comment #6)

Thanks, the JSON suggestion is a good one. Let's continue this conversation on bug 542368 if you don't mind.
Comment 8 Thomas Deutschmann (RETIRED) gentoo-dev 2015-03-09 13:46:15 UTC
As posted in bug 542368#3:

We should add a general block for =sys-apps/util-linu-2.26.

This will solve any problems logger from util-linux-2.26 is causing for any syslogger due to the changed default format.

util-linux-2.26.1 should be released soon which will revert these changes. The new behavior will be the behavior from <util-linux-2.26.

The changes are already present in util-linux git master (can be tested via =sys-apps/util-linu-9999).

For more details please see bug 542368 comment #3.
Comment 9 Thomas Deutschmann (RETIRED) gentoo-dev 2015-03-15 21:01:29 UTC
Closing, sys-apps/util-linux-2.26.1 which includes all the fixes is now available and in tree.