Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 501982 - app-admin/rsyslog-7.6.x - New default configuration
Summary: app-admin/rsyslog-7.6.x - New default configuration
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Server (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Ultrabug
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 490744 501988
  Show dependency tree
 
Reported: 2014-02-21 11:34 UTC by Thomas Deutschmann
Modified: 2014-05-22 11:23 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
New default configuration (rsyslog.conf,2.86 KB, text/plain)
2014-05-06 16:44 UTC, Thomas Deutschmann
Details
New default configuration (rsyslog.conf,3.17 KB, text/plain)
2014-05-09 08:43 UTC, Thomas Deutschmann
Details
New default configuration (rsyslog.conf,3.62 KB, text/plain)
2014-05-15 23:42 UTC, Thomas Deutschmann
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Deutschmann gentoo-dev Security 2014-02-21 11:34:45 UTC
Hi,

for the upcoming app-admin/rsyslog-7.6.x series I am proposing the following changes:

1) Include additional configurations from "/etc/rsyslog.d" *after* we set up our defaults.

Our current default configuration loads additional configurations from "/etc/rsyslog.d" before we set up the default logs ("/var/log/{messages,syslog,...}"). This could be a problem if any custom configuration in "/etc/rsyslog.d" will discard a message we want in our default logs. A good default configuration should prevent this problem.


2) Update the syntax.

We are still using syntax from 7.2. A good default configuration should represent the current syntax from upstream, not? So let's update the default configuration.


3) Should we create an additional input socket in "/var/empty/dev/log" by default? See bug 490744.

Reproducible: Always
Comment 1 Ultrabug gentoo-dev 2014-05-04 15:01:21 UTC
Hi Thomas,

I'm fine with the proposed modifications, do you have a relevant config to propose which acheives the listed goals please ?
Comment 2 Thomas Deutschmann gentoo-dev Security 2014-05-06 16:44:12 UTC
Created attachment 376492 [details]
New default configuration

Here's the new configuration:

- Completely rewritten in the new syntax (based on RainerScript)

- Per default, we longer create the "/var/log/syslog" target. Directive is still in the configuration, so users who need this can re-enable it without a hassle

- We will now create another Unix socket in "/var/empty/dev/log" per default (closes bug 490744)

- Additional configurations from "/etc/rsyslog.d" are now included at the end to make our default configuration more robust

- omusrmsg log target is now guarded by a time-rate-limit
Comment 3 Thomas Deutschmann gentoo-dev Security 2014-05-09 08:43:22 UTC
Created attachment 376608 [details]
New default configuration

I noticed that I uploaded a wrong revision of the new default configuration... this is the current revision (commented /var/log/syslog directive to allow the user to re-enable the deprecated log file without a hassle was missing).
Comment 4 Thomas Deutschmann gentoo-dev Security 2014-05-12 13:39:24 UTC
Question regarding #1 (Including /etc/rsyslog.d/*.conf at the end):

The change has one downside: Imagine you are running a firewall. You don't want to clutter up "kern.log" or "messages" with logs from iptables for example.

The only way to prevent this is adding a rule before the default rules, which will discard these messages after processing.

With the new default configuration this isn't possible anymore without touching "/etc/rsyslog.conf" itself. 

So I am not sure if we should revert this change, e.g. move the include statement above the default rules again.

One could argue that if you put something in "/etc/rsyslog.d" that you do this on purpose. If this will affect some "default" log files, well... that's what you expected.

...I am currently tending to revert this change. Other opinions?
Comment 5 Marcin Mirosław 2014-05-12 14:02:27 UTC
/etc/rsyslog.d/myrules.pre.conf
/etc/rsyslogd/myanotherrules.post.conf

But I'm not sure if it follow KISS rule:)
Comment 6 Thomas Deutschmann gentoo-dev Security 2014-05-12 14:18:21 UTC
I had the same idea but how to we deal with packages which only install a *.conf file into "/etc/rsyslog.d"? I think if we will change something, we have to take care about backward compatibility, not?

So one idea would be:

$IncludeConfig /etc/rsyslog.d/*.pre.conf
$IncludeConfig /etc/rsyslog.d/*.conf

<our default rules/actions>

$IncludeConfig /etc/rsyslog.d/*.post.conf

This would allow us to be compatible with current installations (people expecting that their *.conf files will affect default rules won't have to change something; people who explicit want a pre/post rule could do that, now).

The ebuild could scan "/etc/rsyslog.d" in postinst phase to see if there are only (now old) *.conf and no *.{pre,post}.conf files (indicator for an "old" installation) and show a message to the user telling him/her about this change.
Comment 7 Thomas Deutschmann gentoo-dev Security 2014-05-12 14:20:46 UTC
Mh... "*.conf" will also match for "*.{pre,post}.conf" files... :/

Well, it is just the suffix, so we could add a new one like

".post-rule" or ".pre-rule" or something else...
Comment 8 Marcin Mirosław 2014-05-12 15:00:07 UTC
".post-rule" or ".pre-rule" - I prefer suffixes .conf, .cfg etc.
What about extending your proposition: do check in pkg_pretend(), if  there is any old (without pre,post) *.conf file in /etc/rsyslog.d then do die() with information what user should do. User should do simple mv, it shouldn't be very annoying. It will force changing suffix format without unattended breaking configuration.
Comment 9 Thomas Deutschmann gentoo-dev Security 2014-05-12 15:48:55 UTC
(In reply to Marcin Mirosław from comment #8)
> ".post-rule" or ".pre-rule" - I prefer suffixes .conf, .cfg etc.

In general I also prefer *.conf... but would a different suffix really be a problem?

The "local" service for example expects "*.{start,stop}" files in "/etc/local.d" to allow scripts which should only execute on start/stop...


> What about extending your proposition: do check in pkg_pretend(), if  there
> is any old (without pre,post) *.conf file in /etc/rsyslog.d then do die()
> with information what user should do. User should do simple mv, it shouldn't
> be very annoying. It will force changing suffix format without unattended
> breaking configuration.

So you wouldn't include any *.conf file (without pre/post) per default (e.g. you would force users upgrading to v7.6 to adapt the new naming schema)?

I would have said you cannot do that. Think of all the packages which would need to be updated... but it seems like nothing is using "/etc/rsyslog.d" at the moment:

find /usr/portage/ -type f -name "*.ebuild" | xargs grep -F 'rsyslog.d'

...I am tending to agree with you:

$IncludeConfig /etc/rsyslog.d/*.pre.conf

<our default rules/actions>

$IncludeConfig /etc/rsyslog.d/*.post.conf

Documentation would recommend to use *.post.conf per default/

In pkg_postinst we would run a check like

unset RSYSLOG_OLD_CONF_FILES
declare -a RSYSLOG_OLD_CONF_FILES
local i=0 RSYSLOG_OLD_CONF_FILES=
while IFS= read -r -u 3 -d $'\0' RSYSLOG_OLD_CONF_FILE; do
	RSYSLOG_OLD_CONF_FILES[i++]="$RSYSLOG_OLD_CONF_FILE"
done 3< <(find "${config[data_path]}" -maxdepth 1 -name '*.dat' -print0 2>/dev/null)

if [[ ${#RSYSLOG_OLD_CONF_FILES[@]} -gt 0 ]] ; then
	ewarn "Beginning with rsyslog-7.6 we changed the way we are including additional configuration files."
	ewarn "See bug #501982 for details."
	ewarn ""
	ewarn "You have to adapt the new naming schema for the following files:
	ewarn ""
	
	RSYSLOG_OLD_CONF_FILE=
	for RSYSLOG_OLD_CONF_FILE in "${RSYSLOG_OLD_CONF_FILES[@]}" ; do
		ewarn "  - ${RSYSLOG_OLD_CONF_FILE}"
	done
	
	ewarn ""
	ewarn "To keep the old behavior, just change the suffix from \".conf\" to \".pre.conf\""
	ewarn "If you don't do that, these configuration files won't be included anymore."
fi

unset RSYSLOG_OLD_CONF_FILE RSYSLOG_OLD_CONF_FILES


Sounds good for me!

@UltraBug: Should I update the configuration and ebuilds?
Comment 10 Thomas Deutschmann gentoo-dev Security 2014-05-12 15:51:36 UTC
> done 3< <(find "${config[data_path]}" -maxdepth 1 -name '*.dat' -print0 2>/dev/null)

must be

> done 3< <(find "/etc/rsyslog.d" -maxdepth 1 -type f \( -iname "*.conf" ! -iname "*.pre.conf" ! -iname "*.post.conf" \) -print0 2>/dev/null)
Comment 11 Thomas Deutschmann gentoo-dev Security 2014-05-12 16:04:26 UTC
...and I would add a possibility to suppress this check for people running their own configuration and therefore don't care about this warning:

They could create "/etc/rsyslog.d/.I_KNOW_WHAT_I_AM_DOING" to bypass the check.
Comment 12 Marcin Mirosław 2014-05-13 15:10:51 UTC
(In reply to Thomas D. from comment #9)

> I would have said you cannot do that. Think of all the packages which would
> need to be updated... but it seems like nothing is using "/etc/rsyslog.d" at
> the moment:
> 
> find /usr/portage/ -type f -name "*.ebuild" | xargs grep -F 'rsyslog.d'
 
> ...I am tending to agree with you:

Great;)

[...] 
> 	ewarn "To keep the old behavior, just change the suffix from \".conf\" to
> \".pre.conf\""
> 	ewarn "If you don't do that, these configuration files won't be included
> anymore."
> fi

From my personal experience I can say it's easy to miss ewarn...:/ (I know it is my fault) Good news is that etc-update will be also a kind of reminder that something is going to change.

Which template will be used in your conf file (for omfile), with or without high precision timestamp? And why not?;)
Comment 13 Thomas Deutschmann gentoo-dev Security 2014-05-13 15:29:25 UTC
(In reply to Marcin Mirosław from comment #12)
> From my personal experience I can say it's easy to miss ewarn...:/ (I know
> it is my fault) Good news is that etc-update will be also a kind of reminder
> that something is going to change.

Checkout "app-portage/elogv" and "PORTAGE_ELOG_*" stuff.

 
> Which template will be used in your conf file (for omfile), with or without
> high precision timestamp? And why not?;)

See https://github.com/Whissi/gentoo-overlay/blob/master/app-admin/rsyslog/files/7-stable/rsyslog.conf#L54 - it is "RSYSLOG_TraditionalFileFormat".

Why? Well, I don't really have reason. The "RSYSLOG_TraditionalFileFormat" template is the current default. I personally don't see a benefit from using high precisions timestamps on normal systems. But I wouldn't have a problem if we would decide to change template to "RSYSLOG_FileFormat" per default...
Comment 14 Thomas Deutschmann gentoo-dev Security 2014-05-15 23:42:07 UTC
Created attachment 377040 [details]
New default configuration

Here's the new default configuration, which will include "*.{pre,post}.conf" files.
Comment 15 Ultrabug gentoo-dev 2014-05-16 14:17:13 UTC
(In reply to Thomas D. from comment #14)
> Created attachment 377040 [details]
> New default configuration
> 
> Here's the new default configuration, which will include "*.{pre,post}.conf"
> files.

I'm not a big fan of this solution I must say.

I'd prefer to set the gentoo defaults in a /etc/rsyslog.d/50-default.conf so that the users can make their own choices of config placement (50 is a good in-between choice)

We'd then just have to add a warning message saying we added this file et voila

Would that be okay ?
Comment 16 Thomas Deutschmann gentoo-dev Security 2014-05-16 17:24:11 UTC
Like said in IRC, I agree with Ultrabug (KISS).

I am not attaching the new configuration, because now we have 2 files.

/etc/rsyslog.conf:
https://github.com/Whissi/gentoo-overlay/blob/172c517aab667188db50ec290ecefd1f6cd9d1cb/app-admin/rsyslog/files/7-stable/rsyslog.conf


/etc/rsyslog.d/50-default.conf:
https://github.com/Whissi/gentoo-overlay/blob/172c517aab667188db50ec290ecefd1f6cd9d1cb/app-admin/rsyslog/files/7-stable/default.conf
Comment 17 Ultrabug gentoo-dev 2014-05-22 11:23:28 UTC
+*rsyslog-7.6.3 (22 May 2014)
+
+  22 May 2014; Ultrabug <ultrabug@gentoo.org> -rsyslog-7.4.3.ebuild,
+  -rsyslog-7.4.6.ebuild, -rsyslog-7.4.7.ebuild, +rsyslog-7.6.3.ebuild,
+  +files/7-stable/50-default.conf, +files/7-stable/README.gentoo,
+  +files/7-stable/bugfix_52.patch, +files/7-stable/bugfix_73.patch,
+  +files/7-stable/fix-omruleset-default-value.patch,
+  +files/7-stable/rsyslog.conf, +files/7-stable/rsyslog.confd-r1,
+  +files/7-stable/rsyslog.initd-r1, +files/7-stable/rsyslog.logrotate-r1,
+  metadata.xml:
+  version bump fix #501988 #501982 #501986 #490744 thx to @Whissi, drop old
+  versions
+