Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 292919 - app-admin/rsyslog-5.2.0 version bump
Summary: app-admin/rsyslog-5.2.0 version bump
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: High enhancement (vote)
Assignee: Tiziano Müller (RETIRED)
URL: http://www.rsyslog.com/Downloads-req-...
Whiteboard:
Keywords:
: 321537 (view as bug list)
Depends on:
Blocks:
 
Reported: 2009-11-12 14:39 UTC by Martin Böcher
Modified: 2010-10-21 07:39 UTC (History)
8 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge --info for myinstallation of rsyslog (emerge-info-rsyslog.txt,5.04 KB, text/plain)
2010-08-28 15:05 UTC, Mike Nerone
Details
Adds a reload action to the init script (much less disruptive way to re-open logs) and makes logrotate use it. (Obsolete - alternate functionally identical patch applied in overlay.) (0001-Add-reload-action-to-init-script.patch,3.47 KB, patch)
2010-09-10 21:38 UTC, Mike Nerone
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Böcher 2009-11-12 14:39:29 UTC
a new version of rsyslog is out. latest stable version: 5.2.0
a new ebuild/version bump is needed, thanks.

Reproducible: Always
Comment 1 Sebastian Luther (few) 2009-11-12 16:56:19 UTC
@reporter: Please include the full package name and the new version in the summary next time.
Comment 2 svrmarty 2009-12-28 16:53:34 UTC
please add a logrotate script (eg. from debian)

Comment 3 svrmarty 2009-12-28 17:47:31 UTC
which includes /var/log/syslog to rotate
Comment 4 Ultrabug gentoo-dev 2010-05-26 10:14:06 UTC
I proposed an ebuild for rsyslog-5.4.0 on bug #321537. I hope it helps.
Comment 5 Jeroen Roovers (RETIRED) gentoo-dev 2010-05-26 16:50:08 UTC
*** Bug 321537 has been marked as a duplicate of this bug. ***
Comment 6 Opportunist 2010-08-01 18:46:29 UTC
rsyslog 5.5.6 with great performance enhancements is available at homepage ( http://www.rsyslog.com/rsyslog-5-5-6-devel-released/ )
Comment 7 chris 2010-08-05 14:42:57 UTC
Looking for an update to rsyslog as well and found this bug entry.
Looks like even the stable build of rsyslog is up to 4.6.4, compared to the current 3.2.0 in portage (3.22 in ~).

Are there plans to update this in portage anytime soon?  Need to know if I should change my plans.
Comment 8 Ultrabug gentoo-dev 2010-08-05 17:13:22 UTC
According to upstream, here are the current stable releases by branch :
v5 stable: 5.4.0 [2010-03-08]
v4 stable: 4.6.4 [2010-08-05]
v3 stable: 3.22.2 [2010-08-05]

I pushed all those ebuilds bumps into scarabeus overlay which is accessible via layman. Please, feel free to test them and give some feedback. I think they still need some final touch/work and I'll do my best to do it by the end of next week.

Regards
Comment 9 Opportunist 2010-08-06 09:46:45 UTC
thank you, good work!

rsyslog 5.4.0 works perfectly on my AMD64, very good software
Comment 10 chris 2010-08-06 13:20:03 UTC
Much appreciated!  I'll report back with any issues I run into.
Comment 11 Mike Nerone 2010-08-25 15:50:53 UTC
I'd like to ask the Gentoo devs to devote a bit of extra time to maintenance of the rsyslog package. rsyslog has become the de facto standard syslogger in the Linux community, and most other major distributions have already switched to it (1,2) (those that haven't are preparing to (3)). Most of the other distros are on v4, while upstream has considered the 5 series stable for nearly a year. Yet, in the main portage tree, the latest release even *available* is v3.

rsyslog should not be an afterthought in Gentoo - as a drop-in but far superior replacement for sysklogd, it has completely supplanted it. I believe we should actually give it default status in the handbook wording, and perhaps even make a virtual/syslogger package (or similar) preferring it.

[1] http://fedoraproject.org/wiki/Releases/FeatureRsyslog
[2] http://wiki.debian.org/Rsyslog
[3] http://bugs.archlinux.org/task/12314

Basically, consider this a long-winded vote for some prioritization of this version bump and continued love for rsyslog. :)
Comment 12 Ultrabug gentoo-dev 2010-08-25 16:10:43 UTC
I agree with you and I'm really confident that we'll get this done quite soon.

As a sidenote, according to upstream we should also be using >=dev-libs/librelp-0.1.3 with RELP-enabled rsyslog, the stabilization request is ongoing (see bug #333595)

Also, I did push some updates of my proposed ebuilds on scarabeus overlay (via layman), all feedback about it would be appreciated so it gets more chance to enter portage.

Thanks
Comment 13 Mike Nerone 2010-08-27 21:00:06 UTC
BTW, the rsyslog ebuild in the scarabeus overlay won't build because it deps on virtual/postgresql-base. That virtual package no longer exists (ever since the old-style postgresql & libpq packages were removed from the tree, there's only one postgresql-base, so no need for a virtual).
Comment 14 Mike Nerone 2010-08-27 21:01:39 UTC
Ahem - obviously I meant it won't build with USE=postgres.
Comment 15 Mike Nerone 2010-08-28 14:59:33 UTC
I've installed app-admin/rsyslog-5.4.0 from the scarabeus overlay, and I do have some feedback (in addition to the USE=postgres thing above).

1. Set up a config dir

The default config under Gentoo should set up a config dir, such as /etc/rsyslog.d, so that other packages can drop (r)syslog configs in it. This is similar to what other packages do ((/etc/)cron.d, logrotate.d, profile.d, xinet.d, etc.), and is also what other distros are doing. All that's needed is this in the main rsyslog.conf:

    $IncludeConfig /etc/rsyslog.d/

In fact, if it were up to me, I would have that be the *only* (functional) line in the main rsyslog.conf, and split the default config into /etc/rsyslog.d/10-gentoo-default-modules and /etc/rsyslog.d/50-gentoo-default. This would make it easy for users to insert their own configs before or after the default configs as necessary to achieve the behavior they desire without complicated package upgrades, while at the same time not adversely affecting out-of-the-box-config users.

3. Overlay-specific annoyance: the scarabeus overlay contains a couple of packages (dev-embedded/avra-1.3.0 and dev-lang/yasm-1.1.0) with stable keywords. This (particularly yasm) makes it inconvenient to add the overlay when all I want is the rsyslog stuff. I have to package.mask yasm-1.1.0 to avoid the upgrade and hope I notice when yasm-1.1.0 stabilizes in the main portage tree so I can unmask it. Even then, I have to fear that the stable ebuild in the overlay will shadow the stable ebuild in the main tree (since overlays typically have higher priority than the main tree). Due to the concerns, IMHO, overlays should never have stable ebuilds.
Comment 16 Mike Nerone 2010-08-28 15:05:13 UTC
Created attachment 245143 [details]
emerge --info for myinstallation of rsyslog

The previous request-for-enhancement not withstanding, I've gone ahead and put rsyslog as packaged in the scarabeus overlay into production on my boxes, and it is functioning perfectly so far. Attached is the output of "emerge --info rsyslog" on one of my machines.
Comment 17 Ultrabug gentoo-dev 2010-08-30 10:35:27 UTC
Hi, and thanks for all your feedback.

(In reply to comment #13)
> BTW, the rsyslog ebuild in the scarabeus overlay won't build because it deps on
> virtual/postgresql-base. That virtual package no longer exists (ever since the
> old-style postgresql & libpq packages were removed from the tree, there's only
> one postgresql-base, so no need for a virtual).

I replaced the virtual by dev-db/postgresql-base, thanks (not yet commited)

(In reply to comment #15)
> 1. Set up a config dir
> 
> The default config under Gentoo should set up a config dir, such as
> /etc/rsyslog.d, so that other packages can drop (r)syslog configs in it. This
> is similar to what other packages do ((/etc/)cron.d, logrotate.d, profile.d,
> xinet.d, etc.), and is also what other distros are doing. All that's needed is
> this in the main rsyslog.conf:

The difference here between rsyslog and packages like logrotate is that other packages do write their own bit of conf in those package.d directories. In the case of rsyslog, this directory is only used to split rsyslog's conf.

What would you think of a more "apache2-like" way to handle this, like :

/etc/rsyslog : main rsyslog conf dir
/etc/rsyslog/rsyslog.conf : main config file
/etc/rsyslog/include.d : keepdir for user convenience splitted configuration

NB: apart from the include.d directory, that's how it is handled by the latest revision of the ebuild in the scarabeus overlay.

>     $IncludeConfig /etc/rsyslog.d/
> 
> In fact, if it were up to me, I would have that be the *only* (functional) line
> in the main rsyslog.conf, and split the default config into
> /etc/rsyslog.d/10-gentoo-default-modules and /etc/rsyslog.d/50-gentoo-default.
> This would make it easy for users to insert their own configs before or after
> the default configs as necessary to achieve the behavior they desire without
> complicated package upgrades, while at the same time not adversely affecting
> out-of-the-box-config users.

I think it is best for user convenience to stick with the default upstream way of handling rsyslog config in a unique and fairly simple file.
I've already added some more config directives to the default config file we provide with the ebuild so I could aswell add a comment about how to include config like:

# $IncludeConfig /etc/rsyslog/include.d/ # include more specific config files

What would you think of that ?

Regards
Comment 18 Ultrabug gentoo-dev 2010-09-03 17:09:12 UTC
Commited to scarabeus overlay :

- postgresql dependency fix
- /etc/rsyslog directory tree as proposed in comment #17

Thanks.
Comment 19 Mike Nerone 2010-09-07 06:04:41 UTC
I find forever having to dispatch config changes to a packaged file to preserve a single custom line during upgrades is pretty annoying, especially when the app in question provides a clean way to avoid that. If we placed the "$IncludeConfig /etc/rsyslog/include.d/" in the Gentoo default config, it doesn't hurt the standard functionality in any way (the directory is empty by default), solves that problem.

Also, if it's not on, maintainers of other packages will certainly not make use of the include dir. We're missing an opportunity for other packages to configure their own logging for the user (enabled, presumably, by an rsyslog USE flag), Having it on would enhance not only customization for the end-user, but extend those benefits to other packages, as well.

I feel pretty strongly that the directory-include, wherever you decide is the best location for it, should be on, and I don't see a downside.
Comment 20 Ultrabug gentoo-dev 2010-09-09 17:32:52 UTC
(In reply to comment #19)
> I find forever having to dispatch config changes to a packaged file to preserve
> a single custom line during upgrades is pretty annoying, especially when the
> app in question provides a clean way to avoid that. If we placed the
> "$IncludeConfig /etc/rsyslog/include.d/" in the Gentoo default config, it
> doesn't hurt the standard functionality in any way (the directory is empty by
> default), solves that problem.
> 
> Also, if it's not on, maintainers of other packages will certainly not make use
> of the include dir. We're missing an opportunity for other packages to
> configure their own logging for the user (enabled, presumably, by an rsyslog
> USE flag), Having it on would enhance not only customization for the end-user,
> but extend those benefits to other packages, as well.
> 
> I feel pretty strongly that the directory-include, wherever you decide is the
> best location for it, should be on, and I don't see a downside.
> 

Alright so we agree :) Added the default inclusion of the files placed in /etc/rsyslog/include.d/ :

$IncludeConfig /etc/rsyslog/include.d/*

Commited to scarabeus overlay !
Comment 21 Mike Nerone 2010-09-10 01:26:20 UTC
Sweet! One more thing. :)

Anyone putting in a custom config is quite likely to want to discard messages they've already handled to prevent them from getting to the standard logs at all so as to avoid unnecessary duplication. With the current location of the $IncludeConfig line, there's no opportunity to do that short of customizing the file, thereby introducing the aforementioned annoyance. I think putting the $IncludeConfig line *before* the standard rules (right after the $ModLoad's) provides the flexibility to do this without hurting anything (if someone *wants* the duplication, all they have to do is *not* discard the message - i.e. don't add a "~" rule).
Comment 22 Ultrabug gentoo-dev 2010-09-10 09:57:09 UTC
(In reply to comment #21)
> Sweet! One more thing. :)
> 
> Anyone putting in a custom config is quite likely to want to discard messages
> they've already handled to prevent them from getting to the standard logs at
> all so as to avoid unnecessary duplication. With the current location of the
> $IncludeConfig line, there's no opportunity to do that short of customizing the
> file, thereby introducing the aforementioned annoyance. I think putting the
> $IncludeConfig line *before* the standard rules (right after the $ModLoad's)
> provides the flexibility to do this without hurting anything (if someone
> *wants* the duplication, all they have to do is *not* discard the message -
> i.e. don't add a "~" rule).
> 

Sorry, my bad indeed, this was what I was thinking too, commited !
Comment 23 Mike Nerone 2010-09-10 21:38:10 UTC
Created attachment 246760 [details, diff]
Adds a reload action to the init script (much less disruptive way to re-open logs) and makes logrotate use it.

(Obsolete - alternate functionally identical patch applied in overlay.)

The logrotate script is currently configures to do a full restart of rsyslog. All that's needed is for rsyslog to re-open log files, which rsyslog is happy to do when it receives a HUP. See http://blog.gerhards.net/2008/10/new-rsyslog-hup-processing.html . This is also good because rsyslog does *not* re-read its config (god forbid you might be in the middle of editing those configs when logrotate runs). The attached patch adds a "reload" action to the init script, per the common convention, and configures logrotate to make use of that.

Note: since you use git, I git-formatted the patch so you can "git am" it. I wasn't sure whether I'm supposed to include the manifest update - I did so.
Comment 24 Ultrabug gentoo-dev 2010-09-11 12:29:51 UTC
(In reply to comment #23)
> Created an attachment (id=246760) [details]
> Adds a reload action to the init script (much less disruptive way to re-open
> logs) and makes logrotate use it.
> 

Thanks for submitting a patch !

> The logrotate script is currently configures to do a full restart of rsyslog.
> All that's needed is for rsyslog to re-open log files, which rsyslog is happy
> to do when it receives a HUP. See
> http://blog.gerhards.net/2008/10/new-rsyslog-hup-processing.html . This is also
> good because rsyslog does *not* re-read its config (god forbid you might be in
> the middle of editing those configs when logrotate runs). The attached patch
> adds a "reload" action to the init script, per the common convention, and
> configures logrotate to make use of that.
> 

I indeed removed the reload functionality which is present in the current portage stable rsyslog-3 init script and logrotate file because upstream now says it is deprecated (2009-07-15) :
http://www.rsyslog.com/doc/v4compatibility.html

Quote:
Please note that restart-type HUP is depricated and will go away in rsyslog v5. So it is a good idea to become ready for the new version now and also enjoy some of the benefits of the "real restart", like the better error-reporting capability.

> Note: since you use git, I git-formatted the patch so you can "git am" it. I
> wasn't sure whether I'm supposed to include the manifest update - I did so.
> 

No need to patch the Manifest since the contributor always update it before commiting.

Thanks again for all your feedback and ideas.
Comment 25 Mike Nerone 2010-09-12 06:17:36 UTC
Hello again,

As the quote says, they're only deprecating the "restart-type" HUP, not HUP completely.

There are two types of HUP responses. The old response is to actually restart the daemon. This full restart was like "/etc/init.d/rsyslog restart", but was suboptimal because it doesn't provide the daemon a way to output any errors to your screen. What Gerhards meant in the quote you cited was that if you want a full restart like that, it's beneficial to do a "real restart" with the init script (he explains this earlier in the article). Thus, this old behavior of responding to a HUP signal with a full restart has indeed been deprecated in v5.

The *new* response to a HUP, i.e. v5's response, is simply to re-open log files. This is much less disruptive, and remains the proper way to re-open log files for rotation purposes. Another quote from the same article:

"On the contrary, a HUP is typically needed for log rotation, and the real desire is to close files. This is a non-disruptive and very lightweigth <sic> operation."

When you want a full restart, re-reading the config, reinitializing every plugin and its dog, etc., use the init script restart. For file rotation, HUP is most definitely the way to go, which is what my patch makes available via an init script "reload".
Comment 26 Ultrabug gentoo-dev 2010-09-13 09:55:43 UTC
(In reply to comment #25)
> Hello again,
> 
> As the quote says, they're only deprecating the "restart-type" HUP, not HUP
> completely.
> 
> There are two types of HUP responses. The old response is to actually restart
> the daemon. This full restart was like "/etc/init.d/rsyslog restart", but was
> suboptimal because it doesn't provide the daemon a way to output any errors to
> your screen. What Gerhards meant in the quote you cited was that if you want a
> full restart like that, it's beneficial to do a "real restart" with the init
> script (he explains this earlier in the article). Thus, this old behavior of
> responding to a HUP signal with a full restart has indeed been deprecated in
> v5.
> 
> The *new* response to a HUP, i.e. v5's response, is simply to re-open log
> files. This is much less disruptive, and remains the proper way to re-open log
> files for rotation purposes. Another quote from the same article:
> 
> "On the contrary, a HUP is typically needed for log rotation, and the real
> desire is to close files. This is a non-disruptive and very lightweigth <sic>
> operation."
> 
> When you want a full restart, re-reading the config, reinitializing every
> plugin and its dog, etc., use the init script restart. For file rotation, HUP
> is most definitely the way to go, which is what my patch makes available via an
> init script "reload".
> 

Hi Mike,

I stand corrected indeed I did not read/understand this that way, thank you for clarifying this !

So I pushed back the reload functionality and tested it. It's commited in the overlay.
Comment 27 Mike Nerone 2010-09-13 19:26:52 UTC
I've tested again, including the directory-include functionality, and everything is still working swimmingly. This ebuild is looking pretty worthy to me.
Comment 28 Ultrabug gentoo-dev 2010-10-13 08:12:53 UTC
Hi,

I had the chance to have a talk with dev-zero (thanks again for your time) and he rightly pointed out that other distros have a different and more upstream-compliant config file hierarchy (fedora for example) :

- main conf file : /etc/rsyslog.conf
- include conf dir : /etc/rsyslog.d
- ssl certs dir : /etc/ssl/rsyslog

So I modified and pushed on scarabeus overlay the revised version of the ebuild and config files to adjust to this hierarchy. I hope this adjustment is fine with everyone and will get this ebuild a clear path to portage.

Regards
Comment 29 Mike Nerone 2010-10-13 15:57:14 UTC
I totally agree (it's what I was suggesting in comment #15).
Comment 30 Tiziano Müller (RETIRED) gentoo-dev 2010-10-21 07:39:24 UTC
Finally done.
Thanks a lot, Ultrabug and Mike Nerone.