Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 106651 - logrotate loses log output if sharedscripts, postrotate and compress options used together
Summary: logrotate loses log output if sharedscripts, postrotate and compress options ...
Status: RESOLVED TEST-REQUEST
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: No maintainer - Look at https://wiki.gentoo.org/wiki/Project:Proxy_Maintainers if you want to take care of it
URL:
Whiteboard:
Keywords:
Depends on: 192468
Blocks:
  Show dependency tree
 
Reported: 2005-09-20 00:18 UTC by Gerard Neil
Modified: 2012-06-08 13:08 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
Longer description of bug details (details.txt,1.86 KB, text/plain)
2005-09-20 00:20 UTC, Gerard Neil
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Gerard Neil 2005-09-20 00:18:26 UTC
If you use the compress option of logrotate along with sharedscripts option 
and a postrotate script intended to force the logging program to close its log 
files, it is possible to lose log ouput. 

The problem occurs because the sharedscripts option causes the shared 
postrotate script to be run AFTER all log files are compressed. This means 
that the log files are compressed while the logging program can still be 
writing to them. There is a window for losing output between the time a given 
file is compressed and and the time the (shared) postrotate script is run.

That's probably too abstract to make sense, so see the details.txt attachment 
for a worked example.

This isn't a hypothetical problem, it actually occurs with the current stable 
distributions of logrotate (3.7.1-r1) and syslog-ng (1.6.8-r1) as of Sep 05 
(see steps to reproduce).

You can work around the problem by simply not using compress and sharedscripts 
together, or by using the delaycompress option which postpones the compression 
operations to the next rotation cycle, or just by not using the sharedscripts 
option and putting up with running the postrotate script multiple times. 
That's the old, non-optimal but safe behaviour.

IMHO, in a correct implementation, the sharedscripts option would cause 
logrotate to run the postrotate script after rotating all the log files, but 
*before* compressing them. At the least, there needs to be a proper warning in 
the man page. Logrotate is supposed to help you safely rotate log files, but 
as it stands, if you do what seems natural from reading the man page, you will 
get bitten and lose log output.

Reproducible: Always
Steps to Reproduce:
1. Use a default configuration of app-addmin/logrotate-3.7.1-r1 and app-
admin/syslog-ng-1.6.8-r1. The /etc/logrotate.conf from the logrotate ebuild 
sets the compress option, and /etc/logrotate.d/syslog-ng from the syslog-ng 
ebuild rotates /var/log/messages using the sharedscripts option.
2. Force a rotation of log files using sudo logrotate -f /etc/logrotate.conf
3. Comment out the sharedscripts option in /etc/logrotate.d/syslog-ng
4. Force another rotation as in step 2.

You can also confirm the order of operations as stated in the details by using 
the -d option of logrotate.
Actual Results:  
You will now have /var/log/messages, /var/log/messages.1.gz 
and /var/log/messages.2.gz. 

The last line of /var/log/messages.2.gz reflect what happened *with* the 
sharedscripts option. The last thing in the log is a sudo entry for the 
command in step 2:

Sep 20 10:56:37 goblin sudo:      gid : TTY=pts/0 ; PWD=/etc/logrotate.d ; 
USER=root ; COMMAND=/usr/sbin/logrotate -f /etc/logrotate.conf

The last lines of /var/log/messages.1.gz reflect what happened *without* the 
shared scripts option. There is an entry for the the sudo logrotate in step 4. 
The final entry in the log is from syslog-ng itself, and is the result of 
executing the postrotate script.

Sep 20 16:53:28 goblin sudo:      gid : TTY=pts/0 ; PWD=/var/log ; USER=root ; 
COMMAND=/usr/sbin/logrotate -f /etc/logrotate.conf
Sep 20 16:53:35 goblin syslog-ng[5527]: SIGHUP received, restarting syslog-ng


Expected Results:  
The last line of /var/log/messages.2.gz should be the "restarting syslog-ng" 
message, which was lost.
Comment 1 Gerard Neil 2005-09-20 00:20:58 UTC
Created attachment 68844 [details]
Longer description of bug details
Comment 2 Daniel Gryniewicz (RETIRED) gentoo-dev 2005-09-29 08:59:17 UTC
This is exceptionally difficult, based on the structure of the program. 
Basically, without forking and re-writing parts of it, it's not possible.  (the
original file is compressed, then moved, one at a time.  This is done in a loop,
then the shared postrotate is run after they're all done)  I suspect this is why
delaycompress was added in the first place.

I may yet fork and re-write, as the program is not well maintained (a loose,
unoffical coalition of distros maintains it)  for the moment, I'd suggest you
either don't use sharedscripts with compress, or use delaycompress, as you
suggested.  I will leave this bug open, so that I can updated it when I get a
better fix in place.
Comment 3 Jaco Kroon 2009-06-10 12:27:21 UTC
Alternative suggestion seeing as I just ran into this as well:

if sharedscripts is specified, make it imply delaycompress?  This is a bad compromise, but it does "fix" the problem.

The alternative mechanism would be to use a "queue" of sorts to just queue the to be compressed files until logrotate is done with everything else.  I'll take a quick peek at the code to see how easy/difficult this would be.

/me is almost praying that logrotate is written with C++ and not pure C, even if just for the stl.
Comment 4 Pacho Ramos gentoo-dev 2012-06-08 13:08:33 UTC
Still valid with 3.8.1?