If you use the compress option of logrotate along with sharedscripts option and a postrotate script intended to force the logging program to close its log files, it is possible to lose log ouput. The problem occurs because the sharedscripts option causes the shared postrotate script to be run AFTER all log files are compressed. This means that the log files are compressed while the logging program can still be writing to them. There is a window for losing output between the time a given file is compressed and and the time the (shared) postrotate script is run. That's probably too abstract to make sense, so see the details.txt attachment for a worked example. This isn't a hypothetical problem, it actually occurs with the current stable distributions of logrotate (3.7.1-r1) and syslog-ng (1.6.8-r1) as of Sep 05 (see steps to reproduce). You can work around the problem by simply not using compress and sharedscripts together, or by using the delaycompress option which postpones the compression operations to the next rotation cycle, or just by not using the sharedscripts option and putting up with running the postrotate script multiple times. That's the old, non-optimal but safe behaviour. IMHO, in a correct implementation, the sharedscripts option would cause logrotate to run the postrotate script after rotating all the log files, but *before* compressing them. At the least, there needs to be a proper warning in the man page. Logrotate is supposed to help you safely rotate log files, but as it stands, if you do what seems natural from reading the man page, you will get bitten and lose log output. Reproducible: Always Steps to Reproduce: 1. Use a default configuration of app-addmin/logrotate-3.7.1-r1 and app- admin/syslog-ng-1.6.8-r1. The /etc/logrotate.conf from the logrotate ebuild sets the compress option, and /etc/logrotate.d/syslog-ng from the syslog-ng ebuild rotates /var/log/messages using the sharedscripts option. 2. Force a rotation of log files using sudo logrotate -f /etc/logrotate.conf 3. Comment out the sharedscripts option in /etc/logrotate.d/syslog-ng 4. Force another rotation as in step 2. You can also confirm the order of operations as stated in the details by using the -d option of logrotate. Actual Results: You will now have /var/log/messages, /var/log/messages.1.gz and /var/log/messages.2.gz. The last line of /var/log/messages.2.gz reflect what happened *with* the sharedscripts option. The last thing in the log is a sudo entry for the command in step 2: Sep 20 10:56:37 goblin sudo: gid : TTY=pts/0 ; PWD=/etc/logrotate.d ; USER=root ; COMMAND=/usr/sbin/logrotate -f /etc/logrotate.conf The last lines of /var/log/messages.1.gz reflect what happened *without* the shared scripts option. There is an entry for the the sudo logrotate in step 4. The final entry in the log is from syslog-ng itself, and is the result of executing the postrotate script. Sep 20 16:53:28 goblin sudo: gid : TTY=pts/0 ; PWD=/var/log ; USER=root ; COMMAND=/usr/sbin/logrotate -f /etc/logrotate.conf Sep 20 16:53:35 goblin syslog-ng[5527]: SIGHUP received, restarting syslog-ng Expected Results: The last line of /var/log/messages.2.gz should be the "restarting syslog-ng" message, which was lost.
Created attachment 68844 [details] Longer description of bug details
This is exceptionally difficult, based on the structure of the program. Basically, without forking and re-writing parts of it, it's not possible. (the original file is compressed, then moved, one at a time. This is done in a loop, then the shared postrotate is run after they're all done) I suspect this is why delaycompress was added in the first place. I may yet fork and re-write, as the program is not well maintained (a loose, unoffical coalition of distros maintains it) for the moment, I'd suggest you either don't use sharedscripts with compress, or use delaycompress, as you suggested. I will leave this bug open, so that I can updated it when I get a better fix in place.
Alternative suggestion seeing as I just ran into this as well: if sharedscripts is specified, make it imply delaycompress? This is a bad compromise, but it does "fix" the problem. The alternative mechanism would be to use a "queue" of sorts to just queue the to be compressed files until logrotate is done with everything else. I'll take a quick peek at the code to see how easy/difficult this would be. /me is almost praying that logrotate is written with C++ and not pure C, even if just for the stl.
Still valid with 3.8.1?
