Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 426242

Summary: log files for media-tv/mythtv-0.25.1_p20120708 not rotated correctly
Product: Gentoo Linux Reporter: Myk Taylor <myk002>
Component: Current packagesAssignee: MythTV Maintainers <mythtv+disabled>
Status: RESOLVED FIXED    
Severity: normal    
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
URL: http://www.mythtv.org/wiki/Logrotate_-_all_applications
Whiteboard: http://www.mythtv.org/wiki/Logrotate_-_all_applications
Package list:
Runtime testing required: ---

Description Myk Taylor 2012-07-12 03:21:30 UTC
the current logrotate file for mythtv (media-tv/mythtv/files/mythtv.logrotate.d-r2) fails to account for some mythtv log files.

Reproducible: Always

Steps to Reproduce:
1. fire up mythtv-0.25.1 and perform some actions (like transcoding or flagging commercials)
2. see log files appear in /var/log/mythtv that are not handled in /etc/logrotate.d/mythtv
Comment 1 Myk Taylor 2012-07-12 03:23:45 UTC
These are the log files I've accumulated so far:

mythbackend.20120626020843.3689.log
mythbackend.20120626021153.3975.log
mythbackend.20120627104337.2381.log
mythbackend.20120709235741.2377.log
mythbackend.20120710141722.4293.log
mythcommflag.20120626022809.4142.log
mythcommflag.20120626031019.5682.log
mythcommflag.20120628195856.5214.log
mythcommflag.20120709202835.23012.log
mythcommflag.20120710000057.2607.log
mythcommflag.20120710000107.2642.log
mythcommflag.20120710000119.2682.log
mythcommflag.20120710000129.2716.log
mythcommflag.20120710000139.2756.log
mythcommflag.20120710142238.4412.log
mythfilldatabase.20120626032307.5880.log
mythfilldatabase.20120627025750.7975.log
mythfilldatabase.20120628061817.4144.log
mythfilldatabase.20120628173612.5004.log
mythfilldatabase.20120630145826.8783.log
mythfilldatabase.20120701153420.10919.log
mythfilldatabase.20120701183633.11162.log
mythfilldatabase.20120703004229.13565.log
mythfilldatabase.20120704061806.16056.log
mythfilldatabase.20120704184136.17000.log
mythfilldatabase.20120706102910.20267.log
mythfilldatabase.20120707122220.22397.log
mythfilldatabase.20120707220859.23141.log
mythfilldatabase.20120709045534.25811.log
mythfilldatabase.20120710150310.4578.log
mythfilldatabase.20120711023559.5519.log
mythmetadatalookup.20120626022804.4125.log
mythmetadatalookup.20120626031014.5222.log
mythmetadatalookup.20120710000052.2592.log
mythmetadatalookup.20120710000102.2627.log
mythmetadatalookup.20120710000114.2667.log
mythmetadatalookup.20120710000124.2701.log
mythmetadatalookup.20120710000134.2741.log
mythpreviewgen.20120626022915.4157.log
mythpreviewgen.20120626030509.4323.log
mythpreviewgen.20120626040504.5946.log
mythpreviewgen.20120626040509.5961.log
mythpreviewgen.20120628203101.5280.log
mythpreviewgen.20120628203108.5297.log
mythpreviewgen.20120628210503.5348.log
mythpreviewgen.20120709210503.23079.log
mythpreviewgen.20120709210508.23094.log
mythpreviewgen.20120710000058.2615.log
mythpreviewgen.20120710000112.2654.log
mythpreviewgen.20120710000119.2689.log
mythpreviewgen.20120710000132.2728.log
mythpreviewgen.20120710000358.2784.log
mythpreviewgen.20120710142033.4368.log
mythpreviewgen.20120710142239.4423.log
mythpreviewgen.20120710142536.4447.log
mythtranscode.20120710142033.4365.log

It looks like every mythtv binary produces its own log file.  Autocompleting on 'myth', I see the following binaries:

mythavtest          mythffplay          mythmediaserver     mythtranscode
mythbackend         mythfilldatabase    mythmetadatalookup  mythtv-setup
mythccextractor     mythfrontend        mythpreviewgen      mythutil
mythcommflag        mythjobqueue        mythreplex          mythwelcome
mythffmpeg          mythlcdserver       mythshutdown

do these all produce separate log files?
Comment 2 Myk Taylor 2012-07-12 03:26:08 UTC
mythtv proposes the following at:
http://www.mythtv.org/wiki/Logrotate_-_all_applications

/var/log/mythtv/*.log {
        weekly
        rotate 8
        missingok
        ifempty
        nocreate
        nocompress
        sharedscripts
        olddir /var/log/mythtv/old
        lastaction
                killall -HUP mythbackend
                killall -HUP mythfrontend
                killall -HUP mythlcdserver
                find /var/log/mythtv/old -name 'mythfrontend*' -type f -mtime +30 -delete
        endscript
}


I am not an expert in logrotate, but will this not work?
Comment 3 Doug Goldstein (RETIRED) gentoo-dev 2012-07-12 05:55:33 UTC
(In reply to comment #2)
> mythtv proposes the following at:
> http://www.mythtv.org/wiki/Logrotate_-_all_applications
> 
> /var/log/mythtv/*.log {
>         weekly
>         rotate 8
>         missingok
>         ifempty
>         nocreate
>         nocompress
>         sharedscripts
>         olddir /var/log/mythtv/old
>         lastaction
>                 killall -HUP mythbackend
>                 killall -HUP mythfrontend
>                 killall -HUP mythlcdserver
>                 find /var/log/mythtv/old -name 'mythfrontend*' -type f
> -mtime +30 -delete
>         endscript
> }
> 
> 
> I am not an expert in logrotate, but will this not work?

Not really because logrotate expects a daemon to write to. /var/log/mythtv/example.log continually and it will move the file aside to /var/log/mythtv/example.log.1 and create a new /var/log/mythtv/example.log once a week. So in the current case it will see each of those files as independent and not grouped so you will just have more and more messy files while HUPing the daemons a lot for each file. This is less than ideal.
Comment 4 Myk Taylor 2012-07-14 07:05:49 UTC
I ran some tests, and I found that the logrotate file on the mythtv wiki works as advertised -- almost.  It performs the following actions:

1) moves all log files in /var/log/mythtv to /var/log/mythtv/old (rotating as necessary if a current log file has the same name as a previously-rotated one)
2) HUPs persistent mythtv processes to reopen log files
3) removes log files in /var/log/mythtv/old that are older than 30 days

There is apparently a small bug in the file on the wiki, though.  It only removes old mythfrontend log files instead of any old log file.  I corrected it below:

/var/log/mythtv/*.log {
        weekly
        rotate 8
        missingok
        ifempty
        nocreate
        nocompress
        sharedscripts
        olddir /var/log/mythtv/old
        lastaction
                killall -HUP mythbackend
                killall -HUP mythfrontend
                killall -HUP mythlcdserver
                find /var/log/mythtv/old -type f -mtime +30 -delete
        endscript
}


logrotate will not create /var/log/mythtv/old, so we should create that directory in the ebuild.  It can be owned by root (as opposed to /var/log/mythtv, which is owned by the mythtv user).
Comment 5 Doug Goldstein (RETIRED) gentoo-dev 2012-07-15 22:39:11 UTC
(In reply to comment #4)
> I ran some tests, and I found that the logrotate file on the mythtv wiki
> works as advertised -- almost.  It performs the following actions:
> 
> 1) moves all log files in /var/log/mythtv to /var/log/mythtv/old (rotating
> as necessary if a current log file has the same name as a previously-rotated
> one)
> 2) HUPs persistent mythtv processes to reopen log files
> 3) removes log files in /var/log/mythtv/old that are older than 30 days
> 
> There is apparently a small bug in the file on the wiki, though.  It only
> removes old mythfrontend log files instead of any old log file.  

A logging system shouldn't ever delete files automatically. I left the 30 day delete for mythfrontend simply because it gets very loud.

Try the current solution for now, otherwise I'll use a script written by Wagner (MythTV maintainer) and drop the logrotate script entirely.

If the current ebuild (20120715) isn't good enough reopen or let me know by e-mail.
Comment 6 Myk Taylor 2012-07-16 01:29:46 UTC
(In reply to comment #5)
> Try the current solution for now, otherwise I'll use a script written by
> Wagner (MythTV maintainer) and drop the logrotate script entirely.

If the logrotate script will be dropped entirely, that's fine.  MythTV's choice of log file names really doesn't easily lend itself to integration with logrotate.
Comment 7 Myk Taylor 2012-07-22 04:33:11 UTC
The current mythtv.logrotate.d-r2 file doesn't work adequately (i.e. it doesn't rotate the majority of the files in /var/log/mythtv), so I'm reopening so we don't lose track of this.

I contacted the author of the logrotate script at http://www.mythtv.org/wiki/Logrotate_-_all_applications (MythTV's Sphery) and cleared up a bug in the script listed there.  His logrotate script does indeed work now.  If it will be a while before Raymond Wagner's more accurate python script is integrated, perhaps we could just use Sphery's in the interim?
Comment 8 Doug Goldstein (RETIRED) gentoo-dev 2012-08-26 02:40:18 UTC
Should be addressed in mythtv-0.25.2_p20120825. The more I deal with the new logging format, the more I hate it. Ubuntu ended up switching to syslog, but they force everyone to use rsyslog so they can provide a config for it. We unfortunately don't have that luxury but maybe I can generate a config for the 3 logging daemons supported.