I have a small problem that really drives me crazy! I found out that when stopping / restarting Metalog the buffered log is thrown away, instead to be flushed into file. Also I found no way to configure lines / interval until a flush occurs. So I unmerged Metalog to give syslog-ng a try. I managed to configure lines buffered until written to disk. I found out that syslog-ng flushes every 10 seconds even if buffer line size is not reached, but when stopping/restarting daemon while the most recent log data is still in output buffer the data is dropped. My question is WHY?! It should be self-evident for such an important program to flush their buffer before terminating! PS: I tested it using kernels: gentoo-2.6.25-r7 and gentoo-2.6.20-r6. PPS: Im not sure, but I think I can remember that Metalog were flushing to file when using /etc/init.d/Metalog restart some time ago. But since that time it worked for me I updated my gentoo multiple times (kernel, libriaries, ...) Reproducible: Always Steps to Reproduce: 1. tail -f /var/log/... 2. Create script: #!/bin/bash logger Test1 logger Test2 logger Test3 logger Test4 logger Test5 logger Test6 logger Test7 logger Test8 logger Test9 logger Test0 /etc/init.d/syslog-ng restart (or) /etc/init.d/metalog restart 3. Run script 4. Watch tail output Actual Results: Log data is being lost... Expected Results: Buffered data flushed to disk, then terminating.
Please post your `emerge --info' too. Also, does it help to call `sync'?
Portage 2.1.4.4 (default/linux/x86/2008.0/server, gcc-4.1.2, glibc-2.6.1-r0, 2.6.25-gentoo-r7-k1 i686) ================================================================= System uname: 2.6.25-gentoo-r7-k1 i686 Pentium III (Katmai) Timestamp of tree: Fri, 01 Aug 2008 22:15:01 +0000 app-shells/bash: 3.2_p33 dev-lang/python: 2.4.4-r13, 2.5.2-r5 dev-python/pycrypto: 2.0.1-r6 sys-apps/baselayout: 1.12.11.1 sys-apps/sandbox: 1.2.18.1-r2 sys-devel/autoconf: 2.61-r2 sys-devel/automake: 1.5, 1.7.9-r1, 1.10.1 sys-devel/binutils: 2.18-r3 sys-devel/gcc-config: 1.4.0-r4 sys-devel/libtool: 1.5.26 virtual/os-headers: 2.6.23-r3 ACCEPT_KEYWORDS="x86" CBUILD="i686-pc-linux-gnu" CFLAGS="-march=pentium3 -pipe -O3" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc" CONFIG_PROTECT_MASK="/etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/terminfo /etc/udev/rules.d" CXXFLAGS="-O2 -march=i686 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="distlocks metadata-transfer sandbox sfperms strict unmerge-orphans userfetch" GENTOO_MIRRORS="ftp://ftp.uni-erlangen.de/pub/mirrors/gentoo ftp://ftp.join.uni-muenster.de/pub/linux/distributions/gentoo ftp://ftp.wh2.tu-dresden.de/pub/mirrors/gentoo" LDFLAGS="-Wl,-O1" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.de.gentoo.org/gentoo-portage" USE="acl apache2 berkdb bzip2 cli cracklib crypt cups dri fortran gdbm gpm iconv isdnlog kerberos ldap mailwrapper midi mudflap mysql ncurses nptl nptlonly openmp pam pcre perl pppd python readline reflection sasl session snmp spl ssl sysfs tcpd truetype unicode x86 xml xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="fbdev glint i810 mach64 mga neomagic nv r128 radeon savage sis tdfx trident vesa vga via vmware voodoo" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
(In reply to comment #1) > Please post your `emerge --info' too. Also, does it help to call `sync'? > I tried to call sync before and after restart, it did´t help.
So, the call to /etc/init.d/syslog-ng restart is part of your script? I don't see the issue that you're talking about here. All logger lines appear in /var/log/messages as expected. Please attach your /etc/syslog-ng/syslog-ng.conf file as text/plain.
# $Header: /var/cvsroot/gentoo-x86/app-admin/syslog-ng/files/syslog-ng.conf.gentoo,v 1.7 2007/08/02 04:52:18 mr_bones_ Exp $ # # Syslog-ng default configuration file for Gentoo Linux # contributed by Michael Sterrett options { chain_hostnames(off); sync(50); log_fifo_size(1000); # The default action of syslog-ng 1.6.0 is to log a STATS line # to the file every 10 minutes. That's pretty ugly after a while. # Change it to every 12 hours so you get a nice daily update of # how many messages syslog-ng missed (0). stats(43200); }; source src { unix-stream("/dev/log" max-connections(256)); internal(); file("/proc/kmsg"); }; destination messages { file("/var/log/messages"); }; # By default messages are logged to tty12... destination console_all { file("/dev/tty12"); }; # ...if you intend to use /dev/console for programs like xconsole # you can comment out the destination line above that references /dev/tty12 # and uncomment the line below. #destination console_all { file("/dev/console"); }; log { source(src); destination(messages); }; log { source(src); destination(console_all); };
(In reply to comment #4) > So, the call to /etc/init.d/syslog-ng restart is part of your script? I don't > see the issue that you're talking about here. All logger lines appear in > /var/log/messages as expected. > > Please attach your /etc/syslog-ng/syslog-ng.conf file as text/plain. > Yes /etc/init.d/syslog-ng restart is part of the script is it called directly after the output via logger. If I add a sleep(11) between logger and the restart the lines are flushed before the daemon is restarted (because it seams this "flush all 10 seconds" is the default behaviour from syslog-ng) , but otherwise these lines are just dropped. Metalog did not flush time dependant, it waits until enough (no idea how much) data is there. If I restart system some of the last lines are dropped. Maybe there is a bug in one of the libraries / system calls which prevent the flush/sync when daemon is stopped.
You've changed the default config from sync(0) to sync(50). Go ahead and change it back and try again.
Well, I've tested with your script on my computer: 1. syslog-ng works if you set sync(0) in config file 2. metalog works our of box So it WORKSFORME...
sync(0) is the default setting for syslog-ng with the Gentoo-installed config so WORKSFORME as well.
Created attachment 178541 [details, diff] metalog-0.8-flush_on_exit.patch This actually *is* a bug in metalog itself. In case it is killed while being in buffered mode, it looses all its unflushed messages. See patch.
Created attachment 178542 [details] metalog-0.8-r2.ebuild This is a slightly modified and rev bumped ebuild to apply the patch.
Created attachment 178546 [details] metalog.initd There also is an error in metalog's init script which results in the inability to actually write the pid file. See modified metalog.initd Btw. all changes can also be found in my overlay at http://repo.or.cz/w/gentoo-overlay-ibormuth.git?a=tree;f=app-admin/metalog
Created attachment 178553 [details, diff] metalog-0.8-flush_on_exit.patch Replace former patch by a simpler version. Sorry.
should be fixed with metalog-0.9
