When you set mark_freq(0), instead of disabling the -- MARK -- messages in the log, it send them as fast as it can until it runs out of disk space. There is one machine on which it works as expected. On the other three, I can see the above behaviuor Reproducible: Always Steps to Reproduce: 1. edit /etc/syslog-ng/syslog-ng.conf and add the line options { .. mark_freq(0); .. } 2. run /etc/init.d/syslog-ng reload 3. watch /var/log/messages fill up Actual Results: Jun 20 12:54:38 mahaweli -- MARK -- Jun 20 12:54:38 mahaweli -- MARK -- Jun 20 12:54:38 mahaweli -- MARK -- Jun 20 12:54:38 mahaweli -- MARK -- Jun 20 12:54:38 mahaweli -- MARK -- Jun 20 12:54:38 mahaweli -- MARK -- Jun 20 12:54:38 mahaweli -- MARK -- Jun 20 12:54:38 mahaweli -- MARK -- Jun 20 12:54:38 mahaweli -- MARK -- Expected Results: Not have any -- MARK -- messages emerge --info out from three machines are pasted below. Machines 1 and 2 exhibit the bug and machine 3 doesn't. Machines 2 (with bug) and 3 (without bug) are almost identical in terms of hardware and software. Machine 1: exhibits above bug Portage 2.1.4.4 (default/linux/x86/2008.0, gcc-4.1.2, glibc-2.6.1-r0, 2.6.24-gentoo-r8 i686) ================================================================= System uname: 2.6.24-gentoo-r8 i686 Intel(R) Pentium(R) 4 CPU 2.26GHz Timestamp of tree: Wed, 18 Jun 2008 19:15:01 +0000 app-shells/bash: 3.2_p33 dev-lang/python: 2.4.4-r13 dev-python/pycrypto: 2.0.1-r6 sys-apps/baselayout: 1.12.11.1 sys-apps/sandbox: 1.2.18.1-r2 sys-devel/autoconf: 2.61-r1 sys-devel/automake: 1.10.1 sys-devel/binutils: 2.18-r1 sys-devel/gcc-config: 1.4.0-r4 sys-devel/libtool: 1.5.26 virtual/os-headers: 2.6.23-r3 ACCEPT_KEYWORDS="x86" CBUILD="i686-pc-linux-gnu" CFLAGS="-march=pentium4 -O2 -pipe -fomit-frame-pointer" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc" CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/terminfo /etc/udev/rules.d" CXXFLAGS="-march=pentium4 -O2 -pipe -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="ccache distlocks metadata-transfer parallel-fetch sandbox sfperms strict unmerge-orphans userfetch userpriv usersandbox" GENTOO_MIRRORS="http://mirror.csclub.uwaterloo.ca/gentoo-distfiles http://adelie.polymtl.ca ttp://gentoo.mirrors.tera-byte.com http://distfiles.gentoo.org http://www.ibiblio.o rg/pub/Linux/distributions/gentoo" LINGUAS="en" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://kala.eng.uwo.ca/gentoo-portage" USE="acl apache2 berkdb bzip2 cli cracklib crypt dri fortran gdbm gpm iconv ipv6 isdnlog midi mudflap ncurses nls nptl nptlonly openmp pam pcre perl pppd python readline reflection session spl ssl tcpd unicode vhosts x86 xml xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en" USERLAND="GNU" VIDEO_CARDS="fbdev glint i810 mach64 mga neomagic nv r128 radeon savage sis tdfx trident vesa vga via vmware voodoo" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY -------------------- Machine 2: exhibits above bug Portage 2.1.4.4 (default-linux/amd64/2007.0, gcc-4.1.2, glibc-2.6.1-r0, 2.6.24-gentoo-r8 x86_64) ================================================================= System uname: 2.6.24-gentoo-r8 x86_64 Intel(R) Core(TM)2 Quad CPU @ 2.40GHz Timestamp of tree: Mon, 16 Jun 2008 09:15:01 +0000 app-shells/bash: 3.2_p33 dev-lang/python: 2.4.4-r13 dev-python/pycrypto: 2.0.1-r6 sys-apps/baselayout: 1.12.11.1 sys-apps/sandbox: 1.2.18.1-r2 sys-devel/autoconf: 2.61-r1 sys-devel/automake: 1.9.6-r2, 1.10.1 sys-devel/binutils: 2.18-r1 sys-devel/gcc-config: 1.4.0-r4 sys-devel/libtool: 1.5.26 virtual/os-headers: 2.6.23-r3 ACCEPT_KEYWORDS="amd64" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=nocona -O2 -pipe -fomit-frame-pointer" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc" CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c /etc/udev/rules.d" CXXFLAGS="-march=nocona -O2 -pipe -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="ccache distlocks metadata-transfer parallel-fetch sandbox sfperms strict unmerge-orphans userfetch userpriv usersandbox" GENTOO_MIRRORS="http://mirror.csclub.uwaterloo.ca/gentoo-distfiles http://adelie.polymtl.ca ttp://gentoo.mirrors.tera-byte.com http://distfiles.gentoo.org http://www.ibiblio.o rg/pub/Linux/distributions/gentoo" LINGUAS="en" MAKEOPTS="-j6" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://kala.eng.uwo.ca/gentoo-portage" USE="acl amd64 apache2 berkdb bzip2 cli cracklib crypt dri fortran gdbm gpm iconv ipv6 isdnlog midi mmx mudflap ncurses nls nptl nptlonly openmp pam pcre perl postgres pppd python readline reflection session spl sse sse2 ssl tcpd unicode vhosts xml xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en" USERLAND="GNU" VIDEO_CARDS="apm ark chips cirrus cyrix dummy fbdev glint i128 i810 mach64 mga neomagic nv r128 radeon rendition s3 s3virge savage siliconmotion sis sisusb tdfx tga trident tseng v4l vesa vga via vmware voodoo" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY -------------------- Machine 3: does not exhibit the above bug Portage 2.1.4.4 (default-linux/amd64/2007.0/desktop, gcc-4.1.2, glibc-2.6.1-r0, 2.6.24-gentoo-r8 x86_64) ================================================================= System uname: 2.6.24-gentoo-r8 x86_64 Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz Timestamp of tree: Mon, 16 Jun 2008 09:15:01 +0000 app-shells/bash: 3.2_p33 dev-java/java-config: 1.3.7, 2.1.6 dev-lang/python: 2.4.4-r13 dev-python/pycrypto: 2.0.1-r6 sys-apps/baselayout: 1.12.11.1 sys-apps/sandbox: 1.2.18.1-r2 sys-devel/autoconf: 2.13, 2.61-r1 sys-devel/automake: 1.7.9-r1, 1.9.6-r2, 1.10.1 sys-devel/binutils: 2.18-r1 sys-devel/gcc-config: 1.4.0-r4 sys-devel/libtool: 1.5.26 virtual/os-headers: 2.6.23-r3 ACCEPT_KEYWORDS="amd64" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=nocona -O2 -pipe -fomit-frame-pointer" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config" CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/udev/rules.d" CXXFLAGS="-march=nocona -O2 -pipe -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="ccache distlocks metadata-transfer parallel-fetch sandbox sfperms strict unmerge-orphans userfetch userpriv usersandbox" GENTOO_MIRRORS="http://gentoo.mirrors.tera-byte.com http://adelie.polymtl.ca http://distfiles.gentoo.org http://www.ibiblio.o rg/pub/Linux/distributions/gentoo" LINGUAS="en si" MAKEOPTS="-j6" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://kala.eng.uwo.ca/gentoo-portage" USE="X acl acpi alsa amd64 berkdb branding cairo cdr cli cracklib crypt cups dbus dri dvd dvdr dvdread eds emboss encode esd evo fam firefox fortran gdbm gif gpm gstreamer hal iconv ipv6 isdnlog jpeg kde kdeenablefinal kdehiddenvisibility ldap mad midi mikmod mmx mp3 mpeg mudflap ncurses nls nptl nptlonly nvidia ogg opengl openmp oss pam pcre pdf perl png pppd python qt3 qt3support qt4 quicktime readline reflection sdl session spell spl sse sse2 ssl svg tcpd tiff truetype unicode vorbis xml xorg xv zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en si" USERLAND="GNU" VIDEO_CARDS="nvidia" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
What makes you expect that mark_freq(0) disables MARK messages?
(In reply to comment #1) > What makes you expect that mark_freq(0) disables MARK messages? > http://forums.gentoo.org/viewtopic-t-639827-highlight-syslogng+mark.html
I'm not able to reproduce this behavior. Go ahead and post the complete config please. Also, please post the lines from the log file that show the reload of the config file.
(In reply to comment #3) > I'm not able to reproduce this behavior. Go ahead and post the complete config > please. Also, please post the lines from the log file that show the reload of > the config file. > Apparently all three machines that exhibited this behavior decided to make me look like a fool. I can't reproduce it on any of the machines now. i.e. When I set mark_freq(0) and reload the config, all is quiet. So, please feel free to close this. I'll keep an eye and see if I can reproduce it reliably.
ok, reopen if you can get more information on it.
(In reply to comment #4) > So, please feel free to close this. I'll keep an eye and see if I can reproduce > it reliably. > Got it (I'm vindicated:). It looks like it takes a while for it to start pumping MARK messages. Heres the config and the log: --------syslog-ng.conf--------- ## $Header: /var/cvsroot/gentoo-x86/app-admin/syslog-ng/files/syslog-ng.conf.gentoo,v 1.5 2005/05/12 05:46:10 mr_bones_ Exp $ # # Syslog-ng default configuration file for Gentoo Linux # contributed by Michael Sterrett options { chain_hostnames(off); sync(0); mark_freq(0); # The default action of syslog-ng 1.6.0 is to log a STATS line # to the file every 10 minutes. That's pretty ugly after a while. # Change it to every 12 hours so you get a nice daily update of # how many messages syslog-ng missed (0). stats(43200); }; ## -- source where to read log source src { unix-stream("/dev/log"); internal(); file("/proc/kmsg"); }; ## -- define destinations destination cron { file("/var/log/cron.log"); }; destination imap { file("/var/log/imap.log"); }; destination emerge { file("/var/log/emerge-info.log"); }; destination sudo { file("/var/log/sudo.log"); }; destination messages { file("/var/log/messages"); }; destination console { usertty("root"); }; destination console_all { file("/dev/tty12"); }; destination xconsole { pipe("/dev/xconsole"); }; destination sshd { file("/var/log/sshd.log"); }; destination sshd_attack { file("/var/log/sshd_attack.log"); }; destination sshd_ban { program("/sbin/blacklist_js.py"); }; destination rsync { file("/var/log/rsync.log"); }; destination coursedb { file("/var/log/coursedb.log");}; ## -- create filters filter f_cron { facility(cron); }; filter f_emerge { program(portage); }; filter f_imap { program(dovecot); }; filter f_urgent { level(emerg); }; filter f_sudo { program(sudo); }; filter f_sshd { program(sshd); }; filter f_sshd_attack { program(sshd) and ( match('Did not receive identification string from') or match('invalid user') or match('Invalid user') or match('Failed password for root') ); }; filter f_rsync { program(rsync); }; filter f_coursedb { program(coursedb); }; ## -- connect filter and destination (may be order dependant) log { source(src); filter(f_urgent); destination(console); }; log { source(src); filter(f_cron); destination(cron); flags(final); }; log { source(src); filter(f_imap); destination(imap); flags(final); }; log { source(src); filter(f_emerge); destination(emerge); flags(final); }; log { source(src); filter(f_sudo); destination(sudo); flags(final); }; log { source(src); filter(f_sshd_attack); destination(sshd_attack); }; log { source(src); filter(f_sshd_attack); destination(sshd_ban); }; log { source(src); filter(f_sshd); destination(sshd); flags(final); }; log { source(src); filter(f_rsync); destination(rsync); flags(final); }; log { source(src); filter(f_coursedb); destination(coursedb); flags(final); }; log { source(src); destination(messages); }; #default log log { source(src); destination(console_all); }; --------------end-syslog.conf------------ ---------/var/log/messages------ Jun 24 09:37:49 kala syslog-ng[3647]: Configuration reload request received, reloading configuration; Jun 24 09:37:49 kala blacklist: Reading initial list: 219.94.133.76, 121.8.104.3, 141.85.3.66 Jun 24 13:56:08 kala -- MARK -- Jun 24 13:56:08 kala -- MARK -- Jun 24 13:56:08 kala -- MARK -- ... removed about 100 million similar lines Jun 24 15:21:01 kala -- MARK -- Jun 24 15:21:01 kala -- MARK -- Jun 24 15:21:01 kala -- MARK -- Jun 24 15:21:01 kala -- MARK -- Jun 24 15:21:01 kala -- MARK -- Jun 24 11:21:02 kala syslog-ng[3647]: Configuration reload request received, reloading configuration; Jun 24 11:21:03 kala blacklist: Reading initial list: 219.94.133.76, 121.8.104.3, 141.85.3.66 --------end-/var/log/messages-------- kala log # grep MARK /var/log/messages|wc -l 100066795 Another funny thing I noticed was that the timestamp was wrong. I reloaded the log at 9:37. First MARK message has the timestamp of 13:56:08 and the last message has the timestamp of 15:21:01 I noticed it at around 11:20, immediately commented out the mark_freq(0) line and reloaded config. The reload message has the correct timestamp of 11:21:02. I'll be happy to help debug this if you can't reproduce it.
This happened to me about 30 minutes after setting mark_freq(0) in app-admin/syslog-ng-2.1.4.
