With the last sarg's ebuild (2.2.5-r1) you get the next error: SARG: getword loop detected. SARG: searching for 'x20' SARG: Maybe you have a broken record or garbage in your access.log file. If I reemerge the previous version (2.2.5) with the same configuration and access.log file, all works ok... Reproducible: Always Steps to Reproduce: 1. emerge the last sarg (2.2.5-r1) 2. create an access.log with a line like this (is only an example, I guess it fails always) 1237962353.542 478 10.58.118.38 TCP_MISS/200 325 GET http://ui.skype.com/ui/0/3.2.0.148.141/es/getlatestversion? - DIRECT/204.9.163.158 text/html 3. run sarg here is my sarg.conf: language Spanish access_log /var/log/squid/mensual.squid graphs yes graph_days_bytes_bar_color orange title "Reporte de paginas accedidas por los usuarios" font_face Tahoma,Verdana,Arial header_color darkblue header_bgcolor blanchedalmond font_size 10px header_font_size 11px title_font_size 14px background_color white text_color #000000 text_bgcolor lavender title_color green output_dir /var/www/localhost/htdocs/squid-reports resolve_ip yes user_ip yes date_format e index_sort_order D max_elapsed 28800000 report_type topusers topsites sites_users users_sites date_time denied auth_failures site_user_time_date downloads usertab none long_url no date_time_by bytes charset UTF-8 denied_report_limit 0 user_report_limit 0 download_report_limit 0
Well, I failed to reproduce this bug both on amd64 and x86. Could you post your emerge --info, may be it gives some hint... camobap-unstable sarg-2.2.5 # cat access.log 1237962353.542 478 10.58.118.38 TCP_MISS/200 325 GET http://ui.skype.com/ui/0/3.2.0.148.141/es/getlatestversion? - DIRECT/204.9.163.158 text/html camobap-unstable sarg-2.2.5 # ./sarg -l ./access.log -o /var/tmp/portage/net-analyzer/sarg-2.2.5-r1/work/sarg-2.2.5/report SARG: Records in file: 1, reading: 100.00% SARG: Successful report generated on /var/tmp/portage/net-analyzer/sarg-2.2.5-r1/work/sarg-2.2.5/report/2009Mar25-2009Mar25
> emerge --info, may be it gives some hint... Sure... here you have: Portage 2.1.6.7 (default/linux/x86/2008.0, gcc-4.1.2, glibc-2.8_p20080602-r1, 2.6.25-gentoo-r4 i686) ================================================================= System uname: Linux-2.6.25-gentoo-r4-i686-AMD_Athlon-tm-with-glibc2.0 Timestamp of tree: Wed, 25 Mar 2009 18:00:01 +0000 app-shells/bash: 3.2_p39 dev-lang/python: 2.4.4-r13, 2.5.2-r7 dev-python/pycrypto: 2.0.1-r6 sys-apps/baselayout: 1.12.11.1 sys-apps/sandbox: 1.2.18.1-r2 sys-devel/autoconf: 2.63 sys-devel/automake: 1.8.5-r3, 1.10.2 sys-devel/binutils: 2.18-r3 sys-devel/gcc-config: 1.4.0-r4 sys-devel/libtool: 1.5.26 virtual/os-headers: 2.6.27-r2 ACCEPT_KEYWORDS="x86" CBUILD="i686-pc-linux-gnu" CFLAGS="-march=athlon -O3 -pipe -fomit-frame-pointer" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /var/bind" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/udev/rules.d" CXXFLAGS="-march=athlon -O3 -pipe -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="distlocks fixpackages parallel-fetch protect-owned sandbox sfperms strict unmerge-orphans userfetch" GENTOO_MIRRORS="http://gentoo-euetib.upc.es/mirror/gentoo/ http://gentoo.osuosl.org/ http://gentoo.mirrors.tds.net/gentoo" LANG="es_ES.UTF-8" LC_ALL="es_ES.UTF-8" LDFLAGS="-Wl,-O1" LINGUAS="es" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp/portage" PORTDIR="/usr/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="aalib acl acpi acpi4linux apache2 berkdb bzip2 cli cracklib crypt cups dri ethereal fortran gd gdbm gpm hardened hardenedphp iconv imq innodb isdnlog jpeg l7filter loop-aes midi mudflap ncurses nls nptl nptlonly openmp pam pcre perl php png pppd python readline reflection sd1 session spl ssl svga sysfs tcpd truetype unicode x86 xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="es" USERLAND="GNU" VIDEO_CARDS="fbdev glint i810 intel mach64 mga neomagic nv r128 radeon savage sis tdfx trident vesa vga via vmware voodoo" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY Best regards...
well, I still fail to reproduce problem with the line you posted here. Are you 100% sure that sarg fails with access.log with this line only? I've tested on x86 both in stable and unstables sytems, with different CFLAGS/LDFLAGS and sarg works perfectly here...
And yes, I've tried to change locale/use your config file...
(In reply to comment #4) > And yes, I've tried to change locale/use your config file... > Ummm... I must test it again this night... Maybe is it failing because I've not deleted the /tmp/sarg files of the other tests??... Thank you very much... I'll give my reports this night...
deleted /tmp/sarg folder deleted /var/www/localhost/htdocs/squid-reports/25Mar2009-25Mar2009 folder execute sarg -x: # sarg -z SARG: TAG: language Spanish SARG: TAG: access_log /var/log/squid/mensual.squid SARG: TAG: graphs yes SARG: TAG: graph_days_bytes_bar_color orange SARG: TAG: title "Reporte de paginas accedidas por los usuarios" SARG: TAG: font_face Tahoma,Verdana,Arial SARG: TAG: header_color darkblue SARG: TAG: header_bgcolor blanchedalmond SARG: TAG: font_size 10px SARG: TAG: header_font_size 11px SARG: TAG: title_font_size 14px SARG: TAG: background_color white SARG: TAG: text_color #000000 SARG: TAG: text_bgcolor lavender SARG: TAG: title_color green SARG: TAG: output_dir /var/www/localhost/htdocs/squid-reports SARG: TAG: resolve_ip yes SARG: TAG: user_ip yes SARG: TAG: date_format e SARG: TAG: index_sort_order D SARG: TAG: max_elapsed 28800000 SARG: TAG: report_type topusers topsites sites_users users_sites date_time denied auth_failures site_user_time_date downloads SARG: TAG: usertab none SARG: TAG: long_url no SARG: TAG: date_time_by bytes SARG: TAG: charset UTF-8 SARG: TAG: denied_report_limit 0 SARG: TAG: user_report_limit 0 SARG: TAG: download_report_limit 0 SARG: (util) tbuf=25Mar2009eading: 0.00% SARG: (util) period=25Mar2009- SARG: Records in file: 1, reading: 100.00% SARG: (util) data=25/03/2009 SARG: (util) tbuf=25Mar2009 SARG: (util) period=25Mar2009-25Mar2009 SARG: (util) dirname=/var/www/localhost/htdocs/squid-reports/25Mar2009-25Mar2009 SARG: (util) wdir=/var/www/localhost/htdocs/squid-reports/25Mar2009-25Mar2009 SARG: getword loop detected. SARG: searching for 'x20' SARG: Maybe you have a broken record or garbage in your access.log file. mensual.squid file contains only: 1237962353.542 478 10.58.118.38 TCP_MISS/200 325 GET http://ui.skype.com/ui/0/3.2.0.148.141/es/getlatestversion? - DIRECT/204.9.163.158 text/html (mensual.squid file is my access.log file)... Can I do another test for you??
Ehh!!!... I've founded something... I've renamed the destination folder and create a new one, and now it create me the report without error... Why??
(In reply to comment #7) > Ehh!!!... I've founded something... > > I've renamed the destination folder and create a new one, and now it create me > the report without error... > > Why?? > This is my folder content: torpedo squid-reports # ll total 16K drwxr-xr-x 66 root root 4,0K mar 1 03:23 01Feb2009-01Mar2009 drwxr-xr-x 99 root root 4,0K feb 1 03:27 01Jan2009-01Feb2009 drwxr-xr-x 70 root root 4,0K ene 1 03:11 16Dec2008-01Jan2009 drwxr-xr-x 2 root root 4,0K mar 26 22:41 images torpedo squid-reports # If I move or delete the older report folders, all works fine. If I restore it, the sarg fails again... When sarg fail, it creates a file named "index.unsort" with the next content: torpedo squid-reports # cat index.unsort 26224508;25Mar2009-25Mar2009;jue mar 26 22:45:08 CET 2009;1;325;325;20090325 Hope it helps...
same problems here .... sarg -x SARG: Init SARG: Loading configuration from: /etc/sarg//sarg.conf SARG: Parametri: SARG: SARG: Hostname o indirizzo IP (-a) = SARG: Useragent log (-b) = SARG: File Exclude (-c) = SARG: Data from-until (-d) = SARG: Repporto spedito all'indirizzo Email (-e) = renatogallo@unixproducts.com SARG: File di Configurazione (-f) = /etc/sarg//sarg.conf SARG: Formato della Data (-g) = USA (mm/dd/yyyy) SARG: Rapporto IP (-i) = No SARG: Input log (-l) = /var/log/squid/cache.log SARG: Risoluzione IP Address (-n) = Si SARG: Directory di destinazione (-o) = /tmp/sarg/ SARG: Usa l'indirizzo Ip invece della userid (-p) = Si SARG: Accessed site (-s) = SARG: Tempo (-t) = SARG: Utente (-u) = SARG: Directory Temporanea (-w) = /tmp SARG: Messaggi di Debug (-x) = Si SARG: Messaggi di Processo (-z) = No SARG: SARG: sarg versione: 2.2.5 Mar-03-2008 SARG: Maximum file descriptor: cur=1024 max=1024, changed to cur=20000 max=20000 SARG: Lettura access log file: /var/log/squid/cache.log SARG: Records in file: 989, reading: 100.00% SARG: Records letti: 408, scritto: 201, esclusi: 77 SARG: Formato Squid log SARG: Periodo: 1970Jan01-1970Jan01 SARG: pre-sorting files SARG: Creazione del file del periodo SARG: Creazione del file: /tmp/sarg/Finished... SARG: Creazione del file: /tmp/sarg/With... SARG: Creazione del file: /tmp/sarg/Starting... SARG: Creazione del file: /tmp/sarg/DNS... SARG: Creazione del file: /tmp/sarg/Reconfiguring... SARG: Creazione del file: /tmp/sarg/httpReadReply:... SARG: Creazione del file: /tmp/sarg/Done... SARG: getword loop detected. SARG: searching for 'x20' SARG: Maybe you have a broken record or garbage in your access.log file. sarg -z SARG: TAG: language Italian SARG: TAG: access_log /var/log/squid/cache.log SARG: TAG: graphs yes SARG: TAG: graph_days_bytes_bar_color orange SARG: TAG: title "Squid User Access Reports" SARG: TAG: font_face Tahoma,Verdana,Arial SARG: TAG: header_color darkblue SARG: TAG: header_bgcolor blanchedalmond SARG: TAG: font_size 9px SARG: TAG: header_font_size 9px SARG: TAG: title_font_size 11px SARG: TAG: background_color white SARG: TAG: text_color #000000 SARG: TAG: text_bgcolor lavender SARG: TAG: title_color green SARG: TAG: logo_image http://www.unixproducts.com/zen/includes/templates/classic/images/logo.gif SARG: TAG: image_size 80 45 SARG: TAG: output_dir /var/www/localhost/htdocs/squid-reports SARG: TAG: output_email renatogallo@unixproducts.com SARG: TAG: resolve_ip yes SARG: TAG: user_ip yes SARG: TAG: index yes SARG: TAG: records_without_userid ip SARG: TAG: privacy no SARG: TAG: weekdays 0,1,2,3,4,5,6 SARG: (util) tbuf=1970Jan01 reading: 0.00% SARG: (util) period=1970Jan01- SARG: Records in file: 989, reading: 100.00% SARG: (util) data=01/01/1970 SARG: (util) tbuf=1970Jan01 SARG: (util) period=1970Jan01-1970Jan01 SARG: (util) dirname=/tmp/sarg/1970Jan01-1970Jan01 SARG: (util) wdir=/tmp/sarg/1970Jan01-1970Jan01 SARG: getword loop detected. SARG: searching for 'x20' SARG: Maybe you have a broken record or garbage in your access.log file. emerge --info Portage 2.1.6.7 (default/linux/x86/2008.0/desktop, gcc-4.1.2, glibc-2.8_p20080602-r1, 2.6.28.8 i686) ================================================================= System uname: Linux-2.6.28.8-i686-Intel-R-_Pentium-R-_4_CPU_2.80GHz-with-glibc2.0 Timestamp of tree: Wed, 25 Mar 2009 14:00:01 +0000 app-shells/bash: 3.2_p39 dev-lang/python: 2.4.4-r13, 2.5.2-r7 dev-python/pycrypto: 2.0.1-r8 dev-util/cmake: 2.6.2-r1 sys-apps/baselayout: 1.12.11.1 sys-apps/sandbox: 1.6 sys-devel/autoconf: 2.13, 2.63 sys-devel/automake: 1.5, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.2 sys-devel/binutils: 2.18-r3 sys-devel/gcc-config: 1.4.0-r4 sys-devel/libtool: 1.5.26 virtual/os-headers: 2.6.27-r2 ACCEPT_KEYWORDS="x86" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -march=i686 -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /var/bind" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/sandbox.d /etc/splash /etc/terminfo /etc/udev/rules.d" CXXFLAGS="-O2 -march=i686 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="distlocks fixpackages parallel-fetch protect-owned sandbox sfperms strict unmerge-orphans userfetch" GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo" LDFLAGS="-Wl,-O1" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="X acl acpi alsa apache2 authfile ban bash-completion berkdb bluetooth branding bzip2 cairo case cdr clamav cli cracklib crypt ctype cups curl dbus deflate dlz dri dvd dvdr dvdread eds emboss encode esd evo fam fbcondecor filter firefox fortran gdbm gif gnome gpm gstreamer gtk hal iconv idn ifsession innodb ipv6 isdnlog jpeg kde ldap libnotify mad midi mikmod mng mp3 mpeg mudflap mysql ncurses nls nptl nptlonly odbc ogg opengl openmp opensslcrypt pam pcre pdf perl php png postgres ppds pppd python qt3 qt3support qt4 quicktime radius readline reflection resolvconf rewrite samba sdl session shaper sitemisc softquota spell spl ssl startup-notification svg sysfs tcpd tiff truetype unicode urandom usb vorbis vroot webkit win32codecs x86 xml xml2 xorg xulrunner xv zip zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87xca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logiomem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="nvidia" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LANG, LC_ALL, LINGUAS,MAKEOPTS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY
I too got this error. Now I will attach my config and log file.
Created attachment 186761 [details] emerge --info
Created attachment 186763 [details] sarg error and config
Created attachment 186764 [details] access.log access.log cause sarg fail.
(In reply to comment #13) > Created an attachment (id=186764) [edit] > access.log > > access.log cause sarg fail. > Could you check with the same access.log but creating a new destination folder of your sarg report?? (rename your original destination folder with other name and create a new empty one)... I guess is no access.log problem. I guess is index.htlm creating problem.
Result not changed. # mv squid-reports{,.0} # mkdir squid-reports # chown root:apache squid-reports # ls -la squid-reports итого 8 drwxr-sr-x 2 root apache 4096 Мар 30 21:06 . drwxr-sr-x 6 root apache 4096 Мар 30 21:06 .. # sarg -x SARG: Init SARG: Loading configuration from: /etc/sarg//sarg.conf SARG: Parameters: SARG: SARG: Hostname or IP address (-a) = SARG: Useragent log (-b) = SARG: Exclude file (-c) = SARG: Date from-until (-d) = SARG: Email address to send reports (-e) = SARG: Config file (-f) = /etc/sarg//sarg.conf SARG: Date format (-g) = Europe (dd/mm/yyyy) SARG: IP report (-i) = No SARG: Input log (-l) = /var/log/squid/access.log SARG: Resolve IP Address (-n) = No SARG: Output dir (-o) = /srv/adm.winkhaus.com.ua/www/htdocs/squid-reports/ SARG: Use Ip Address instead of userid (-p) = Yes SARG: Accessed site (-s) = SARG: Time (-t) = SARG: User (-u) = SARG: Temporary dir (-w) = /tmp SARG: Debug messages (-x) = Yes SARG: Process messages (-z) = No SARG: SARG: sarg version: 2.2.5 Mar-03-2008 SARG: Maximum file descriptor: cur=1024 max=1024, changed to cur=20000 max=20000 SARG: Reading access log file: /var/log/squid/access.log SARG: Records in file: 95134, reading: 100.00% SARG: Records read: 95134, written: 95134, excluded: 0 SARG: Squid log format SARG: Period: 23Mar2009-30Mar2009 SARG: pre-sorting files SARG: Making period file SARG: Making file: /tmp/sarg/10_248_0_133 SARG: Making file: /tmp/sarg/10_248_0_85 SARG: Making file: /tmp/sarg/10_248_0_101 SARG: Making file: /tmp/sarg/10_248_0_112 SARG: Making file: /tmp/sarg/10_248_0_242 SARG: Making file: /tmp/sarg/10_248_0_122 SARG: Making file: /tmp/sarg/10_248_0_143 SARG: Making file: /tmp/sarg/10_248_0_115 SARG: Making file: /tmp/sarg/10_248_0_113 SARG: Making file: /tmp/sarg/10_248_0_118 SARG: Making file: /tmp/sarg/10_248_0_240 SARG: Making file: /tmp/sarg/10_248_0_117 SARG: Making file: /tmp/sarg/10_248_0_201 SARG: Making file: /tmp/sarg/10_248_0_116 SARG: Making file: /tmp/sarg/127_0_0_1 SARG: Making file: /tmp/sarg/10_248_0_129 SARG: Making file: /tmp/sarg/10_248_0_114 SARG: Sorting file: /tmp/sarg/10_248_0_242 SARG: Sorting file: /tmp/sarg/10_248_0_117 SARG: Sorting file: /tmp/sarg/10_248_0_118 SARG: Sorting file: /tmp/sarg/10_248_0_122 SARG: Sorting file: /tmp/sarg/10_248_0_115 SARG: Sorting file: /tmp/sarg/10_248_0_101 SARG: Sorting file: /tmp/sarg/10_248_0_129 SARG: Sorting file: /tmp/sarg/10_248_0_133 SARG: Sorting file: /tmp/sarg/10_248_0_113 SARG: Sorting file: /tmp/sarg/10_248_0_201 SARG: Sorting file: /tmp/sarg/10_248_0_116 SARG: Sorting file: /tmp/sarg/10_248_0_143 SARG: Sorting file: /tmp/sarg/10_248_0_240 SARG: Sorting file: /tmp/sarg/10_248_0_85 SARG: Sorting file: /tmp/sarg/127_0_0_1 SARG: Sorting file: /tmp/sarg/10_248_0_112 SARG: Sorting file: /tmp/sarg/10_248_0_114 SARG: Making report: 127.0.0.1 SARG: Making report: 10.248.0.143 SARG: Making report: 10.248.0.85 SARG: Making report: 10.248.0.116 SARG: Making report: 10.248.0.201 SARG: Making report: 10.248.0.242 SARG: Making report: 10.248.0.118 SARG: Making report: 10.248.0.133 SARG: Making report: 10.248.0.240 SARG: Making report: 10.248.0.115 SARG: Making report: 10.248.0.101 SARG: Making report: 10.248.0.112 SARG: Making report: 10.248.0.117 SARG: Making report: 10.248.0.129 SARG: Making report: 10.248.0.113 SARG: Making report: 10.248.0.122 SARG: Making report: 10.248.0.114 SARG: Making index.html SARG: getword loop detected. SARG: searching for 'x20' SARG: Maybe you have a broken record or garbage in your access.log file. # ls -la squid-reports итого 16 drwxr-sr-x 4 root apache 4096 Мар 30 21:07 . drwxr-sr-x 6 root apache 4096 Мар 30 21:06 .. drwxr-sr-x 19 root apache 4096 Мар 30 21:07 23Mar2009-30Mar2009 drwxr-sr-x 2 root apache 4096 Мар 30 21:07 images -rw-r--r-- 1 root apache 0 Мар 30 21:07 index.unsort
same here (In reply to comment #15) > Result not changed. > > # mv squid-reports{,.0} > # mkdir squid-reports > # chown root:apache squid-reports > # ls -la squid-reports > итого 8 > drwxr-sr-x 2 root apache 4096 Мар 30 21:06 . > drwxr-sr-x 6 root apache 4096 Мар 30 21:06 .. > # sarg -x > SARG: Init > SARG: Loading configuration from: /etc/sarg//sarg.conf > SARG: Parameters: > SARG: > SARG: Hostname or IP address (-a) = > SARG: Useragent log (-b) = > SARG: Exclude file (-c) = > SARG: Date from-until (-d) = > SARG: Email address to send reports (-e) = > SARG: Config file (-f) = /etc/sarg//sarg.conf > SARG: Date format (-g) = Europe (dd/mm/yyyy) > SARG: IP report (-i) = No > SARG: Input log (-l) = /var/log/squid/access.log > SARG: Resolve IP Address (-n) = No > SARG: Output dir (-o) = > /srv/adm.winkhaus.com.ua/www/htdocs/squid-reports/ > SARG: Use Ip Address instead of userid (-p) = Yes > SARG: Accessed site (-s) = > SARG: Time (-t) = > SARG: User (-u) = > SARG: Temporary dir (-w) = /tmp > SARG: Debug messages (-x) = Yes > SARG: Process messages (-z) = No > SARG: > SARG: sarg version: 2.2.5 Mar-03-2008 > SARG: Maximum file descriptor: cur=1024 max=1024, changed to cur=20000 > max=20000 > SARG: Reading access log file: /var/log/squid/access.log > SARG: Records in file: 95134, reading: 100.00% > SARG: Records read: 95134, written: 95134, excluded: 0 > SARG: Squid log format > SARG: Period: 23Mar2009-30Mar2009 > SARG: pre-sorting files > SARG: Making period file > SARG: Making file: /tmp/sarg/10_248_0_133 > SARG: Making file: /tmp/sarg/10_248_0_85 > SARG: Making file: /tmp/sarg/10_248_0_101 > SARG: Making file: /tmp/sarg/10_248_0_112 > SARG: Making file: /tmp/sarg/10_248_0_242 > SARG: Making file: /tmp/sarg/10_248_0_122 > SARG: Making file: /tmp/sarg/10_248_0_143 > SARG: Making file: /tmp/sarg/10_248_0_115 > SARG: Making file: /tmp/sarg/10_248_0_113 > SARG: Making file: /tmp/sarg/10_248_0_118 > SARG: Making file: /tmp/sarg/10_248_0_240 > SARG: Making file: /tmp/sarg/10_248_0_117 > SARG: Making file: /tmp/sarg/10_248_0_201 > SARG: Making file: /tmp/sarg/10_248_0_116 > SARG: Making file: /tmp/sarg/127_0_0_1 > SARG: Making file: /tmp/sarg/10_248_0_129 > SARG: Making file: /tmp/sarg/10_248_0_114 > SARG: Sorting file: /tmp/sarg/10_248_0_242 > SARG: Sorting file: /tmp/sarg/10_248_0_117 > SARG: Sorting file: /tmp/sarg/10_248_0_118 > SARG: Sorting file: /tmp/sarg/10_248_0_122 > SARG: Sorting file: /tmp/sarg/10_248_0_115 > SARG: Sorting file: /tmp/sarg/10_248_0_101 > SARG: Sorting file: /tmp/sarg/10_248_0_129 > SARG: Sorting file: /tmp/sarg/10_248_0_133 > SARG: Sorting file: /tmp/sarg/10_248_0_113 > SARG: Sorting file: /tmp/sarg/10_248_0_201 > SARG: Sorting file: /tmp/sarg/10_248_0_116 > SARG: Sorting file: /tmp/sarg/10_248_0_143 > SARG: Sorting file: /tmp/sarg/10_248_0_240 > SARG: Sorting file: /tmp/sarg/10_248_0_85 > SARG: Sorting file: /tmp/sarg/127_0_0_1 > SARG: Sorting file: /tmp/sarg/10_248_0_112 > SARG: Sorting file: /tmp/sarg/10_248_0_114 > SARG: Making report: 127.0.0.1 > SARG: Making report: 10.248.0.143 > SARG: Making report: 10.248.0.85 > SARG: Making report: 10.248.0.116 > SARG: Making report: 10.248.0.201 > SARG: Making report: 10.248.0.242 > SARG: Making report: 10.248.0.118 > SARG: Making report: 10.248.0.133 > SARG: Making report: 10.248.0.240 > SARG: Making report: 10.248.0.115 > SARG: Making report: 10.248.0.101 > SARG: Making report: 10.248.0.112 > SARG: Making report: 10.248.0.117 > SARG: Making report: 10.248.0.129 > SARG: Making report: 10.248.0.113 > SARG: Making report: 10.248.0.122 > SARG: Making report: 10.248.0.114 > SARG: Making index.html > SARG: getword loop detected. > SARG: searching for 'x20' > SARG: Maybe you have a broken record or garbage in your access.log file. > > # ls -la squid-reports > итого 16 > drwxr-sr-x 4 root apache 4096 Мар 30 21:07 . > drwxr-sr-x 6 root apache 4096 Мар 30 21:06 .. > drwxr-sr-x 19 root apache 4096 Мар 30 21:07 23Mar2009-30Mar2009 > drwxr-sr-x 2 root apache 4096 Мар 30 21:07 images > -rw-r--r-- 1 root apache 0 Мар 30 21:07 index.unsort >
Created attachment 186794 [details, diff] util.c.patch Eh whatever I did, I still fail to reproduce this problem. With Dmitriy access.log, created directories like cpa posted here, took exactly same configuration as you have guys, but the end is always the same - sarg successfully generates report here. On the other hand if somebody of you knows what's gdb you could set break point on the line 'printf("SARG: getword loop detected.\n");' in util.c and print bt from that point (also it's could be useful to print line - 20). Another possibility to help is to use attached patch and run sarg with it. Probably this'll help to find out what fails. Or may be some other test cases allow me to reproduce this problem...
cat sarg-2.2.5-getword.patch|patch -p0 patching file sarg-2.2.5/util.c Hunk #1 FAILED at 58. Hunk #2 succeeded at 69 (offset -1 lines). 1 out of 2 hunks FAILED -- saving rejects to file sarg-2.2.5/util.c.rej VAJRA distfiles # cd sarg-2.2.5 sarg-2.2.5/ sarg-2.2.5-getword.patch sarg-2.2.5-patchset-1.tar.bz2 sarg-2.2.5.tar.gz VAJRA distfiles # cd sarg-2.2.5/ VAJRA sarg-2.2.5 # cat util.c.rej *************** *** 58,66 **** void getword(char *word, int limit, char *line, int stop) { int x = 0, y = 0; - //char wline[MAXLEN]; - //strcpy(wline,line); if(strlen(line) < 3) { word[0]='\0'; --- 58,66 ---- void getword(char *word, int limit, char *line, int stop) { int x = 0, y = 0; + char wline[MAXLEN]; + strcpy(wline,line); if(strlen(line) < 3) { word[0]='\0'; (In reply to comment #17) > Created an attachment (id=186794) [edit] > util.c.patch > > Eh whatever I did, I still fail to reproduce this problem. With Dmitriy > access.log, created directories like cpa posted here, took exactly same > configuration as you have guys, but the end is always the same - sarg > successfully generates report here. > > On the other hand if somebody of you knows what's gdb you could set break point > on the line 'printf("SARG: getword loop detected.\n");' in util.c and print bt > from that point (also it's could be useful to print line - 20). > > Another possibility to help is to use attached patch and run sarg with it. > Probably this'll help to find out what fails. > > Or may be some other test cases allow me to reproduce this problem... >
(In reply to comment #18) > cat sarg-2.2.5-getword.patch|patch -p0 > patching file sarg-2.2.5/util.c > Hunk #1 FAILED at 58. Renato, you need that patch it after all other patches. And please, don't press reply button if you don't cite anything. It just makes bug longer and thus harder to follow without any new information.
Don't think that it is very useful... but here is backtrace you asked for. Breakpoint 2, getword (word=0x5b495c18 "�\237н03", limit=4, line=0x5b4a9870 "�\237нд �\234а�\200 30 21:07:12 EEST 2009", stop=32) at util.c:72 72 printf("SARG: getword loop detected.\n"); (gdb) bt #0 getword (word=0x5b495c18 "�\237н03", limit=4, line=0x5b4a9870 "�\237нд �\234а�\200 30 21:07:12 EEST 2009", stop=32) at util.c:72 #1 0x17f3f871 in make_index () at index.c:209 #2 0x17f3325a in gerarel () at report.c:383 #3 0x17f2f2f5 in main (argc=1, argv=0x5b556ba4) at log.c:1431
(In reply to comment #20) > Don't think that it is very useful... but here is backtrace you asked for. > > Breakpoint 2, getword (word=0x5b495c18 "�\237н03", limit=4, > line=0x5b4a9870 "�\237нд �\234а�\200 30 21:07:12 EEST 2009", > stop=32) But actually it is useful, thank you Dmitriy. Now we see where it's broken: It tries to get day of a week, which normally takes tree bytes, but since localized name takes more space (�\237нд) sarg fails. Well, that's said I've put 2.2.5-r2 which makes two things: 1. modified sarg not to exit(1) in case limit passed to getword() is not enough to handle word and issue warning instead. 2. increased buffer to store localized day name and month name (so Dmitriy's problem should go away). Before I close this bug, guys, please test sarg-2.2.5-r2 and report back if it works for you. It should not issue any "Warning: limit reached at getword while searching for...
thanks I will try to sync portage now and try it (In reply to comment #21) > (In reply to comment #20) > > Don't think that it is very useful... but here is backtrace you asked for. > > > > Breakpoint 2, getword (word=0x5b495c18 "�\237н03", limit=4, > > line=0x5b4a9870 "�\237нд �\234а�\200 30 21:07:12 EEST 2009", > > stop=32) > > But actually it is useful, thank you Dmitriy. Now we see where it's broken: It > tries to get day of a week, which normally takes tree bytes, but since > localized name takes more space (�\237нд) sarg fails. > > Well, that's said I've put 2.2.5-r2 which makes two things: > > 1. modified sarg not to exit(1) in case limit passed to getword() is not enough > to handle word and issue warning instead. > 2. increased buffer to store localized day name and month name (so Dmitriy's > problem should go away). > > Before I close this bug, guys, please test sarg-2.2.5-r2 and report back if it > works for you. It should not issue any "Warning: limit reached at getword while > searching for... >
there is a sarg-2.2.5-r3 but no sarg-2.2.5-r2 in tree where can I find the patch ? (In reply to comment #21) > (In reply to comment #20) > > Don't think that it is very useful... but here is backtrace you asked for. > > > > Breakpoint 2, getword (word=0x5b495c18 "�\237н03", limit=4, > > line=0x5b4a9870 "�\237нд �\234а�\200 30 21:07:12 EEST 2009", > > stop=32) > > But actually it is useful, thank you Dmitriy. Now we see where it's broken: It > tries to get day of a week, which normally takes tree bytes, but since > localized name takes more space (�\237нд) sarg fails. > > Well, that's said I've put 2.2.5-r2 which makes two things: > > 1. modified sarg not to exit(1) in case limit passed to getword() is not enough > to handle word and issue warning instead. > 2. increased buffer to store localized day name and month name (so Dmitriy's > problem should go away). > > Before I close this bug, guys, please test sarg-2.2.5-r2 and report back if it > works for you. It should not issue any "Warning: limit reached at getword while > searching for... >
(In reply to comment #23) > there is a sarg-2.2.5-r3 but no sarg-2.2.5-r2 in tree where can I find the > patch ? It's the same, but a bit improved. BTW, renato, please, don't press 'reply' button if you don't site anything specifically. Take a look at comment #16, #22 and #23, they add only 5 lines of new information and about 150 lines of duplicate text which does not help but makes reading this bug much harder. If you wish to tell us that you are replying to specific comment, just keep only *one* line: "(In reply to comment #23)" and delete everything else. Also obviously comments like "me too" or similar does not help to resolve the issue, so generally it's better to avoid such messages. Just subscribe to the bug and/or vote if you wish. Thank you.
I have upgraded to sarg-2.2.5-r3. Now it working. ^_^ Big thanks.
Thank you Dmitriy. Fixed.
I have a lot of warnings like: SARG: Warning: limit reached at getword while searching for 'x20'. SARG: context: limit = 20, line - alberto-villan.paramowifix.net 01:09:17 08/04/2009 000000000010030 000000000636112 000000000090861 SARG: Maybe you have a broken record or garbage in your access.log file. SARG: If you think access.log is Ok, please, report this issue at bugs.gentoo.org. SARG: Warning: limit reached at getword while searching for 'x20'. SARG: context: limit = 20, line - jul-jav-villan.paramowifix.net 12:21:53 05/04/2009 000000000002126 000000000000000 000000000006648 SARG: Maybe you have a broken record or garbage in your access.log file. SARG: If you think access.log is Ok, please, report this issue at bugs.gentoo.org. With the r2, it was not ending the process (it was failing like in the r1).. now, with the r3, it finally works, but what does mean this warnings?? I've compared the reports of the previous version and this one, and seems the same report anyway... but I would like to know about this warnings... Best regards and thank you for the fix!!!
cpa, please, attach or send me directly access.log.
(In reply to comment #28) > cpa, please, attach or send me directly access.log. > I'm doing right now... Best regards...
cpa, thank you for report. That regression was caused by very long names (from sarg point of view) of local hosts. It should be fixed in sarg-2.2.5-r5. Now if anybody wants to report another similar regression, please, open new bug. It's easier to track separate issues.
(In reply to comment #30) > cpa, thank you for report. Thanks to you... I'm glad to help... Best regards... ;)
> It should be fixed in sarg-2.2.5-r5. FIXED IN SARG-2.2.5-r5... It's working right now without warnings... THANK YOU!!! :-)
really thanks pals :D
It seems still broken to me.. Version 2.2.5-r5 still has the same error. Portage 2.1.6.7 (default/linux/x86/2008.0, gcc-4.1.2, glibc-2.8_p20080602-r1, 2.6.27-gentoo-r10 i686) ================================================================= System uname: Linux-2.6.27-gentoo-r10-i686-Intel-R-_Pentium-R-_4_CPU_1.80GHz-with-glibc2.0 Timestamp of tree: Wed, 13 May 2009 01:45:03 +0000 distcc 3.0 i486-pc-linux-gnu [disabled] app-shells/bash: 3.2_p39 dev-lang/python: 2.4.4-r13, 2.5.2-r7 dev-python/pycrypto: 2.0.1-r8 sys-apps/baselayout: 2.0.0 sys-apps/openrc: 0.4.3-r1 sys-apps/sandbox: 1.2.18.1-r2 sys-devel/autoconf: 2.63 sys-devel/automake: 1.7.9-r1, 1.9.6-r2, 1.10.2 sys-devel/binutils: 2.18-r3 sys-devel/gcc-config: 1.4.0-r4 sys-devel/libtool: 1.5.26 virtual/os-headers: 2.6.27-r2 ACCEPT_KEYWORDS="x86" CBUILD="i486-pc-linux-gnu" CFLAGS="-O2 -mtune=i686 -pipe" CHOST="i486-pc-linux-gnu" CONFIG_PROTECT="/etc /var/bind" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/terminfo" CXXFLAGS="-O2 -mtune=i686 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="distlocks fixpackages parallel-fetch protect-owned sandbox sfperms strict unmerge-orphans userfetch" GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1" LINGUAS="ru en" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://192.168.21.50/gentoo-portage/" USE="7zip acl apache2 berkdb bzip2 cli cracklib crypt cups dri glibc-omitfp gpm iconv isdnlog kerberos ldap midi mudflap mysql mysqli ncurses nls nptl nptlonly openmp pam pcre perl postgres pppd python readline reflection samba sasl session slang spl sse sse2 ssl sysfs unicode x86 xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="ru en" USERLAND="GNU" VIDEO_CARDS="fbdev glint i810 intel mach64 mga neomagic nv r128 radeon savage sis tdfx trident vesa vga via vmware voodoo" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY ----------------------------------------------------------------------------------------- Sample error message: SARG: Warning: limit reached at getword while searching for 'x20'. SARG: context: limit = 15, line - 43D-à432à435à44 - - GET REDIRECT SARG: Maybe you have a broken record or garbage in your access.log file. SARG: If you think access.log is Ok, please, report this issue at bugs.gentoo.org. SARG: Warning: limit reached at getword while searching for 'x20'. SARG: context: limit = 15, line - 0à442à44C%20à43 - - GET REDIRECT SARG: Maybe you have a broken record or garbage in your access.log file. SARG: If you think access.log is Ok, please, report this issue at bugs.gentoo.org. SARG: Warning: limit reached at getword while searching for 'x20'. SARG: context: limit = 15, line - 41Dà435à442%20à ----------------------------------------------------------------------------------------- I couldn't find any lines in my access.log that make sarg warning.
Damir, do you reproduce this error with full access.log then? Could you send me it then?
(In reply to comment #35) > Damir, do you reproduce this error with full access.log then? Could you send me > it then? Yes. It is reproducable. Here is error message: SARG: Warning: limit reached at getword while searching for 'x20'. SARG: context: limit = 25, line - u1243446/33744755/x_c66b91b1.jpg 192.168.21.33/- - GET REDIRECT Here is chunk of log: 1243427198.269 88 192.168.21.32 TCP_MISS/200 4884 GET http://cs4198.userapi.com/u1243446/b_72dfda45.jpg - DIRECT/93.186.239.102 image/jpeg Real log is very 'big'. It is about 50mb. Maybe my sarg configured with wrong access log format? But I didn't change it.. [I] net-analyzer/sarg Available versions: 2.2.5 2.2.5-r1 2.2.5-r5 Installed versions: 2.2.5-r5(08:21:21 AM 04/29/2009) [U] net-proxy/squid Available versions: 2.7.6-r1 3.0.13!t (~)3.0.14-r2!t (~)3.0.15!t {elibc_uclibc ...} Installed versions: 3.0.13!t(05:20:37 PM 04/30/2009)(epoll ...)
(In reply to comment #36) > (In reply to comment #35) Sorry, I putted wrong line of access.log. But I didn't find any occurrence of 'u1243446' and/or '33744755' and/or 'x_c66b91b1.jpg' (words from error message) in log...
