Bug 98125

Summary:	rrdtool consumes 100% cpu when zoomed into detail using cacti
Product:	Gentoo Linux	Reporter:	Jelle Smet <forums>
Component:	Current packages	Assignee:	Gentoo Netmon project <netmon>
Status:	VERIFIED UPSTREAM
Severity:	critical	CC:	frederic.mangeant
Priority:	High
Version:	unspecified
Hardware:	x86
OS:	Linux
Whiteboard:
Package list:		Runtime testing required:	---
Attachments:	rrd archive, using this makes rrdtool hang with 100% cpu

Description Jelle Smet 2005-07-06 08:01:35 UTC

In cacti there is a zoom tool where you can zoom into rrdtool generated graphs.
For some reason if you keep zooming, to a resolution of 5 minutes,  no graphs
are generated anymore and the rrdtool process hangs in the background while
consuming 100% cpu.
rrdtool needs to: kill -9
There is also a post on the cacti forums:

http://forums.cacti.net/viewtopic.php?p=36336#36336

This makes an ideal DOS attack,....



Reproducible: Always
Steps to Reproduce:
1. Select a graph
2. click on the zoom tool
3. select a region that is +- 5 minutes

Actual Results:  
on the webbased userinterface no graph is produced.
Meanwhile rrdtool is still active as a process consuming 100% cpu forever.


Expected Results:  
It should keep zooming and generating graphs without consequences.

Strange but, apparently all users that are reporting this issue on the cacti.net
board are Gentoo Linux users,......
The error can be reproduced on a default installation.
The only thing that is changed in this default installation are the RRA values:
The ones used by me to generate this problem:

Name   	                      Steps     Rows    Timespan
Daily (5 Minute Average) 	1 	600 	86400 	
Weekly (5 Minute Average) 	1 	2200 	604800 	
Yearly (5 Minute Average) 	1 	105120 	33053184 	
Monthly (5 Minute Average) 	1 	9000 	2678400 	

cacti 0.8.6e 
rrdtool 1.2.6-r1

Comment 1 Frederic Mangeant 2005-07-06 09:21:03 UTC

Hi

I'm having this problem too (with Cacti 0.8.6f and RRDtool 1.2.10) on Gentoo x86.

Here's my post on the Cacti forum : http://forums.cacti.net/viewtopic.php?t=8357
and on the RRDtool ML : http://lists.ee.ethz.ch/rrd-users/msg10007.html



Portage 2.0.51.19 (default-linux/x86/2005.0, gcc-3.3.5-20050130,
glibc-2.3.4.20041102-r1, 2.6.11.5 i686)
=================================================================
System uname: 2.6.11.5 i686 Intel(R) Pentium(R) 4 CPU 2.80GHz
Gentoo Base System version 1.6.12
Python:              dev-lang/python-2.3.5 [2.3.5 (#1, May  4 2005, 09:59:21)]
dev-lang/python:     2.3.5
sys-apps/sandbox:    [Not Present]
sys-devel/autoconf:  2.13, 2.59-r6
sys-devel/automake:  1.5, 1.7.9-r1, 1.6.3, 1.8.5-r3, 1.4_p6, 1.9.5
sys-devel/binutils:  2.15.92.0.2-r10
sys-devel/libtool:   1.5.16
virtual/os-headers:  2.6.8.1-r4
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CFLAGS="-O2 -march=pentium4 -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config
/usr/lib/X11/xkb /usr/share/config /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-O2 -march=pentium4 -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs autoconfig ccache distlocks sandbox sfperms strict"
GENTOO_MIRRORS="http://distfiles.gentoo.org
http://distro.ibiblio.org/pub/Linux/distributions/gentoo"
LC_ALL="fr_FR"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.de.gentoo.org/gentoo-portage"
USE="x86 alsa apache2 apm arts avi bash-completion berkdb bitmap-fonts cdr crypt
cups emboss encode flash foomaticdb fortran gd gdbm gif gpm gtk2 imlib innodb
ipv6 jpeg ldap libg++ libwww mad mikmod motif mp3 mpeg mysql ncurses nls odbc
oggvorbis opengl oss pam pdflib perl php png python quicktime readline sdl slang
snmp spell ssl svga tcpd tiff truetype truetype-fonts type1-fonts xml2 xmms xv
zlib userland_GNU kernel_linux elibc_glibc"
Unset:  ASFLAGS, CBUILD, CTARGET, LANG, LDFLAGS, LINGUAS, PORTDIR_OVERLAY

Comment 2 Jelle Smet 2005-07-15 05:22:32 UTC

Created attachment 63461 [details]
rrd archive, using this makes rrdtool hang with 100% cpu

Comment 3 Jelle Smet 2005-07-15 05:23:28 UTC

Comment on attachment 63461 [details]
rrd archive, using this makes rrdtool hang with 100% cpu

/usr/bin/rrdtool graph - \
--imgformat=PNG \
--start=1121167317 \
--end=1121167660 \
--title="Localhost - Processes" \
--base=1000 \
--height=120 \
--width=500 \
--alt-autoscale-max \
--lower-limit=0 \
COMMENT:"From 2005/07/12 13\:21\:57 To 2005/07/12 13\:27\:40\c" \
COMMENT:" \n" \
--vertical-label="processes" \
--slope-mode \
DEF:a="/var/www/localhost/htdocs/cacti/rra/localhost_proc_7.rrd":proc:AVERAGE \
AREA:a#F51D30:"Running Processes" \
GPRINT:a:LAST:"Current\:%8.0lf" \
GPRINT:a:AVERAGE:"Average\:%8.0lf" \
GPRINT:a:MAX:"Maximum\:%8.0lf"

Comment 4 Jelle Smet 2005-07-15 12:19:04 UTC

when I strace this :

strace /usr/bin/rrdtool graph - \
--imgformat=PNG \
--start=1121167317 \
--end=1121167660 \
--title="Localhost - Processes" \
--base=1000 \
--height=120 \
--width=500 \
--alt-autoscale-max \
--lower-limit=0 \
COMMENT:"From 2005/07/12 13\:21\:57 To 2005/07/12 13\:27\:40\c" \
COMMENT:" \n" \
--vertical-label="processes" \
--slope-mode \
DEF:a="/var/www/localhost/htdocs/cacti/rra/localhost_proc_7.rrd":proc:AVERAGE \
AREA:a#F51D30:"Running Processes" \
GPRINT:a:LAST:"Current\:%8.0lf" \
GPRINT:a:AVERAGE:"Average\:%8.0lf" \
GPRINT:a:MAX:"Maximum\:%8.0lf" 


I get an infinite loop:

open("/etc/localtime", O_RDONLY)        = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0
close(3)                                = 0
open("/etc/localtime", O_RDONLY)        = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0
close(3)                                = 0
open("/etc/localtime", O_RDONLY)        = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0
close(3)                                = 0
open("/etc/localtime", O_RDONLY)        = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0
close(3)                                = 0
open("/etc/localtime", O_RDONLY)        = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0
close(3)                                = 0
open("/etc/localtime", O_RDONLY)        = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0
{st_mode=S_IFREG|0644, st_size=1067, ...}) = 0

Comment 5 Daniel Black (RETIRED) gentoo-dev

2005-07-15 16:47:29 UTC

looks as though its been worked upstream. please advice when a new 
release/workaround is released by the author.

Comment 6 Jelle Smet 2005-07-17 09:17:07 UTC

(In reply to comment #5)
> looks as though its been worked upstream. 

Could you point me to your source where you get this information from please?

Comment 7 Jelle Smet 2005-07-23 08:52:33 UTC

upgrading to sys-libs/glibc-2.3.5 solves the problem