Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 233455 - app-misc/beagle-0.3.7-r1 eats all the memory and make the system swapping.
Summary: app-misc/beagle-0.3.7-r1 eats all the memory and make the system swapping.
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Cédric Krier
Depends on:
Reported: 2008-07-31 07:48 UTC by Juergen Rose
Modified: 2009-03-03 04:50 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Note You need to log in before you can comment on or make changes to this bug.
Description Juergen Rose 2008-07-31 07:48:48 UTC
My system slows down and beagled appear to be the reason. Top shows me:
top - 06:47:21 up 2 days, 19:55,  2 users,  load average: 5.20, 4.56, 3.66
Tasks: 245 total,   4 running, 240 sleeping,   0 stopped,   1 zombie
Cpu(s): 10.5%us,  2.1%sy,  0.3%ni, 52.4%id, 34.7%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   7073148k total,  6964236k used,   108912k free,   199068k buffers
Swap: 35672304k total, 35639652k used,    32652k free,   175340k cached

 9108 rose      27   7 38.7g 5.0g 2024 S    1 73.6   2361:38 beagled

I.e., beagled uses 38.7GB virtual and 5GB real memory.

Any Hint?

Reproducible: Always

root@condor:/usr/src/Graphics/ptplot5.2(11)# emerge --info
Portage 2.2_rc5 (default-linux/amd64/2007.0, gcc-4.2.4, glibc-2.8_p20080602-r0, 2.6.26-gentoo x86_64)
System uname: Linux-2.6.26-gentoo-x86_64-Intel-R-_Core-TM-2_CPU_6600_@_2.40GHz-with-glibc2.2.5
Timestamp of tree: Thu, 31 Jul 2008 05:36:01 +0000
distcc 2.18.3 x86_64-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
app-shells/bash:     3.2_p39
dev-java/java-config: 1.3.7, 2.1.6-r1
dev-lang/python:     2.4.4-r13, 2.5.2-r5
dev-python/pycrypto: 2.0.1-r6
sys-apps/baselayout: 2.0.0
sys-apps/openrc:     0.2.5
sys-devel/autoconf:  2.13, 2.62-r1
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.1-r1
sys-devel/binutils:  2.18-r3
sys-devel/gcc-config: 1.4.0-r4
sys-devel/libtool:   2.2.4
virtual/os-headers:  2.6.25-r4
ACCEPT_KEYWORDS="amd64 ~amd64"
CFLAGS="-march=nocona -O2 -pipe"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-march=nocona -O2 -pipe"
FEATURES="distlocks fixpackages parallel-fetch preserve-libs sandbox sfperms strict unmerge-orphans userfetch"
LINGUAS="de fr"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTDIR_OVERLAY="/usr/local/portage/layman/arcon /usr/local/portage/layman/science /usr/local/portage/layman/sajinet /usr/local/portage"
USE="X Xaw3d a52 aac acl acpi administrator afs aiglx alsa amd64 amrr ao aotuv apache2 asf atlas auctex audacious audiofile automount bash-completion beagle berkdb bjam blas bluetooth bonobo boo boost bzip2 c++ cairo cdda cddb cdf cdio cdparanoia cdr cgi chm cli corba cracklib cran crypt cups curl cxx daap dbus device-mapper dga dia divx djvu dlloader dmi dri ds3490 dv dvb dvd dvdr dvi dxr3 dynagraph eds effects elf emacs emboss encode epiphany esd evo examples exif expat extra fam fame ffmpeg fftw firefox fits flac fltk foomaticdb fortran fpx galago gd gdal gdbm geos gif gimp gimpprint ginac glade glitz gml gmp gnokii gnome gnuplot gnutls gphoto2 gpm graphics graphviz gs gsl gsm gstreamer gtk guile hal hardened hddtemp hdf hdf5 hlapi http hvm iconv icq icu id3 imagemagick imap innodb ipv6 irda irmc isdnlog ithreads jabber java jbig john jpeg jpeg2k kerberos kexi keyring ladspa lame lapack latex lcms ldap libffi libgda libsamplerate lirc lm_sensors lua lzo lzw mad maildir math matroska midi mmx mmxext mng mod mono motif mozdevelop mozilla mozsvg mozxmlterm mp3 mp4 mp4live mpeg mpeg2 mplayer mudflap musicbrainz mysql mysqli nautilus ncurses neXt netcdf netpbm network networkmanager nfs nls nntp nptl nptlonly nsplugin ntfs numarray numeric nvidia obex ocaml octave odbc ogdi ogg ole opengl openmp pae pam pcre pda pdf perl plotutils plugins png posix postgres postscript pppd preview-latex proj projectx pstricks python qhull quicktime readline reflection reiserfs rhythmbox rle rpc rrdcgi rrdtool samba sasl science sdl session slang slp smp sndfile snmp soup sox speex spell spl sqlite sse sse2 ssl stlport subtitles subversion suexec svg t1lib tcl tcpd tex theora threads thunderbird tidy tiff tk truetype unicode userlocales utempter v4l2 vorbis wmf wxwindows xattr xemacs xext xine xml xmlreader xmlrpc xorg xpm xv xvid xvmc zlib zvbi" ALSA_CARDS="intel8x0" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse                " KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="de fr" USERLAND="GNU" VIDEO_CARDS="nv nvidia vesa"
Comment 1 Arun Raghavan (RETIRED) gentoo-dev 2009-01-18 06:50:04 UTC
Could you check ~/.beagle/Log/current-* to see if Beagle is getting stuck while indexing a particular file?
Comment 2 Juergen Rose 2009-03-02 09:10:26 UTC
In the last weeks I had no problem with beagle. It looks if I have current beagle files:

rose@condor:/home/rose(5)$ ll  ~/.beagle/Log/current-*
lrwxrwxrwx 1 rose users 49 Mar  1 11:13 /home/rose/.beagle/Log/current-Beagle -> /home/rose/.beagle/Log/2009-03-01-11-13-10-Beagle
lrwxrwxrwx 1 rose users 56 Mar  2 08:58 /home/rose/.beagle/Log/current-BeagleConsole -> /home/rose/.beagle/Log/2009-03-01-11-13-10-BeagleConsole
lrwxrwxrwx 1 rose users 59 Mar  1 11:13 /home/rose/.beagle/Log/current-BeagleExceptions -> /home/rose/.beagle/Log/2009-03-01-11-13-10-BeagleExceptions
lrwxrwxrwx 1 rose users 54 Mar  2 08:58 /home/rose/.beagle/Log/current-IndexHelper -> /home/rose/.beagle/Log/2009-03-02-08-58-44-IndexHelper

rose@condor:/home/rose(7)$ ll /home/rose/.beagle/Log/2009-03-01-11-13-10-Beagle  /home/rose/.beagle/Log/2009-03-01-11-13-10-BeagleConsole  /home/rose/.beagle/Log/2009-03-01-11-13-10-BeagleExceptions  /home/rose/.beagle/Log/2009-03-02-08-58-44-IndexHelper
-rw-r--r-- 1 rose users 334635 Mar  2 08:58 /home/rose/.beagle/Log/2009-03-01-11-13-10-Beagle
-rw-r--r-- 1 rose users      0 Mar  2 08:58 /home/rose/.beagle/Log/2009-03-01-11-13-10-BeagleConsole
-rw-r--r-- 1 rose users  47024 Mar  1 11:27 /home/rose/.beagle/Log/2009-03-01-11-13-10-BeagleExceptions
-rw-r--r-- 1 rose users   5505 Mar  2 09:34 /home/rose/.beagle/Log/2009-03-02-08-58-44-IndexHelper

And I can recognize something special in those files:
rose@condor:/home/rose(11)$ tail .beagle/Log/2009-03-01-11-13-10-Beagle
20090302 08:53:28.8423 05346 Beagle DEBUG: Could not resolve unique id of '.Xauthority-c' in '/home/rose' for removal -- it is probably already gone
20090302 08:53:28.8429 05346 Beagle DEBUG: Could not resolve unique id of '.Xauthority-l' in '/home/rose' for removal -- it is probably already gone
20090302 08:58:44.1762 05346 Beagle DEBUG: Caught ResponseMessageException: Connection refused
20090302 08:58:44.1763 05346 Beagle DEBUG: InnerException is SocketException -- we probably need to launch a helper
20090302 08:58:44.1764 05346 Beagle DEBUG: Launching helper process
20090302 08:58:44.1857 05346 Beagle DEBUG: IndexHelper PID is 15492
20090302 08:58:45.1865 05346 Beagle DEBUG: Found IndexHelper (15492) in 1.00s
For log files check /home/rose/.beagle/Log/current-Beagle.

rose@condor:/home/rose(12)$ tail .beagle/Log/2009-03-01-11-13-10-BeagleExceptions
20090301 11:27:03.3354 05346 Beagle  WARN EX: (Repeated) Caught exception in DoTaskReal ...
20090301 11:27:03.8365 05346 Beagle  WARN EX: (Repeated) Caught exception in DoTaskReal ...
20090301 11:27:05.9294 05346 Beagle  WARN EX: (Repeated) Caught exception in DoTaskReal ...
20090301 11:27:06.4295 05346 Beagle  WARN EX: (Repeated) Caught exception in DoTaskReal ...
20090301 11:27:09.4485 05346 Beagle  WARN EX: (Repeated) Caught exception in DoTaskReal ...
20090301 11:27:09.9494 05346 Beagle  WARN EX: (Repeated) Caught exception in DoTaskReal ...
20090301 11:27:11.9994 05346 Beagle  WARN EX: (Repeated) Caught exception in DoTaskReal ...
20090301 11:27:12.5007 05346 Beagle  WARN EX: (Repeated) Caught exception in DoTaskReal ...
20090301 11:27:14.8495 05346 Beagle  WARN EX: (Repeated) Caught exception in DoTaskReal ...
20090301 11:27:17.8246 05346 Beagle  WARN EX: (Repeated) Caught exception in DoTaskReal ...

rose@condor:/home/rose(13)$ tail  /home/rose/.beagle/Log/2009-03-02-08-58-44-IndexHelper
20090302 09:04:31.1105 15492 IndexH DEBUG: +file:///home/rose/scripts/
20090302 09:34:33.0329 15492 IndexH DEBUG: No activity for 30.0 minutes, shutting down
20090302 09:34:33.0341 15492 IndexH  INFO: Shutdown requested
20090302 09:34:33.0356 15492 IndexH DEBUG: (1) Waiting for 1 worker...
20090302 09:34:33.0357 15492 IndexH DEBUG: waiting for server '/home/rose/.beagle/socket-helper'
20090302 09:34:33.0364 15492 IndexH DEBUG: Server '/home/rose/.beagle/socket-helper' shut down
20090302 09:34:33.0366 15492 IndexH DEBUG: All workers have finished.  Exiting main loop.
20090302 09:34:33.0380 15492 IndexH DEBUG: Joining thread 1 of 1: EHT 15498 [15492 IndexHelper] Beagle.IndexHelper.IndexHelperTool:DaemonMonitorWorker
20090302 09:34:33.0398 15492 IndexH DEBUG: Exiting signal handler thread
20090302 09:34:33.3226 15492 IndexH  INFO: Index helper process shut down cleanly.
Comment 3 Arun Raghavan (RETIRED) gentoo-dev 2009-03-03 04:49:04 UTC
If things are working fine now, the problematic file(s) probably got removed sometime along the way. The exception that you show is not informative, since the exception stack is not there.

The next time you see a bug like this, please check the logs to see where Beagle is stuck immediately, so it's easier to debug. Running beagled with the --debug flag might also be useful.

Closing for now as the problem seems to have gone away.