Hence, the ability to search for things like Documentation, Installed Applications (.desktop files), and ebuild, Changelog and metadata.xml using beagle is off by default, though /etc/beagle/crawl-* can be used to turn it on. The Changelog does not seem to say why it was switched off, and I could not find any corresponding bug on Bugzilla. The previous maintainer metalgod also does not remember why it was switched off, though he says it had something to do with CPU usage complaints :) Using an old portage sync I had, I was able to trace the switching off back to at least 0.2.5 In any case, Beagle's CPU usage problems have been largely addressed, and there seems to be no reason for the default switching off. Besides, if I'm not wrong, this goes against Gentoo's policy of sticking to upstream as closely as possible. Also, please note that the other distributions I checked (Ubuntu, Debian, and Foresight) have it on by default, so it doesn't seem like its hard-disk-killingly harmful ;)
Created attachment 130526 [details, diff] Patch to re-enable system wide crawling by default Whoops! Forgot emerge --info.. -------- Portage 2.1.3.9 (default-linux/x86/2007.0/desktop, gcc-4.2.0/vanilla, glibc-2.6.1-r0, 2.6.22-11-generic i686) ================================================================= System uname: 2.6.22-11-generic i686 Intel(R) Pentium(R) M processor 1.86GHz Timestamp of tree: Mon, 10 Sep 2007 15:50:01 +0000 app-admin/eselect-compiler: 2.0.0_rc2-r1 app-shells/bash: 3.2_p17-r1 dev-java/java-config: 1.3.7, 2.0.33-r1 dev-lang/python: 2.5.1-r2 dev-python/pycrypto: 2.0.1-r6 sys-apps/baselayout: 2.0.0_rc4-r1 sys-apps/sandbox: 1.2.18.1 sys-devel/autoconf: 2.13, 2.61-r1 sys-devel/automake: 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10 sys-devel/binutils: 2.18.50.0.1 sys-devel/gcc-config: 2.0.0_rc1 sys-devel/libtool: 1.5.24 virtual/os-headers: 2.6.22-r2 ACCEPT_KEYWORDS="x86 ~x86" CBUILD="i686-pc-linux-gnu" CFLAGS="-march=native -O2 -pipe -fomit-frame-pointer" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/X11/xkb" CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/eselect/compiler /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/splash /etc/terminfo /etc/texmf/web2c" CXXFLAGS="-march=native -O2 -pipe -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig ccache collision-protect distlocks metadata-transfer sandbox sfperms strict unmerge-orphans userfetch userpriv usersandbox" GENTOO_MIRRORS="ftp://172.31.76.254/gentoo" LANG="en_IN" LDFLAGS="-Wl,-O1 -Wl,--enable-new-dtags -Wl,--sort-common" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_EXTRA_OPTS="--compress-level=0" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --filter=H_**/files/digest-*" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/portage/local/layman/navya-overlay /usr/portage/local/layman/gnome-experimental /usr/portage/local/layman/gentopia /usr/portage/local/layman/sunrise /usr/portage/local/layman/science /usr/portage/local/layman/voip /usr/portage/local/layman/xeffects /usr/portage/local/layman/enlightenment /usr/local/portage" SYNC="rsync://172.31.76.254/gentoo-portage" USE="X a52 aac acl acpi alsa ao avahi avi bash-completion beagle bitmap-fonts bonjour branding bzip2 cairo cddb cdparanoia cdr cli cracklib crypt cscope daap dbus dga directfb dri dvd dvdr dvdread emboss encode esd exif fam fbcon ffmpeg firefox flac flash fortran galago gd gdbm gif gimp glitz gnome gnutls gphoto2 gstreamer gtk gzip hal hddtemp iconv ieee1349 ipod ipv6 isdnlog jabber jack java javascript jingle jpeg jpeg2k kdexdeltas lame ldap libnotify mad matroska midi mime mjpeg mmx mng mp3 mpeg mudflap musepack musicbrainz nautilus ncurses nfs nls noamazon nptl nptlonly nsplugin offensive ogg openal opengl openmp pam pango pcre pdf perl png pppd python quicktime rdesktop readline real reflection ruby samba sdl session speex spell spl sse sse2 ssl startup-notification svg svga tcpd theora threads tiff totem truetype truetype-fonts type1-fonts unicode usb vcd vorbis wifi win32codecs wmf x264 x86 xine xml xml2 xorg xpm xv xvid zeroconf zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" ELIBC="glibc" INPUT_DEVICES="keyboard mouse synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="radeon fbdev" Unset: CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS
please check metadata.xml before assigning bugs. Thanks. FYI, I think system wide crawling is disabled on purpose. It's easy to re-enable it later on ... "safe defaults" if you will.
Yeah it really slows the system down, cf: http://blog.funtoo.org/2007/08/resolving-sabayon-and-gentoo-peformance.html
(In reply to comment #3) > Yeah it really slows the system down, cf: > http://blog.funtoo.org/2007/08/resolving-sabayon-and-gentoo-peformance.html > Okay, first off, drobbin's post is very vague on what exactly was the impact of beagle on portage's speed. Was the dependency resolution slower, was the compiling and installing slower, or was the emerge --sync slower? Assuming that the problem is ${PORTDIR} crawling related, specifically that feature should be disabled by default (/etc/beagle/crawl-portage), and a bug filed for the ebuild filter/indexing in beagle.
(In reply to comment #4) > Okay, first off, drobbin's post is very vague on what exactly was the impact of > beagle on portage's speed. Was the dependency resolution slower, was the > compiling and installing slower, or was the emerge --sync slower? Well I have to be honest and say i don't run any of these disk indexing things (including slocate) but I thought it was a general system wide comment as opposed to a specific issue with beagle and portage. (It got mixed up with a portage bug in the article.) > Assuming that the problem is ${PORTDIR} crawling related, specifically that > feature should be disabled by default (/etc/beagle/crawl-portage), and a bug > filed for the ebuild filter/indexing in beagle. > Makes sense. It should definitely not index /usr/portage as a default.
(In reply to comment #5) > (including slocate) but I thought it was a general system wide comment as > opposed to a specific issue with beagle and portage. (It got mixed up with a > portage bug in the article.) Well, he says, quote, "I noticed that Portage was horrendously slow, and discovered the first culprit - Beagle, a search tool that is enabled by default under Sabayon" so, I guess it _is_ related to Portage :)
(In reply to comment #6) > Well, he says, quote, > > "I noticed that Portage was horrendously slow, and discovered the first culprit > - Beagle, a search tool that is enabled by default under Sabayon" > > so, I guess it _is_ related to Portage :) > OK, you got me ;-) I still think it's better not to have system-wide crawling by default; if it's /etc/crawl-portage, great. But would the average user really want eg /var/log indexed in such a manner for desktop search? (Apologies if I am missing something in default beagle, I don't use it as I said.)
(In reply to comment #7) > OK, you got me ;-) I still think it's better not to have system-wide crawling > by default; if it's /etc/crawl-portage, great. But would the average user > really want eg /var/log indexed in such a manner for desktop search? (Apologies > if I am missing something in default beagle, I don't use it as I said.) I guess calling it 'system-wide indexing' is a misnomer :P Beagle only crawls according to what is present in the /etc/beagle/crawl-* files. A full list of the default directories crawled is: * /etc/beagle/crawl-applications: /usr/share/applications /usr/local/share/applications /opt/gnome/share/applications /opt/kde3/share/applications /usr/kde/*/share/applications * /etc/beagle/crawl-documentation /usr/share/doc /usr/local/share/doc /opt/kde3/share/doc /opt/gnome/share/gnome/help /usr/share/gnome/help /opt/gnome/share/gtk-doc/html /usr/share/gtk-doc/html /usr/share/gnome/html /usr/kde/*/share/doc * /etc/beagle/crawl-portage (not upstream) ${PORTDIR} /var/db/pkg These directories (except for portage stuff) do not change frequently, and hence are just a one-time index. Also, installed applications and documentation are indeed relevant for the average desktop user, and have a case for being searchable.
(In reply to comment #8) > * /etc/beagle/crawl-portage (not upstream) > ${PORTDIR} > /var/db/pkg > > These directories (except for portage stuff) do not change frequently, and > hence are just a one-time index. Also, installed applications and documentation > are indeed relevant for the average desktop user, and have a case for being > searchable. > Ah thanks for the info! Yeah I agree with all but the portage stuff, I just don't see that as being useful. You could argue /var/db/pkg for installed apps, but I'd disagree; /usr/portage is just crazy imo: that's what eix or qgrep is for. Anyhow, it clearly affects stuff (/var/db/pkg will i imagine have less effect but I really don't think it should be set for the average user-- benchmarks would be nice I suppose.) What do you mean a bug for "ebuild filter/indexing in beagle"? Isn't that what this is (effectively-- since this seems to be the nub of the issue)?
(In reply to comment #9) > Ah thanks for the info! Yeah I agree with all but the portage stuff, I just > don't see that as being useful. You could argue /var/db/pkg for installed apps, Ah, the nuisances of misunderstandings :) /var/db/pkg (and hence crawl-portage) is not used for searching installed applications, crawl-applications is; which searches menu items for installed applications. > but I'd disagree; /usr/portage is just crazy imo: that's what eix or qgrep is > for. Anyhow, it clearly affects stuff (/var/db/pkg will i imagine have less > effect but I really don't think it should be set for the average user-- > benchmarks would be nice I suppose.) Yes, I agree, ${PORTDIR} and /var/db/pkg won't be of use to the average user and they should be disabled by default - most of them won't be using it anyways :) > What do you mean a bug for "ebuild filter/indexing in beagle"? Isn't that what > this is (effectively-- since this seems to be the nub of the issue)? > Well, I'd say this bug is for getting documentation and application indexing back on by default. I think a separate bug for the ebuild filter should be there because beagle is usually very very efficient at indexing things that are constantly changing (think downloading files, torrents, Direct Connect), and it usually doesn't result in a performance loss. So if something in beagle does indeed have a huge impact on system performance, it is definitely a major bug, and should be addressed as such.
I think we can close this bug
