Long back on version 0.4.x eupdatedb (updating the database) was blazingly fast with the counter of ebuilds just reeling down to zero in no time. Now, on version 0.5.2 eupdatedb crawls along at an *extremely slow* speed on the same machines. Just to give you an idea of how slow it is the counter decreases by one every 2-3 seconds. I have no idea what could have caused this drastic decrease in speed. It has happened on both my machines which are reasonably high spec in every way. Perhaps some investigation is required here on what may have caused this because if I can't build the database then the use of the tool becomes more unlikely. Thanks. $ emerge info Portage 2.0.49-r20 (default-x86-1.4, gcc-3.2.3, glibc-2.3.2-r3, 2.4.24) ================================================================= System uname: 2.4.24 i686 AMD Athlon(tm) processor Gentoo Base System version 1.4.3.10p1 distcc 2.11.1 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [enabled] ccache version 2.3 [enabled] ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CFLAGS="-march=athlon-tbird -O3 -pipe" CHOST="i686-pc-linux-gnu" COMPILER="gcc3" CONFIG_PROTECT="/etc /opt/tomcat/conf /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/env.d" CXXFLAGS="-march=athlon-tbird -O3 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs buildpkg ccache distcc notitles sandbox" GENTOO_MIRRORS="http://212.219.56.162/sites/www.ibiblio.org/gentoo/ http://212.219.56.152/sites/www.ibiblio.org/gentoo/ http://212.219.56.131/sites/www.ibiblio.org/gentoo/ http://194.83.57.3/sites/www.ibiblio.org/gentoo/" MAKEOPTS="-j4" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="" SYNC="rsync://rsync.uk.gentoo.org/gentoo-portage" USE="acpi acpi4linux apache2 berkdb crypt curl dedicated fax foomaticdb gd gd-external gif imap innodb jabber java jpeg junit libwww mad maildir md5sum msn mysql nas ncurses oggvorbis oscar pam parse-clocks pdflib perl php png python readline samba slang spell ssl tcpd tetex tiff truetype x86 xml xml2 yahoo zlib" $ emerge info Portage 2.0.49-r21 (default-x86-1.4, gcc-3.2.3, glibc-2.3.2-r1, 2.6.1-gentoo) ================================================================= System uname: 2.6.1-gentoo i686 Intel(R) Pentium(R) 4 CPU 2.66GHz Gentoo Base System version 1.4.3.12 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CFLAGS="-march=pentium4 -O3 -funroll-loops -fprefetch-loop-arrays -pipe -fomit-frame-pointer" CHOST="i686-pc-linux-gnu" COMPILER="gcc3" CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/env.d" CXXFLAGS="-march=pentium4 -O3 -funroll-loops -fprefetch-loop-arrays -pipe -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs ccache sandbox" GENTOO_MIRRORS="http://ftp.gentoo.skynet.be/pub/gentoo/ http://ftp.snt.utwente.nl/pub/os/linux/gentoo http://gentoo.linux.no/" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="" SYNC="rsync://rsync.uk.gentoo.org/gentoo-portage" USE="X aalib acpi acpi4linux apache2 avi berkdb cdr cgi crypt cups directfb dvd dvdr encode ethereal expat faad fam fbcon foomaticdb gdbm gif gimpprint gtk gtk2 imagemagick imlib imlib2 jabber java javamail javascript jpeg libwww mad maildir mmx motif mpeg mpeg4 msn ncurses oggvorbis openal opengl oscar oss pam parse-clocks pdflib perl png python quicktime readline sdl slang sse ssl svga tcpd truetype x86 xalan xerces xml xml2 xmms xv yahoo zlib"
Perhaps it was manifested by load on the machines as one of them it has resumed normal speed. If it works for you all it may indicate a rare scenario in my case where it was slow. You may mark as WORKSFORME if you think so.
I've noticed a speed decrease some time ago, eupdatedb was/is running for 2 hours. Haven't really investigated it further as I thought it's caused by my NFS setup.
well, since esearch uses portage as it's backend, any speed hits is more of a portage problem isnt it ?
I tried eupdatedb twice from the commandline. The first time, I happened to also emerge other packages (not during the whole update though) and it took 19,5 minutes. I run top, and I saw, besides the eupdatedb command, a lot of /usr/sbin/ebuild.sh depend processes running one after the other (i.e. not concurrently). The second time, eupdatedb was running with no other load, and it completed within 1,5 minutes. I run top, but no ebuild.sh was noticed running. I own a 1,5GHz Pentium-M (Centrino). HTH
right, all that `/usr/sbin/ebuild.sh depend` stuff is from portage, not eupdatedb
Hi, I did some benchmarks on eupdate some weeks ago, and I found out, that this is a (really annoying) problem of portage. I think the problem is the save_eclassdb() method in portage.py. This function is called ~10000 times during an eupdatedb. I have not found out what it is doing exactly, but it takes a lot of time. I hope this will be fixed in some of the following portage updates altough it's not that important for me, because I run eupdatedb (esync) once a day, and it only takes me about 2 minutes averagely. David
I solved it by executing emerge metadata Kind regards, DQ
and metadata should be properly created after `emerge sync`
Created attachment 509014 [details] Screenshot whhich shows now load but zombie With esearch 1.3-r1 i see the problem again. It starts with >100/s but after a few thousands it goes to 1/s max 3/s. In this case a ebuild.sh zombie can be found. There will no CPU time consumed this seems only waiting times. I think there is a bug in the part which is calling ebuild.sh. It seems to die. This produces the zombie and the part which called the crashed part is waiting of its child. This slow down the whole process of updating the database. cpuinfo: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
If it's taking a log time then that means that metadata cache needs to be generated. You can use `emerge --regen --jobs=NUM` to generate the cache using multiple cpu cores.
