Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 239243

Summary: app-portage/eix-0.13.3-r1: takes incredible long time to update
Product: Gentoo Linux Reporter: Albert Zeyer <albzey>
Component: Current packagesAssignee: Stefan Schweizer (RETIRED) <genstef>
Status: RESOLVED CANTFIX    
Severity: normal CC: martin
Priority: High    
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---

Description Albert Zeyer 2008-10-01 12:12:00 UTC
acompneu ~ # update-eix
Reading Portage settings ..
Building database (/var/cache/eix) ..
[0] "gentoo" /usr/portage/ (cache: metadata-flat)
     Reading 100%
[1] "kde4-overlay" /usr/portage/local/layman/kdesvn-portage (cache: parse|ebuild*)
     Reading  47%

Building the database for gentoo takes about 200ms (just an estimation, it's perhaps even faster), then it hangs for about 400ms on 11% of kde4-overlay and then it hangs at 47% about some minutes or longer. It seems to me as a bug because I have 100% CPU (only one core) and no single disk activitiy while it hangs at 47%. And this is a 3.16 GHz Core2Duo CPU, so it really should not take that long.

The bug is reproducible, I have always the same behaviour.

I don't know if there is something messed up in the kde4-overlay which is the reason for this behaviour. But even then, shouldn't eix be able to handle this better?
Comment 1 Albert Zeyer 2008-10-01 12:12:18 UTC
acompneu ~ # emerge --info
Error during set creation: Redefinition of set 'live-rebuild' (sections: 'live-rebuild', 'kde4-overlay sets')
Error during set creation: Redefinition of set 'live-rebuild' (sections: 'live-rebuild', 'kde4-overlay sets')
Portage 2.2_rc8 (default/linux/x86/2008.0/desktop, gcc-4.1.2, glibc-2.6.1-r0, 2.6.27-rc4 i686)
=================================================================
System uname: Linux-2.6.27-rc4-i686-Intel-R-_Core-TM-2_CPU_E8500_@_3.16GHz-with-glibc2.0
Timestamp of tree: Wed, 01 Oct 2008 11:30:01 +0000
distcc 2.18.3 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
app-shells/bash:     3.2_p33
dev-java/java-config: 1.3.7, 2.1.6
dev-lang/python:     2.4.4-r13, 2.5.2-r6
dev-python/pycrypto: 2.0.1-r6
sys-apps/baselayout: 1.12.11.1
sys-apps/sandbox:    1.2.18.1-r2
sys-devel/autoconf:  2.13, 2.61-r2
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.1
sys-devel/binutils:  2.15.92.0.2-r10, 2.16.1-r3, 2.17-r1, 2.18-r3
sys-devel/gcc-config: 1.4.0-r4
sys-devel/libtool:   1.5.26
virtual/os-headers:  2.6.26
ACCEPT_KEYWORDS="x86"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=nocona -ggdb -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/kde/svn/env /usr/kde/svn/share/config /usr/kde/svn/shutdown /usr/share/config"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-O2 -march=nocona -ggdb -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="candy distlocks parallel-fetch preserve-libs sandbox sfperms splitdebug unmerge-orphans userfetch"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"
LANG="en_US.UTF-8"
LC_ALL="en_US.UTF-8"
LDFLAGS="-Wl,-O1"
LINGUAS="de"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/portage/local/layman/kdesvn-portage /usr/portage/local/layman/mpd /usr/portage/local/private"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="GAPING_SECURITY_HOLE X X509 a52 aac aalib acl acpi acpi4linux adns afs agg aiglx alsa amd amuled ao aqua_theme archive ares asf asm async asyncns atm auctex avahi bash-completion berkdb bidi binary binary-drivers bindist bittorrent bl bluetooth bonjour bookmarks bootsplash branding bzip2 bzlib cairo captury cdda cddb cdio cdparanoia cdr cdrom cgi cli clisp clucene cmake codecs corba cracklib crypt cups curl custreloc dbus dedicated dga dhcp disk-partition divx4linux dlopen dmi dnd dri dts dv dvd dvdnav dvdr dvdread dxr3 eds emboss encode esd evo exif faad fam fame ffmpeg firefox flac flash font-server foomaticdb fortran fpx freetype ftp galago gcj gcl gd gdbm ggi gif gimp gl glib glitz glslideshow glut gnome gphoto2 gpm gprof graphviz gs gstreamer gtk gtk2 gzip h323 hal haskell howl howl-compat html httpd hyperestraier iconv icq id3 id3tag imagemagick imap imlib imlib2 inotify insecure-drivers ipv6 isdnlog ithreads jack java jbig jikes jit jpeg jpeg2k kde kerberos lame latex lcms ldap libcaca libnotify libwww linuxthreads-tls live lzo mad matroska mbox mdnsresponder-compat midi mikmod mjpeg mmx mmxext mng mod modplug mozbranding mozcalendar mozilla mp2 mp3 mp4 mp4live mpeg mpeg2 mpeg4 mplayer mppe-mppc mudflap multislot musepack musicbrainz mysql nautilus ncurses nepomuk net network njb nls nntp nptl nptlonly nsplugin objc offensive ogg oggvorbis openal opengl openmp oss pam pascal pch pcre pdf perl php plasma png pop posix ppds pppd prebuilt-system pthreads pulseaudio python qt qt3 qt3support qt4 quicktime rar rdesktop readline real realmedia reflection rogue rplay rtc rtsp ruby samba sasl screen sdl semantic-desktop server session shorten shout silvercity silverxp skins slp sndfile sockets solver speedo speex spell spl sqlite sse sse2 ssh ssl ssse3 startup-notification stream subversion svg sysfs tagwriting tcltk tcpd teamarena tetex tga themes theora threads threadsafe thumbnail tiff transcode truetype type1 unicode urandom usb v4l v4l2 vcd vdr vidix visualization vlm vorbis win32codecs wma wmf wxwindows x264 x86 xanim xatrix xcomposite xext xface xforms xine xml xorg xosd xpm xprint xrandr xrender xscreensaver xsettings xterm xulrunner xv xvid xvmc zeroconf zip zlib zvbi" ALSA_CARDS="emu10k1" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="de" USERLAND="GNU" VIDEO_CARDS="nvidia nv apm"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

Comment 2 Sebastian Luther (few) 2008-10-01 12:33:20 UTC
I can reproduce this but I don't know if this is a bug. eix has to use another cache method for overlays then for portage. You might want to change the cache method. This can be done with the  OVERLAY_CACHE_METHOD and CACHE_METHOD variables in ~./eixrc or /etc/eixrc. See "man eix".
Comment 3 Martin Väth 2008-10-01 14:06:50 UTC
This is the time which has to be expected with cache method parse|ebuild*
for this particular overlay, because in this case the fast cache method
"parse" always fails (because all ebuilds rely on eclasses):
So eix has to execute bash for each single ebuild - the disk cache will buffer
these calls, but bash is just terribly slow.
The percentage status is explained by the fact that it just relies on the
categories: Since almost the whole overlay is contained in one category, the
number is somewhat misleading in this case.

As Sebastian suggested, a workaround might be to force cache method "parse" for
this particular overlay, but then some data (like e.g. slot information) which
can be obtained only through eclasses might be wrong/missing.

A further alternative might be to store that overlay in a separate eix database
file and to use cache method "eix:..." for that overlay; this will be incredibly
fast, but it means that the overlay is actually not updated in the eix database
but only the previous state of the overlay will be stored.