I've just spent half an hour waiting for java-sdk-docs to emerge. Which is silly, since essentially there is nothing to build, just a bunch of files to extract and copy. But it's that copying, done by doins during the install phase, which was so slow. Some output from genlop -t dev-java/java-sdk-docs: Thu Jan 24 10:43:22 2013 >>> dev-java/java-sdk-docs-1.7.0.11 merge time: 1 minute and 35 seconds. Mon Mar 11 07:16:00 2013 >>> dev-java/java-sdk-docs-1.7.0.17 merge time: 1 minute and 55 seconds. Tue May 21 12:49:35 2013 >>> dev-java/java-sdk-docs-1.7.0.21 merge time: 3 minutes and 30 seconds. Wed Jun 26 09:55:58 2013 >>> dev-java/java-sdk-docs-1.7.0.25 merge time: 23 minutes and 23 seconds. Sat Sep 21 15:16:02 2013 >>> dev-java/java-sdk-docs-1.7.0.40-r1 merge time: 27 minutes and 26 seconds. Sat Nov 2 17:28:52 2013 >>> dev-java/java-sdk-docs-1.7.0.45 merge time: 34 minutes and 22 seconds. I can correlate that to genlop portage: Mon May 6 09:50:17 2013 >>> sys-apps/portage-2.2.0_alpha174 Tue May 21 17:37:49 2013 >>> sys-apps/portage-2.2.0_alpha176 Mon Jun 3 18:47:49 2013 >>> sys-apps/portage-2.2.0_alpha177 Wed Jun 12 19:38:10 2013 >>> sys-apps/portage-2.2.0_alpha179 Sun Jun 16 21:27:11 2013 >>> sys-apps/portage-2.2.0_alpha180 Wed Jun 19 20:41:14 2013 >>> sys-apps/portage-2.2.0_alpha181 Mon Jun 24 10:23:37 2013 >>> sys-apps/portage-2.2.0_alpha184 Wed Jun 26 11:52:32 2013 >>> sys-apps/portage-2.2.0_alpha185 So apparently alpha174 was used for the last fast install, while alpha184 was used for the first slow install. I'd say this problem was likely introduced somewhere in that window. Of course it might as well have been a config change on my machine, don't recall. I have a vague recollection that I observed similar behavior once in the past, and perhaps even mentioned it somewhere, but I don't recall when or where, and I couldn't find a matching bug report yet. Judging from pstree and strace, the time is mostly spent in MANY instances of python3.3 running /usr/lib64/portage/bin/install.py, one after the other. Perhaps that script can be replaced by a compiled binary? Or perhaps we can at least get Python bytecode for it? Another approach would be reducing the number of times that script has to be invoked, by moving recursion into it. Or perhaps one can even get rid of that script altogether? Portage 2.2.7 (default/linux/amd64/13.0/desktop/kde, gcc-4.7.3, glibc-2.17, 3.11.4-gentoo x86_64) ================================================================= System uname: Linux-3.11.4-gentoo-x86_64-AMD_Phenom-tm-_II_X4_945_Processor-with-gentoo-2.2 KiB Mem: 15672840 total, 1150884 free KiB Swap: 6836220 total, 6832004 free Timestamp of tree: Fri, 01 Nov 2013 23:15:01 +0000 ld GNU ld (GNU Binutils) 2.23.2 distcc 3.1 x86_64-pc-linux-gnu [disabled] ccache version 3.1.9 [disabled] app-shells/bash: 4.2_p45 dev-java/java-config: 2.2.0::java dev-lang/python: 2.6.8-r3, 2.7.5-r4, 3.2.5-r3, 3.3.2-r2 dev-util/ccache: 3.1.9-r3 dev-util/cmake: 2.8.12 dev-util/pkgconfig: 0.28 sys-apps/baselayout: 2.2 sys-apps/openrc: 0.12.4 sys-apps/sandbox: 2.6-r1 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.6, 1.12.6, 1.14 sys-devel/binutils: 2.23.2 sys-devel/gcc: 3.4.6-r2, 4.1.2, 4.5.4, 4.6.4, 4.7.3-r1, 4.8.1-r1 sys-devel/gcc-config: 1.8 sys-devel/libtool: 2.4.2 sys-devel/make: 3.82-r4 sys-kernel/linux-headers: 3.11 (virtual/os-headers) sys-libs/glibc: 2.17 Repositories: sunrise-enabled ldc bump gentoo java generated sage-on-gentoo bugfix mvg-java-experimental mvgLocal ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="* -@EULA dlj-1.1 sun-bcla-java-vm Oracle-BCLA-JavaSE skype-eula skype-4.0.0.7-copyright googleearth AdobeFlash-10.3 AdobeFlash-11.x PUEL" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=amdfam10 -O2 -ggdb -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/maven-bin-3.1/conf /usr/share/themes/oxygen-gtk/gtk-2.0 /var/lib/hsqldb" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/apache2-php5.5/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cgi-php5.5/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/php/cli-php5.5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=amdfam10 -O2 -ggdb -pipe" DISTDIR="/usr/portage/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs collision-protect config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://mirror.opteamax.de/gentoo/ http://ftp-stud.hs-esslingen.de/pub/Mirrors/gentoo/ http://mirror.netcologne.de/gentoo/" LANG="de_DE.utf8" LDFLAGS="-Wl,--as-needed" MAKEOPTS="-j5" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/portage/local/sunrise-enabled /usr/portage/local/layman/ldc /usr/portage/local/bump /usr/portage /usr/portage/local/layman/java /usr/portage/local/generated /usr/portage/local/layman/sage-on-gentoo /usr/portage/local/bugfix /usr/portage/local/mvg-java /usr/portage/local/mvg" SYNC="rsync://rsync.de.gentoo.org/gentoo-portage" USE="X a52 aac acl acpi alsa amd64 apache2 audacious audiofile avahi bash-completion bcmath berkdb branding bzip2 c++ cairo cdda cdparanoia cdr chroot cli consolekit cracklib crypt css cups curl cxx dba dbus declarative dhcp doc dri dts dv dvd dvdr emacs emboss encode escreen exif fam fastcgi ffmpeg fftw firefox flac flatfile fortran ftp gcc-libffi gd gdbm gif gimp gnutls gphoto2 gpm graphviz gs gstreamer gtk gtk3 hal hbci html iconv icu idn imagemagick ipv6 iso14755 ithreads jabber jack java jpeg jpeg2k kde kerberos kipi kpathsea kvm ladspa latex lcms ldap leim libnotify lirc lm_sensors logrotate lzma lzo mad maildir mhash midi mikmod mime mjpeg mmx mng modules mozxmlterm mp3 mp4 mpeg mpeg2 mplayer mudflap multilib mysql ncurses network nls nptl nsplugin numpy objc odbc ofx ogg openexr opengl openmp pam pango pcre pdf perl phonon php plasma plotutils png policykit povray ppds procmail python qt3support qt4 quicktime rdesktop readline real recode sasl scanner sdl session sip sipim smime sndfile snmp sockets socks5 sox speex spell sqlite sse sse2 ssl startup-notification subversion svg tcl tcpd theora threads thunderbird tiff timidity tokenizer transcode translator truetype type1 udev udisks unicode upower usb userlocales v4l v4l2 vhosts vorbis vpx win32codecs wmf wxwidgets x264 xanim xattr xcb xcomposite xine xinerama xinetd xml xmp xprint xscreensaver xulrunner xv xvid xvmc zlib" ABI_X86="64" ALSA_CARDS="bt87x emu10k1x hda-intel usb-audio via82xx via82xx-modem" APACHE2_MODULES="actions alias asis auth_basic auth_digest authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter filter headers include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_connect proxy_ftp proxy_http rewrite setenvif speling status unique_id userdir usertrack vhost_alias" APACHE2_MPMS="prefork" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DRACUT_MODULES="lvm" ELIBC="glibc" FRITZCAPI_CARDS="fcpci" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev joystick keyboard mouse wacom" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en de en_US en_GB" LIRC_DEVICES="devinput" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-3" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_6 python2_7 python3_2 python3_3" RUBY_TARGETS="ruby19 ruby20 jruby" USERLAND="GNU" VIDEO_CARDS="nouveau nv radeon fbdev v4l vesa vga" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" USE_PYTHON="2.6 2.7 3.2 3.3" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
The observed time window agrees with the introduction of install.py, which was added to address bug #465000. There was some discussion in that bug about bash vs. python, and in bug #465000 comment #11 SpanKY wrote: “getting people to write more scripts in python would be good.” I'm not sure whether bash would be any faster, but I guess it might. So at least for scripts which will be called REALLY OFTEN, performance considerations should be taken into account.
*** This bug has been marked as a duplicate of bug 482290 ***
