Portage's install phase takes ages, as files slowly appear one by one in the install image. Reproducible: Always Steps to Reproduce: Emerge a package containing many files in the install image, watch the time it takes, see the files slowly drip into the install image. Actual Results: >>> Install rocksndiamonds-3.3.0.1-r1 into /var/tmp/portage/games-arcade/rocksndiamonds-3.3.0.1-r1/image/ category games-arcade [seems to do nothing for ages] Files appear in install image at a rate of few a second, even tiny ones: $ while true; do sudo find /var/tmp/portage/games-arcade/rocksndiamonds-3.3.0.1-r1/image | wc -l; sleep 1s; done 18656 18663 18671 18678 18685 18692 18699 18706 18714 18721 …and so on. Expected Results: Files should appear in the install image in a sensible amount of time. Portage 2.2.7 (default/linux/amd64/13.0, gcc-4.7.3, glibc-2.17, 3.10.26 x86_64) ================================================================= System uname: Linux-3.10.26-x86_64-Intel-R-_Core-TM-_i7_CPU_M_620_@_2.67GHz-with-gentoo-2.2 KiB Mem: 8038948 total, 251620 free KiB Swap: 0 total, 0 free Timestamp of tree: Mon, 27 Jan 2014 09:00:01 +0000 ld GNU ld (GNU Binutils) 2.23.2 app-shells/bash: 4.2_p45 dev-java/java-config: 2.1.12-r1 dev-lang/python: 2.7.5-r3, 3.2.5-r3, 3.3.3 dev-util/cmake: 2.8.11.2 dev-util/pkgconfig: 0.28 sys-apps/baselayout: 2.2 sys-apps/openrc: 0.12.4 sys-apps/sandbox: 2.6-r1 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.11.6, 1.12.6, 1.13.4 sys-devel/binutils: 2.23.2 sys-devel/gcc: 4.7.3-r1 sys-devel/gcc-config: 1.7.3 sys-devel/libtool: 2.4.2 sys-devel/make: 3.82-r4 sys-kernel/linux-headers: 3.9 (virtual/os-headers) sys-libs/glibc: 2.17 Repositories: gentoo emacs java gbin antoni lisp Installed sets: @pulseaudio ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/easy-rsa /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=native -O2 -pipe" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--nospinner --keep-going --load-average 8 --quiet-build=n" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_GB.UTF-8" LC_ALL="en_GB.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j3" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_EXTRA_OPTS="-4 --progress" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/var/lib/layman/emacs /var/lib/layman/java /var/lib/layman/gbin /usr/local/portage/antoni /usr/local/portage/lisp" USE="256-color 3rd-party-plugins 64bit 7zip X a-like-o a52 aac aalib acl acpi adns afs alsa amd64 amr amrnb amrwb anonres ares bash-completion berkdb bzip2 cairo caps cdda cdio cdparanoia cjk cleartype cli color-console colorio common-lisp cpudetection cracklib crypt css cuda cups curl cxx cycles cyrillic device-mapper djvu dmx dri dv dvd dvdr dvdread emacs exif faac faad fat fgpanel flac fortran ftdi ftp g3dvl gd gdbm gif gimp glitz gpg gpgme gpgmeidn gpm gtk gtk3 gudev hdri hfs hwdb hybrid iconv icu id3tag idn imap inkjar iproute2 ipv6 iso14755 ithreads jabber jack javascript jfs jingle jpeg kipi kpathsea kqemu ladspa latex lcdfilter lcms ldap libedit libsamplerate loop-aes mad maildir matroska mbox mdnsresponder-compat metalink mikmod minizip mjpeg mmx mmxext mng modules monolithic-build mp3 mplayer multilib musepack musibrainz musicbrainz nat ncurses net nethack networking nls nntp nodrm nptl ntfs ntfsprogs ntp offensive ogg opencore-amr opengl openmp otr pam pcre pcsc-lite pdf pg-intdatetime plotutils png postscript ppds preview-latex pulseaudio pypy1_8 python3 qt3support quicktime radeon rdesktop-vrdp readline reiser4 rfc3779 rle rtc ruby sbcl secure-delete session sip sipim skey slang slp smartcard smi smime smp sndfile socks5 srt sse sse2 sse3 sse4 ssl ssse3 subtitles svg system-sqlite tcl tcpd tex theora thread threads tiff tk tkimg toolkit-scroll-bars tordns truetype tty-helpers unicode unicode3 urandom usb v4l v4l2 vaapi vboxwebsrv vcd vde vdpau video vim vim-syntax vim-with-x vorbis vpx wav wmf x264 xattr xetex xft xinerama xinetd xmp xnest xpm xterm-color xv xvid zlib" ABI_X86="64" ALSA_CARDS="hda-intel usb-audio virmidi indigoio" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DRACUT_MODULES="caps cifs crypt crypt-gpg crypt-loop dash dmraid dmsquash-live livenet lvm mdraid multipath ssh-client syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="pc" INPUT_DEVICES="evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en en_GB pl cs" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3 python3_2" QEMU_SOFTMMU_TARGETS="i386" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby18 ruby19 ruby20" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="account chaos condition delude dhcpmac fuzzy geoip iface ipmark ipp2p ipv4options length2 logmark lscan pknock psd quota2 rawnat steal sysrq tarpit echo gradm" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, SYNC, USE_PYTHON
Same thing on ~amd64 Portage 2.2.8-r1 (default/linux/amd64/13.0, gcc-4.8.2, glibc-2.18-r1, 3.12.8-gentoo x86_64) ================================================================= System uname: Linux-3.12.8-gentoo-x86_64-Intel-R-_Core-TM-_i7-4700MQ_CPU_@_2.40GHz-with-gentoo-2.2 KiB Mem: 8096596 total, 7603296 free KiB Swap: 16800764 total, 16800764 free Timestamp of tree: Mon, 27 Jan 2014 02:00:01 +0000 ld GNU ld (GNU Binutils) 2.24 distcc 3.1 x86_64-pc-linux-gnu [disabled] ccache version 3.1.9 [disabled] app-shells/bash: 4.2_p45-r1 dev-java/java-config: 2.2.0 dev-lang/python: 2.7.6, 3.3.3 dev-util/ccache: 3.1.9-r3 dev-util/cmake: 2.8.12.1-r4 dev-util/pkgconfig: 0.28 sys-apps/baselayout: 2.2 sys-apps/openrc: 0.12.4 sys-apps/sandbox: 2.6-r1 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.11.6, 1.14.1 sys-devel/binutils: 2.24-r2 sys-devel/gcc: 4.8.2 sys-devel/gcc-config: 1.8 sys-devel/libtool: 2.4.2 sys-devel/make: 4.0-r1 sys-kernel/linux-headers: 3.13 (virtual/os-headers) sys-libs/glibc: 2.18-r1 Repositories: gentoo local ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=core-avx2 -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /var/bind" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo" CXXFLAGS="-march=core-avx2 -O2 -pipe" DISTDIR="/usr/portage/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs collision-protect compress-build-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms split-log splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LDFLAGS="-Wl,-z,combreloc -Wl,-O1 -Wl,--hash-style=both -Wl,--as-needed" MAKEOPTS="-j7" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" USE="X acl alsa amd64 avx avx2 cxx dmx fbcon gif iconv ipv6 ithreads jpeg lcms mmx mmxext mng multilib ncurses nls nptl offensive opengl openmp openssl oss pam png profile sse sse2 sse3 sse4_1 ssl ssse3 tcpd threads truetype udev unicode xattr xcb xft xinerama xv zlib" ABI_X86="64" ALSA_CARDS="hda-intel dummy seq-dummy loopback virmidi serial-u16550 intel8x0m" ELIBC="glibc" ENLIGHTENMENT_MODULES="*" INPUT_DEVICES="evdev keyboard mouse joystick void synaptics" KERNEL="linux" LINGUAS="en en_US en_GB pl ja" PYTHON_SINGLE_TARGET="python3_3" PYTHON_TARGETS="python2_7 python3_3" RUBY_TARGETS="ruby21" USERLAND="GNU" VIDEO_CARDS="nvidia nouveau nv intel modesetting none dummy" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, SYNC, USE_PYTHON
The actual time it took to emerge above mentioned games-arcade/rocksndiamonds-3.3.0.1-r1 is: real 61m55.856s user 50m52.656s sys 10m52.247s compilation itself took about a minute, and the distfiles were all predownloaded.
*** This bug has been marked as a duplicate of bug 498478 ***
ok some more info: portage xattr support is slow on both python2 and python3 (and yes i have pyxattr installed for python2) [ebuild R ] sys-apps/portage-2.2.8-r1 USE="(ipc) python3 -build -doc -epydoc (-pypy2_0) -python2 (-selinux) -xattr" LINGUAS="-ru" PYTHON_TARGETS="python2_7 python3_3 -pypy2_0 -python2_6 -python3_2 (-python3_4)" 0 kB <= merges fast [ebuild R ] sys-apps/portage-2.2.8-r1 USE="(ipc) python2 -build -doc -epydoc (-pypy2_0) -python3 (-selinux) -xattr" LINGUAS="-ru" PYTHON_TARGETS="python2_7 python3_3 -pypy2_0 -python2_6 -python3_2 (-python3_4)" 0 kB <= merges fast too [ebuild R ] sys-apps/portage-2.2.8-r1 USE="(ipc) python3 -build -doc -epydoc (-pypy2_0) -python2 (-selinux) xattr" LINGUAS="-ru" PYTHON_TARGETS="python2_7 python3_3 -pypy2_0 -python2_6 -python3_2 (-python3_4)" 0 kB <= slow install phase [ebuild R ] sys-apps/portage-2.2.8-r1 USE="(ipc) python2 -build -doc -epydoc (-pypy2_0) -python3 (-selinux) xattr" LINGUAS="-ru" PYTHON_TARGETS="python2_7 python3_3 -pypy2_0 -python2_6 -python3_2 (-python3_4)" 0 kB <= slow install phase
*** P.S. This isnt a duplicate of bug 498478 - happens on both python2+pyxattr and python3 when portage have xattr support enabled, while 498478 is performance issues on python2 without pyxattr ***
