After upgrade from sys-apps/portage-2.1.10.11 to sys-apps/portage-2.1.10.65 I noticed that the verification of package checksum is extremely slow with emerge process taking nearly 100% CPU. The qt is already checking over half hour. If I strace the python process I see it goes roughly 26kb per second. I don't believe this is normal for AMD Athlon(tm) 64 Processor 3000+ (3600 bogomips, 3dnowext, 3dnow, mmxext, sse, sse2), not speaking about fact that 32bit chroot on same computer runs at normal speeds. I would suspect some fallback to slow version of algorithm happened but have no idea to verify that. Tried to upgrade pycrypto to dev-python/pycrypto-2.6, didn't helped. Portage 2.1.10.65 (hardened/linux/amd64/desktop, gcc-4.3.4-hardenednopie, glibc-2.11.2-r3, 3.0.6-gentoo-64 x86_64) ================================================================= System uname: Linux-3.0.6-gentoo-64-x86_64-AMD_Athlon-tm-_64_Processor_3000+-with-gentoo-1.12.11.1 Timestamp of tree: Sat, 28 Jul 2012 10:15:01 +0000 app-shells/bash: 4.1_p9 dev-java/java-config: 1.3.7-r1, 2.1.10 dev-lang/python: 2.4.6, 2.5.4-r3, 2.6.4 dev-util/cmake: 2.6.4-r3 dev-util/pkgconfig: 0.26 sys-apps/baselayout: 1.12.11.1 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13::<unknown repository>, 2.68 sys-devel/automake: 1.4_p6::<unknown repository>, 1.5::<unknown repository>, 1.6.3::<unknown repository>, 1.7.9-r1::<unknown repository>, 1.8.5-r3::<unknown repository>, 1.9.6-r2::<unknown repository>, 1.10.2, 1.11.1 sys-devel/binutils: 2.16.1-r3::<unknown repository>, 2.18-r3, 2.20.1-r1 sys-devel/gcc: 3.4.6-r2, 4.3.4 sys-devel/gcc-config: 1.4.0-r4 sys-devel/libtool: 2.4-r1 sys-devel/make: 3.81 sys-kernel/linux-headers: 2.6.39 (virtual/os-headers) sys-libs/glibc: 2.11.2-r3 Repositories: gentoo x-portage ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -mtune=athlon64" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-O2 -pipe -mtune=athlon64" DISTDIR="/usr/portage/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles metadata-transfer news parallel-fetch parse-eapi-ebuild-head protect-owned sandbox sfperms unknown-features-warn unmerge-logs unmerge-orphans userfetch" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="ftp://ftp.tu-clausthal.de/pub/linux/gentoo http://ftp.sh.cvut.cz/MIRRORS/gentoo/gentoo/ http://www.mirror.ac.uk/sites/www.ibiblio.org/gentoo/ http://gentoo.oregonstate.edu http://www.ibiblio.org/pub/Linux/distributions/gentoo" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LINGUAS="en cs en_GB" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="3dnow 3dnowex X Xaw3d a52 aac aalib acl acpi alsa amd64 apache2 apm audiofile avi berkdb bluetooth branding browserplugin bzip2 bzlib cairo caps cdda cdr cli consolekit cracklib crypt cups curl cxx dbase dbm dbus dbx dga directfb divx4linux doc dri dts dvd dvdr emboss encode esd ethereal exif fam fbcon firefox flac flash gd gdbm ggi gif gpm gtk hardened iconv imagemagick imlib innodb ipv6 java jpeg justify lcms ldap lesstif libcaca libnotify libwww lirc lzma lzo mad mailwrapper mbox mcal memlimit mhash mikmod mime ming mmap mmx mmx2 mng modules motif mozilla mp3 mp4 mpeg mudflap multilib multislot mysql ncurses nls nptl offensive ogg oggvorbis openal opengl openmp oss pam pango pax_kernel pcntl pcre pdf pdflib perl php plotutils png policykit posix ppds pppd python qt qt3support qt4 quicktime readline rtc samba sdl session shared sharedmem slang sndfile snmp sockets spell sqlite sse sse2 ssl startup-notification svg sysvipc tcpd tetex theora tiff truetype udev udisks unicode upower urandom usb v4l v4l2 vhosts videos vorbis wmf wxwidgets x264 xcb xinerama xml xosd xpm xsl xv xvid zip zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="access auth auth_dbm auth_anon auth_digest alias file-cache echo charset-lite cache disk-cache mem-cache ext-filter case_filter case-filter-in deflate mime-magic cern-meta expires headers usertrack unique-id proxy proxy-connect proxy-ftp proxy-http info include cgi cgid dav dav-fs vhost-alias speling rewrite log_config logio env setenvif mime status autoindex asis negotiation dir imap actions userdir so filter unique_id authz_host auth_basic authn_dbd authn_file authz_owner authz_user charset_lite dbd mem_cache" APACHE2_MPMS="prefork" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en cs en_GB" PHP_TARGETS="php5-3" PYTHON_TARGETS="python3_2 python2_7" RUBY_TARGETS="ruby18 ruby19" USERLAND="GNU" VIDEO_CARDS="nv vesa r128 fbdev radeon nouveau" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON Selected toolchain parts: binutils: x86_64-pc-linux-gnu-2.20.1 gcc: x86_64-pc-linux-gnu-4.3.4-hardenednopie
Ok, tried few other packages and seems that only the WHIRLPOOL takes so long. (links doesn't have it and was checked fast). Can I somehow select that sha512 is enough for me?
Looking at http://archives.gentoo.org/gentoo-portage-dev/msg_545bb5944498d07c53e1878caea491ca.xml ... can portage TELL user what hash implementation is it using? On some -v flag or something ... trying to recompile mhash, pycrypto, python-fchksum and python-2.6 just in case ...
Looking at compiled filenames, dev-python/pycrypto-2.6 does not compile whirlpool. Emerge of (~amd64 keyworded) dev-python/python-mhash-1.4 helped A LOT.qt-everywhere-opensource-src-4.8.2.tar.gz finally verified in matter of minutes, instead of previous not-finished-after-hour ...
... so, I would propose: 1) stabilize dev-python/python-mhash-1.4 2) add a check for available hash functions in portage postinst and warning that emerging will be slow as hell if no accelerated version is available with list of packages providing it (starting with dev-python/python-mhash).
Latest portage skips the whirlpool hashes when necessary: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=63e25313d785f40abff3197fe0f3a3eeaa617f7d *** This bug has been marked as a duplicate of bug 425046 ***
