Since I have a significant number of cores available at some locations (here: 8 local, further 16 via distcc) I'd like to reduce the wall-clock time of emerges by having emerge build enough jobs in parallel to saturate the cores. After initially starting out with building a number of jobs in parallel, emerge quickly tends to devolve to emerging and installing one package at a time, linearly. This is long-standing, I'm seeing this behavior for at least 18 months. Reproducible: Always Steps to Reproduce: 1. configure emerge for parallel builds (make.conf): EMERGE_DEFAULT_OPTS="--with-bdeps y --jobs=24 --load-average=6" MAKEOPTS="--jobs=32 --load-average=6" 2. emerge -bDNauvt @system [66 packages to build] 3. Observe emerge output: >>> Starting parallel fetch >>> Emerging (1 of 66) media-libs/libpng-1.5.8 >>> Emerging (2 of 66) media-libs/libogg-1.3.0 >>> Emerging (3 of 66) sys-libs/timezone-data-2011n >>> Emerging (4 of 66) sys-apps/sdparm-1.06 >>> Emerging (5 of 66) media-libs/jpeg-8c-r1 >>> Emerging (6 of 66) dev-util/byacc-1.9-r3 >>> Emerging (7 of 66) virtual/modutils-0 >>> Installing (6 of 66) dev-util/byacc-1.9-r3 >>> Installing (3 of 66) sys-libs/timezone-data-2011n >>> Installing (2 of 66) media-libs/libogg-1.3.0 >>> Installing (1 of 66) media-libs/libpng-1.5.8 >>> Installing (4 of 66) sys-apps/sdparm-1.06 >>> Installing (7 of 66) virtual/modutils-0 >>> Installing (5 of 66) media-libs/jpeg-8c-r1 >>> Emerging (8 of 66) media-libs/tiff-4.0.0-r1 >>> Installing (8 of 66) media-libs/tiff-4.0.0-r1 >>> Emerging (9 of 66) sys-libs/slang-2.2.4 >>> Installing (9 of 66) sys-libs/slang-2.2.4 >>> Emerging (10 of 66) sys-kernel/linux-headers-3.1 >>> Installing (10 of 66) sys-kernel/linux-headers-3.1 >>> Emerging (11 of 66) dev-perl/Net-Daemon-0.480.0 >>> Installing (11 of 66) dev-perl/Net-Daemon-0.480.0 >>> Emerging (12 of 66) perl-core/Time-Local-1.200.0 >>> Installing (12 of 66) perl-core/Time-Local-1.200.0 >>> Emerging (13 of 66) virtual/perl-Time-Local-1.200.0-r1 >>> Installing (13 of 66) virtual/perl-Time-Local-1.200.0-r1 >>> Emerging (14 of 66) perl-core/Test-Harness-3.230.0 >>> Installing (14 of 66) perl-core/Test-Harness-3.230.0 >>> Emerging (15 of 66) virtual/perl-Test-Harness-3.230.0-r2 >>> Installing (15 of 66) virtual/perl-Test-Harness-3.230.0-r2 >>> Emerging (16 of 66) dev-libs/libusb-1.0.9_rc3 >>> Installing (16 of 66) dev-libs/libusb-1.0.9_rc3 >>> Emerging (17 of 66) app-text/libpaper-1.1.24-r1 >>> Installing (17 of 66) app-text/libpaper-1.1.24-r1 >>> Emerging (18 of 66) dev-libs/openssl-1.0.0g >>> Installing (18 of 66) dev-libs/openssl-1.0.0g >>> Emerging (19 of 66) media-libs/libdvdread-4.2.0 >>> Installing (19 of 66) media-libs/libdvdread-4.2.0 >>> Emerging (20 of 66) media-libs/libdvdnav-4.2.0 >>> Installing (20 of 66) media-libs/libdvdnav-4.2.0 >>> Emerging (21 of 66) kde-base/kde-env-4.7.4 >>> Installing (21 of 66) kde-base/kde-env-4.7.4 >>> Emerging (22 of 66) dev-libs/pth-2.0.7-r3 >>> Installing (22 of 66) dev-libs/pth-2.0.7-r3 >>> Emerging (23 of 66) sys-apps/attr-2.4.46-r1 >>> Installing (23 of 66) sys-apps/attr-2.4.46-r1 >>> Emerging (24 of 66) sys-apps/coreutils-8.14 >>> Installing (24 of 66) sys-apps/coreutils-8.14 >>> Emerging (25 of 66) dev-util/intltool-0.50.0 >>> Jobs: 24 of 66 complete, 1 running Load avg: 0.28, 0.88, 0.53 Actual Results: emerge, after typically one round of parallel emerges, degrades to 'emerge one package, install one package' in a linear fashion. The load average shows that the available system (8 local cores, further 16 via distcc) is not utilized. Expected Results: Packages should be emerged in parallel achieving a load average matching that configured using --load-average. Portage 2.1.10.44 (default/linux/amd64/10.0/desktop/kde, gcc-4.5.3, glibc-2.13-r4, 3.1.6-gentoo x86_64) ================================================================= System uname: Linux-3.1.6-gentoo-x86_64-Intel-R-_Core-TM-_i7-2630QM_CPU_@_2.00GHz-with-gentoo-2.0.3 Timestamp of tree: Wed, 15 Feb 2012 12:15:01 +0000 distcc 3.1 x86_64-pc-linux-gnu [enabled] app-shells/bash: 4.1_p9 dev-java/java-config: 2.1.11-r3 dev-lang/python: 2.7.2-r3, 3.1.4-r3 dev-util/cmake: 2.8.6-r4 dev-util/pkgconfig: 0.26 sys-apps/baselayout: 2.0.3 sys-apps/openrc: 0.9.4 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.68 sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.1 sys-devel/binutils: 2.21.1-r1 sys-devel/gcc: 4.5.3-r1 sys-devel/gcc-config: 1.4.1-r1 sys-devel/libtool: 2.4-r1 sys-devel/make: 3.82-r1 sys-kernel/linux-headers: 3.1 (virtual/os-headers) sys-libs/glibc: 2.13-r4 Repositories: gentoo LOCAL jmbreuer ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA dlj-1.1 PUEL skype-eula sun-bcla-java-vm" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=core2 -mtune=generic -O2 -pipe -momit-leaf-frame-pointer" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/openvpn/easy-rsa /var/lib/hsqldb" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /usr/share/X11/xkb" CXXFLAGS="-march=core2 -mtune=generic -O2 -pipe -momit-leaf-frame-pointer" DISTDIR="/mnt/data/Archive/linux/gentoo/distfiles" EMERGE_DEFAULT_OPTS="--with-bdeps y --jobs=24 --load-average=6" FEATURES="assume-digests binpkg-logs buildpkg collision-protect distcc distlocks ebuild-locks fixlafiles news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox" FFLAGS="" GENTOO_MIRRORS="http://de-mirror.org/distro/gentoo/ http://gentoo.mneisen.org/ http://gentoo.wheel.sk/ http://gentoo.mirror.pw.edu.pl/" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LINGUAS="en de" MAKEOPTS="--jobs=32 --load-average=6" PKGDIR="/mnt/data/Archive/linux/gentoo/packages/x64cd" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage /var/lib/layman/jmbreuer" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="X Xaw3d a52 aac accessibility acl acpi alsa amd64 apache2 berkdb bluetooth branding bzip2 cairo cdda cdr cjk cli consolekit cracklib crypt css cups curl cvs cxx dbus declarative dga dri dts dv dvd dvdr emboss encode exif fam fame fastcgi ffmpeg firefox flac flash fontconfig foomaticdb fortran fuse gd gdbm gdu gif glitz glut gphoto2 gpm gtk iconv ieee1394 imap imlib innodb ipv6 java jce jpeg kde kerberos kipi kpathsea lame lcms leim libgda libnotify libsamplerate lzo mad mbox mjpeg mmx mmxext mng modplug modules mono motif mozilla mp3 mp4 mpeg mplayer mudflap mule multilib ncurses network network-cron networkmanager nls nptl nptlonly nsplugin offensive ogg openal openexr opengl openmp pam pango pcap pcmcia pcre pda pdf phonon plasma plotutils png policykit ppds pppd qt3support qt4 readline rtsp samba sdl semantic-desktop sensord session slang slp sound spell sse sse2 ssl ssse3 startup-notification stream subversion svg sysfs tcpd theora tiff truetype trusted udev unicode usb v4l v4l2 vhosts vim-syntax vlm vorbis webdav wifi wmf x264 xanim xcb xcomposite xemacs xinerama xml xorg xscreensaver xulrunner xv xvid xvmc zlib zsh-completion" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en de" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" SANE_BACKENDS="net" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS I'll happily provide further information / help in debugging, as requested.
To forestall one possible argument regarding the low load average: No, I/O bandwidth is NOT the issue here - the disk is mostly idle during the emerge, there only are occasional accesses.
Given a particular depgraph, you cannot ALWAYS have multiple parallel "chains" of emerges going. eg. if 10 packages depend on foo, you must wait for foo to finish emerging.
I fully understand darkside's argument, but: A bit further down in the list there are, for example, the following packages: - eselect - xvid - openmotif - man - curl - libvorbis And they also built one after the other. I do not see how these packages would have a linear dependency. On proper reflection, especially openmotif, xvid and libvorbis seem a bit out-of-place in a @system build... To pick an arbitrary example, openmotif is required by: app-editors/xemacs-21.4.22-r2 (motif ? >=x11-libs/openmotif-2.3:0) app-text/xdvik-22.84.16 (motif ? >=x11-libs/openmotif-2.3:0) dev-util/ddd-3.3.12-r2 (>=x11-libs/openmotif-2.3:0) media-libs/mesa-7.11.2 (motif ? x11-libs/openmotif) sci-electronics/pcb-20100929 (!gtk ? >=x11-libs/openmotif-2.3:0) I cannot readily figure out why this would be built as part of / a dependency for the system set in the first place. Is there something like 'equery depgraph' going in the OPPOSITE direction, to help figure out where @system dependencies come from? Or something which would in effect be 'equery depgraph @system'? Currently, the @world build underway seems to be reasonably well-behaved (load avgs 4.95, 5.73, 5.10); but I've seen @world build degrade in a similar fashion as well.
(I'm not a portage dev, so I'll probably stop following this bug) (In reply to comment #3) > Is there something like 'equery depgraph' going in the OPPOSITE direction, to > help figure out where @system dependencies come from? > Or something which would in effect be 'equery depgraph @system'? emerge -p --tree world, would probably help you.
Created attachment 302053 [details] Output of emerge -pe --tree @system @darkside: Of course, thanks. Always forget that one ;-) I still think I'm onto something here: # emerge -pe --tree @system These are the packages that would be merged, in reverse order: Calculating dependencies .... done! ... [nomerge ] media-plugins/gst-plugins-soup-0.10.30 [nomerge ] net-libs/libsoup-2.36.1 [nomerge ] net-libs/glib-networking-2.30.1-r1 [nomerge ] net-libs/libproxy-0.4.7 [nomerge ] kde-base/kdelibs-4.7.4 [nomerge ] kde-base/khelpcenter-4.7.4 [nomerge ] kde-base/kdesu-4.7.4 [nomerge ] x11-libs/qt-webkit-4.7.4 [nomerge ] media-libs/phonon-4.5.1-r1 [nomerge ] media-libs/phonon-gstreamer-4.5.0 [nomerge ] media-plugins/gst-plugins-meta-0.10-r6 [nomerge ] media-plugins/gst-plugins-dv-0.10.30 [nomerge ] media-libs/libdv-1.0.0-r2 [nomerge ] media-libs/libsdl-1.2.14-r6 [nomerge ] virtual/opengl-7.0 [ebuild R ] media-libs/mesa-7.11.2 [ebuild R ] sys-devel/llvm-2.9-r2 ... [nomerge ] media-libs/phonon-gstreamer-4.5.0 [ebuild R ] media-plugins/gst-plugins-soup-0.10.30 [ebuild R ] net-libs/libsoup-2.36.1 [ebuild R ] net-libs/glib-networking-2.30.1-r1 [ebuild R ] net-libs/libproxy-0.4.7 ... To me, this LOOKS LIKE the circular dependency between gst-plugins-soup and phonon-gstreamer bloats my @system set with all the stuff required by that sub-tree. I did not see a package requiring either gst-plugins-soup and phonon-gstreamer, apart from those related to the circular dependency. I'm attaching the full output, perhaps I'm just blind and do not see the "real" dependency.
*** This bug has been marked as a duplicate of bug 383439 ***
I've found a bug that's triggered by the --load-average option: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=fe03b5fd790fc09bd6594c55bb174e80aaac1e5a
This is fixed in 2.1.11.2 and 2.2.0_alpha113.
