| Summary: | sys-apps/portage-2.1.10.44: emerge with parallel jobs quickly degrades to building only one package at a time | ||
|---|---|---|---|
| Product: | Portage Development | Reporter: | Joe Breuer <gentoo> |
| Component: | Unclassified | Assignee: | Portage team <dev-portage> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | Keywords: | InVCS |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | |||
| Bug Blocks: | 184128, 423075 | ||
| Attachments: | Output of emerge -pe --tree @system | ||
|
Description
Joe Breuer
2012-02-15 13:18:23 UTC
To forestall one possible argument regarding the low load average: No, I/O bandwidth is NOT the issue here - the disk is mostly idle during the emerge, there only are occasional accesses. Given a particular depgraph, you cannot ALWAYS have multiple parallel "chains" of emerges going. eg. if 10 packages depend on foo, you must wait for foo to finish emerging. I fully understand darkside's argument, but: A bit further down in the list there are, for example, the following packages: - eselect - xvid - openmotif - man - curl - libvorbis And they also built one after the other. I do not see how these packages would have a linear dependency. On proper reflection, especially openmotif, xvid and libvorbis seem a bit out-of-place in a @system build... To pick an arbitrary example, openmotif is required by: app-editors/xemacs-21.4.22-r2 (motif ? >=x11-libs/openmotif-2.3:0) app-text/xdvik-22.84.16 (motif ? >=x11-libs/openmotif-2.3:0) dev-util/ddd-3.3.12-r2 (>=x11-libs/openmotif-2.3:0) media-libs/mesa-7.11.2 (motif ? x11-libs/openmotif) sci-electronics/pcb-20100929 (!gtk ? >=x11-libs/openmotif-2.3:0) I cannot readily figure out why this would be built as part of / a dependency for the system set in the first place. Is there something like 'equery depgraph' going in the OPPOSITE direction, to help figure out where @system dependencies come from? Or something which would in effect be 'equery depgraph @system'? Currently, the @world build underway seems to be reasonably well-behaved (load avgs 4.95, 5.73, 5.10); but I've seen @world build degrade in a similar fashion as well. (I'm not a portage dev, so I'll probably stop following this bug) (In reply to comment #3) > Is there something like 'equery depgraph' going in the OPPOSITE direction, to > help figure out where @system dependencies come from? > Or something which would in effect be 'equery depgraph @system'? emerge -p --tree world, would probably help you. Created attachment 302053 [details]
Output of emerge -pe --tree @system
@darkside: Of course, thanks. Always forget that one ;-)
I still think I'm onto something here:
# emerge -pe --tree @system
These are the packages that would be merged, in reverse order:
Calculating dependencies .... done!
...
[nomerge ] media-plugins/gst-plugins-soup-0.10.30
[nomerge ] net-libs/libsoup-2.36.1
[nomerge ] net-libs/glib-networking-2.30.1-r1
[nomerge ] net-libs/libproxy-0.4.7
[nomerge ] kde-base/kdelibs-4.7.4
[nomerge ] kde-base/khelpcenter-4.7.4
[nomerge ] kde-base/kdesu-4.7.4
[nomerge ] x11-libs/qt-webkit-4.7.4
[nomerge ] media-libs/phonon-4.5.1-r1
[nomerge ] media-libs/phonon-gstreamer-4.5.0
[nomerge ] media-plugins/gst-plugins-meta-0.10-r6
[nomerge ] media-plugins/gst-plugins-dv-0.10.30
[nomerge ] media-libs/libdv-1.0.0-r2
[nomerge ] media-libs/libsdl-1.2.14-r6
[nomerge ] virtual/opengl-7.0
[ebuild R ] media-libs/mesa-7.11.2
[ebuild R ] sys-devel/llvm-2.9-r2
...
[nomerge ] media-libs/phonon-gstreamer-4.5.0
[ebuild R ] media-plugins/gst-plugins-soup-0.10.30
[ebuild R ] net-libs/libsoup-2.36.1
[ebuild R ] net-libs/glib-networking-2.30.1-r1
[ebuild R ] net-libs/libproxy-0.4.7
...
To me, this LOOKS LIKE the circular dependency between gst-plugins-soup and phonon-gstreamer bloats my @system set with all the stuff required by that sub-tree. I did not see a package requiring either gst-plugins-soup and phonon-gstreamer, apart from those related to the circular dependency.
I'm attaching the full output, perhaps I'm just blind and do not see the "real" dependency.
*** This bug has been marked as a duplicate of bug 383439 *** I've found a bug that's triggered by the --load-average option: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=fe03b5fd790fc09bd6594c55bb174e80aaac1e5a This is fixed in 2.1.11.2 and 2.2.0_alpha113. |
