Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 391769 - revdep-rebuild should parallelise rebuilds
Summary: revdep-rebuild should parallelise rebuilds
Status: IN_PROGRESS
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Third-Party Tools (show other bugs)
Hardware: All Linux
: Normal enhancement (vote)
Assignee: Portage Tools Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-11-24 10:39 UTC by lalebarde
Modified: 2011-11-27 15:04 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description lalebarde 2011-11-24 10:39:25 UTC
revdep-rebuild (belonging to app-portage/gentoolkit) may be very long to execute - say tens of minutes to hours. I observe while it has tens or hundreds of packages to rebuild, it uses only ONE CPU, though 8 cores are available.

I assume it emerges packages in sequence.

Proposal : It should build one full emerge command and run it to benefit from emerge parallelisation.

Counter argument & solution : if the sequence is important, revdep-rebuild will fail at some point. run something like << emerge $list -u || while ! emerge --resume --skipfirst; do :; done >> in loop until full success or a given number of trials.
Comment 1 Brian Dolbec (RETIRED) gentoo-dev 2011-11-26 17:27:54 UTC
The new python version of revdep-rebuild already does this.  All the pkgs needing rebuild are passed to emerge in one call, so any EMERGE_DEFAULT_OPTS will be used and take advantage of any --jobs settings.  It also passes any options meant for emerge that are added after a "--".

eg:  revdep-rebuild -- --jobs 8

To use and test it, emerge gentoolkit-9999.  The original bash revdep-rebuild is also still included, but has been renamed to revdep-rebuild.sh.  You will notice a significant speed improvement in the new python version :)
Comment 2 lalebarde 2011-11-27 10:40:03 UTC
Good job Brian ! Thank you.
Comment 3 Brian Dolbec (RETIRED) gentoo-dev 2011-11-27 15:04:50 UTC
I didn't do all the work.  I just directed it somewhat to ensure it was done with a usable api.

The main credit goes to Slawek Lis.