A few of the bigger ebuilds around the shop, e.g. Mozilla and XFree, use get_number_of_jobs from eutils.eclass to modify MAKEOPTS so that the number of jobs spawned by make is appropriate for the number of CPUs on the local machine. For those of us who have a couple of extra machines which run as distcc servers, this is a bit of a pain. For instance, here I run distcc on two Athlons to speed up compile times on my G3 Powerbook. When I go to emerge a package like XFree, get_number_of_jobs correctly detects that my Powerbook has only one CPU, and reduces MAKEOPTS from -j5 to -j2. Thus make does not spawn enough jobs to use all my available (distributed) CPU power. Would it be feasible for get_number_of_jobs to account for the number of distcc hosts when it calculates the number of jobs? Or would this only recreate the problem which get_number_of_jobs tries to solve? My simple solution would be to examine the list in /etc/distcc/hosts, try a test compile on each of these machines, and increment jobs accordingly. However, if one of those hosts becomes unavailable during the build, then I fear that the job number will be incorrect, possibly causing the build to fail. Can anyone comment on this? Is it worth coding up something like I have described, to see how it goes? Reproducible: Always Steps to Reproduce: 1. 2. 3.
i propose you just set up your MAKEOPTS in make.conf the way you should and forget about trying to make portage sufficiently smart ... but thats just me :)
From asking around it seems that this function is supposed to ensure that the build will complete. The best solution would be to disable parallel makes when they fail (such in the case of Xfree and Mozilla) instead of trying to guess what -j would be the "best." I'll ask around some more to find the true intention of the function, but I'm hesitant to change it right now, without more info.
That's cool, it was just a thought . . . you know, when you're compiling a big package on a slow machine, thoughts about how to improve the compile time seem really important . . . but once the package is compiled, compile time seems less important and stability moreso. I guess the real fix would be to fix the build systems for the packages which fail with high job numbers, but I suspect that'd be rather more difficult. Bug 13565 contains more discussion about this function, and a suggestion that those of us who don't like it can disable it via the portage overlay in /usr/local/portage.
Right. I think the only packages that have problems with parallel makes use this function. I don't really see any reason or need to do anything with this function at this time.
