Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 29752 - Gleaning -jN from DISTCC_HOSTS
Summary: Gleaning -jN from DISTCC_HOSTS
Status: RESOLVED WONTFIX
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Unclassified (show other bugs)
Hardware: All Linux
: High enhancement
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2003-09-27 09:06 UTC by Tobias Klausmann (RETIRED)
Modified: 2011-10-30 22:21 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Tobias Klausmann (RETIRED) gentoo-dev 2003-09-27 09:06:53 UTC
It would be nice if portage optionally gleaned the value for -jN from the number
of machines in DISTCC_HOSTS (or distcc-config --get-hosts). I update the latter
from a script that checks which hosts are up and running distccd (in a lab that
sometimes is used for Windows stuff). Unfortunately, I also have to adjust
MAKEOPTS so the machines don't slow themselves down by using -j22 on three
machines. 

One way might be to make MAKEOPTS more intelligent, like being able to specify
MAKEOPTS="-jAUTO" and the replacing the AUTO string with the number of
DISTCC_HOSTS. Escaping this ("-j{NJOBS}" etc.) could/should be done in a
portage-firendly way. 

Reproducible: Always
Steps to Reproduce:
Comment 1 Markus Nigbur (RETIRED) gentoo-dev 2003-09-27 10:00:17 UTC
The MAKEOPTS value depends on two things:
1. cpus used to compile
2. load you want to set on them.

So if you have 20 dual xeon boxes, you don't want to run with -j21, but -j41
or even 61.
CPUs+1 is a (unwritten?) rule
Personally i prefer CPUs*2+1.

Lisa should decide this one ;)
Comment 2 Tobias Klausmann (RETIRED) gentoo-dev 2003-09-27 10:08:41 UTC
That's why my setup adds a hostname twice for dual-cpu machines. This also
makes sure that the smp machines get double the load of the uniprocessor
machines.

As for the amoutn of N, I have benchmarked using 2.4 and 2.6 kernels and
usually, compiles does not speed up after N=NUM_CPUS. YMMV, of course.
Comment 3 Lisa Seelye (RETIRED) gentoo-dev 2003-09-27 10:09:20 UTC
Markus is right.

Why not fix your script so it updates make.conf on its own?

grep -v '^MAKEOPTS=' /etc/make.conf > /tmp/make.conf.tmp
echo "MAKEOPTS='-jN'" >> /tmp/make.conf.tmp
mv --force /tmp/make.conf.tmp /etc/make.conf

Setting MAKEOPTS should be up to the user.
Comment 4 Tobias Klausmann (RETIRED) gentoo-dev 2003-09-27 11:19:20 UTC
I had in mind that the user sets auto-mode for -j explcitly, then never tiuches
it again. So they can still override at the cmd line with MAEKOPTS="..."
but in the default case, the amount of parallelism is gleaned from the no.
of hosts.

I feel a bit uneasy when fiddling with make.conf directly, but ok. 'twas
just an idea.
Comment 5 Markus Nigbur (RETIRED) gentoo-dev 2003-09-28 07:36:15 UTC
closing