The user docs - Distcc Howto - state that -jN should be set to number of processors plus one. This is drawn from portage's /etc/make.conf which states the same thing. They are both incorrect. According to the official distcc website "You should use about twice the total number of CPUs available, but it depends on your network, program being compiled, available memory, etc. Experiment with different values." This information can be viewed at http://distcc.samba.org/faq.html#dash-j It also logically makes no sense. If one processor will be doing two jobs, then they should all be able to handle two jobs. Reproducible: Always Steps to Reproduce: 1.Look at http://www.gentoo.org/doc/en/distcc.xml 2. 3.
sure it makes sense the idea is to keep the cpu's always loaded ... slightly overloading them is better than underloading ... plus with the network latency and misc i/o, the attempt at doubling up CPU's usually doesnt happen ... you just keep them loaded with compiling jobs
If the idea was to keep them always loaded, then there should be a compile job ready for each cpu as it finishes the last. In the worst case this means you should have a job ready for each cpu as they simultaneously finish one job. Thus N ought to have the value of twice the number of CPU's. In other words, each CPU is working on one job. As it finishes that job it has another job it continues working on while the overhead of send/receive/preprocess is going on. You also fail to justify why distcc.samba.org ought to be ignored on this matter. The idea that N ought to be only one more than the number of CPU's is silly. Take this example: You have three machines. Each machine is assigned one job (N = number of CPUs). As each finishes a job, they must wait IDLY for a new job to be sent to them. This is obviously a waste of processing power. Well, how about if each machine is assigned one job, and then another floats around (N = number of CPUs + 1) Ok, in this case, that one job doesnt just sit around and wait, it gets assigned to a CPU (if you dont believe me, watch the distccmon-gnome program). Now when that CPU finishes processing, it will wait for another job, but in the meantime it will work on the other compilation job. However, all the other CPUs will be stuck waiting IDLY for a new job. This also wastes processing power. Now, look at each machine getting assigned 2 jobs (N = 2 x number of CPUs). Instead of being forced to wait idly for new jobs, each machine has two jobs, meaning that when one is finished, it will be sent back, and a new one will be prepared and delivered while it CONTINUES to compile a different job. Therefore we don't waste our processing power on idle CPU cycles. If that isn't convincing reasoning and for some reason gentoo continues to believe that distcc.samba.org is incorrect in the evaluation of their own software, please at least post some sort of benchmarks that prove that N = CPUs + 1 is a better choice than N = CPUs x 2 Otherwise, it is incorrect and ought to be modified in both /etc/make.conf and the DistCC HowTo
i wasnt backing up why our suggestion or the samba suggestion is correct, just why using an N greater than the actual # of cpu's is correct we deal in real world problems, that is, de facto if you have issues with how the algorithm works (in other words, with how it isnt perfect), then feel free to go talk to the distcc people we'll continue with the solution that works, that is, to suggest N being greater than the actual number of CPU's this bug is this and this only: the distcc howto should be updated to suggest an N value of 2 * # of cpu's instead of just 1 + # of cpu's
David, you're probably right. However, this is a guide designed to get users familiar and comfortable with Distcc. You will notice that I do not go into great detail on how to set up distcc to work over SSH, and I don't mention lzo compression at all. For an exhaustive guide the user is encouraged to read the man pages. It is simply beyond the scope of the document to discuss at length the pros and cons of doing #CPU 2*#CPU, or whatever scheme you may come up with. For the next revision, I will probably add a note to the effect of: "You will have to tweak N for your setup" as I do in the automake section.
SpanKY, I think that we're trying to say the same thing, but it's getting lost through the text somehow :) I was just giving my reasoning for why N = CPU * 2 is better than N = CPU + 1 considering the way distcc works right now, not explaining how distcc was flawed in some way. I agree that it should be set higher than the number of CPU's, I simply disagree on how much higher it should have been set ;) Lisa, I think that making a note that the user may need to tweak the N value to get maximum performance is a good idea. However, I also think that you ought to at least mention that the distcc creators recommend the value of N = 2 * CPU. It just struck me as odd right off the bat the first time I read it, and upon checking distcc's homepage I found the suggestion of N = 2 * CPU. It would certainly give the guide more credibility if you said something like: "I am aware that distcc.samba.org suggests a value of N = CPU * 2, yet I recommended N = CPU + 1 nonetheless for reasons too complex to be discussed in an introductory guide. If you wish to learn more about improving the efficiency of your distcc setup, you are encouraged to read the documentation available at http://distcc.samba.org/doc.html." Instead, one wonders if you are not aware of the site's suggestion, which is not the case. It is otherwise a great introductory guide :)
Interestingly enough, Samba used to reccomend CPU*3 + 1.
Understandable, with 2 compile jobs per CPU it is still conceivable that they're both done at the same time. With three jobs, that chance is pretty darn low, and it looks like they threw that "+1" in there just to make extra sure, though I personally don't think the +1 would be relevant when this example is taken to a case where you have more than 5 or 6 CPUs working on a build. I'm satisfied :) Thanks all. Er, at the risk of sounding noobish, who closes bugs and how exactly. For now I'll assume I don't do that, and leave it as is. ;)
When I get the guide updated I'll reassign this to docs-team. *yoink*
Lisa, while you are at it, can you remove the notes that PORTAGE_TMPDIR needs to be uncommented? It seems that it is already available (through /etc/make.globals). It's just a small change of course...
Attaching updated docs... reassigning back to docs team
Created attachment 23504 [details] Updated distcc docs
Lisa, you don't happen to have a diff between this version and the previous one? I know I've asked it previously and you provided me with a link to something on thedoh.com but I can't seem to find it :/
Sure: http://www.thedoh.com/viewcvs/distcc/html/distcc.xml
Thanks. Committed.
