when attempting to spawn lots of jobs but keep under a certain --load-average, emerge automatically delays the spawning of new jobs. The default settings work great for my 4 core laptop, but when trying to hit --load-average=32 it chokes. I get 2-5, sometimes even 6 running jobs, but load average hovers around 8, maxing around 14-16. This is caused by jobs completing too quickly but not being allowed to spawn more jobs fast enough to increase the load average. With the help of zmedico I messed with /usr/lib/portage/pym/_emerge/Scheduler.py specifically self._job_delay_max and _loadavg_latency. I found that setting these to 5 was able to nearly max my cpu, while setting 2 was maxing out my crappy hdd IO and slowed down build by a small percentage. The IO issue is really my issue, need to take this crappy hdd out but the ssd I ordered came broken. I only mention my IO issue for completeness in this report.
A little more details of my testing: time emerge ettercap aircrack-ng wireshark irssi xchat @x11-module-rebuild -vt1 both variables set to 5 seconds real 6m22.386s user 51m30.286s sys 21m34.309s both set to 2 seconds (my IO hit max) real 6m38.912s user 52m8.455s sys 23m47.197s both set to 2 seconds and using tmpfs for /var/tmp/portage/ real 5m54.879s user 51m11.274s sys 18m50.599s both set to 2 seconds and using tmpfs for /var/tmp/portage/ real 5m54.672s user 51m23.374s sys 18m54.213s and then I got bored
Hopefully this is better: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=fdb856ae456a559fa74066704cd19858d317ed33
(In reply to comment #2) > Hopefully this is better: > > http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit; > h=fdb856ae456a559fa74066704cd19858d317ed33 This patch is in 2.1.11.29 and 2.2.0_alpha140.
Portage before fix: real 7m51.775s user 50m31.985s sys 18m4.628s Portage after fix: real 5m55.700s user 52m3.284s sys 19m3.982s My best time was 5m54 when I was editing things, so I'd have to give this my stamp of approval!
Thanks for testing. I guess we can consider this fixed for now (though we may come up with additional tweaks later).
