Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 641604 - MAKEOPT="-j8 -l8" and EMERGE_DEFAULT_OPTS="--jobs 8 --load-average 7.2" are not effective.
Summary: MAKEOPT="-j8 -l8" and EMERGE_DEFAULT_OPTS="--jobs 8 --load-average 7.2" are n...
Status: RESOLVED INVALID
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: AMD64 Linux
: Normal major (vote)
Assignee: Gentoo Linux bug wranglers
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-12-18 20:24 UTC by Frank Daniel
Modified: 2017-12-18 20:53 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Frank Daniel 2017-12-18 20:24:09 UTC
I am doing an emerge -avDN @world
Some of my make.conf settings are:

MAKEOPTS="-j8 -l8"
EMERGE_DEFAULT_OPTS="--jobs 8 --load-average 7.2"

For the smaller files it works well.  The emerge processes are spread out. But,
for larger processes, like emerging sys-devel/gcc-6.4.0, the palatalization does not work.

Per 'top' and 'system monitor' the aggregate of CPU usage is less than 15%.

Upon commenting out EMERGE_DEFAULT_OPTS statement in make.conf, the aggregate for the CPUs increased to ~80%.
Comment 1 Ian Stakenvicius (RETIRED) gentoo-dev 2017-12-18 20:49:03 UTC
Not every package will parallelize, and not every part of every package will parallelize.  

In particular, the emerge dependency resolution and package ordering will explicitly block later packages from emerging until all of their necessary dependencies are resolved; toolchain dependencies like gcc are a bottleneck for that.  This is a design feature, without it bad things would happen with 'emerge -j'

BTW, it is worth noting that your "--load-average 7.2" in EMERGE_DEFAULT_OPTS, being lower than "-l8" in MAKEOPTS, means that emerge will not start a new job if the current load on the system is above 7.2 (which MAKE within a single package can cause).  Also note that load is not the same as CPU busyness, rather its the number of active processes (in 'top' its the three numbers in the top-right corner); I have had a load of 10 while my CPUs were around at 10% active before, it's rare and needs a very specific workload but it can happen.


If there really is an issue with EMERGE_DEFAULT_OPTS then you will need to perform re-emerges of the exact same package tree with and without the options set, and provide the /var/log/emerge.log for each run.  If you can do that and do indeed find differences then please reopen.
Comment 2 Brian Evans (RETIRED) gentoo-dev 2017-12-18 20:53:35 UTC
(In reply to Frank Daniel from comment #0)
> I am doing an emerge -avDN @world
> Some of my make.conf settings are:
> 
> MAKEOPTS="-j8 -l8"
> EMERGE_DEFAULT_OPTS="--jobs 8 --load-average 7.2"
> 
> For the smaller files it works well.  The emerge processes are spread out.
> But,
> for larger processes, like emerging sys-devel/gcc-6.4.0, the palatalization
> does not work.
> 
> Per 'top' and 'system monitor' the aggregate of CPU usage is less than 15%.
> 
> Upon commenting out EMERGE_DEFAULT_OPTS statement in make.conf, the
> aggregate for the CPUs increased to ~80%.

I don't think this is a bug as it works well for me, once you understand the differences between them.

MAKEOPTS affects make (and ninja in Gentoo when the eclass is used) during the compile and, possibly, install phases.  It will not be used for configure or other phases.  Some ebuilds may override this setting when they will fail to build.

emerge's options affect how many packages build simultaneously *if* no dependencies are scheduled.  New leaf package installs will not benefit too much here, but updates may more so.  sys-devel/gcc is a poor example as almost everything depends on it.