Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 715542 - waf-utils.eclass respects jobs set in MAKEOPTS only for compile phase, not configure or install phases
Summary: waf-utils.eclass respects jobs set in MAKEOPTS only for compile phase, not co...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Eclasses (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: No maintainer - Look at https://wiki.gentoo.org/wiki/Project:Proxy_Maintainers if you want to take care of it
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: makeopts-ignored
  Show dependency tree
 
Reported: 2020-03-31 01:56 UTC by Jacob Floyd
Modified: 2023-05-19 02:38 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jacob Floyd 2020-03-31 01:56:52 UTC
I have tuned the number of jobs I want building so that there are not build errors from the heat on some older hardware and so that there is less impact on using the system while compiling.

In watching top during a build of samba, I was surprised to see all 8 cores used for the install phase instead of 6, 3, or even 1. The compile phase is accurately limited according to the MAKEOPTS -j seeing, but other phases aren't.

For me, that meant that ld failed because of hardware issues when this particular system overheats. I don't believe that's a bug in samba, because I had similar issues with other packages until I lowered the jobs level to better manage the heat. But samba continued to cause me errors.

Today I found a pointer in the ebuild to #683148 which discovered that waf-utils.eclass is not passing the jobs arg to waf during configure phase. That didn't matter for the big in question because they had to cap samba at 1 job during the configure phase.

Inspecting the eclass, the jobs arg is not passed in either configure or install phase.

For this machine editing the samba rebuild in my local overlay to add `--jobs 1` to the waf install line allowed the build to complete. So, if waf-utils.eclass respected my jobs setting I could avoid the hardware settings until I can afford to replace this machine.

Reproducible: Always

Steps to Reproduce:
1. Use a lower than normal jobs setting in MAKEOPTS (like -j3 or -j1)
2. Build samba while watching with top
3. Count number of active jobs during install phase.
Actual Results:  
The install phase used 8 jobs (following number of effective cores) during install phase. It did this no matter the MAKEOPTS setting (I tried with -j6, -j3, and -j1).

Expected Results:  
The max number of jobs should respect MAKEOPTS in all phases, not just during compile.

While testing this I disabled all system services that weren't necessary so that as few things were running as possible.

The hardware issues are not what this issue is about. I'm asking for a change of the waf-utils.eclass.
Comment 1 Jacob Floyd 2020-03-31 02:02:15 UTC
Link to related bug#683148 that found that configure phase was not getting jobs setting.
Comment 2 Larry the Git Cow gentoo-dev 2022-11-25 08:56:16 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=c8bbd4cf6d3b4f87a78dddf7d85fabe68fc27b20

commit c8bbd4cf6d3b4f87a78dddf7d85fabe68fc27b20
Author:     Matthew Smith <matthew@gentoo.org>
AuthorDate: 2022-10-17 17:49:38 +0000
Commit:     Matthew Smith <matthew@gentoo.org>
CommitDate: 2022-11-25 08:55:31 +0000

    waf-utils.eclass: set --jobs=1 for configure and install
    
    MAKEOPTS was only respected for src_compile, leading to out-of-memory
    issues for some users. Force number of concurrent jobs to 1 in src_configure
    and src_install phases as it doesn't provide much of a benefit anyway.
    
    Closes: https://bugs.gentoo.org/715542
    Signed-off-by: Matthew Smith <matthew@gentoo.org>

 eclass/waf-utils.eclass | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
Comment 3 Larry the Git Cow gentoo-dev 2023-05-19 02:38:21 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=00f0517beddf7efb61f9f18f36f22dd76fc4a7aa

commit 00f0517beddf7efb61f9f18f36f22dd76fc4a7aa
Author:     Mike Gilbert <floppym@gentoo.org>
AuthorDate: 2023-01-28 19:21:00 +0000
Commit:     Mike Gilbert <floppym@gentoo.org>
CommitDate: 2023-05-19 02:38:03 +0000

    waf-utils.eclass: enable parallel install
    
    This gives a nice speedup to net-fs/samba, which (re)links several
    hundred files in its install phase.
    
    Bug: https://bugs.gentoo.org/715542
    Signed-off-by: Mike Gilbert <floppym@gentoo.org>

 eclass/waf-utils.eclass | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)