Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 927285

Summary: sci-libs/caffe2-2.2.1-r1: MAKEOPTS uses -j instead of -l threads
Product: Gentoo Linux Reporter: APN-Pucky <alexander>
Component: Current packagesAssignee: Tupone Alfredo <tupone>
Status: RESOLVED INVALID    
Severity: minor CC: alexander, ionen, tupone
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 654178    

Description APN-Pucky 2024-03-19 11:07:33 UTC
My default make.conf has MAKEOPTS="-j113 -l16 -s". If I install caffe2 (even with FEATURES="-distcc") it will use 113 threads and come near to an OOM.

https://wiki.gentoo.org/wiki/Distcc says that this is the right setting for MAKEOPTS and it normally works well. Maybe this bug better belongs to upstream?
Comment 1 Tupone Alfredo gentoo-dev 2024-03-30 21:12:18 UTC
ninja (or make) is allowed to start 113 compilation jobs and will stop adding  jobs if the load is more than 16 %.

-j is jobs and -l is load

at the beginning the load is near 0 so it start all the jobs he can.

But then the load goes very high.

When one jobs terminate it will not start another jobs if the load is still over the threshold.

Try with -j10 and -l1 and you will see 50 jobs started, and then only one will remains
Comment 2 Ionen Wolkens gentoo-dev 2024-03-30 21:20:06 UTC
There are plenty of packages that either do not understand/respect load-average limitations or implement it poorly and may still spawn a lot of jobs at once as noted above, and then may not be able to use distcc either (don't believe it's handled for nvcc) to spawn these jobs on another machine.

In general I don't think these MAKEOPTS are going to be safe and it may not be worth trying to handle this ebuild-side on every problematic packages.

If want to keep using this, I'd suggest to set per-package exceptions using package.env (albeit that won't stop bad surprises the first time).
Comment 3 Tupone Alfredo gentoo-dev 2024-03-30 21:24:42 UTC
> 
> Try with -j10 and -l1 and you will see 50 jobs started, and then only one
> will remains

you will see 10 jobs started (sorry)