Summary: | emerge lacks option to limit -j by memory usage | ||
---|---|---|---|
Product: | Portage Development | Reporter: | korte |
Component: | Enhancement/Feature Requests | Assignee: | Portage team <dev-portage> |
Status: | UNCONFIRMED --- | ||
Severity: | enhancement | CC: | a.v.popov, bjoern, esigra, gentoo, sam |
Priority: | Normal | ||
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
See Also: | https://bugs.gentoo.org/show_bug.cgi?id=692576 | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Bug Depends on: | |||
Bug Blocks: | 184128 |
Description
korte
2021-11-06 15:34:29 UTC
Maybe we want something like --memsuspend in GNU parallel (https://www.gnu.org/software/parallel/man.html)? (In reply to Tee KOBAYASHI from comment #1) > Maybe we want something like --memsuspend in GNU parallel > (https://www.gnu.org/software/parallel/man.html)? If it is easier to implement, that could be helpful. While one or more jobs are stopped, no new jobs should be started. Imho the job(s) with the least RAM usage should be stopped first. I have no clue how to calculate the swapping load. Is the sum of real memory usage of the running emerge processes plus the memory of the running system of relevance? Is there a recent swapping load number in the system available? Swapping could be less, when using memsuspend since some stopped jobs stay in the swap. The only prerequisite would be to have enough RAM+swap available, which should be imho possible. RAM+swap could be: X=make -j x Y=emerge -j y default_size=2GB (of one compile process, e.g.) BIGGEST=13GB # RAM usage of the biggest package that will be installed, e.g.firefox? RAM+swap = X*Y*default_size+BIGGEST # having at least 1*default_size for safety reasons, since BIGGEST already includes the default_size and big package may have a lower -j1 in package.env . This could avoid hitting the memory limiting part of smallest(CPUs,RAM/2GB) when using emerge -j JOBS and JOBS>1. On my Pi400 there could be several packages compiled during the time a big package with MAKEOPTS="-j1" is using less then 2GB. As workaroud I use: MAKEOPTS="-j2" emerge --jobs=2 --exclude "`grep one-make-thread.conf /etc/portage/package.env | grep -v "^#" | cut -f 1 | tr '\n' ' '`" -DvuaN @world MAKEOPTS="-j1" emerge --jobs=1 -DvuaN @world Would be nice to get some feedback in irc, if this works for others, too. |