Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 287327 - How to raise the process limit in Gentoo Linux
Summary: How to raise the process limit in Gentoo Linux
Status: VERIFIED WORKSFORME
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Gentoo's Team for Core System packages
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-10-02 09:37 UTC by Robert Bradbury
Modified: 2009-10-10 18:26 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Bradbury 2009-10-02 09:37:34 UTC
The default Gentoo limit for number of open files (1024) is too low (for Opera when restarting a large session which may reopen a large number of cache and/or RSS files).

The default Gentoo limit for number of processes (300) is too low (for Google Chrome because the master parent fails to wait() for and collect "defunct" processes (presumably associated with closed tabs/windows).

Reproducible: Always

Steps to Reproduce:
1. Restart a large opera session which may have thousands of cached HTML / Image / RSS file handles open.
2. Start a chrome session and open and close a lot of windows/tabs/URLs.


Actual Results:  
Opera will output messages of the form:
  Failed to open file (please check with "ulimit") [fopen]: Too many open files

Chrome will cause shells to fail with:
  bash: fork: Resource temporarily unavailable

Also note that when the fork problem takes place it is user-specific but a simple counting of user processes (e.g. ps -u username | wc -l) comes up with a number like 191, of which 89 are defunct chrome processes.  But that number is still less than the max user processes limit of 300.  My suspicion is that the some of the defunct processes are holding multiple defunct threads which may be counting towards the 300 limit but I may be wrong.

Expected Results:  
Both opera and chrome should not encounter "arbitrary" process limits (or should adjust to them).  They should also not cause other programs to malfunction.

Opera should not re-open so many files when it restarts.  Chrome should collect its children which have exited.

This can obviously be fixed by adjusting the open files ulimit (ulimit -n 4096 seems to work) and the max user processes ulimit (ulimit -u 1000 should work).

It would be nice to fix this on a system-wide basis but a search of the files in /etc/conf.d or /etc/*rc for general ulimit settings doesn't seem to indicate where this should be done.  The user can set ulimit -u in his .profile but changes to ulimit -n seem to require root privilege (presumably before login).

Ultimately, bug reports on these problems should be filed with Opera and Google but they don't (to my knowledge) use standard "bugzilla" databases.  If Gentoo developers do file bug reports, please post the URLs to track them in this bug report.
Comment 1 Justin Lecher (RETIRED) gentoo-dev 2009-10-02 18:22:00 UTC
(In reply to comment #0)
> It would be nice to fix this on a system-wide basis but a search of the files
> in /etc/conf.d or /etc/*rc for general ulimit settings doesn't seem to indicate
> where this should be done.  The user can set ulimit -u in his .profile but
> changes to ulimit -n seem to require root privilege (presumably before login).

You can change this systemwide in /etc/security/limits.conf.
Comment 2 Justin Lecher (RETIRED) gentoo-dev 2009-10-09 19:33:30 UTC
I assume this fixed this issue for you.
Comment 3 Robert Bradbury 2009-10-10 00:34:23 UTC
Justin, I have made the changes to limits.conf and rebooted (thank you for pointing this out).  And from reading the documentation I agree that in theory it should work, but in practice it doesn't appear to.  Some investigation suggests this may be because there appear to be no references to pam_limits.so in either /etc/pam.d/login or /etc/pam.d/other.  There are no bash, sh, opera or chrome entries in /etc/pam.d.  The only references I have to pam_limits.so are in cron, squid, start-stop-daemon, system-auth and system-services and those would not seem to be involved in standard user programs (login shell, browsers, etc.).

Of course I may completely misunderstand how this all works as none of this existed back in the '70s when I first started using UNIX (I think ulimit was added later perhaps in System 3).

I have filed a chrome Issue report about this and apparently the defunct process problem may be dependent on chrome version (which is of course in a rapid state of flux) and may be fixed in the most recent versions.  I haven't reported the opera problem because they don't have an open bug reporting system in which one can monitor comments/progress.

My current workarounds involve starting chrome from a shell which bumps ulimit -u and opera from a su'ed shell which bumps ulimit -n.
Comment 4 Justin Lecher (RETIRED) gentoo-dev 2009-10-10 07:29:12 UTC
Hi Robert,

this isn't a support forum and I am not really sure whether you are reporting something real. I will forward the bug to the maintainers, and let them judge.

Comment 5 Jeroen Roovers (RETIRED) gentoo-dev 2009-10-10 16:51:14 UTC
Nothing to do with Opera.
Comment 6 SpanKY gentoo-dev 2009-10-10 17:04:52 UTC
process limits are a login issue, not a per-application issue.  assuming you have USE=pam, then the aforementioned file is where you can configure default or per-user limits.
Comment 7 Robert Bradbury 2009-10-10 18:26:12 UTC
Ok, there is still some confusion (for me) between the limits.conf man page, the comments in limits.conf itself and the gentoo documentation on limits.conf, perhaps relating to the lack of clarity regarding what exact limits are imposed by the kernel itself (really hard limits) and what limits are pseudo-hard (imposed by limits.conf?) but I'm going to try working with the following and will assume it works.

# make core file size unlimited (firefox can easily run 1+GB)
*               soft    core            0
# increase nofile (1024 -> 2048-8192) for opera
*		hard	nofile		8192
*		soft	nofile		2048
# increase nproc (300 -> 500-1000) for chrome
*		hard	nproc		1000
*		soft	nproc		500
# allow rtprio setting from 0-32 to allow work around for emerges degrading desktop performance.
# Use "chrt -f -p prio real-time-process-id" to change user process real time priorities where necessary.
# "Real-time" process is a relative concept as it can mean everything from Xorg to mplayer to bash.
*		hard	rtprio		 31
*		soft	rtprio		 20