Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 419733

Summary: net-mail/dovecot - modify init script to dynamically determine inotify max_user_instances
Product: Gentoo Linux Reporter: tanstaafl <tanstaafl>
Component: [OLD] ServerAssignee: Eray Aslan <eras>
Status: RESOLVED WONTFIX    
Severity: enhancement CC: mjo, net-mail+disabled
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---

Description tanstaafl@libertytrek.org 2012-06-05 10:10:31 UTC
We just switched from courier-imap to dovecot, and it was a very smooth transition (and the performance improvement is amazing), but we are having one warning in the logs since we flipped the switch:

Jun  4 10:42:40 myhost dovecot: imap(validuser@example.com): Warning: Inotify instance limit for user ### (UID aaaaaa) exceeded, disabling. Increase /proc/sys/fs/inotify/max_user_instances

Getting a lot of these, not quite flooding the logs, but sometimes close...

I chatted with Timo (dovecot dev), and he confirmed that this is a known issue for most people switching from courier to dovecot (and possibly just anyone using dovecot period). He told me how to fix it temporarily (echo 1024 > /proc/sys/fs/inotify/max_user_instances), but I have to do this after every restart of dovecot (and/or reboot?). He also mentioned I could modify the init script, but init scripts are one thing on gentoo t hat I try not to ever modify myself (its hard enough to keep up with config file changes).

He then said that he had spent the day writing an RPM for dovecot, and was going to be fixing this issue in the init script for it, so hopefully this will provide an easy way for gentoo to do the same thing - it basically dynamically calculates the necessary size based on the current config, so if you make config changes that would require raising the limit, the init script figures this out and automatically adjusts it...

Quote

btw i've today been building RPMs for dovecot
and in that init.d script i'm also adding this
i'll make it somewhat automatic though:

 if [ "$INOTIFY_INSTANCES" != "" ]; then
   if [ "$INOTIFY_INSTANCES" = "auto" ]; then
     INOTIFY_INSTANCES=`$doveadm config -h service/imap/process_limit`
     if [ "$INOTIFY_INSTANCES" = 0 ]; then
       INOTIFY_INSTANCES=`$doveadm config -h default_process_limit`
     fi
     increase_proc_value /proc/sys/fs/inotify/max_user_watches $INOTIFY_INSTANCES
     increase_proc_value /proc/sys/fs/inotify/max_user_instances $INOTIFY_INSTANCES
   fi
 fi

it looks up a good value from doveconf

EndQuote

So, I'm hoping the dovecot ebuild maintainer would please do the same for the dovecot ebuild...

Thanks,

Charles
Comment 1 Michael Orlitzky gentoo-dev 2012-06-05 12:54:35 UTC
In the meantime, you can set the value using sysctl:

  $ sudo sysctl -a | grep inotify
  fs.inotify.max_user_instances = 128
  fs.inotify.max_user_watches = 8192
  fs.inotify.max_queued_events = 16384

So just add,

  fs.inotify.max_user_instances = whatever

to your /etc/sysctl.conf and it will take effect upon boot.
Comment 2 tanstaafl@libertytrek.org 2012-06-05 13:18:59 UTC
Thanks Michael, I had googled and didn't run across how to do that (I was thinking it may be a kernel config thing)...

But I still think it makes much more sense to fix this permanently in the init script so everyone benefits from a silent dynamic (in that it increases/decreases the limit at dovecot startup based on the current config)...
Comment 3 Michael Orlitzky gentoo-dev 2012-06-05 13:54:56 UTC
(In reply to comment #2)
> Thanks Michael, I had googled and didn't run across how to do that (I was
> thinking it may be a kernel config thing)...
> 
> But I still think it makes much more sense to fix this permanently in the
> init script so everyone benefits from a silent dynamic (in that it
> increases/decreases the limit at dovecot startup based on the current
> config)...

Well, it would probably prevent a few headaches, but might cause other problems. For example my default was max_user_instances = 128. Is that,

  a) 128 and I don't care what is this thing
  b) I want it to be 128

?

In the second case, the init script shouldn't change the value. The situation is even less clear if I've changed it from the default.

Moreover, unless you provide each virtual dovecot user with its own UID, the increase isn't really necessary. We run everyone off one system user here, so the dovecot process_limit is irrelevant. Assuming the default is sane (someone smarter than I came up with it), it's probably better to leave it where it is on systems that don't need it.

A warning on startup would be annoying if there could be false positives...
Comment 4 tanstaafl@libertytrek.org 2012-06-05 14:39:46 UTC
Timo should be commenting when he gets time (at least I asked him to and he said he would), but...

(In reply to comment #3)
> (In reply to comment #2)
>> But I still think it makes much more sense to fix this permanently in the
>> init script so everyone benefits from a silent dynamic (in that it
>> increases/decreases the limit at dovecot startup based on the current
>> config)...

> Well, it would probably prevent a few headaches, but might cause other
> problems. For example my default was max_user_instances = 128. Is that,

What do you mean 'your default'? I think that is the system default, because I've never changed mine (never heard of it before), and also my sysctl.conf didn't have *anything* for inotify in it...

>   a) 128 and I don't care what is this thing
>   b) I want it to be 128
> 
> ?

Valid concern... maybe Timo could add a config option, so that it only dynamically sets this value if you haven't set the config option?

> In the second case, the init script shouldn't change the value. The
> situation is even less clear if I've changed it from the default.
> 
> Moreover, unless you provide each virtual dovecot user with its own UID, the
> increase isn't really necessary. We run everyone off one system user here,
> so the dovecot process_limit is irrelevant.

I don't pretend to know the real answer here, but what I do know is that we only run one virtual (vmail) user, and our server is fairly lightly loaded, and we ran into this immediately. Timo said that almost everyone hits it... hopefully he will shed some light on this when he comments...

> Assuming the default is sane (someone smarter than I came up with it),
> it's probably better to leave it where it is on systems that don't need it.
> 
> A warning on startup would be annoying if there could be false positives...

Agree and agree, but it sounds like something is pretty much always needed under certain conditions, so hopefully Timo can figure out the best way to handle it for all cases...
Comment 5 Michael Orlitzky gentoo-dev 2012-06-05 15:11:55 UTC
(In reply to comment #4)
> 
> What do you mean 'your default'? I think that is the system default, because
> I've never changed mine (never heard of it before), and also my sysctl.conf
> didn't have *anything* for inotify in it...
> 

That's all I meant. It was 128 on a machine where I hadn't touched it.


> I don't pretend to know the real answer here, but what I do know is that we
> only run one virtual (vmail) user, and our server is fairly lightly loaded,
> and we ran into this immediately. Timo said that almost everyone hits it...
> hopefully he will shed some light on this when he comments...

Sorry, I had the two cases switched. max_user_instances is the limit on the number of inotify instances that any user can have open. So it should only be a problem if you share the UID, not the other way around.
Comment 6 tanstaafl@libertytrek.org 2012-06-05 16:04:05 UTC
(In reply to comment #5)
> That's all I meant. It was 128 on a machine where I hadn't touched it.

ah, ok...

> Sorry, I had the two cases switched. max_user_instances is the limit on the
> number of inotify instances that any user can have open. So it should only
> be a problem if you share the UID, not the other way around.

gotcha... but i think since most people run with shared UID (running with individual UIDs is much more complicated), that should be the 'default' assumption...
Comment 7 Michael Orlitzky gentoo-dev 2012-06-05 16:15:52 UTC
(In reply to comment #6)
> (In reply to comment #5)
> > That's all I meant. It was 128 on a machine where I hadn't touched it.
> 
> ah, ok...
> 
> > Sorry, I had the two cases switched. max_user_instances is the limit on the
> > number of inotify instances that any user can have open. So it should only
> > be a problem if you share the UID, not the other way around.
> 
> gotcha... but i think since most people run with shared UID (running with
> individual UIDs is much more complicated), that should be the 'default'
> assumption...

Indeed, I did not have that part mixed up: we run a shared user, so I've probably already had to set this on our production server.

My original goal wasn't to fight about whether it was a good/bad idea, just to let you know about the sysctl option =)
Comment 8 tanstaafl@libertytrek.org 2012-06-05 17:03:12 UTC
(In reply to comment #7)
> My original goal wasn't to fight about whether it was a good/bad idea, just
> to let you know about the sysctl option =)

And it is appreciated... :)
Comment 9 Jeroen Roovers (RETIRED) gentoo-dev 2012-06-06 03:42:03 UTC
I would say that on any server with enough users, you should tweak the default system limits to match its expected use.

Putting that in an init.d script is not a solution. So just tweak the system to match what you expect it to deliver.

We could probably talk about raising the defaults more generally, but that shouldn't rely on a single package maxing out some of them.
Comment 10 Eray Aslan gentoo-dev 2012-06-06 08:00:59 UTC
Please use the sysctl commands that Micheal mentioned to adjust your settings.  Setting system wide defaults in dovecot init script in an ad hoc manner is not a good idea.