We just switched from courier-imap to dovecot, and it was a very smooth transition (and the performance improvement is amazing), but we are having one warning in the logs since we flipped the switch: Jun 4 10:42:40 myhost dovecot: imap(validuser@example.com): Warning: Inotify instance limit for user ### (UID aaaaaa) exceeded, disabling. Increase /proc/sys/fs/inotify/max_user_instances Getting a lot of these, not quite flooding the logs, but sometimes close... I chatted with Timo (dovecot dev), and he confirmed that this is a known issue for most people switching from courier to dovecot (and possibly just anyone using dovecot period). He told me how to fix it temporarily (echo 1024 > /proc/sys/fs/inotify/max_user_instances), but I have to do this after every restart of dovecot (and/or reboot?). He also mentioned I could modify the init script, but init scripts are one thing on gentoo t hat I try not to ever modify myself (its hard enough to keep up with config file changes). He then said that he had spent the day writing an RPM for dovecot, and was going to be fixing this issue in the init script for it, so hopefully this will provide an easy way for gentoo to do the same thing - it basically dynamically calculates the necessary size based on the current config, so if you make config changes that would require raising the limit, the init script figures this out and automatically adjusts it... Quote btw i've today been building RPMs for dovecot and in that init.d script i'm also adding this i'll make it somewhat automatic though: if [ "$INOTIFY_INSTANCES" != "" ]; then if [ "$INOTIFY_INSTANCES" = "auto" ]; then INOTIFY_INSTANCES=`$doveadm config -h service/imap/process_limit` if [ "$INOTIFY_INSTANCES" = 0 ]; then INOTIFY_INSTANCES=`$doveadm config -h default_process_limit` fi increase_proc_value /proc/sys/fs/inotify/max_user_watches $INOTIFY_INSTANCES increase_proc_value /proc/sys/fs/inotify/max_user_instances $INOTIFY_INSTANCES fi fi it looks up a good value from doveconf EndQuote So, I'm hoping the dovecot ebuild maintainer would please do the same for the dovecot ebuild... Thanks, Charles
In the meantime, you can set the value using sysctl: $ sudo sysctl -a | grep inotify fs.inotify.max_user_instances = 128 fs.inotify.max_user_watches = 8192 fs.inotify.max_queued_events = 16384 So just add, fs.inotify.max_user_instances = whatever to your /etc/sysctl.conf and it will take effect upon boot.
Thanks Michael, I had googled and didn't run across how to do that (I was thinking it may be a kernel config thing)... But I still think it makes much more sense to fix this permanently in the init script so everyone benefits from a silent dynamic (in that it increases/decreases the limit at dovecot startup based on the current config)...
(In reply to comment #2) > Thanks Michael, I had googled and didn't run across how to do that (I was > thinking it may be a kernel config thing)... > > But I still think it makes much more sense to fix this permanently in the > init script so everyone benefits from a silent dynamic (in that it > increases/decreases the limit at dovecot startup based on the current > config)... Well, it would probably prevent a few headaches, but might cause other problems. For example my default was max_user_instances = 128. Is that, a) 128 and I don't care what is this thing b) I want it to be 128 ? In the second case, the init script shouldn't change the value. The situation is even less clear if I've changed it from the default. Moreover, unless you provide each virtual dovecot user with its own UID, the increase isn't really necessary. We run everyone off one system user here, so the dovecot process_limit is irrelevant. Assuming the default is sane (someone smarter than I came up with it), it's probably better to leave it where it is on systems that don't need it. A warning on startup would be annoying if there could be false positives...
Timo should be commenting when he gets time (at least I asked him to and he said he would), but... (In reply to comment #3) > (In reply to comment #2) >> But I still think it makes much more sense to fix this permanently in the >> init script so everyone benefits from a silent dynamic (in that it >> increases/decreases the limit at dovecot startup based on the current >> config)... > Well, it would probably prevent a few headaches, but might cause other > problems. For example my default was max_user_instances = 128. Is that, What do you mean 'your default'? I think that is the system default, because I've never changed mine (never heard of it before), and also my sysctl.conf didn't have *anything* for inotify in it... > a) 128 and I don't care what is this thing > b) I want it to be 128 > > ? Valid concern... maybe Timo could add a config option, so that it only dynamically sets this value if you haven't set the config option? > In the second case, the init script shouldn't change the value. The > situation is even less clear if I've changed it from the default. > > Moreover, unless you provide each virtual dovecot user with its own UID, the > increase isn't really necessary. We run everyone off one system user here, > so the dovecot process_limit is irrelevant. I don't pretend to know the real answer here, but what I do know is that we only run one virtual (vmail) user, and our server is fairly lightly loaded, and we ran into this immediately. Timo said that almost everyone hits it... hopefully he will shed some light on this when he comments... > Assuming the default is sane (someone smarter than I came up with it), > it's probably better to leave it where it is on systems that don't need it. > > A warning on startup would be annoying if there could be false positives... Agree and agree, but it sounds like something is pretty much always needed under certain conditions, so hopefully Timo can figure out the best way to handle it for all cases...
(In reply to comment #4) > > What do you mean 'your default'? I think that is the system default, because > I've never changed mine (never heard of it before), and also my sysctl.conf > didn't have *anything* for inotify in it... > That's all I meant. It was 128 on a machine where I hadn't touched it. > I don't pretend to know the real answer here, but what I do know is that we > only run one virtual (vmail) user, and our server is fairly lightly loaded, > and we ran into this immediately. Timo said that almost everyone hits it... > hopefully he will shed some light on this when he comments... Sorry, I had the two cases switched. max_user_instances is the limit on the number of inotify instances that any user can have open. So it should only be a problem if you share the UID, not the other way around.
(In reply to comment #5) > That's all I meant. It was 128 on a machine where I hadn't touched it. ah, ok... > Sorry, I had the two cases switched. max_user_instances is the limit on the > number of inotify instances that any user can have open. So it should only > be a problem if you share the UID, not the other way around. gotcha... but i think since most people run with shared UID (running with individual UIDs is much more complicated), that should be the 'default' assumption...
(In reply to comment #6) > (In reply to comment #5) > > That's all I meant. It was 128 on a machine where I hadn't touched it. > > ah, ok... > > > Sorry, I had the two cases switched. max_user_instances is the limit on the > > number of inotify instances that any user can have open. So it should only > > be a problem if you share the UID, not the other way around. > > gotcha... but i think since most people run with shared UID (running with > individual UIDs is much more complicated), that should be the 'default' > assumption... Indeed, I did not have that part mixed up: we run a shared user, so I've probably already had to set this on our production server. My original goal wasn't to fight about whether it was a good/bad idea, just to let you know about the sysctl option =)
(In reply to comment #7) > My original goal wasn't to fight about whether it was a good/bad idea, just > to let you know about the sysctl option =) And it is appreciated... :)
I would say that on any server with enough users, you should tweak the default system limits to match its expected use. Putting that in an init.d script is not a solution. So just tweak the system to match what you expect it to deliver. We could probably talk about raising the defaults more generally, but that shouldn't rely on a single package maxing out some of them.
Please use the sysctl commands that Micheal mentioned to adjust your settings. Setting system wide defaults in dovecot init script in an ad hoc manner is not a good idea.
