I run qmail with vpopmail. For some reason I have/need courier-imap. Yesterday morning I upgraded courier-imap from 3.0.8 to 4.0.1. The emerge finished successfully and I did etc-update, writing over old files as I did not see anything installation specific. I restarted courier-imapd-ssl and courier-pop3d-ssl and went to work. Received a call a short time later that a user was not able to retrieve mail. Not having time to debug while at work, I logged in and emerged courier-imap-3.0.8. I received another call just as the emerge was finishing -- before I had even done etc-update or restarted the daemons -- that mail was working again. Not wishing to take more time away from work, I left everything as it was. When I got home, I tried emerging 4.0.1 again. When it finished I realized that I hadn't updated the config files after the downgrade, so I kept the "non-._cfg" files. I stopped imapd-ssl, pop3d-ssl and authdaemon and then started authlib, imapd-ssl, and pop3d-ssl. Was unable to check mail and there was no error message in the logs. In fact, throughout this whole frustrating affair, I was never given a useful error message. I looked through all the courier-imap/authlib config files for potential missettings, but it seemed that all I really needed was "authvchkpw" and I had that. I read through the 4.0.1 readme file (which, like so much gentoo documentation, is pretty much useless if things don't go as planned) and checked all the settings. After struggling in vain for a few hours to get 4.0.1 working, I decided to give up and go back to 3.0.8 again. So, I did the downgrade, updated config files, and restarted the daemons (incl. authdaemond instead of authlib) and was extremely depressed to find that I could not check mail. When I tried to start authdaemond, I got the error that /usr/sbin/courierlogger could not be found... and indeed, it was missing. I copied that from my backup server and the error went away, but I still could not check my mail. I went through config files several times and restored the previous day's config files from backup. Restarted the daemons several times. Went through the qmail/imap/vpopmail how-to on the forums several times. Finally, after restarting the daemons again, it did start working... though I'm not sure exactly why. But, I'm still on 3.0.8... I've masked any courier-imap > 4 and I pretty much don't want to touch anything that has anything to do with email on my server again. However, maybe someone else has had this problem and better understands the interplay between qmail, vpopmail, courier-imap, and authdaemon/authlib. Reproducible: Always Steps to Reproduce: 1.start with qmail, vpopmail and courier-imap-3.0.8 2.upgrade to courier-imap-4.0.1 3.try to check your mail Actual Results: Everything's in the details... Expected Results: Everything's in the details... Here's my emerge info: Portage 2.0.51-r15 (default-linux/x86/2004.0, gcc-3.3.5, glibc-2.3.4.20040808-r1, 2.4.23_pre8-gss-r2 i686) ================================================================= System uname: 2.4.23_pre8-gss-r2 i686 Intel(R) Celeron(R) CPU 1.70GHz Gentoo Base System version 1.4.16 Python: dev-lang/python-2.2.3-r5,dev-lang/python-2.3.4 [2.3.4 (#1, Dec 5 2004, 12:46:48)] dev-lang/python: 2.2.3-r5, 2.3.4 sys-devel/autoconf: 2.59-r6, 2.13 sys-devel/automake: 1.8.5-r2, 1.5, 1.4_p6, 1.6.3, 1.7.9, 1.9.4 sys-devel/binutils: 2.15.92.0.2-r1 sys-devel/libtool: 1.5.10-r4 virtual/os-headers: 2.4.19-r1, 2.4.21-r1 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CFLAGS="-O2 -march=pentium4 -fprefetch-loop-arrays -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/bind /var/qmail/alias /var/qmail/control /var/vpopmail/domains /var/vpopmail/etc" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-O2 -march=pentium4 -fprefetch-loop-arrays -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs autoconfig ccache distlocks fixpackages sandbox sfperms" GENTOO_MIRRORS="ftp://mirrors.tds.net/gentoo http://mirror.datapipe.net/gentoo ftp://gentoo.ccccom.com" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="X adns apache2 berkdb crypt curl doc flash gd gdbm gif imagemagick imap innodb ipv6 java jpeg kerberos libg++ libwww maildir mbox mcal mmx mysql ncurses nls odbc pam pdflib perl png postgres prelude python qt quicktime readline sasl snmp socks5 spell ssl tcltk tcpd tiff truetype unicode vhosts x86 xml xml2 zeo zlib" Unset: ASFLAGS, CBUILD, CTARGET, LANG, LC_ALL, LDFLAGS, PORTDIR_OVERLAY
I see from bug 70874 that I should assign this to netmail@g.o
The emerge finished successfully and I did etc-update, writing over old files as I did not see anything installation specific. I restarted courier-imapd-ssl and courier-pop3d-ssl and went to work. > did you start courier-authlib.
obviuosly you missed the postinst messages * Authdaemond is no longer provided this package. * athentication libraries are from courier-authlib * for a quick start please refer to * /usr/share/doc/courier-imap-4.0.1/courier-imap-gentoo.readme.gz sound like a misconfig. We are idle in #gentoo-netmail on irc.freenode.net. Join the channel if you need help.
Look, I read the post-install stuff, I read the courier-imap site, I read the forum howto. I know that courier now uses courier-authlib instead of authdaemond. I tried the upgrade... twice. The first time I may have missed something, but the second time I tried everything I could think of. I spent several hours trying to get 4.0.1 to work and then several more hours trying to get 3.0.8 to work again. And during that time, several dozen email users could not retrieve their email. I find that unacceptable. If there were at least some error reporting, some indication of what was wrong while it wasn't working, that would have been a little better. But I found the error reporting non-existent and the documentation useless. It was, without a doubt the most frustrating experience I have ever had with gentoo. I know there may have been configuration issue, however if the documentation fails to warn users of a common enough potential pitfall, that is still a bug in my book. I opened this bug knowing that it may have just been a configuration issue, but I know of at least one other user (from bug 70874) that has had the problem. Perhaps others have as well and with enough comments we can figure out whether there's some improvements to documentation to be made, there's a real bug in the code, or if I am simply thickheaded.
> I've masked any courier-imap > 4 and I pretty much don't want to touch anything that has anything to do with email on my server again. sound like you are not willing help us to help to resolve the issue. reopen when you are ready.
Tuan Van: These errors are definitely misconfiguration-based. Perhaps the post-install warnings need to be much more explicit. Please have a look at some of suggestions in the forums and consider adding them to ewarn or /usr/share/doc/courier-imap-4.0.1/courier-imap-gentoo.readme.gz (like rc-update stuff etc.) http://forums.gentoo.org/viewtopic.php?t=290170 There are also many other threads concerning this, so it seems to me that people need some big red shiny warning, otherwise they just ignore that. :-(
>------- Additional Comment #5 From Tuan Van 2005-02-05 13:47 PST ------- > >sound like you are not willing help us to help to resolve the issue. reopen >when you are ready. No, I opened the bug to see if others were having issues with this... and it would seem that they are. I followed the post-install instructions and found that I was unable to retrieve mail... and without anything in the logs to indicate why. So, okay, I'm willing to help you resolve the issue... if you're willing to help me resolve it. So: What would cause mail retrieval to fail silently? Is there a specific order I should stop the old daemons/start the new daemons? In the forum thread sigmalll wrote that he invoked /usr/lib/courier/courier-authlib/authdaemond directly... why? And could the fact that I did not do this have been a problem for me? If I'm using qmail and vpopmail (with horde-imp for webmail) I should only need authvchkpw... right? For that matter, why do I need courier at all if I'm running qmail? Doesn't qmail provide everything I need? I didn't decide to install courier for fun... something else wanted it... although doing emerge -p qmail vpopmail horde-imp on my home box (with no email capability currently) does not show courier at all. Anyway, there's a bunch of questions to start with. I'm willing to try the emerge again tonight, though I'm a bit spooked by the fact that reverting to 3.0.8 last night did not fix the problem initially. That's why I'd like some additional information before risking hosing my mail server again. thanks...
Ben: the basic procedure for upgrading courier-imap (when using vpopmail+qmail) should be the following: 1. emerge -B courier-authlib courier-imap # build new version, don't install yet 2. stop ALL components of courier-imap, INCLUDING authdaemon! 3. emerge -k courier-authlib courier-imap # install new version 4. check authlib configurations in new location that vchkpw is still set (it should be) [see authmodulelist in /etc/courier/authlib/authdaemonrc]. 5. restart courier-imap services, authlib service should be brought up automatically. 6. follow the instructions to test IMAP login as described in the doc others have pointed out. qmail only provides pop3, so you need courier-imap if you want an IMAP server (which is MUCH better than POP3).
Hmmm... No, I didn't do all that. I don't think I've actually ever even used the -B and -k options to emerge... seems like a good approach to something sensitive like a mail server. So, perhaps not specifically stopping authdaemond before emerging authlib could have caused a problem. As far as imap vs pop3 goes, none of the people who have email accounts on my server are accessing them using imap. At the time I set up the server, my only experience with imap was with my school email and that was excrutiatingly slow. I've since been told that that was because they were using mbox instead of maildir, but whatever... it doesn't change the fact that imap, I believe, is unused on my server. In fact, as an experiment I just went and tried to set up thunderbird to retrieve mail from one of my accounts on the server using imap-ssl... and it failed. The log says "imapd-ssl: LOGIN FAILED". Seems that I can't use the imap capabilities even if I want. Anyway, I'd like to keep things simple... five or six installs just for email does not seem simple (qmail, courier, authlib, vpopmail, qmailadmin, vqadmin... oh, and spamassassin, clamav, and qmail-scanner too). So, the question is: is there some way I can I retrieve email over ssl using a desktop client or through horde-imp with only qmail/vpopmail (i.e. without courier)? thanks... PS: if there are any other docs that are not out-of-date or incorrect for gentoo, I'd love to hear about them.
I ran into a different problem - since I have the postgres and ldap flags set in my USE, authdaemon was trying to use those methods first, and failing because of configuration issues with those modules - it would then fail to use any of the other modules. I would recommend putting only pam and pw authentication in the default list of modules to use, and allowing users who want to use other authentication mechanisms to add them to the list.
For anything really critical to a system it can be wise to either use quickpkg to make a snapshot of the files that'll be removed/overwritten during an upgrade -B is handy even just as a dry-run of an emerge, though since it does not really install any files, none of the packages that are to be installed after it will be able to see it or know it exists or build against any of the libraries it provides. In this case though, courier-imap-4 will not even begin building as authlib is a firm requirement for its configure to even proceed. courier-authlib can be built/installed before the old courier-imap parts are taken out. It puts the migrated daemon and config files into non-overlapping locations so it isn't going to step on your old program/config. if in trying to start an authdaemond, it complains about the logger, then you are trying to run the wrong authdaemond. with courier-imap-4 and above, the old /etc/init.d/authdaemond script is no longer used. Delete it. The one that authlib installs is /etc/init.d/courier-authlib. You do NOT need to add it to any runlevel by way of rc-update. The init scripts that need it will start it all by themselves when they are starting. And since the old authdaemond script no longer points to files that exist, you should at the very least do an "rc-update del authdaemond". The thread suggestions to run some authdaemond directly or making a symlink to it or anything along those lines are incorrect. If doing that fixes the problem, then someone probably neglected to allow the files in /etc/init.d get updated to point to require courier-authlib instead of authdaemond, or they didn't stop/remove the old authdaemond. Due to the nature of the imap protocol, and that it supports nondestructive reads of a users mailbox, supports subfolders, has commands to request a list of messages, read an individual message... it is exactly what is needed to offer webmail, as pop3 would waste time fetching copies of each message into a temp location, process them, etc - imap is the only really practical way to offer webmail. And as for needing 9 components for email, one could also ask why you mess around with all those when courier-mta handles pop3/imap/smtp and ssl/tls encryption for every one of those and has builtin support for virtual users in ldap/mysql/postgres and integrated handling of mailing lists as well as webmail (admittedly not the prettiest webmail but no reason you couldn't use your favorite one, since imap is of course already available) and of course will let everything be passed thru spamassassin, and so on. So, no, a 9-part set is not the *only* way to get the functionality you are asking for. But thats beside the point. And as for the last comment, try adding "courier-authlib -ldap -whateverelse" to /etc/portage/package.use to disable components you dont need/want. That is advisable for any package, not just this one. Having, for example, "ldap" in your use flags will cause openldap and likely a cascade of another dozen packages to also be installed as well. If you don't want something, do not add it as a use flag. The whole point of a use flag is to tell a package that you WANT to make use that feature. Because of that, a module will not only be installed, but it will be activated as well, in the presence of its use flag.
Thanks for all the info Scott... very helpful. I will definitely look into switching to just courier-imap. The one thing that I'm still puzzled about, however, is why -- if I'm running courier-imapd-ssl anyway -- I can't retrieve my mail with thunderbird using an account configured to use imap-ssl? Other than that, I must admit that this bug has degenerated into basically a forum thread... hoping for a little more guidance, but close it if you must.
run into the same problem after upgrading to 4.0.1: Can't login anymore. Symptom: nc localhost 143 * BYE imaplogin expected exactly two arguments. (no entries in syslog...) Problem was /etc/courier-imap/imapd --> MAILDIR I'm using a virtual maildir setup, an MAILDIR should point to the current directory. With 3.0.x it was enough to set it empty, e.g. MAILDIR= Now that does'nt work anymore, but MAILDIR=./ fixed that.
I disagree that having a global USE flag set means that I want to use that for every package. It may mean I want to build support for that, but not necessarily that I want to use it immediately.
Feel free to disagree. Use flags exist to activate features, and they can be activated and deactivated on a package-by-package basis by way of the /etc/portage/package.use file. If you don't like how a package is built with a certain set of use flags, then change the set of use flags that ebuild sees. re: comment #12 each listener (unencrypted imap, imap+ssl, and so on) each have their own config file, each of which usually have a line near the bottom that needs set to "yes" to start that daemon, in addition to adding their startup script by way of rc-update. Also, setting the tls method from TLS1 to SSL3 for the imapd-ssl config might be worth a shot. Some mail clients automatically change the port they try to hit on the server when you request they use tls/ssl though others don't. The imapd-ssl (and pop3d-ssl and esmtpd-ssl) all expect encryption to happen from the beginning, and a plaintext session starting to them will not work. The standard ports for imap/pop3/smtp typically support the STARTTLS extension, which means they have the plaintext session like everything used to, but the client can send the STARTTLS message, at which point the encryption handshake takes place on the same socket. When you tell a mail client to "always" encrypt, it generally expects the encryption to take place from the beginning, on the imapd-ssl port. When a client has "encrypt if available" or something to that effect, it usually tries the standard imapd port, and, if the server says it supports STARTTLS, it'll encrypt, otherwise it won't, but that "if available" setting won't usually get it to try to connect to the imapd-ssl port. Looking at your syslog, the -ssl is listed if that is the port the connection came in on. One could happily encrypt to thunderbird while the other doesn't. Need to first find out which ones are being used to troubleshoot further...
Well, I finally worked up the courage to try this again. Followed instructions from comment #8. Unfortunately it didn't work right away. It turned out that I still had old pid files in var run, so authlib was never really starting. At least I think that's what happened. In any case, I deleted all the courier related pid files and started the imap/pop3d daemons and authlib started up just fine. It would be nice to put a warning about the pid files in the docs somewhere.
