Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 42185 - Cyrus-imapd + Berkeley db fail to open/create db-s in /var/imap
Summary: Cyrus-imapd + Berkeley db fail to open/create db-s in /var/imap
Status: RESOLVED CANTFIX
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: x86 Linux
: High critical (vote)
Assignee: Net-Mail Packages
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-02-19 13:30 UTC by borut.kersevan
Modified: 2005-02-14 14:24 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description borut.kersevan 2004-02-19 13:30:50 UTC
I'm using everything latest&greatest (stable) from Gentoo 1.4 however when emerging and running cyrus-imapd I get the following messages:

Feb 19 22:38:36 havelock master[11595]: about to exec /usr/cyrus/bin/ctl_cyrusdb
Feb 19 22:38:36 havelock ctl_cyrusdb[11595]: recovering cyrus databases
Feb 19 22:38:54 havelock ctl_cyrusdb[11595]: DBERROR db4: unable to join the environment
Feb 19 22:38:54 havelock ctl_cyrusdb[11595]: DBERROR db4: write: 0xbfffc890, 8192: Invalid argument
Feb 19 22:38:54 havelock ctl_cyrusdb[11595]: DBERROR: dbenv->open '/var/imap/db' failed: Invalid argument
Feb 19 22:38:54 havelock ctl_cyrusdb[11595]: DBERROR: init /var/imap/db: cyrusdb error
Feb 19 22:38:54 havelock ctl_cyrusdb[11595]: DBERROR db4: environment not yet opened
Feb 19 22:38:54 havelock ctl_cyrusdb[11595]: DBERROR: opening /var/imap/mailboxes.db: Invalid argument
Feb 19 22:38:54 havelock ctl_cyrusdb[11595]: DBERROR: opening /var/imap/mailboxes.db: cyrusdb error
Feb 19 22:38:54 havelock master[11593]: process 11595 exited, status 75
Feb 19 22:38:54 havelock master[11593]: ready for work

and then in intervals:

Feb 19 22:39:12 havelock ctl_deliver[11611]: DBERROR: init /var/imap/db: cyrusdb error
Feb 19 22:39:12 havelock master[11593]: process 11611 exited, status 1
Feb 19 22:39:12 havelock ctl_cyrusdb[11612]: DBERROR db4: unable to join the environment
Feb 19 22:39:12 havelock ctl_cyrusdb[11612]: DBERROR: dbenv->open '/var/imap/db' failed: Resource temporarily unavail
able

I managed to build and run the latest vanilla cyrus and Berkeley db 4.x on RH9 so I'm baffled about the ebuild
 
Best,

BPK

Reproducible: Always
Steps to Reproduce:
I did nothing at all to the e-merges (ok, several things but I also tried to reproduce it directly..).
1.emerge postfix, cyrus-sasl, cyrus... 

 



Expected Results:  



I can paste the RH9 log which says:
Aug  6 01:46:39 havelock master[28349]: about to exec /usr/cyrus/bin/ctl_cyrusdb
Aug  6 01:46:39 havelock ctl_cyrusdb[28349]: recovering cyrus databases
Aug  6 01:46:39 havelock ctl_cyrusdb[28349]: done recovering cyrus databases
Aug  6 01:46:39 havelock master[28348]: ready for work
Aug  6 01:46:39 havelock master[28350]: about to exec /usr/cyrus/bin/ctl_cyrusdb
Aug  6 01:46:39 havelock ctl_cyrusdb[28350]: checkpointing cyrus databases
Aug  6 01:46:39 havelock ctl_cyrusdb[28350]: archiving log file:
/var/imap/db/log.0000000001
Aug  6 01:46:39 havelock ctl_cyrusdb[28350]: archiving database file:
/var/imap/mailboxes.db
Aug  6 01:46:39 havelock ctl_cyrusdb[28350]: archiving log file:
/var/imap/db/log.0000000001
Aug  6 01:46:39 havelock ctl_cyrusdb[28350]: done checkpointing cyrus databases
Aug  6 01:46:39 havelock master[28348]: process 28350 exited, status 0
Aug  6 01:47:00 havelock master[28370]: about to exec /usr/cyrus/bin/imapd
Aug  6 01:47:00 havelock imap[28370]: executed
Aug  6 01:46:39 havelock master[28349]: about to exec /usr/cyrus/bin/ctl_cyrusdb
Aug  6 01:46:39 havelock ctl_cyrusdb[28349]: recovering cyrus databases
Aug  6 01:46:39 havelock ctl_cyrusdb[28349]: done recovering cyrus databases
Aug  6 01:46:39 havelock master[28348]: ready for work
Aug  6 01:46:39 havelock master[28350]: about to exec /usr/cyrus/bin/ctl_cyrusdb
Aug  6 01:46:39 havelock ctl_cyrusdb[28350]: checkpointing cyrus databases
Aug  6 01:46:39 havelock ctl_cyrusdb[28350]: archiving log file:
/var/imap/db/log.0000000001
Aug  6 01:46:39 havelock ctl_cyrusdb[28350]: archiving database file:
/var/imap/mailboxes.db
Aug  6 01:46:39 havelock ctl_cyrusdb[28350]: archiving log file:
/var/imap/db/log.0000000001
Aug  6 01:46:39 havelock ctl_cyrusdb[28350]: done checkpointing cyrus databases
Aug  6 01:46:39 havelock master[28348]: process 28350 exited, status 0
Aug  6 01:47:00 havelock master[28370]: about to exec /usr/cyrus/bin/imapd
Aug  6 01:47:00 havelock imap[28370]: executed
Comment 1 borut.kersevan 2004-02-20 11:17:04 UTC
Actually, it is most probably a kernel/glibc issue (NPTL?).. the same code works without problems when working in a chrooted env on RH9!! Any comments regarding gentoo-source kernel patches? (on RH mailing discussion it is said also to be an non-intel issue (I have athlonXP...)..?
Comment 2 Max Kalika (RETIRED) gentoo-dev 2004-02-25 16:47:54 UTC
so any chance you can try another kernel (vanilla-sources) and see if problems go away?  if so, we'll re-assign to the kernel folk.
Comment 3 Max Kalika (RETIRED) gentoo-dev 2004-03-15 10:24:44 UTC
any word on this?
Comment 4 Denny Schierz 2004-06-17 03:22:19 UTC
Same problem here,

the system was running for ~8 months, but in the last weeks i have this error:

Jun 17 08:44:57 s15144503 master[8909]: about to exec
/usr/cyrus/bin/imapd
Jun 17 08:44:57 s15144503 imaps[8909]: executed
Jun 17 08:44:57 s15144503 imapd[8909]: DBERROR db4: fatal region error
detected; run recovery
Jun 17 08:44:57 s15144503 imapd[8909]: DBERROR: dbenv->open
'/var/imap/db' failed: DB_RUNRECOVERY: Fatal error, run database
recovery

I had to remove the *db and the system is working again for a few days (2-3).
Comment 5 Roberto P.R. 2004-10-26 05:51:05 UTC
I had the same problem here. The server is an Athlon-XP 2800, and Gentoo version is 2004.2, with gentoo-dev-sources 2.6.8 and NPTL. I was using cyrus 2.1.3 and everything was working fine. Then, when I tried to upgrade to cyrus 2.2.8, I had the same problems described above, on the same machine and same operational environment. 

What happens is that the cyrus documentation tells us to convert the mailboxes from the Berkeley DB format, to the new "skiplist" format. Then, it seems that cyrus itself doesn't take that in consideration, and try to use its db4 interfaces to access those "skiplist" DB files ! 

If you run db_verify (from the Berkeley DB utils) over one of the converted "skiplist" DB files, it will give the same error message for db4 showed above !

So, I'm wondering what the heck is going on, since cyrus 2.2.8 doesn't use Berkeley DBs anymore, but it's trying to use db4 functions to access its databases ! 

Meanwhile, I'll try to find some solution myself...

Thanks
Comment 6 Paul Slinski 2004-11-25 14:00:42 UTC
I've tried various options to solve this riddle but it's still not functioning.

Here are a few I've tried so you don't waste your time:

One mailing list post said to build cyrus "--with-bdb=no" - that failed as some of the tools seem to rely on db.

I've tried directives in the imapd.conf file:
annotation_db:          skiplist
duplicate_db:           skiplist
mboxlist_db:            skiplist
ptscache_db:            skiplist
quota_db:               skiplist
seenstate_db:           skiplist
lscache_db:             skiplist
subscription_db:        skiplist

That fails. It still tries to use db4.

So, there are a couple of things I've tried.
Anyone else having any luck?
Comment 7 Paul Slinski 2004-11-25 14:03:27 UTC
Oh, and I also read on a redhat mailing list that db4 is no good with NPTL enabled.
Comment 8 Paul Slinski 2004-11-26 08:12:22 UTC
For me, it seemed that I had a db file that couldn't be converted to skiplist.
Unfortunatly, it was my mailboxes.db file. I had to remove it and create a new file to get cyrus to operate correctly.

You'll need to "search and destroy" the broken db files (or at least move them to a backup folder).

Also, in the upgrade docs they mention adding a couple of config options in order to keep the old functionality:

"Specify --with-mboxlist-db=berkeley and --with-seen-db=flat to configure. This will instruct Cyrus to continue to use the previous defaults."

I assume this could be added to the ebuild to get the job done. For me, it was too late to turn back. (maybe a use flag is in order?)

Also note that the .seen and .sub files need to be converted from flat to skiplist.

Needless to say, this was the worst upgrade I've done in a long time.

It's not a Gentoo bug. It's a cyrus problem. If there were a cleaner way to upgrade we wouldn't have had this problem.
Comment 9 Steven Green 2004-12-07 18:10:10 UTC
I had many of the problems mentioned here... one important step is that all the database converting should be done as the cyrus user (not as root).  

Once I realised this I ran "chown -R cyrus:mail /var/imap" and cyrus-imapd is now working fine.
Comment 10 Paul Slinski 2004-12-08 06:30:06 UTC
RE: Comment #9

That wasn't the case at this paricular location. All of the conversion was performed as the cyrus user, but the upgrade still did not go as planned.

It's a fairly complex task, and simple at the same time. It would help however if the docs from Project Cyrus were more than a changelog.

Regardless, maybe there should be some docs for this created by some generous Gentoo user ;-)
Comment 11 Tuan Van (RETIRED) gentoo-dev 2005-01-12 19:56:28 UTC
if any of you still having this problem, please try the solution in this archive:
http://asg.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&searchterm=skiplist&msg=32337
and report back, I'll put more info into pkg_postinst msg.
Comment 12 Paul Slinski 2005-01-19 09:11:23 UTC
Thanks, testing now.
Comment 13 Fernando J. Pereda (RETIRED) gentoo-dev 2005-02-14 14:24:08 UTC
We discussed this one and concluded that this one can't be fixed. As ticho said: "we would if we could, but since we can't, we won't".

Regards,
Ferdy