Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 460820 - app-office/akonadi-server disable sqlite use flag
Summary: app-office/akonadi-server disable sqlite use flag
Status: RESOLVED WONTFIX
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] KDE (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo KDE team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-03-08 20:12 UTC by William L. Thomson Jr.
Modified: 2013-04-29 21:21 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description William L. Thomson Jr. 2013-03-08 20:12:49 UTC
I am not sure sqlite should be used with akonadi anymore, or if it ever should. It looks like allot of the problems I had with kmail and akonadi stems more from sqlite, than anything else. After update to KDE 4.10, I could not get kmail or akonadi to work right at all. Not till I found a post in the forum, http://forums.gentoo.org/viewtopic-t-950578-start-0.html, where they switched to mysql from sqlite. Not that I am a fan really of either, I use Firebird, but at least I am not having the issues I was, and kmail/akonadi seems a bit faster and more responsive. I did not do all the rest of the stuff in that forum post, I just switched back ends, at first there were no emails, but mysql very quickly indexed them and they were present, always there on disk, just not in db.

In my case I had to switch, sqlite was not working at all. I even deleted the dbs, and they did not get re-created and other issues. Really not sure this use flag should be enabled ever, or allow anyone to use sqlite as the backend for akonadi. Just my take on it, from my experience, and seems others had to switch as well. Now I was using akonadi-server-1.9.0, I see 1.9.1 is out and about to merge that. However given past problems with akonadi, lost emails etc, I am really starting to believe sqlite was the problem the whole time. Being forced to switch to get things working seems to be a good thing.
Comment 1 Johannes Huber (RETIRED) gentoo-dev 2013-04-21 11:01:33 UTC
I dont like the idea to drop a feature as long it is supported by upstream

How about to change the if statement so that the warning about sqlite backend is always showing when sqlite is selected:

> if has_version "<=${CATEGORY}/${PN}-1.9.0[sqlite]"; then
> ...
> big fat warning
> ...

to 

> if use sqlite; then
> ...
> big fat warning
> ...
Comment 2 William L. Thomson Jr. 2013-04-21 13:04:11 UTC
Either way, just had lots of issues with sqlite and thought it was kmail and akonadi, but turns out its just akonadi's back end. But a warning is likely fine if others are not having issues with it.
Comment 3 Maciej Mrozowski gentoo-dev 2013-04-22 23:43:58 UTC
(In reply to comment #2)
> Either way, just had lots of issues with sqlite and thought it was kmail and
> akonadi, but turns out its just akonadi's back end. But a warning is likely
> fine if others are not having issues with it.

Over my 'dead cvs access' :P QSqlite3 work fine here.
Comment 4 William L. Thomson Jr. 2013-04-23 00:02:49 UTC
I don't think I ever tried sqlite3. I never cared initially, I just could not understand the issues I was dealing with. I filed bugs upstream, and really about lost it at times when during a business day I lost several important emails within a few hours of each other. I have had tons of issues since switching from Gnome and Evolution back in October or so of 2011, all kde 4.x stuff with akonadi etc. In fact in the process of switching to mysql, I lost all my contacts, birthdays, etc. I have my old sqlite database, and copy of my entire home directory and that stuff is MIA. I really hate mysql, I run Firebird, maintained it on Gentoo for a while. But I have never had a single issue since I switched to mysql. I didn't even think of that, I noticed the posting in the forum where other users had the same issue and thats where I got the idea to switch.

Glad it works for developers, but when it not for end users, something to take into consideration, coming from a former developer.

BTW talking about upstream, if your going to adhere to them, then mysql should be the default for desktops, according to upstream. Sqlite for embedded and mobile systems, and has limitations and known issues, reciepe for issues there.
http://techbase.kde.org/Projects/PIM/Akonadi/Database

Thus it was a Gentoo decision to switch the default. One that cost me and other users likely dearly. What do I mean?

Oh just losing 20+ work emails in a day
https://bugs.kde.org/show_bug.cgi?id=313697

Or how about losing all email in a folder, deleted from disk not from akonadi db...
https://bugs.kde.org/show_bug.cgi?id=313702

Leave options, warning or not, I would at least change the akonadi backend to adhere to upstream defaults! The clearly exist for a reason, and again I hate mysql so really rather it not be on my system. But if its a matter of having it and not losing emails, I can get over my hate to make a living and pay my bills. :)
Comment 5 Maciej Mrozowski gentoo-dev 2013-04-23 19:12:46 UTC
Right, but what you're after here? To disable sqlite USE flag which (for me for instance) never ceased to work?

> I would at least change the akonadi backend to adhere to upstream defaults! 
For some time, mysql is already preferred backend in Gentoo when both mysql and sqlite are selected.

As for problems with backend switching... 
Akonadi in theory just keeps the cache, no email database, whether local (maildir/mbox) or remote (imap/pop3) should be lost because of backend switching - ever. If it was (bugs you linked suggest it was) - it's generic akonadi problem not <particular backend> problem.
Comment 6 William L. Thomson Jr. 2013-04-25 23:19:18 UTC
You already stated you had no issues with sqlite or sqlite three, despite known issues upstream, enough with that. It clearly causes issues for others, not just me. The others did not file a bug, I did. Guess you missed the discussion in the forum I was not part of in my initial post.

(In reply to comment #5)
> > I would at least change the akonadi backend to adhere to upstream defaults! 
> For some time, mysql is already preferred backend in Gentoo when both mysql
> and sqlite are selected.

Well that doesn't do much for anyone who installed it prior to that change. They will still be using what ever. Understand that I never configured this, just one day after an upgrade things did not work. When I looked into why I came across others posts, which lead me to switching my back end to mysql.

Not sure how an ebuild can pickup on that, maybe checking for existence of users home directories, and a grep to see what backend they are using and maybe some warning if using one other the default.

> As for problems with backend switching... 
> Akonadi in theory just keeps the cache,

In theory just a cache, however I have seen it delete emails entirely from disk, like as mentioned in the bugs I filed to KDE. Which I never would have filed if the backend was always defaulted to mysql, because I would have been running mysql not sqlite. I never chose sqlite, it was just that way after first install/start. Also likely why nepomuk never worked, because it always failed to make it through all my stuff, then would start over, and over, so I never was able to use that. Not that I care much, just wanted auto completion in emails.

Though clearly others have had the same issues, even some using mysql, so you can say it won't delete emails, but it does and has, not just for me. My bug was a duplicate of this one. Lots of people commenting and even some saying stopping akonadi stopped emails from being truncated.
https://bugs.kde.org/show_bug.cgi?id=298962

> no email database, whether local
> (maildir/mbox) or remote (imap/pop3) should be lost because of backend
> switching - ever. If it was (bugs you linked suggest it was) - it's generic
> akonadi problem not <particular backend> problem.

I am glad you feel that way, but reality is otherwise. I have NEVER had those issues since switching to mysql. Never lost a single email or had any issues. Though seem others have in the bug link above. 

Either way I lost all my contacts for good, and I have no idea where that was stored, what happened or anything. I know that was akonadi, I spent time dialing in my contacts initally and all their birth dates for ones that mattered. All gone and no idea where it went to or where akonadi stored that stuff.

This conversation though is really indicative of the present state of Gentoo and attitudes of newer developers. Which again I used to be developer 06-08, bug 135927, might want to listen a bit to those that have been around for many years... Some what wishing I never filed the bug in the first place. Exact reasons why I contribute and do very little with Gentoo these days, sad....
Comment 7 Chris Reffett (RETIRED) gentoo-dev Security 2013-04-27 05:48:22 UTC
mysql has been the default for Gentoo for several months now. I like johu's suggested change, but since we have both mysql and sqlite enabled by default right now (sqlite so that people don't have their backend support disappear without warning during the transition between default backends), it will have to wait until the next version bump when we will presumably change the ebuild to only mysql enabled by default. Either way, however, people are being given fair warning, since anyone who had sqlite support for the previous version of akonadi-server will get a warning, and anyone who doesn't get that warning either already is using mysql/postgres or is installing it fresh and so will get the mysql backend. The sqlite backend, while it has known problems, is a configuration supported by upstream, and as such we will not be removing the option to use it.
Comment 8 William L. Thomson Jr. 2013-04-27 13:14:36 UTC
Once again another new developer, while I can appreciate having both for a transition. Fact is for some those in the forum post, me, and others, sqlite stopped working at all. I did not switch because sqlite was working. It broke entirely, I had no access to my email, or anything. Till I switched back ends. I assume having both will make that not be an issue. But again that you all have not experienced the issue, does not really give confidence that it would be a viable solution, and what will happen for someone when they have both set and sqlite stops working, if it does like it did for me and others. It broke with a kde update.

From forum post no on read

"Few minutes ago I just reemerged a new kde 4.10.0. Unfortunatley, when I am trying to run kmail or kontact when program starts, few seconds after this I have a meassage:The akonadi personal information management service is not operational"
http://forums.gentoo.org/viewtopic-t-950578-start-0.html


Resolved as won't fix is understandable given the comments and stance toward this issue thus far. However this is quite confirmed by many, even upstream, so that aspect of the status I find interesting. This problem is most definitely confirmed, verified and many have run into it, just not some thick Gentoo Developers maintaining the KDE herd.

While upstream does support sqlite, its ONLY for embedded NOT for desktop regular use, much less high volume. Therefore I might recommend, though clearly falling on deaf ears, that in the desktop profile, sqlite is disabled. Upstream is NOT supporting sqlite as the backend for desktops.

Upstream even has FAQs on this

"Why not use sqlite?
We tried. Really. It just can't handle the concurrent access very well.
Please refer to [2] for more information on this subject."

http://techbase.kde.org/Projects/PIM/Akonadi#Why_not_use_sqlite.3F

And again since no one is paying much attention

Sqlite

Status: working with limitations, default backend for mobile systems

Supported modes: Embedded

Known issues:

Requires newer version than the default Qt one.
Requires patched QtSql driver to fix concurrency issues (Akonadi ships a fork of the driver with the necessary changes)
Slower than MySQL (see e.g. http://bertjan.broeksemaatjes.nl/node/53), based on our measurments about 2-5x on the query level, but still in the milliseconds range, so that's probably not a big issue.
Mutex-based transaction serialization in Akonadi (required because SQlite's support of concurrency might not be enough) can block clients while background operations such as a IMAP sync are going on. This is the real killer here, the more clients (including background processes like indexing) are accessing the data, the more blocking you experience. Background http://www.sqlite.org/faq.html#q5 , http://www.somacon.com/p369.php , http://www.sqlite.org/lockingv3.html , http://www.mail-archive.com/sqlite-users%40sqlite.org/msg43873.html


http://techbase.kde.org/Projects/PIM/Akonadi/Database#Sqlite

Do you not see the mention of performance difference with mysql over sqlite. Upstream has made it quite clear, there are issues, mysql performs better, etc. But new generation of Gentoo developers seems to go against upstream and say this stuff is not confirmed, when it is well known.

I will let the issue rest, just once again sorry I even filed a bug report. Now that I can see how the newer generation of Gentoo Developers are maintaining things. Much less interacting with users and former developers. Really not impressed what so ever. Doesn't even make me want to get more involved, just motivates me to want to move away from Gentoo. This is a joke for real! Not surprised that other users ran into this issue or others. Clearly people are doing what ever, and so long as it works for them, that is good enough. Very sad!
Comment 9 Andreas K. Hüttel archtester gentoo-dev 2013-04-27 13:19:26 UTC
Please cut out the &%$/$%/. It's enough by now. Noone of us really wants to involve userrel (luckily devrel is not needed anymore).
Comment 10 William L. Thomson Jr. 2013-04-27 13:31:46 UTC
And another new developer who all three commenting became developers years after I left after stepping down from the Board of Trustees, and an issue with Devrel were THEY drove me away. I left Gentoo by choice cause entites like that are asshats other projects do not have them as I mentioned on my developer bug.

https://bugs.gentoo.org/show_bug.cgi?id=135927

FYI those on develrel and recruiting have prevented me from coming back because they fear I will get rid of those entities and negate their power and control over Gentoo.

Don't even go there, I fear not userrel or devrel or any of them. You all are clearly not aware of Gentoo history that predates you all. Nor any respect for those who came before you. Again I did not have to file a bug. Typically that is encouraged for users to file bugs and help report issues to Developers. Clearly others with this issue were not. I rarely file any bugs these days, and I surely won't be filing any more KDE ones.

Now back to things that MATTER!

http://www.sqlite.org/faq.html#q5

(5) Can multiple applications or multiple instances of the same application access a single database file at the same time?

Multiple processes can have the same database open at the same time. Multiple processes can be doing a SELECT at the same time. But only one process can be making changes to the database at any moment in time, however.

....

However, client/server database engines (such as PostgreSQL, MySQL, or Oracle) usually support a higher level of concurrency and allow multiple processes to be writing to the same database at the same time. This is possible in a client/server database because there is always a single well-controlled server process available to coordinate access. If your application has a need for a lot of concurrency, then you should consider using a client/server database.

....

http://www.sqlite.org/faq.html#q5

So forget KDE, sqlite itself states its own known issues. Even they say if your application has a need for high concurrency then you should consider a client/server database. 

They even referring to it in that above link in italics as embedded database. Which can be used for sure in desktop environments, but its really best for other uses, systems with limited resources, etc. For desktops on modern machines with plenty of power, it just causes problems with heavy usage. Know by both KDE and sqlite, just not some gentoo developers for some reason.

Understand database is what I have done for a living for over a decade. I maintained Firebird on Gentoo for some time. I even have an ebuild for the Firebird ODBC driver I was thinking about contributing to Gentoo but why?

FYI that would close a bug that predates ALL of you from 2005. I doubt you all care about working old bugs like that
https://bugs.gentoo.org/show_bug.cgi?id=81588
Comment 11 William L. Thomson Jr. 2013-04-27 13:45:37 UTC
BTW since all of you are new, and this should likely be something addressed on the quiz. What power do you think Userrel has over users? Its not like Devrel who can suspend developers commit access and other things. Userrel can do what exactly?

Its pretty funny what is mentioned on userrel page, exactly what I previously stated about encouraging users to file bugs.

"Gentoo Linux Development team: Prospective developers are encouraged to become active on bugzilla. The bug reports are monitored by development recruiters, so start squashing bugs and you will become noticed."
http://www.gentoo.org/proj/en/userrel/

FYI Even devrels actions are not good, most any developer they have interacted with they drive away, which spawned thinks like exherbo, that predates all of you as well. Probably never worked with those on the exherbo project when they were all gentoo developers, or having seen them leave or why. Or even Funtoo, you could credit devrel for creating both projects Exherbo and Funtoo. Those forks would not exist if it wasn't for devrel.

Constructive feedback

You all might look to encourage others to help out and contribute, valuing everyone's comments, opinions, feedback, and experiences. Rather than being close minded, and threatening. That will get you no where, does little, and just will not help Gentoo as a whole in the long run. As the newer generation of Gentoo developers. I hope you take that into consideration and take action to get Gentoo back on the right track. It has not been for years. The amount of packages is increasing year after year, while active developer numbers have been flat. Gentoo adds as many as it loses, and rarely moves ahead in active developer count. Many have done factual analysis of such over the years. Hopefully someday that will be taken into consideration and changes made to get Gentoo growing and thriving again.

In the mean time seems things have gotten worse, since after this I will definitely not be filing any other bugs or contributing. Gentoo has been putting me off for many years, and I have not suffered, but Gentoo has. To bad many do not realize that, but even more do, they just lack ability to bring about change, and/or do not feel like pissing in the wind as I did before.
Comment 12 Andreas K. Hüttel archtester gentoo-dev 2013-04-27 20:11:00 UTC
(In reply to comment #10)
> And another new developer ...
[snip]

Just for your information, I've been a developer considerably longer than you managed to stay on before getting kicked out.
Comment 13 William L. Thomson Jr. 2013-04-27 20:18:07 UTC
You fool I left... I was NEVER kicked out... I own a business I did not have time for stupidity, which clearly remains.
Comment 14 Johannes Huber (RETIRED) gentoo-dev 2013-04-27 20:22:53 UTC
(In reply to comment #12)
> (In reply to comment #10)
> > And another new developer ...
> [snip]
> 
> Just for your information, I've been a developer considerably longer than
> you managed to stay on before getting kicked out.
Comment 15 Maciej Mrozowski gentoo-dev 2013-04-29 21:07:13 UTC
(In reply to comment #6)
> This conversation though is really indicative of the present state of Gentoo
> and attitudes of newer developers.

Not sure who you're hitting at, I'm involved in Gentoo KDE since 2008 for the record..

Please keep it civil, gentlemen.
Comment 16 William L. Thomson Jr. 2013-04-29 21:21:02 UTC
(In reply to comment #15)
> (In reply to comment #6)
> > This conversation though is really indicative of the present state of Gentoo
> > and attitudes of newer developers.
> 
> Not sure who you're hitting at, I'm involved in Gentoo KDE since 2008 for
> the record..

You did not become a developer till 2010, bug 272106, years after I left for a variety of reasons. I have been involved since late 2005, maintaining packages such as Tomcat for months prior to becoming a developer in 2006. I have been running Gentoo for my business since 2003. A developer and fellow business owner who predated me first turned me on to Gentoo, and they were gone by 2003.

> Please keep it civil, gentlemen.

I have tried to keep it technical, not referring to entities within Gentoo that have no power over the matter, as some sort of threat. Rather childish behavior and does nothing to address the overall issue.

Anyway bug has been closed, I have chosen not to reopen, and likely will not be filling or commenting on others. Its amazing how I keep getting put off by people in Gentoo year after year... very sad...