Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 413295 - www-client/chromium: add a way to periodically vacuum sqlite databases
Summary: www-client/chromium: add a way to periodically vacuum sqlite databases
Status: RESOLVED WONTFIX
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal enhancement (vote)
Assignee: Chromium Project
URL: https://groups.google.com/a/chromium....
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-04-23 19:51 UTC by Pacho Ramos
Modified: 2012-12-16 19:16 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Pacho Ramos gentoo-dev 2012-04-23 19:51:26 UTC
Hello

This is a problem I see on a lot of apps relying on sqlite (like firefox, evolution...). When I am using them for some months, they start to take more and more time to startup because their databases looks to not auto-vacuum (not sure if this is a bug or desired effect).

Now, I have run the following:

cd ~/.config/chromium/Default
find -type f -exec file '{}' \; | perl -ne 'print "$1\n" if /(.*?): SQLite/' | while read fname; do
	sqlite3 "${fname}" VACUUM;
        sqlite3 "${fname}" REINDEX;
done

And it starts again much faster :)

The problem is that I am sure I will need to manually rerun this in some weeks. I thought that maybe a more general script could be placed in cron.monthly to run vacuum on all user accounts if chromium process is not running.

What do you think?

Reproducible: Always
Comment 1 Mike Gilbert gentoo-dev 2012-04-23 20:23:22 UTC
There are a couple of open bug reports upstream that mention vacuuming. These have been largely ignored by the chromium developers.

http://code.google.com/p/chromium/issues/list?q=sqlite+vacuum

My 2 cents: I don't really think a cron job is a good idea, but maybe we could document your workaround somewhere.
Comment 2 Pacho Ramos gentoo-dev 2012-04-23 20:29:07 UTC
The problem is that that workarounds are around the web for firefox, evolution, chromium... and then, it makes this a bit annoying as I need to manually run them for all machines I maintain :(
Comment 3 Paweł Hajdan, Jr. (RETIRED) gentoo-dev 2012-04-27 06:22:14 UTC
(In reply to comment #0)
> cd ~/.config/chromium/Default
> find -type f -exec file '{}' \; | perl -ne 'print "$1\n" if /(.*?): SQLite/'
> | while read fname; do
> 	sqlite3 "${fname}" VACUUM;
>         sqlite3 "${fname}" REINDEX;
> done

The discussion on chromium-dev indicates it's unsafe to run this automatically. However, I'd be fine with including such a script in the www-client/chromium package (so it could be run manually).
 
Does that sound good to you?
Comment 4 Pacho Ramos gentoo-dev 2012-04-27 08:41:01 UTC
I am fine with it, but please wait a bit more as maybe we would be able to make a more complex script allowing it to detect if chromium is running and so. I am still around in upstream mailing list but won't probably be able to reply today as I don't have much time now :(

Thanks a lot for your help on this :)
Comment 5 Paweł Hajdan, Jr. (RETIRED) gentoo-dev 2012-04-27 09:15:31 UTC
(In reply to comment #4)
> I am fine with it, but please wait a bit more as maybe we would be able to
> make a more complex script allowing it to detect if chromium is running and
> so.

I'm skeptical about that, and the most serious problem are race conditions.

For manually running the script, such detection is not very helpful or needed anyway. The user knows whether chromium is running or not, and knows when the script finishes running.
Comment 6 Pacho Ramos gentoo-dev 2012-05-07 17:39:14 UTC
OK, thanks for adding the script then :) (it should probably be enhanced a bit to show a warning to the users asking them to kill chromium before running)
Comment 7 Paweł Hajdan, Jr. (RETIRED) gentoo-dev 2012-06-18 07:38:43 UTC
Pushed to http://git.overlays.gentoo.org/gitweb/?p=proj/chromium-tools.git;a=summary
Comment 8 Pacho Ramos gentoo-dev 2012-06-18 20:09:09 UTC
Nice, thanks :)

I see it was moved to a "python version" ;)
Comment 9 Pacho Ramos gentoo-dev 2012-07-21 08:55:21 UTC
(In reply to comment #7)
> Pushed to
> http://git.overlays.gentoo.org/gitweb/?p=proj/chromium-tools.git;a=summary

How should people install that chromium-tools? Looks like they are not supplied in main tree, or I am looking in wrong place?

Thanks for the info
Comment 10 Paweł Hajdan, Jr. (RETIRED) gentoo-dev 2012-07-31 13:07:39 UTC
(In reply to comment #9)
> (In reply to comment #7)
> > Pushed to
> > http://git.overlays.gentoo.org/gitweb/?p=proj/chromium-tools.git;a=summary
> 
> How should people install that chromium-tools? Looks like they are not
> supplied in main tree, or I am looking in wrong place?

Just clone the git repo. Suggestions how to handle this better are welcome (off-bugzilla if possible), but I'm not enthusiastic about creating a package just to install one script.
Comment 11 Richard Freeman gentoo-dev 2012-07-31 13:23:43 UTC
I gave up on using the script in the overlay - I think it was missing some python dependencies/etc and seemed like overkill.  I just stuck the following in a script:


cd ~/.config/chromium/Default
find -type f -exec file '{}' \; | perl -ne 'print "$1\n" if /(.*?): SQLite/' | while read fname; do echo "sqlite3 \"${fname}\" VACUUM ; sqlite3 \"${fname}\" REINDEX" ; done | parallel

(obviously based on previous contributions in this bug, and it does require gnu parallel)

Probably wouldn't hurt to place suggestions on the wiki or a blog/etc.  While a script in the tree would be ideal, all of this is a bit of a nice-to-have, and it is dangerous to run without care.
Comment 12 Pacho Ramos gentoo-dev 2012-08-01 08:42:15 UTC
(In reply to comment #10)
> (In reply to comment #9)
> > (In reply to comment #7)
> > > Pushed to
> > > http://git.overlays.gentoo.org/gitweb/?p=proj/chromium-tools.git;a=summary
> > 
> > How should people install that chromium-tools? Looks like they are not
> > supplied in main tree, or I am looking in wrong place?
> 
> Just clone the git repo. Suggestions how to handle this better are welcome
> (off-bugzilla if possible), but I'm not enthusiastic about creating a
> package just to install one script.

Why not install it under /usr/share/doc? (we do a similar thing to install script for converting xscreensaver screensavers to gnome-screensaver files)
Comment 13 Paweł Hajdan, Jr. (RETIRED) gentoo-dev 2012-08-01 13:45:54 UTC
(In reply to comment #12)
> (In reply to comment #10)
> > (In reply to comment #9)
> > > (In reply to comment #7)
> > > > Pushed to
> > > > http://git.overlays.gentoo.org/gitweb/?p=proj/chromium-tools.git;a=summary
> > > 
> > > How should people install that chromium-tools? Looks like they are not
> > > supplied in main tree, or I am looking in wrong place?
> > 
> > Just clone the git repo. Suggestions how to handle this better are welcome
> > (off-bugzilla if possible), but I'm not enthusiastic about creating a
> > package just to install one script.
> 
> Why not install it under /usr/share/doc? (we do a similar thing to install
> script for converting xscreensaver screensavers to gnome-screensaver files)

The script applies to both chromium and google-chrome. /usr/share/doc location itself is fine, the issue is more about how to handle dependencies and sharing between chromium and google-chrome (and also versioning of the script) while avoiding unnecessary duplication.
Comment 14 Pacho Ramos gentoo-dev 2012-08-01 13:55:14 UTC
(In reply to comment #13)
> (In reply to comment #12)
> > (In reply to comment #10)
> > > (In reply to comment #9)
> > > > (In reply to comment #7)
> > > > > Pushed to
> > > > > http://git.overlays.gentoo.org/gitweb/?p=proj/chromium-tools.git;a=summary
> > > > 
> > > > How should people install that chromium-tools? Looks like they are not
> > > > supplied in main tree, or I am looking in wrong place?
> > > 
> > > Just clone the git repo. Suggestions how to handle this better are welcome
> > > (off-bugzilla if possible), but I'm not enthusiastic about creating a
> > > package just to install one script.
> > 
> > Why not install it under /usr/share/doc? (we do a similar thing to install
> > script for converting xscreensaver screensavers to gnome-screensaver files)
> 
> The script applies to both chromium and google-chrome. /usr/share/doc
> location itself is fine, the issue is more about how to handle dependencies
> and sharing between chromium and google-chrome (and also versioning of the
> script) while avoiding unnecessary duplication.

It is still a big improvement in "accessibility" to install it in /usr/share/doc over needing to figure out that I need to clone that git repo that is mostly unknown for userbase. 

Also, current script looks to work over .chromium, if I don't misremember google-chrome used a different config dir under /home :/
Comment 15 Pacho Ramos gentoo-dev 2012-12-16 18:49:35 UTC
Other option would be to add an ebuild for installing that chromium-tools package... because currently this script is completely hidden to administrators
Comment 16 Mike Gilbert gentoo-dev 2012-12-16 19:16:32 UTC
I think this whole idea is hacky and should not be in the tree at all.