Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 304913 - www-client/mozilla-firefox-3.6-r2: Please make dev-db/sqlite[secure-delete] dependency optional
Summary: www-client/mozilla-firefox-3.6-r2: Please make dev-db/sqlite[secure-delete] d...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Mozilla Gentoo Team
URL: https://bugzilla.mozilla.org/show_bug...
Whiteboard:
Keywords:
: 304949 (view as bug list)
Depends on:
Blocks:
 
Reported: 2010-02-13 17:46 UTC by Alec Meyers
Modified: 2011-06-20 14:49 UTC (History)
16 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
proposed patch for sqlite vs. system-sqlite (mozilla-system-sqlite.patch,3.74 KB, text/plain)
2010-02-13 22:01 UTC, Jory A. Pratt
Details
alternative proposal (for xulrunner) (xulrunner-safe-delete.patch,739 bytes, patch)
2010-02-14 00:32 UTC, Alec Meyers
Details | Diff
firefox with conditional secure-delete use (mozilla-firefox-3.6-r3.ebuild,8.26 KB, text/plain)
2010-02-14 02:43 UTC, Jason A. Donenfeld
Details
xulrunner with conditional secure-delete use (xulrunner-1.9.2-r3.ebuild,6.59 KB, text/plain)
2010-02-14 02:44 UTC, Jason A. Donenfeld
Details
Official Fix to go into tree (xulrunner-fix-system-sqlite.patch,1.28 KB, text/plain)
2010-02-14 18:25 UTC, Jory A. Pratt
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Alec Meyers 2010-02-13 17:46:45 UTC
With version 3.6-r2, mozilla-firefox requires sqlite to be built with the secure-delete flag. The purpose of it is so that when the data in Firefox's sqlite databases (history, cookies, etc) is cleared, no trace is left.

However, because it zeroes out the data on every delete, this may be undesirable, especially considering that this is the system sqlite and it affect all other applications that use it.

Please add a secure-delete flag to firefox to trigger the sqlite dependency so that those of us who are not paranoid about our browsing history do not need to suffer the performance loss.

Reproducible: Always

Steps to Reproduce:




Here is the description from http://www.sqlite.org/compile.html:

SQLITE_SECURE_DELETE
 This compile-time option causes SQLite to overwrite deleted information with zeros in addition to marking the space as available for reuse. Without this option, deleted data might be recoverable from a database using a binary editor. However, there is a performance penalty for using this option.
 This option does not cause deleted data is securely removed from the underlying storage media.
Comment 1 Jason A. Donenfeld gentoo-dev 2010-02-13 19:36:50 UTC
+1
Comment 2 Samuli Suominen (RETIRED) gentoo-dev 2010-02-13 20:21:58 UTC
*** Bug 304949 has been marked as a duplicate of this bug. ***
Comment 3 Jory A. Pratt gentoo-dev 2010-02-13 21:12:30 UTC
Please do not cc me on any bug report, I will pick this up via mozilla alias when it is assigned. Also if I make a useflag for sqlite you will still end up with secure-delete being enabled in mozilla bundled sqlite.
Comment 4 Patrizio Bassi 2010-02-13 21:24:48 UTC
totally true, but it will be the ff bundled only and not the system wide one.

can't we let the user choose the needed security level?
i mean, maybe on my personal laptop i need this, but not on my workstation @ home.

can we allow ff to use system wide, "unsafe" delete?
Comment 5 Jory A. Pratt gentoo-dev 2010-02-13 21:43:55 UTC
(In reply to comment #4)
> totally true, but it will be the ff bundled only and not the system wide one.
> 
> can't we let the user choose the needed security level?
> i mean, maybe on my personal laptop i need this, but not on my workstation @
> home.
> 
> can we allow ff to use system wide, "unsafe" delete?
> 
This would be more hassel then it is worth, we are either gonna make the dep match that of sqlite requirements set by mozilla or allow the user to use mozilla's bundled sqlite. Either way we gonna have the same functionality the upstream has intended for its products.
Comment 6 Patrizio Bassi 2010-02-13 21:51:06 UTC
ok, generally gentoo is intended for particular customization and patching.
anyhow, if your effort estimation says it doesn't worth, i would propose the change in ff ebuild:

+ secure-delete -> add dep on sqlite with +secure-delete use
- secure-delete -> link against bundled version

does it sound feasable?
Comment 7 Jory A. Pratt gentoo-dev 2010-02-13 22:01:34 UTC
Created attachment 219527 [details]
proposed patch for sqlite vs. system-sqlite

This is what I would be willing to compromise at.
Comment 8 Patrizio Bassi 2010-02-13 22:18:09 UTC
that's good :)
even if i would reuse the sqlite flag but it's up to you
Comment 9 Jory A. Pratt gentoo-dev 2010-02-13 22:24:01 UTC
sqlite - Adds support for sqlite - embedded sql database
sqlite3 - Adds support for sqlite3 - embedded sql database

We are not adding support, we are enabling/disabling needs to be a clear difference what we are doing to not confuse the users. Either way sqlite is built weather it be against system sqlite or bundled sqlite.
Comment 10 Patrizio Bassi 2010-02-13 22:25:10 UTC
all right, ACK for me.
Comment 11 Jory A. Pratt gentoo-dev 2010-02-13 23:56:21 UTC
For those of you not wanting to wait for this to land in tree, which will be a while, you can test from the mozilla overlay.

http://git.overlays.gentoo.org/gitweb/?p=proj/mozilla.git;a=summary
Comment 12 Alec Meyers 2010-02-14 00:20:08 UTC
(In reply to comment #5)
> (In reply to comment #4)
> > ...
> This would be more hassel then it is worth, we are either gonna make the dep
> match that of sqlite requirements set by mozilla or allow the user to use
> mozilla's bundled sqlite. Either way we gonna have the same functionality the
> upstream has intended for its products.
> 

While I don't really mind using a bundled sqlite in Firefox, I really don't see how that would be less of a hassle.

If, for example, secure-delete was required by default, unless a flag is unset, you would have just one sqlite configuration to worry about instead of two (secure-delete does not change the API).

Considering that we did not have secure-delete for the past 2 years (since Firefox 3.0 was released), I don't see why you suddenly want to enforce it now.
Comment 13 Alec Meyers 2010-02-14 00:32:17 UTC
Created attachment 219547 [details, diff]
alternative proposal (for xulrunner)
Comment 14 Jason A. Donenfeld gentoo-dev 2010-02-14 01:58:06 UTC
I am in favor of Alex M's patch. Relying on bundled sqlite doesn't solve the problem. The point is that I do not want the performance degradation (and especially not the extra writes, as I have a SSD!), and relying on bundled sqlite only introduces extra [security] problems that are typical of bundled packages. 

So instead of complicating this bug report with a discussion of "system or bundled", let's assume that's already been decided as "system", typical of gentoo practices, and turn instead the question to whether or not secure-delete should be required by firefox.

I say no, but that perhaps it should be enabled by default, so Alex M's patch makes a lot of sense.
Comment 15 Jason A. Donenfeld gentoo-dev 2010-02-14 02:43:35 UTC
Created attachment 219563 [details]
firefox with conditional secure-delete use

could make this +secure-delete in the iuse instead, I suppose
Comment 16 Jason A. Donenfeld gentoo-dev 2010-02-14 02:44:08 UTC
Created attachment 219565 [details]
xulrunner with conditional secure-delete use

could make this +secure-delete in the iuse instead, I suppose
Comment 17 Jory A. Pratt gentoo-dev 2010-02-14 03:45:14 UTC
(In reply to comment #15)
> Created an attachment (id=219563) [details]
> firefox with conditional secure-delete use
> 
> could make this +secure-delete in the iuse instead, I suppose
> 

You are all not understanding, we have done put the checks in place upstream to check for secure-delete and fts3 support in sqlite, it is no longer an option it is required. 

Maybe this will help put it into perspective for you all.

MOZ_NATIVE_SQLITE=
MOZ_ARG_ENABLE_BOOL(system-sqlite,
[  --enable-system-sqlite  Use system sqlite (located with pkgconfig)],
MOZ_NATIVE_SQLITE=1,
MOZ_NATIVE_SQLITE= )

if test -z "$MOZ_NATIVE_SQLITE"
then
    SQLITE_CFLAGS=
    SQLITE_LIBS='$(call EXPAND_LIBNAME_PATH,mozsqlite3,$(DIST)/lib)'
else
    PKG_CHECK_MODULES(SQLITE, sqlite3 >= $SQLITE_VERSION)
    dnl ===================================
    dnl === SQLITE_SECURE_DELETE checks ===
    dnl ===================================
    dnl check to see if the system sqlite package is compiled with
    dnl SQLITE_SECURE_DELETE enabled.
    AC_MSG_CHECKING(for SQLITE_SECURE_DELETE support in system sqlite)
    _SAVE_CFLAGS="$CFLAGS"
    CFLAGS="$CFLAGS $SQLITE_CFLAGS"
    _SAVE_LIBS="$LIBS"
    LIBS="$LIBS $SQLITE_LIBS"
    AC_CACHE_VAL(ac_cv_sqlite_secure_delete,[
        AC_TRY_RUN([
            #include "sqlite3.h"
            #include <stdio.h>
            #include <assert.h>

            int main(int argc, char **argv){
              sqlite3 *db;
              sqlite3_uint64 r;
              char *zFilename;
              FILE *in;
              int i;
              int rc;
              char *zBuf;
              const int bufSize = 1024*3;

              zBuf = malloc(bufSize*sizeof(char));
              assert( zBuf );
              rc = sqlite3_open(":memory:", &db);
              assert( rc==SQLITE_OK );
              sqlite3_close(db);
              sqlite3_randomness(sizeof(r), &r);
              zFilename = sqlite3_mprintf("test_db_%llu.sqlite", r);
              rc = sqlite3_open(zFilename, &db);
              assert( rc==SQLITE_OK );
              rc = sqlite3_exec(db, 
                "BEGIN;"
                "CREATE TABLE t1(x);"
                "INSERT INTO t1 VALUES(zeroblob(1000)||'abcdefghijklmnopqrstuvwxyz');"
                "COMMIT;"
                "DELETE FROM t1;",
                0, 0, 0
              );
              assert( rc==SQLITE_OK );
              sqlite3_close(db);
              in = fopen(zFilename, "r");
              assert( in!=0 );
              rc = fread(zBuf, 1, bufSize, in);
              assert( rc==bufSize );
              fclose(in);
              unlink(zFilename);
              free( zBuf );
              for(i=0; i<bufSize-11; i++){
                if( *(zBuf+i)=='h' && memcmp(zBuf+i, "hijklmnopq", 10)==0 ){
                  return 1;
                }
              }
              return 0;
            }],
            ac_cv_sqlite_secure_delete=yes,
            ac_cv_sqlite_secure_delete=no,
            ac_cv_sqlite_secure_delete=no
        )
    ])
    AC_MSG_RESULT($ac_cv_sqlite_secure_delete)
    CFLAGS="$_SAVE_CFLAGS"
    LIBS="$_SAVE_LIBS"
    if test "x$ac_cv_sqlite_secure_delete" = "xno"; then
        AC_MSG_ERROR([System Sqlite library is not compiled with SQLITE_SECURE_DELETE.])
    fi
fi

AC_SUBST(MOZ_NATIVE_SQLITE)
Comment 18 Richard 2010-02-14 05:03:26 UTC
If it was required, why did Firefox 3.6-r1 lack this requirement? Is it really necessary to force secure deletions to be done for all software running on the system, firefox or not?
Comment 19 Jason A. Donenfeld gentoo-dev 2010-02-14 05:41:59 UTC
(In reply to comment #17)

Makes sense - okay - so now upstream requires secure-delete. I still don't like the idea of using bundled libs -- it's very much against the spirit of Gentoo. What about a patch to remove the autoconf requirement if secure-delete is not set?
Comment 20 Jory A. Pratt gentoo-dev 2010-02-14 05:54:24 UTC
(In reply to comment #19)
> (In reply to comment #17)
> 
> Makes sense - okay - so now upstream requires secure-delete. I still don't like
> the idea of using bundled libs -- it's very much against the spirit of Gentoo.
> What about a patch to remove the autoconf requirement if secure-delete is not
> set?
> 

We are no longer applying patches that are not upstream already, the only exception is build fixes specific to gentoo. I am not gonna budge on this requirement. If you want a firefox that does not require you to use secure-delete in firefox, you will need to do so locally. I really do not see what you all are complaining about, -r1 did not use system-sqlite it used bundled sqlite, which in return used secure-delete. Please let this die down, the discussion is done, there is nothing anyone is gonna be able to do to change my mind, if you want upstream to change the requirement on secure-delete then you will need to speak with them.

Calchan, I prefer you follow this bug so it does not get out of hand please.
Comment 21 Richard 2010-02-14 16:12:25 UTC
(In reply to comment #20)
> (In reply to comment #19)
> > (In reply to comment #17)
> > 
> > Makes sense - okay - so now upstream requires secure-delete. I still don't like
> > the idea of using bundled libs -- it's very much against the spirit of Gentoo.
> > What about a patch to remove the autoconf requirement if secure-delete is not
> > set?
> > 
> 
> We are no longer applying patches that are not upstream already, the only
> exception is build fixes specific to gentoo. I am not gonna budge on this
> requirement. If you want a firefox that does not require you to use
> secure-delete in firefox, you will need to do so locally. I really do not see
> what you all are complaining about, -r1 did not use system-sqlite it used
> bundled sqlite, which in return used secure-delete. Please let this die down,
> the discussion is done, there is nothing anyone is gonna be able to do to
> change my mind, if you want upstream to change the requirement on secure-delete
> then you will need to speak with them.
> 
> Calchan, I prefer you follow this bug so it does not get out of hand please.
> 

You do not understand the implications of having secure-delete applied to the entire system because of firefox. With Firefox doing secure deletes in its own bundled SQLite, all other applications on the system using SQLite are free to use it without the disadvantage of secure delete. On systems where Firefox is a small percentage of SQLite usage, this is a significant difference.

Your change to the Firefox package has implications far beyond that of firefox and affects the system as a whole. There needs to be an option to revert back to the old functionality where the bundled SQLite is used so that the secure deletes are only performed for Firefox.
Comment 22 Jory A. Pratt gentoo-dev 2010-02-14 16:22:17 UTC
(In reply to comment #21)
> (In reply to comment #20)
> > (In reply to comment #19)
> > > (In reply to comment #17)
> > > 
> > > Makes sense - okay - so now upstream requires secure-delete. I still don't like
> > > the idea of using bundled libs -- it's very much against the spirit of Gentoo.
> > > What about a patch to remove the autoconf requirement if secure-delete is not
> > > set?
> > > 
> > 
> > We are no longer applying patches that are not upstream already, the only
> > exception is build fixes specific to gentoo. I am not gonna budge on this
> > requirement. If you want a firefox that does not require you to use
> > secure-delete in firefox, you will need to do so locally. I really do not see
> > what you all are complaining about, -r1 did not use system-sqlite it used
> > bundled sqlite, which in return used secure-delete. Please let this die down,
> > the discussion is done, there is nothing anyone is gonna be able to do to
> > change my mind, if you want upstream to change the requirement on secure-delete
> > then you will need to speak with them.
> > 
> > Calchan, I prefer you follow this bug so it does not get out of hand please.
> > 
> 
> You do not understand the implications of having secure-delete applied to the
> entire system because of firefox. With Firefox doing secure deletes in its own
> bundled SQLite, all other applications on the system using SQLite are free to
> use it without the disadvantage of secure delete. On systems where Firefox is a
> small percentage of SQLite usage, this is a significant difference.
> 
> Your change to the Firefox package has implications far beyond that of firefox
> and affects the system as a whole. There needs to be an option to revert back
> to the old functionality where the bundled SQLite is used so that the secure
> deletes are only performed for Firefox.
> 

Richard, I am asking you to settle down and show some respect, I done said I would make a system-sqlite useflag, if enabled system-sqlite will be used, if not the bundled copy from upstream will be used. Either way you will end up with secure-delete functionality in your mozilla products. Now As far as I am concerned the bug is being addressed. If you do not agree with how it is being addressed, I am sorry. I will not take and remove the support the mozilla has put in place for its browsers/mail client to ensure security for the user. One last time there will be an option to disable system-sqlite and use the bundled sqlite for mozilla products alone.
Comment 23 Alec Meyers 2010-02-14 16:52:39 UTC
Jory, thanks for the explanation. It's definitely an upstream issue, then.

To anyone with SSDs - if you file a bug upstream, please put a link here so that those interested could follow it.
Comment 24 Jory A. Pratt gentoo-dev 2010-02-14 18:25:42 UTC
Created attachment 219673 [details]
Official Fix to go into tree

This is just a bandaid to the problem, I am not happy with this but our policy says systems libs are to be used. If you do not want secure-delete when I get this commited to the tree you will simply add -private to your useflags. and read the warning.
Comment 25 Jason A. Donenfeld gentoo-dev 2010-02-14 22:06:23 UTC
https://bugzilla.mozilla.org/show_bug.cgi?id=546162
Upstream report ^

Still though, if Mozilla does not budge, I vote that we patch the build system. It's a build system patch, not a patch to the actual source of FF, so it's no big deal. Besides, isn't that what we do as Gentooers? patch build systems to meet our high standards of customizability?
Comment 26 Robert Golding 2010-02-15 12:13:17 UTC
All I know is that I updated from 3.6-r1 to 3.6-r2 and now I cannot start FF with this error message;
---->
SQLite Version Error
The application has been updated, but your version of SQLite is too old and the application cannot run.
---->
Comment 27 Jason A. Donenfeld gentoo-dev 2010-02-15 22:08:41 UTC
Upstream won't fix it: https://bugzilla.mozilla.org/show_bug.cgi?id=546162#c3

So I say, being Gentooers, we patch the build system. It's a simple maintainable patch. This gives us the use-flag flexibility we value so much.
Comment 28 coran.fisher@gmail.com 2010-02-18 11:46:28 UTC
After upgrading I get the same message as comment #26.  This is even after recompiling sqlite with the securedelete flag.  I am forced to do a USE="-sqlite" to be able to use firefox 3.6.  This is also true with the ebuild in the mozilla overlay 3.6-r3
Comment 29 Jory A. Pratt gentoo-dev 2010-02-18 14:37:28 UTC
(In reply to comment #26)
> All I know is that I updated from 3.6-r1 to 3.6-r2 and now I cannot start FF
> with this error message;
> ---->
> SQLite Version Error
> The application has been updated, but your version of SQLite is too old and the
> application cannot run.
> ---->
> 

This is a portage bug, you can remove the libsqlite3.so file from xulrunner/firefox install dir and have a proper working system. This issue only effects those on portage 2.2_rcX
Comment 30 Robert Golding 2010-02-19 00:22:48 UTC
(In reply to comment #29)
> (In reply to comment #26)
> > All I know is that I updated from 3.6-r1 to 3.6-r2 and now I cannot start FF
> > with this error message;
> > ---->
> > SQLite Version Error
> > The application has been updated, but your version of SQLite is too old and the
> > application cannot run.
> > ---->
> > 
> 
> This is a portage bug, you can remove the libsqlite3.so file from
> xulrunner/firefox install dir and have a proper working system. This issue only
> effects those on portage 2.2_rcX
> 

Thankyou, That fixed it for this system.

I really didn't mind as I use Chromium, but a couple of other users prefer FF and don't like Chromium.
Comment 31 redneb 2010-02-22 07:09:23 UTC
I think the ideal solution would be if sqlite made the option to zero out deleted data a run-time option (ie a PRAGMA) instead of compile-time option. This way firefox could use the system sqlite without imposing the secure delete to all other programs. Actually, someone has already proposed that ([1]) to the sqlite-dev mailing list a few days ago but got no response.

[1] http://osdir.com/ml/sqlite-dev/2010-02/msg00016.html
Comment 32 Jory A. Pratt gentoo-dev 2010-03-04 14:45:08 UTC
we went with a system-sqlite useflag, if you do not want to have a sqlite build with secure delete globally you can leave it disabled and it will use sqlite bundled with package.
Comment 33 Jason A. Donenfeld gentoo-dev 2010-03-14 01:52:21 UTC
How does
http://www.sqlite.org/pragma.html#pragma_secure_delete
affect things?
Comment 34 redneb 2010-03-15 04:24:24 UTC
(In reply to comment #33)
> How does
> http://www.sqlite.org/pragma.html#pragma_secure_delete
> affect things?
> 

This is great. Now firefox should not care about the SQLITE_SECURE_DELETE  compile-time option. Instead it should issue a "PRAGMA secure_delete=1;" every time it connects to a sqlite database. This way we will be able to use the system's sqlite for firefox without imposing secure delete to all apps that use sqlite. The SSD people will still be unhappy.
Comment 35 Jason A. Donenfeld gentoo-dev 2010-03-16 07:39:17 UTC
(In reply to comment #34)
> 
> This is great. Now firefox should not care about the SQLITE_SECURE_DELETE 
> compile-time option. Instead it should issue a "PRAGMA secure_delete=1;" every
> time it connects to a sqlite database. This way we will be able to use the
> system's sqlite for firefox without imposing secure delete to all apps that use
> sqlite. The SSD people will still be unhappy.
> 

I wish, but it looks like sdwilsh is being difficult -- he kind of refuses to budge. Head on over to https://bugzilla.mozilla.org/show_bug.cgi?id=546162 to voice your support.
Comment 36 Nikoli 2011-06-20 14:34:31 UTC
Patch is now available in https://bugzilla.mozilla.org/show_bug.cgi?id=546162#c16
Comment 37 Jory A. Pratt gentoo-dev 2011-06-20 14:49:52 UTC
(In reply to comment #36)
> Patch is now available in
> https://bugzilla.mozilla.org/show_bug.cgi?id=546162#c16

This will never make it into upstream, it is a complete hack, when someone comes up with a decent patch and post it that makes it threw review will add to gentoo.