Bug 630698 – dev-db/sqlite-3.20.1: csv01.test hangs

• Status: RESOLVED FIXED
• Product: Gentoo Linux

[Description Matt Turner 2017-09-11 16:32:53 UTC]

I let the tests run overnight in a ppc32 and a ppc64 chroot. Both made no progress after cse.test. I believe I have seen this behavior on other architectures as well (but not at least amd64).

[Comment 1 Sergei Trofimovich (RETIRED) 2017-09-13 21:29:28 UTC]

Poked at it a bit today. It's hanging in csv01.test

By adding print statements it hangs on first test:

"""
  CREATE VIRTUAL TABLE temp.t1 USING csv(
    data=
'1,2,3,4
5,6,7,8
9,10,11,12
13,14,15,16
',
    columns=4
  );
  SELECT * FROM t1 WHERE c1=10;
"""

Is there an easy way to perform the same query on sqlite3 binary?
When i paste the command I get the error:

work/sqlite-src-3200100-abi_ppc_32.ppc $ ./sqlite3
SQLite version 3.20.1 2017-08-24 16:21:36
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite>   CREATE VIRTUAL TABLE temp.t1 USING csv(
   ...>     data=
   ...> '1,2,3,4
   ...> 5,6,7,8
   ...> 9,10,11,12
   ...> 13,14,15,16
   ...> ',
   ...>     columns=4
   ...>   );
Error: no such module: csv
sqlite>   SELECT * FROM t1 WHERE c1=10;
Error: no such table: t1

[Comment 2 Sergei Trofimovich (RETIRED) 2017-09-16 16:36:32 UTC]

Found the bug and a workaround \o/.
powerpc's 'char' is 'unsigned char' by default (while most other platforms are 'signed char'.

That makes buggy code like:
    '(char)c == EOF'
sort of work on later platforms, but not on former ones.

The following hack makes tests not hang on powerpc:

diff --git a/ext/misc/csv.c b/ext/misc/csv.c
index 6d99634..34649c5 100644
--- a/ext/misc/csv.c
+++ b/ext/misc/csv.c
@@ -688,3 +688,3 @@ static int csvtabNext(sqlite3_vtab_cursor *cur){
   }
-  if( z==0 || pCur->rdr.cTerm==EOF ){
+  if( z==0 || pCur->rdr.cTerm==(char)EOF ){
     pCur->iRowid = -1;

[Comment 3 Sergei Trofimovich (RETIRED) 2017-09-16 16:44:36 UTC]

A way to reliably reproduce the hangup on amd64 and athers:
    CFLAGS="-O2 -funsigned-char" FEATURE=test emerge -v1 =dev-db/sqlite-3.20.1

[Comment 4 Sergei Trofimovich (RETIRED) 2017-09-16 17:03:49 UTC]

Created attachment 494762 [details, diff]
sqlite-3.20.1-funsigned-char.patch

[Comment 5 Sergei Trofimovich (RETIRED) 2017-09-16 17:15:23 UTC]

Sent patch to http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users with subject "[PATCH] Fix csv handling hangup on -funsigned-char platforms"

Unfortunately it has no public read-only access.

[Comment 6 Arfrever Frehtes Taifersar Arahesis 2017-09-19 03:36:20 UTC]

https://sqlite.org/src/info/42f0777555675875

Does this commit fix problem?

[Comment 7 Sergei Trofimovich (RETIRED) 2017-09-19 07:52:53 UTC]

This patch works as well. A few date-related tests still fail
but they were failing with my hack as well:

! date-2.2c-1 expected: [06:28:00.001]
! date-2.2c-1 got:      [06:28:00.000]
! date-2.2c-4 expected: [06:28:00.004]
! date-2.2c-4 got:      [06:28:00.003]
! date-2.2c-7 expected: [06:28:00.007]
! date-2.2c-7 got:      [06:28:00.006]
! date-2.2c-8 expected: [06:28:00.008]
! date-2.2c-8 got:      [06:28:00.007]

[Comment 8 Matt Turner 2017-09-19 17:37:29 UTC]

(In reply to Sergei Trofimovich from comment #7)
> This patch works as well. A few date-related tests still fail
> but they were failing with my hack as well:
> 
> ! date-2.2c-1 expected: [06:28:00.001]
> ! date-2.2c-1 got:      [06:28:00.000]
> ! date-2.2c-4 expected: [06:28:00.004]
> ! date-2.2c-4 got:      [06:28:00.003]
> ! date-2.2c-7 expected: [06:28:00.007]
> ! date-2.2c-7 got:      [06:28:00.006]
> ! date-2.2c-8 expected: [06:28:00.008]
> ! date-2.2c-8 got:      [06:28:00.007]

I think that's bug 628242.

[Comment 9 Arfrever Frehtes Taifersar Arahesis 2017-09-27 21:24:35 UTC]

Apparently csv extension is not included in sqlite3.c amalgamation in non-full tarball, so no patching will be needed for that case.

[Comment 10 Arfrever Frehtes Taifersar Arahesis 2017-09-28 17:42:43 UTC]

Fixed:

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=5f80a5be1a768e5964f4d4f3399eb3b91912c81f