error backtraces

test 70 error on 1.2.0.mbox (text/plain), 11.66 KB, created by Daniel Black (RETIRED) on 2009-12-14 01:58:16 UTC

From [e-mail] Mon Dec 14 12:36:35 2009
Received: from ipmail03.adl2.internode.on.net (unverified [203.16.214.135]) 
	by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 77048779-1927428 
	for <[e-mail]>; Mon, 14 Dec 2009 12:08:06 +1030 (CDT)
Return-Path: <[e-mail]>
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AvYCACMkJUvQRbF0i2dsb2JhbACbOAEBAQoLChpBtz6CO4FwBIFi
X-IronPort-AV: E=Sophos;i="4.47,392,1257082200"; 
   d="scan'208";a="19396509"
Received: from ns1.qubic.net ([208.69.177.116])
  by ipmailmx03.adl2.internode.on.net with ESMTP; 14 Dec 2009 12:08:00 +1030
Received: from lists.opendkim.org (IDENT:nobody@localhost [127.0.0.1])
	by ns1.qubic.net (8.14.4.Beta0/8.14.4.Beta0) with ESMTP id nBE1bahL025557;
	Sun, 13 Dec 2009 17:37:37 -0800 (PST)
Received: with LISTRIA (v1.0.0; list opendkim-dev); Sun, 13 Dec 2009 17:37:36 -0800 (PST)
Received: from mail.internode.on.net (bld-mail16.adl2.internode.on.net [150.101.137.101])
	by ns1.qubic.net (8.14.4.Beta0/8.14.4.Beta0) with ESMTP id nBE1bPNe015826
	for <[e-mail]>; Sun, 13 Dec 2009 17:37:32 -0800 (PST)
Received: from passivegrunt.localnet (unverified [121.45.202.228]) 
	by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 10003920-1927428 
	for <[e-mail]>; Mon, 14 Dec 2009 12:07:22 +1030 (CDT)
From: Daniel Black <[e-mail]>
To: [e-mail]
Subject: Re: test 70 error on 1.2.0
Date: Mon, 14 Dec 2009 12:36:35 +1100
User-Agent: KMail/1.12.4 (Linux/2.6.31-gentoo-r4; KDE/4.3.4; x86_64; ; )
References: <[e-mail]> <[e-mail]>
In-Reply-To: <[e-mail]>
MIME-Version: 1.0
Content-Type: Text/Plain;
  charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Message-Id: <[e-mail]>
X-archive-position: 267
Sender: [e-mail]
List-Help: <mailto:[e-mail]?Subject=help>
List-Unsubscribe: <mailto:[e-mail]?Subject=unsubscribe>
List-Id: <opendkim-dev.lists.opendkim.org>
List-Subscribe: <mailto:[e-mail]?Subject=subscribe>
List-Owner: <mailto:[e-mail]>
List-Post: <mailto:[e-mail]>
X-list: opendkim-dev
X-Rcpt-To: <[e-mail]>
X-Vpipe: Scanner said clean (/local/app/clamav/bin/vscand-clamav)
X-IP-stats: Incoming Last 5, First 68, in=55, out=0, spam=0 ip=208.69.177.116
X-Originating-IP: 208.69.177.116
Status: R
X-UIDL: 1260754686.7418_5272954.bld-mail09
X-Status: N
X-KMail-EncryptionState:  
X-KMail-SignatureState:  
X-KMail-MDN-Sent:  

On Thursday 10 December 2009 06:06:23 Murray S. Kucherawy wrote:
> On Wed, 9 Dec 2009, Daniel Black wrote:
> > with diffheaders (tre version - 0.7.5)
> >
> > still looking...
> 
> Sorry, missed the attachment:
> > Breakpoint 1, dkim_eom (dkim=0x8a8cd0, testkey=0x0) at dkim.c:3646
> > 3646            else if (sig->sig_error == DKIM_SIGERROR_KEYFAIL)
> > (gdb) p sig->sig_error
> > No symbol "sig" in current context.
> > (gdb) p dkim->dkim_siglist[0]->sig_error
> > $1 = 24
> 
> Yep, that's DKIM_SIGERROR_KEYFAIL.
> 
> Did you run this test specifically (e.g. "./t-test70" from the shell
> prompt)?  If so, the keys it needs to verify probably aren't available.

quite right - this was the case.

> This is a holdover from the old "devtools" test environment.  Maybe the
> right thing to do now is to have that file always present in the "tests"
> directory, and we can drop t-setup and t-cleanup.

sound good.


now that I'm looking at the right error is seems to be an environment specific problem that results in a tre regex structure initialisation problem.

using tre-0.7.5 with --enable-system-abi

in gentoo compile environment where it fails:

/var/tmp/portage/mail-filter/opendkim-1.2.0/work/opendkim-1.2.0/libopendkim/tests $ gdb ./.libs/t-test70
GNU gdb 6.8                                                                                                              
This GDB was configured as "x86_64-pc-linux-gnu"...                                                                      
(gdb) run                                                                                                                
Starting program: /var/tmp/portage/mail-filter/opendkim-1.2.0/work/opendkim-1.2.0/libopendkim/tests/.libs/t-test70       
[Thread debugging using libthread_db enabled]                                                                            
[New Thread 0x7f2cc5c56710 (LWP 5372)]                                                                                   
[New Thread 0x7f2cc3050950 (LWP 5375)]                                                                                   
*** relaxed/simple rsa-sha256 verifying with header diffing                                                              

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f2cc5c56710 (LWP 5372)]     
0x00007f2cc3f6b9a6 in tre_tnfa_run_approx (tnfa=0x1f52930, string=0x1f4d240, len=-1, type=STR_BYTE, match_tags=0x0, match=0x7fff2fa99fb0, 
default_params=
      {cost_ins = 1, cost_del = 1, cost_subst = 2, max_cost = 5, max_ins = 4096, max_del = 4096, max_subst = 4096, max_err = 5},                         
    eflags=<value optimized out>, match_end_ofs=0x7fff2fa9968c) at tre-match-approx.c:353                                                                
353     tre-match-approx.c: No such file or directory.                                                                                                   
        in tre-match-approx.c                                                                                                                            
(gdb) bt                                                                                                                                                 
#0  0x00007f2cc3f6b9a6 in tre_tnfa_run_approx (tnfa=0x1f52930, string=0x1f4d240, len=-1, type=STR_BYTE, match_tags=0x0, match=0x7fff2fa99fb0,
    default_params={cost_ins = 1, cost_del = 1, cost_subst = 2, max_cost = 5, max_ins = 4096, max_del = 4096, max_subst = 4096, max_err = 5},
    eflags=<value optimized out>, match_end_ofs=0x7fff2fa9968c) at tre-match-approx.c:353
#1  0x00007f2cc3f6b3a0 in tre_match_approx (tnfa=0x1f52930, string=0x1f52930, len=32821825, type=2147483647, match=0x7fff2fa99fb0, params=
      {cost_ins = 1, cost_del = 1, cost_subst = 2, max_cost = 5, max_ins = 4096, max_del = 4096, max_subst = 4096, max_err = 5},
    eflags=<value optimized out>) at regexec.c:297
#2  0x00007f2cc586ef9e in dkim_diffheaders (dkim=0x1f4ccd0, maxcost=<value optimized out>, ohdrs=0x7fff2fa9b050, nohdrs=8, 
out=0x7fff2fa9b258,
    nout=0x7fff2fa9b274) at dkim.c:5400
#3  0x00000000004011fb in main (argc=<value optimized out>, argv=<value optimized out>) at t-test70.c:183
(gdb) list
348              been found. */
349           if (match_costs[TRE_M_COST] > 0)
350             {
351               tre_tnfa_transition_t *trans;
352               DPRINT(("  init"));
353               for (trans = tnfa->initial; trans->state; trans++)
354                 {
355                   int id = trans->state_id;
356
357                   /* If this state is not currently in `reach_next', add it
(gdb) p trans
$1 = (tre_tnfa_transition_t *) 0x319
(gdb) p trans->state
Cannot access memory at address 0x321
(gdb) p match_costs
$2 = {2147483647, 2147483647, 2147483647, 2147483647, 2147483647}
(gdb) p tnfa
$3 = (const tre_tnfa_t *) 0x1f52930
(gdb) p *tnfa
$4 = {transitions = 0x1f5b430, num_transitions = 804, initial = 0x319, final = 0x1f7b6e0, submatch_data = 0x0, firstpos_chars = 0x1f7d020 "\001",
  first_char = 33030768, num_submatches = 0, tag_directions = 0x0, minimal_tags = 0x1f5e680, num_tags = 33068592, num_minimals = 0, end_tag = 
33068976,
  num_states = 0, cflags = 33069360, have_backrefs = 0, have_approx = 33069760, params_depth = 0}

Seen here the tnfa->initial isn't a memory location allocated to the program and first_char and other data elements substantially differ from the working 
structure below:

/tmp/opendkim-1.2.0/libopendkim/tests $ gdb .libs/t-test70   
GNU gdb 6.8                                                                   
This GDB was configured as "x86_64-pc-linux-gnu"...
(gdb) run
Starting program: /tmp/opendkim-1.2.0/libopendkim/tests/.libs/t-test70
[Thread debugging using libthread_db enabled]
*** relaxed/simple rsa-sha256 verifying with header diffing

Program exited normally.
(gdb) break tre_tnfa_run_approx
Breakpoint 1 at 0x7fc2bb5346a4: file tre-match-approx.c, line 222. (2 locations)
(gdb) run
Starting program: /tmp/opendkim-1.2.0/libopendkim/tests/.libs/t-test70
[Thread debugging using libthread_db enabled]
*** relaxed/simple rsa-sha256 verifying with header diffing
[New Thread 0x7f4663eb8710 (LWP 11851)]
[Switching to Thread 0x7f4663eb8710 (LWP 11851)]

Breakpoint 1, tre_tnfa_run_approx (tnfa=0x15d0510, string=0x15a1aa0, len=-1, type=STR_BYTE, match_tags=0x0, match=0x7fff8eabf200, 
default_params=
      {cost_ins = 1, cost_del = 1, cost_subst = 2, max_cost = 5, max_ins = 4096, max_del = 4096, max_subst = 4096, max_err = 5}, eflags=0,
    match_end_ofs=0x7fff8eabe90c) at tre-match-approx.c:222
222     {
(gdb) bt
#0  tre_tnfa_run_approx (tnfa=0x15d0510, string=0x15a1aa0, len=-1, type=STR_BYTE, match_tags=0x0, match=0x7fff8eabf200, default_params=
      {cost_ins = 1, cost_del = 1, cost_subst = 2, max_cost = 5, max_ins = 4096, max_del = 4096, max_subst = 4096, max_err = 5}, eflags=0,
    match_end_ofs=0x7fff8eabe90c) at tre-match-approx.c:222
#1  0x00007f4662ca43a0 in tre_match_approx (tnfa=0x15d0510, string=0x15a1aa0, len=18446744073709551615, type=STR_BYTE, 
match=0x7fff8eabf200, params=
      {cost_ins = 1, cost_del = 1, cost_subst = 2, max_cost = 5, max_ins = 4096, max_del = 4096, max_subst = 4096, max_err = 5},
    eflags=<value optimized out>) at regexec.c:297
#2  0x00007f4663ace2fc in dkim_diffheaders (dkim=0x15a1510, maxcost=5, ohdrs=0x7fff8eabf2d0, nohdrs=8, out=0x7fff8eac04e0, 
nout=0x7fff8eac04f4)
    at dkim.c:5400
#3  0x00000000004015b4 in main (argc=1, argv=0x7fff8eac05e8) at t-test70.c:183
(gdb)  p *tnfa
$1 = {transitions = 0x160ab30, num_transitions = 1573, initial = 0x16234c0, final = 0x1623430, submatch_data = 0x0, firstpos_chars = 0x0, first_char 
= 68,
  num_submatches = 1, tag_directions = 0x0, minimal_tags = 0x0, num_tags = 0, num_minimals = 0, end_tag = 0, num_states = 787, cflags = 8,
  have_backrefs = 0, have_approx = 0, params_depth = 0}
(gdb)  p regcomp
$2 = {int (regex_t *, const char *, int)} 0x7f4662ca3f00 <regcomp>
(gdb) p regaexec
$3 = {int (const regex_t *, const char *, regamatch_t *, regaparams_t, int)} 0x7f4662ca4530 <regaexec>


trying tre-0.8.0 results in the same SEGV only in the gentoo build environment

tre-0.7.5 compiled without --enable-system-abi wouldn't compile with opendkim

tre-0.8.0 compiled without --enable-system-abi passes tests in both gentoo build environment and outside.

I'm therefore assuming that this is a odd library intialision problem where the regex structure is intialised by glibc and then tested against libtre.

Murray - think you were 100% right when you posted "Concern about libtre and feature conflict" on 2009-11-23.

options:
1. check for this type of failure in configure.ac
2. not support system-abi (NetBSD and maybe others will start using this as the system default - http://laurikari.net/tre/netbsd-takes-tre-in-the-c-
library/#comments which would be convenient )
3. lots more ugly hacks

I'll ask the gentoo maintainer to disabling --system-abi in future

debian seems to have gone the same approach 0.8.0-1  http://packages.debian.org/changelogs/pool/main/t/tre/tre_0.8.0-2/changelog