From daniel.subs@internode.on.net Mon Dec 14 12:36:35 2009 Received: from ipmail03.adl2.internode.on.net (unverified [203.16.214.135]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 77048779-1927428 for ; Mon, 14 Dec 2009 12:08:06 +1030 (CDT) Return-Path: X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AvYCACMkJUvQRbF0i2dsb2JhbACbOAEBAQoLChpBtz6CO4FwBIFi X-IronPort-AV: E=Sophos;i="4.47,392,1257082200"; d="scan'208";a="19396509" Received: from ns1.qubic.net ([208.69.177.116]) by ipmailmx03.adl2.internode.on.net with ESMTP; 14 Dec 2009 12:08:00 +1030 Received: from lists.opendkim.org (IDENT:nobody@localhost [127.0.0.1]) by ns1.qubic.net (8.14.4.Beta0/8.14.4.Beta0) with ESMTP id nBE1bahL025557; Sun, 13 Dec 2009 17:37:37 -0800 (PST) Received: with LISTRIA (v1.0.0; list opendkim-dev); Sun, 13 Dec 2009 17:37:36 -0800 (PST) Received: from mail.internode.on.net (bld-mail16.adl2.internode.on.net [150.101.137.101]) by ns1.qubic.net (8.14.4.Beta0/8.14.4.Beta0) with ESMTP id nBE1bPNe015826 for ; Sun, 13 Dec 2009 17:37:32 -0800 (PST) Received: from passivegrunt.localnet (unverified [121.45.202.228]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 10003920-1927428 for ; Mon, 14 Dec 2009 12:07:22 +1030 (CDT) From: Daniel Black To: opendkim-dev@lists.opendkim.org Subject: Re: test 70 error on 1.2.0 Date: Mon, 14 Dec 2009 12:36:35 +1100 User-Agent: KMail/1.12.4 (Linux/2.6.31-gentoo-r4; KDE/4.3.4; x86_64; ; ) References: <200912091930.48341.daniel.subs@internode.on.net> <20091209110014.A24444@medusa.blackops.org> In-Reply-To: <20091209110014.A24444@medusa.blackops.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200912141236.36029.daniel.subs@internode.on.net> X-archive-position: 267 Sender: opendkim-dev-bounce@lists.opendkim.org List-Help: List-Unsubscribe: List-Id: List-Subscribe: List-Owner: List-Post: X-list: opendkim-dev X-Rcpt-To: X-Vpipe: Scanner said clean (/local/app/clamav/bin/vscand-clamav) X-IP-stats: Incoming Last 5, First 68, in=55, out=0, spam=0 ip=208.69.177.116 X-Originating-IP: 208.69.177.116 Status: R X-UIDL: 1260754686.7418_5272954.bld-mail09 X-Status: N X-KMail-EncryptionState: X-KMail-SignatureState: X-KMail-MDN-Sent: On Thursday 10 December 2009 06:06:23 Murray S. Kucherawy wrote: > On Wed, 9 Dec 2009, Daniel Black wrote: > > with diffheaders (tre version - 0.7.5) > > > > still looking... > > Sorry, missed the attachment: > > Breakpoint 1, dkim_eom (dkim=0x8a8cd0, testkey=0x0) at dkim.c:3646 > > 3646 else if (sig->sig_error == DKIM_SIGERROR_KEYFAIL) > > (gdb) p sig->sig_error > > No symbol "sig" in current context. > > (gdb) p dkim->dkim_siglist[0]->sig_error > > $1 = 24 > > Yep, that's DKIM_SIGERROR_KEYFAIL. > > Did you run this test specifically (e.g. "./t-test70" from the shell > prompt)? If so, the keys it needs to verify probably aren't available. quite right - this was the case. > This is a holdover from the old "devtools" test environment. Maybe the > right thing to do now is to have that file always present in the "tests" > directory, and we can drop t-setup and t-cleanup. sound good. now that I'm looking at the right error is seems to be an environment specific problem that results in a tre regex structure initialisation problem. using tre-0.7.5 with --enable-system-abi in gentoo compile environment where it fails: /var/tmp/portage/mail-filter/opendkim-1.2.0/work/opendkim-1.2.0/libopendkim/tests $ gdb ./.libs/t-test70 GNU gdb 6.8 This GDB was configured as "x86_64-pc-linux-gnu"... (gdb) run Starting program: /var/tmp/portage/mail-filter/opendkim-1.2.0/work/opendkim-1.2.0/libopendkim/tests/.libs/t-test70 [Thread debugging using libthread_db enabled] [New Thread 0x7f2cc5c56710 (LWP 5372)] [New Thread 0x7f2cc3050950 (LWP 5375)] *** relaxed/simple rsa-sha256 verifying with header diffing Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7f2cc5c56710 (LWP 5372)] 0x00007f2cc3f6b9a6 in tre_tnfa_run_approx (tnfa=0x1f52930, string=0x1f4d240, len=-1, type=STR_BYTE, match_tags=0x0, match=0x7fff2fa99fb0, default_params= {cost_ins = 1, cost_del = 1, cost_subst = 2, max_cost = 5, max_ins = 4096, max_del = 4096, max_subst = 4096, max_err = 5}, eflags=, match_end_ofs=0x7fff2fa9968c) at tre-match-approx.c:353 353 tre-match-approx.c: No such file or directory. in tre-match-approx.c (gdb) bt #0 0x00007f2cc3f6b9a6 in tre_tnfa_run_approx (tnfa=0x1f52930, string=0x1f4d240, len=-1, type=STR_BYTE, match_tags=0x0, match=0x7fff2fa99fb0, default_params={cost_ins = 1, cost_del = 1, cost_subst = 2, max_cost = 5, max_ins = 4096, max_del = 4096, max_subst = 4096, max_err = 5}, eflags=, match_end_ofs=0x7fff2fa9968c) at tre-match-approx.c:353 #1 0x00007f2cc3f6b3a0 in tre_match_approx (tnfa=0x1f52930, string=0x1f52930, len=32821825, type=2147483647, match=0x7fff2fa99fb0, params= {cost_ins = 1, cost_del = 1, cost_subst = 2, max_cost = 5, max_ins = 4096, max_del = 4096, max_subst = 4096, max_err = 5}, eflags=) at regexec.c:297 #2 0x00007f2cc586ef9e in dkim_diffheaders (dkim=0x1f4ccd0, maxcost=, ohdrs=0x7fff2fa9b050, nohdrs=8, out=0x7fff2fa9b258, nout=0x7fff2fa9b274) at dkim.c:5400 #3 0x00000000004011fb in main (argc=, argv=) at t-test70.c:183 (gdb) list 348 been found. */ 349 if (match_costs[TRE_M_COST] > 0) 350 { 351 tre_tnfa_transition_t *trans; 352 DPRINT((" init")); 353 for (trans = tnfa->initial; trans->state; trans++) 354 { 355 int id = trans->state_id; 356 357 /* If this state is not currently in `reach_next', add it (gdb) p trans $1 = (tre_tnfa_transition_t *) 0x319 (gdb) p trans->state Cannot access memory at address 0x321 (gdb) p match_costs $2 = {2147483647, 2147483647, 2147483647, 2147483647, 2147483647} (gdb) p tnfa $3 = (const tre_tnfa_t *) 0x1f52930 (gdb) p *tnfa $4 = {transitions = 0x1f5b430, num_transitions = 804, initial = 0x319, final = 0x1f7b6e0, submatch_data = 0x0, firstpos_chars = 0x1f7d020 "\001", first_char = 33030768, num_submatches = 0, tag_directions = 0x0, minimal_tags = 0x1f5e680, num_tags = 33068592, num_minimals = 0, end_tag = 33068976, num_states = 0, cflags = 33069360, have_backrefs = 0, have_approx = 33069760, params_depth = 0} Seen here the tnfa->initial isn't a memory location allocated to the program and first_char and other data elements substantially differ from the working structure below: /tmp/opendkim-1.2.0/libopendkim/tests $ gdb .libs/t-test70 GNU gdb 6.8 This GDB was configured as "x86_64-pc-linux-gnu"... (gdb) run Starting program: /tmp/opendkim-1.2.0/libopendkim/tests/.libs/t-test70 [Thread debugging using libthread_db enabled] *** relaxed/simple rsa-sha256 verifying with header diffing Program exited normally. (gdb) break tre_tnfa_run_approx Breakpoint 1 at 0x7fc2bb5346a4: file tre-match-approx.c, line 222. (2 locations) (gdb) run Starting program: /tmp/opendkim-1.2.0/libopendkim/tests/.libs/t-test70 [Thread debugging using libthread_db enabled] *** relaxed/simple rsa-sha256 verifying with header diffing [New Thread 0x7f4663eb8710 (LWP 11851)] [Switching to Thread 0x7f4663eb8710 (LWP 11851)] Breakpoint 1, tre_tnfa_run_approx (tnfa=0x15d0510, string=0x15a1aa0, len=-1, type=STR_BYTE, match_tags=0x0, match=0x7fff8eabf200, default_params= {cost_ins = 1, cost_del = 1, cost_subst = 2, max_cost = 5, max_ins = 4096, max_del = 4096, max_subst = 4096, max_err = 5}, eflags=0, match_end_ofs=0x7fff8eabe90c) at tre-match-approx.c:222 222 { (gdb) bt #0 tre_tnfa_run_approx (tnfa=0x15d0510, string=0x15a1aa0, len=-1, type=STR_BYTE, match_tags=0x0, match=0x7fff8eabf200, default_params= {cost_ins = 1, cost_del = 1, cost_subst = 2, max_cost = 5, max_ins = 4096, max_del = 4096, max_subst = 4096, max_err = 5}, eflags=0, match_end_ofs=0x7fff8eabe90c) at tre-match-approx.c:222 #1 0x00007f4662ca43a0 in tre_match_approx (tnfa=0x15d0510, string=0x15a1aa0, len=18446744073709551615, type=STR_BYTE, match=0x7fff8eabf200, params= {cost_ins = 1, cost_del = 1, cost_subst = 2, max_cost = 5, max_ins = 4096, max_del = 4096, max_subst = 4096, max_err = 5}, eflags=) at regexec.c:297 #2 0x00007f4663ace2fc in dkim_diffheaders (dkim=0x15a1510, maxcost=5, ohdrs=0x7fff8eabf2d0, nohdrs=8, out=0x7fff8eac04e0, nout=0x7fff8eac04f4) at dkim.c:5400 #3 0x00000000004015b4 in main (argc=1, argv=0x7fff8eac05e8) at t-test70.c:183 (gdb) p *tnfa $1 = {transitions = 0x160ab30, num_transitions = 1573, initial = 0x16234c0, final = 0x1623430, submatch_data = 0x0, firstpos_chars = 0x0, first_char = 68, num_submatches = 1, tag_directions = 0x0, minimal_tags = 0x0, num_tags = 0, num_minimals = 0, end_tag = 0, num_states = 787, cflags = 8, have_backrefs = 0, have_approx = 0, params_depth = 0} (gdb) p regcomp $2 = {int (regex_t *, const char *, int)} 0x7f4662ca3f00 (gdb) p regaexec $3 = {int (const regex_t *, const char *, regamatch_t *, regaparams_t, int)} 0x7f4662ca4530 trying tre-0.8.0 results in the same SEGV only in the gentoo build environment tre-0.7.5 compiled without --enable-system-abi wouldn't compile with opendkim tre-0.8.0 compiled without --enable-system-abi passes tests in both gentoo build environment and outside. I'm therefore assuming that this is a odd library intialision problem where the regex structure is intialised by glibc and then tested against libtre. Murray - think you were 100% right when you posted "Concern about libtre and feature conflict" on 2009-11-23. options: 1. check for this type of failure in configure.ac 2. not support system-abi (NetBSD and maybe others will start using this as the system default - http://laurikari.net/tre/netbsd-takes-tre-in-the-c- library/#comments which would be convenient ) 3. lots more ugly hacks I'll ask the gentoo maintainer to disabling --system-abi in future debian seems to have gone the same approach 0.8.0-1 http://packages.debian.org/changelogs/pool/main/t/tre/tre_0.8.0-2/changelog