Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 72109 - Perl RegExp stack segfault with SpamAssassin
Summary: Perl RegExp stack segfault with SpamAssassin
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All All
: High normal (vote)
Assignee: Gentoo Perl team
URL: http://dev.gentoo.org/~jaervosz/evil_...
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-11-22 11:09 UTC by Sune Kloppenborg Jeppesen
Modified: 2006-07-23 07:24 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Perl segfaulter. (perl_segfault.pl,145 bytes, text/plain)
2004-11-23 07:08 UTC, Dan Margolis (RETIRED)
Details
ugly-ass patch (proposed.patch,846 bytes, patch)
2005-01-03 13:11 UTC, Dan Margolis (RETIRED)
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Sune Kloppenborg Jeppesen gentoo-dev 2004-11-22 11:09:40 UTC
A specially crafted email with a lot of email addresses in the to field, causes SA/Perl to segfault:

spamassassin < evil_spam
Segmentation fault
Comment 1 solar (RETIRED) gentoo-dev 2004-11-22 18:11:16 UTC
Can you provide an example email? Know at how many bytes? Or what in what function in perl?
Comment 2 Dan Margolis (RETIRED) gentoo-dev 2004-11-22 18:37:35 UTC
OK, I'm going to start by trying to narrow down what perl code is causing this. Hopefully that'll then allow me some idea of what part of Perl is (presumably) causing this. 

For anyone looking, NoMailAudit.pm->parse_headers() is where it's happening.
Comment 3 Dan Margolis (RETIRED) gentoo-dev 2004-11-22 19:08:54 UTC
Scratch that. After somebody calls parse_headers(), something dies. Still trying to figure out where the call comes from. Using the vaunted print("foo") method of figuring out how a program works. 
Comment 4 Sune Kloppenborg Jeppesen gentoo-dev 2004-11-22 23:03:11 UTC
Solar an example is provided for download in the URL field.

Comment 5 Florian Schilhabel (RETIRED) gentoo-dev 2004-11-23 04:28:17 UTC
hi,
i was able to do a little backtrace on that:
first:
i think, the problem is _not_ located in spamassassin but in perl:

[root@leela][tmp:exec](~) # spamassassin < /home/flo/evil_spam
Segmentation fault (core dumped)

that basically means, perl accepts a syntactically correct program, accepts user supplied input and finally segfaults...

a backtrace gives the following:
Program terminated with signal 11, Segmentation fault.
Cannot access memory at address 0x400168c0
#0  0x08110c0b in S_regmatch (prog=Cannot access memory at address 0xbfe012a0
) at regexec.c:3732
3732    regexec.c: No such file or directory.
        in regexec.c
(gdb) bt
#0  0x08110c0b in S_regmatch (prog=Cannot access memory at address 0xbfe012a0
) at regexec.c:3732
Cannot access memory at address 0xbfe0129c
(gdb)

so i would say, there's a bug in regexec.c, thats part of the regular expression handling routines in perl itself...
tracing through spamassassin under the perl debugger, i get the following:

 main::(/usr/bin/spamassassin:245):        if ($@) { warn $@; }
   DB<2> n
 main::(/usr/bin/spamassassin:247):        if ($doing_whitelist_operation) {
   DB<2> n
 main::(/usr/bin/spamassassin:276):        my $status = $spamtest->check ($mail);
   DB<2> n
 main::(/usr/bin/spamassassin:277):        $status->rewrite_mail ();
   DB<2> n
 Segmentation fault (core dumped)
 hmmmm...
   DB<5> s
 Mail::SpamAssassin::NoMailAudit::get_pristine_header(/usr/lib/perl5/vendor_perl/5.8.4/Mail/SpamAssassin/NoMailAudit.pm:183):
 183:      my ($self, $hdr) = @_;
   DB<5> s
 Mail::SpamAssassin::NoMailAudit::get_pristine_header(/usr/lib/perl5/vendor_perl/5.8.4/Mail/SpamAssassin/NoMailAudit.pm:184):
 184:      my(@ret) = $self->{headers_pristine} =~ /^(?:$hdr:[ ]+(.*\n(?:\s+\S.*\n)*))/mig;
   DB<5> s
 Segmentation fault (core dumped)
i guess, the regular expression it dies is here:
my(@ret) = $self->{headers_pristine} =~ /^(?:$hdr:[ ]+(.*\n(?:\s+\S.*\n)*))/mig;

i think, perl upstream should be contacted about this issue.
_if_ the regular expression engine of perl itself is vulnerable to this issue, all perl based daemons (like webmin) _could_ eventually also be affected...

best regards

florian


 
Comment 6 Sune Kloppenborg Jeppesen gentoo-dev 2004-11-23 05:17:58 UTC
Robert please take a look and advise.
Comment 7 Dan Margolis (RETIRED) gentoo-dev 2004-11-23 06:42:17 UTC
Florian; 

you're right that the program is syntactically correct; what I was trying to figure out is where the input is going that results in *this* input being able to crash perl. It would seem, on first guess, to be the excessive To: headers, which is why I thought the culprit might be that array everything gets pushed into (I was wrong, however). That regex is not itself broken, either, since it seems that it would work on *other* input. 

Very strange. But I don't really have a strong desire to read so much Perl source code. :P
Comment 8 Dan Margolis (RETIRED) gentoo-dev 2004-11-23 07:07:32 UTC
The specific perl code that does it on my linux box is attached. As an interesting note, this does NOT segfault Perl 5.8 on MacOSX. 
Comment 9 Dan Margolis (RETIRED) gentoo-dev 2004-11-23 07:08:24 UTC
Created attachment 44568 [details]
Perl segfaulter.
Comment 10 Robert Coie (RETIRED) gentoo-dev 2004-11-23 13:39:58 UTC
It appears to be a recursion-caused stack overflow in the regexp engine.  The
first mention I find of it is http://perl.com/pub/a/2000/11/p5pdigest/THISWEEK-20001107.html#The_Regex_Stack_Problem
One thing that we could try is to decrease REG_INFTY, which is in regcomp.h.  I
successfully got this to stop segfaulting by dropping it to 1024, but that seems
maybe unfairly low.  Would also appreciate input from people on other CPU arches,
and using ithreads, because both those factors would seem to influence an optimal
value.  The other thing I can think to do is to increase the stack size.  Both
of these alternatives would seem to be tradeoffs that would be unwanted in other
situations, so I don't see a clear-cut winning solution at this point.
Comment 11 Robert Coie (RETIRED) gentoo-dev 2004-11-23 15:28:50 UTC
I've looked at Dominic Dunlop's original stack checking patches described in
http://groups.google.com/groups?q=+regex+stack+check+redux&hl=en&lr=&selm=v03110705b415758253a8%40%5B212.24.192.141%5D&rnum=1
and since they've been out for so long and were not adopted upstream, I'm not
inclined to just go stashing them into Gentoo perl.  Would it be possible to 
change that part of SpamAsasssin to use perl looping constructs instead of riding
the regular expression engine so hard?
Comment 12 Dan Margolis (RETIRED) gentoo-dev 2004-11-23 17:13:06 UTC
First, I was unable to cause a segfault on any other distributions (I tried SuSE 5.8, Debian stable 5.6, MacOSX 5.8, and Solaris God Only Knows). So I would not go upstream with this yet. Further, when I compiled the Gentoo-patched sources by hand, I didn't get this to happen, either, which is just bizarre!

Clearly, I should be running a debugger on this shit instead of playing Halo 2 all the time. 

Regarding Comment #11, I'd be slightly leery of modifying SA, although that'd clearly be quite trivial, especially compared to the task of modifying Perl itself. But what I really would prefer--if I can avoid said video game and put some time in ;)--is to figure out why Gentoo's is segfaulting as well. 
Comment 13 Dan Margolis (RETIRED) gentoo-dev 2004-11-24 11:25:16 UTC
Nevermind. I can get it to crash on MacOSX and Debian by increasing the multiplier that dictates the size of the string being run through the regex. So this is apparently just depending on the stack size or somesuch. 

And I believe rac is right about the cause of this and not fixing it (didn't get to read those links before). It doesn't sound like it can lead to RCE, either, if it's just excessive recursion. So I don't see anything wrong with patching case-by-case. I'll take a look at patching SpamAssassin now, and we can send the patch upstream as well. 
Comment 14 Dan Margolis (RETIRED) gentoo-dev 2005-01-03 13:11:05 UTC
Well, I wrote what looks like it might be a suitable replacement for this ugly regex, but due to conflicting Perl versions and libraries, I can't easily test it on my Mac, and I haven't got a Linux box available since I'm on vacation right now. 

Can someone please test this patch and tell me if it works (it's extremely ugly and very well may not)? If it does, I'll just file a bug upstream. I don't see the need for continued secrecy or for a GLSA or anything, but security, feel free to vote (me off the island). 

Thanks. 
Comment 15 Dan Margolis (RETIRED) gentoo-dev 2005-01-03 13:11:54 UTC
Created attachment 47519 [details, diff]
ugly-ass patch
Comment 16 Dan Margolis (RETIRED) gentoo-dev 2005-01-15 15:56:56 UTC
Looks like the new version of SA breaks my patch, and has this regex a few other places. I may as well just file a bug with SA, 'cause I don't feel like patching every instance, nor do I have the time. 
Comment 17 Dan Margolis (RETIRED) gentoo-dev 2005-01-15 16:04:09 UTC
http://bugzilla.spamassassin.org/show_bug.cgi?id=4079
Comment 18 Malte S. Stretz 2005-01-16 09:52:35 UTC
Changed Subject so this bug appears on my SA-radar :)
Comment 19 Malte S. Stretz 2005-01-16 10:08:41 UTC
A fix for long headers went already into the trunk (aka will-be-3.1-when-its-done);  IIRC was it a patch against excessive memory consumption but as a side effect it fixed this issue, too.  I currently can't remember which patch it was, we've got to identify it and decide wether we'll put it into 3.0.3 (probably).  Stay tuned :)
Comment 20 Malte S. Stretz 2005-01-16 17:31:18 UTC
I found an even simpler testcase here <http://www.livejournal.com/community/perl/61223.html>.

One comment on that posting says "I think this bug got introduced somewhere between 5.8.1 and 5.8.3, since it doesn't show up in 5.8.1, and 5.8.2 hangs for a little bit, but doesn't crash. [...]"

We'll try our best to work around this issue upstream but in the end it's a Perl problem which should be fixed there if possible.
Comment 21 Dan Margolis (RETIRED) gentoo-dev 2005-01-20 09:57:46 UTC
Re: Comment #20 

- That version info is incorrect; I can replicate the segfault on Debian stable (Perl 5.6). If you cannot replicate this, it is most likely an issue with your default stack size. 

- It doesn't look like the Perl guys intend to fix this anytime soon. I can't really blame them. 

As a result--while I'd prefer to fix Perl, too, this bug is only a DoS, so I think we can afford to fix the few apps affected on a case-by-case basis--I think this should just be fixed in SA. But I don't *really* want to be the one to do it (fortunately, I don't have to :). 

Thanks. 
Comment 22 Justin Mason 2005-01-21 17:59:41 UTC
I'm having a bitch of a time reproducing this... has anyone got a reliable recipe to do so on perl 5.8.4?  even with ulimit -s 64 the message (eventually) gets processed.
Comment 23 Justin Mason 2005-01-24 13:35:53 UTC
btw I also have perl 5.6.1 and 5.8.0 here, if anyone has a recipe for those.
Comment 24 Malte S. Stretz 2005-01-24 14:03:08 UTC
From the link I posted:
| $really_long_string = "X" x $ARGV[0];
| if ($really_long_string =~ /^(X|YZ)+/) {
|         print "Match succeeded\n";
| }

my own Gentoo box:
| mss@otherland ~/tmp $ perl -v | grep v5
| This is perl, v5.8.5 built for i686-linux
| mss@otherland ~/tmp $ perl test.pl 3179
| Match succeeded
| mss@otherland ~/tmp $ perl test.pl 3180
| Segmentation fault

some Debian box:
| mss@quickstop ~/tmp $ perl -v | grep v5
| This is perl, v5.8.4 built for i386-linux-thread-multi   
| mss@quickstop ~/tmp $ perl test.pl 20963
| Match succeeded
| mss@quickstop ~/tmp $ perl test.pl 20964
| Segmentation fault

another Debian box:
| mstretz@paul ~/tmp $ perl -v | grep v5
| This is perl, v5.6.1 built for i386-linux
| mstretz@paul ~/tmp $ time perl test.pl 10000000
| Match succeeded
| real    0m0.815s
| user    0m0.280s
| sys     0m0.270s
| mstretz@paul ~/tmp $ time perl test.pl 100000000
| Match succeeded
| real    1m13.379s
| user    0m1.590s
| sys     0m3.080s
Comment 25 Sune Kloppenborg Jeppesen gentoo-dev 2005-04-23 07:36:04 UTC
rac, any news on this one?
Comment 26 Tavis Ormandy (RETIRED) gentoo-dev 2005-07-16 13:46:53 UTC
Reassigning to perl@g.o, no apparent security impact.
Comment 27 Michael Cummings (RETIRED) gentoo-dev 2005-07-19 07:55:13 UTC
Malte - came across this bug in my list of neglected^H^H bugs. Is this still
relevant?
Comment 28 Sune Kloppenborg Jeppesen gentoo-dev 2005-07-19 08:04:40 UTC
It still segfaults here. 
Comment 29 Malte S. Stretz 2005-07-19 09:39:46 UTC
We hope that we put enough workarounds in SpamAssassin that it will be fixed in 
3.1.  Perl itself still has the problems described. 
 
jaervosz, you might want to try the current 3.1 prerelease if you're  
interested.  
Comment 30 Michael Cummings (RETIRED) gentoo-dev 2005-07-26 16:56:39 UTC
Don't suppose someone would mind looking at bug 65264 with thoughts? I *think*
its the same as this bug, but am too tired to grok (hoping for clearer, smarter
heads :)
Comment 31 Jakub Moc (RETIRED) gentoo-dev 2005-09-04 04:12:19 UTC
Mass re-assign.
Comment 32 Jakub Moc (RETIRED) gentoo-dev 2005-11-09 09:11:05 UTC
Mass re-assign.
Comment 33 Sune Kloppenborg Jeppesen gentoo-dev 2005-12-18 12:19:46 UTC
This is no longer an issue with newest Perl and Spamassassin. So I guess this one could be closed.
Comment 34 Jakub Moc (RETIRED) gentoo-dev 2006-07-23 07:21:53 UTC
Closing per reporter request.
Comment 35 Jakub Moc (RETIRED) gentoo-dev 2006-07-23 07:24:00 UTC
*** Bug 65264 has been marked as a duplicate of this bug. ***