Bug 122940 – sys-devel/flex: possible insecure generated code

• Product: Gentoo Security
• Status: RESOLVED FIXED

[Description Tavis Ormandy (RETIRED) 2006-02-15 10:02:46 UTC]

Chris Moore <[e-mail]> reports flex may generate exploitable code, making a potentially large number of packages vulnerable, or at least needing to regerate parsers with a non-vulnerable flex.

generated parser.c:
--------------------- snip --------------------
/* Size of default input buffer. */
#ifndef YY_BUF_SIZE
#define YY_BUF_SIZE 16384
#endif

[...]
            (yy_state_buf) = (yy_state_type *)yyalloc(YY_BUF_SIZE + 2  );
[...]
                (yy_state_ptr) = (yy_state_buf);
                *(yy_state_ptr)++ = yy_current_state;

yy_match:
                do
                        {
                        register YY_CHAR yy_c = yy_ec[YY_SC_TO_UI(*yy_cp)];
                        while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state
)
                                {
                                yy_current_state = (int) yy_def[yy_current_state];
                                if ( yy_current_state >= 711 )
                                        yy_c = yy_meta[(unsigned int) yy_c];
                                }
                        yy_current_state = yy_nxt[yy_base[yy_current_state] + (unsigned int)
yy_c];
                        *(yy_state_ptr)++ = yy_current_state; <===== OVERFLOW
                        ++yy_cp;
                        }
                while ( yy_base[yy_current_state] != 4394 );
--------------------- snip --------------------

The code ends up in a loop which runs for as long as the input file
contains the right characters; i. e. the amount of memory overwritten
by the loop is entirely dependant on user input. If the input file
contains more than 4096 characters in a certain token, the buffer will
overflow.

[Comment 1 SpanKY 2006-02-15 19:37:20 UTC]

any chance of getting some examples here ?  like what packages has this been seen in, what version of flex are we talking about, what is the source .y used to generate that parser.c, etc...

[Comment 2 Tavis Ormandy (RETIRED) 2006-02-16 00:15:13 UTC]

Created attachment 79904 [details]
xboard file to generate flex parser bug


More details from martin pitt:

"At least 2.5.31, but I guess it affects more versions.

However, it doesn't affect all generated parsers, but only those which
are generated by grammars which use either REJECT, or rules with a
'variable trailing context', e. g. a*1 (i. e. an arbitrary number of
a's only if they are followed by an 1). In these rules, the parser
has to keep all backtracking paths"

Attached is an xboard file to reproduce this, also from Martin Pitt.

[Comment 3 Tavis Ormandy (RETIRED) 2006-02-16 10:29:05 UTC]

update: upstream has been informed and are investigating the issue.

[Comment 4 Tavis Ormandy (RETIRED) 2006-02-16 10:49:56 UTC]

From: John Millaway <[e-mail]>
Subject: Re: [vendor-sec] Buffer overflow in generated flex parsers

Just a caution here. Correctly identifing the list of affected packages is
not possible by a script, but you can identify the packages which are
absolutely NOT affected. To do so,  grep the .l file for  REJECT. If you do
NOT find REJECT, then the scanner is immune to this bug. (However, if you do
find REJECT in the .l file, the scanner is not necessarily affected.)

   if ! grep -qf REJECT scanner.l
   then
       echo 'Scanner not affected.'
   fi

-John

[Comment 5 Thierry Carrez (RETIRED) 2006-02-16 12:38:19 UTC]

.

[Comment 6 Thierry Carrez (RETIRED) 2006-02-21 10:27:33 UTC]

Maybe we should do some automated source code audit to determine which packages in Portage are affected. Tavis ? 

[Comment 7 Tavis Ormandy (RETIRED) 2006-02-23 11:43:53 UTC]

this is fixed in flex-2.5.33

[Comment 8 SpanKY 2006-02-23 18:03:59 UTC]

2.5.33 now in portage

[Comment 9 Stefan Cornelius (RETIRED) 2006-02-24 08:07:59 UTC]

Security liaisons, pls test and mark stable. Added flameeyes for blubb and spb/agriffis for kloeri, because they are away visiting fosdem.

[Comment 10 Markus Rothe (RETIRED) 2006-02-24 08:14:10 UTC]

stable on ppc64

[Comment 11 Diego Elio Pettenò (RETIRED) 2006-02-24 08:28:13 UTC]

Going to test now... a jump from 2.5.4 to 2.5.33 is likely to create problems.

[Comment 12 Diego Elio Pettenò (RETIRED) 2006-02-24 08:41:46 UTC]

Okay just to explain a bit. Flex 2.5.30 and later has a lot of changes that makes many syntaxes previously accepted no more supported.
Unfortunately, quite a few packages used to rely on those syntaxes. I had a bad experience about that on Gentoo/FreeBSD. Although ~arch seems to be all safe, I'm not sure what is the status on 'arch' versions.

As I said, I wouldn't be surprised to know that some stable packages died because of that.

Also note that flex had LOTS of internal changes, and flex-2.5.33 ebuild still misses a sys-devel/m4 RDEPEND (GNU m4 is used internally for parsers generation)...

[Comment 13 Stephen Bennett (RETIRED) 2006-02-24 08:42:51 UTC]

Alpha done.

[Comment 14 Diego Elio Pettenò (RETIRED) 2006-02-24 13:23:32 UTC]

Okay emerge -e world and emerge of a few misc packages depending on flex seems fine.

Let's hope there aren't packages missing that dep that are going to break :)

amd64 done.

[Comment 15 Mark Loeser (RETIRED) 2006-02-24 15:58:56 UTC]

x86 done

[Comment 16 SpanKY 2006-02-25 20:37:18 UTC]

> Okay just to explain a bit. Flex 2.5.30 and later has a lot of changes that
> makes many syntaxes previously accepted no more supported.

except that we did a hell of a lot of testing with 2.5.31 before it left package.mask ... for example, i ran 2.5.31 on all my systems for ~8 months before removing it from package.mask

i make no claims on 2.5.33 though :P

[Comment 17 Diego Elio Pettenò (RETIRED) 2006-02-26 02:45:14 UTC]

You did them also on arch system instead of ~arch ?

[Comment 18 Thierry Carrez (RETIRED) 2006-02-26 04:04:50 UTC]

CVE-2006-0459
Still no coordinated release date, pending more affected source code analysis.

[Comment 19 Tobias Scherbaum (RETIRED) 2006-02-26 12:01:50 UTC]

ppc stable

[Comment 20 Gustavo Zacarias (RETIRED) 2006-02-27 07:14:08 UTC]

sparc done for.

[Comment 21 Thierry Carrez (RETIRED) 2006-03-03 10:28:16 UTC]

Looks public : 
http://secunia.com/advisories/19071/

Ready for GLSA

[Comment 22 Tavis Ormandy (RETIRED) 2006-03-06 02:20:42 UTC]

here's the patch ubuntu are using if any architectures need to patch rather than bump http://patches.ubuntu.com/patches/flex.CVE-2006-0459.diff

[Comment 23 Thierry Carrez (RETIRED) 2006-03-10 13:20:19 UTC]

GLSA 200603-07
ppc-macos and mips should mark stable to benefit from GLSA

[Comment 24 Fabian Groffen 2006-03-11 02:10:53 UTC]

ppc-macos stable

[Comment 25 Jasper Bryant-Greene 2006-03-27 17:03:53 UTC]

*** Bug 127800 has been marked as a duplicate of this bug. ***