Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 122940 - sys-devel/flex: possible insecure generated code
Summary: sys-devel/flex: possible insecure generated code
Alias: None
Product: Gentoo Security
Classification: Unclassified
Component: Vulnerabilities (show other bugs)
Hardware: All Linux
: High major (vote)
Assignee: Gentoo Security
Whiteboard: A2? [glsa]
: 127800 (view as bug list)
Depends on:
Reported: 2006-02-15 10:02 UTC by Tavis Ormandy (RETIRED)
Modified: 2019-12-19 00:39 UTC (History)
8 users (show)

See Also:
Package list:
Runtime testing required: ---

xboard file to generate flex parser bug (xboard-flex-testcase,4.17 KB, text/plain)
2006-02-16 00:15 UTC, Tavis Ormandy (RETIRED)
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Tavis Ormandy (RETIRED) gentoo-dev 2006-02-15 10:02:46 UTC
Chris Moore <> reports flex may generate exploitable code, making a potentially large number of packages vulnerable, or at least needing to regerate parsers with a non-vulnerable flex.

generated parser.c:
--------------------- snip --------------------
/* Size of default input buffer. */
#ifndef YY_BUF_SIZE
#define YY_BUF_SIZE 16384

            (yy_state_buf) = (yy_state_type *)yyalloc(YY_BUF_SIZE + 2  );
                (yy_state_ptr) = (yy_state_buf);
                *(yy_state_ptr)++ = yy_current_state;

                        register YY_CHAR yy_c = yy_ec[YY_SC_TO_UI(*yy_cp)];
                        while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state
                                yy_current_state = (int) yy_def[yy_current_state];
                                if ( yy_current_state >= 711 )
                                        yy_c = yy_meta[(unsigned int) yy_c];
                        yy_current_state = yy_nxt[yy_base[yy_current_state] + (unsigned int)
                        *(yy_state_ptr)++ = yy_current_state; <===== OVERFLOW
                while ( yy_base[yy_current_state] != 4394 );
--------------------- snip --------------------

The code ends up in a loop which runs for as long as the input file
contains the right characters; i. e. the amount of memory overwritten
by the loop is entirely dependant on user input. If the input file
contains more than 4096 characters in a certain token, the buffer will
Comment 1 SpanKY gentoo-dev 2006-02-15 19:37:20 UTC
any chance of getting some examples here ?  like what packages has this been seen in, what version of flex are we talking about, what is the source .y used to generate that parser.c, etc...
Comment 2 Tavis Ormandy (RETIRED) gentoo-dev 2006-02-16 00:15:13 UTC
Created attachment 79904 [details]
xboard file to generate flex parser bug

More details from martin pitt:

"At least 2.5.31, but I guess it affects more versions.

However, it doesn't affect all generated parsers, but only those which
are generated by grammars which use either REJECT, or rules with a
'variable trailing context', e. g. a*1 (i. e. an arbitrary number of
a's only if they are followed by an 1). In these rules, the parser
has to keep all backtracking paths"

Attached is an xboard file to reproduce this, also from Martin Pitt.
Comment 3 Tavis Ormandy (RETIRED) gentoo-dev 2006-02-16 10:29:05 UTC
update: upstream has been informed and are investigating the issue.
Comment 4 Tavis Ormandy (RETIRED) gentoo-dev 2006-02-16 10:49:56 UTC
From: John Millaway <>
Subject: Re: [vendor-sec] Buffer overflow in generated flex parsers

Just a caution here. Correctly identifing the list of affected packages is
not possible by a script, but you can identify the packages which are
absolutely NOT affected. To do so,  grep the .l file for  REJECT. If you do
NOT find REJECT, then the scanner is immune to this bug. (However, if you do
find REJECT in the .l file, the scanner is not necessarily affected.)

   if ! grep -qf REJECT scanner.l
       echo 'Scanner not affected.'

Comment 5 Thierry Carrez (RETIRED) gentoo-dev 2006-02-16 12:38:19 UTC
Comment 6 Thierry Carrez (RETIRED) gentoo-dev 2006-02-21 10:27:33 UTC
Maybe we should do some automated source code audit to determine which packages in Portage are affected. Tavis ? 
Comment 7 Tavis Ormandy (RETIRED) gentoo-dev 2006-02-23 11:43:53 UTC
this is fixed in flex-2.5.33
Comment 8 SpanKY gentoo-dev 2006-02-23 18:03:59 UTC
2.5.33 now in portage
Comment 9 Stefan Cornelius (RETIRED) gentoo-dev 2006-02-24 08:07:59 UTC
Security liaisons, pls test and mark stable. Added flameeyes for blubb and spb/agriffis for kloeri, because they are away visiting fosdem.
Comment 10 Markus Rothe (RETIRED) gentoo-dev 2006-02-24 08:14:10 UTC
stable on ppc64
Comment 11 Diego Elio Pettenò (RETIRED) gentoo-dev 2006-02-24 08:28:13 UTC
Going to test now... a jump from 2.5.4 to 2.5.33 is likely to create problems.
Comment 12 Diego Elio Pettenò (RETIRED) gentoo-dev 2006-02-24 08:41:46 UTC
Okay just to explain a bit. Flex 2.5.30 and later has a lot of changes that makes many syntaxes previously accepted no more supported.
Unfortunately, quite a few packages used to rely on those syntaxes. I had a bad experience about that on Gentoo/FreeBSD. Although ~arch seems to be all safe, I'm not sure what is the status on 'arch' versions.

As I said, I wouldn't be surprised to know that some stable packages died because of that.

Also note that flex had LOTS of internal changes, and flex-2.5.33 ebuild still misses a sys-devel/m4 RDEPEND (GNU m4 is used internally for parsers generation)...
Comment 13 Stephen Bennett (RETIRED) gentoo-dev 2006-02-24 08:42:51 UTC
Alpha done.
Comment 14 Diego Elio Pettenò (RETIRED) gentoo-dev 2006-02-24 13:23:32 UTC
Okay emerge -e world and emerge of a few misc packages depending on flex seems fine.

Let's hope there aren't packages missing that dep that are going to break :)

amd64 done.
Comment 15 Mark Loeser (RETIRED) gentoo-dev 2006-02-24 15:58:56 UTC
x86 done
Comment 16 SpanKY gentoo-dev 2006-02-25 20:37:18 UTC
> Okay just to explain a bit. Flex 2.5.30 and later has a lot of changes that
> makes many syntaxes previously accepted no more supported.

except that we did a hell of a lot of testing with 2.5.31 before it left package.mask ... for example, i ran 2.5.31 on all my systems for ~8 months before removing it from package.mask

i make no claims on 2.5.33 though :P
Comment 17 Diego Elio Pettenò (RETIRED) gentoo-dev 2006-02-26 02:45:14 UTC
You did them also on arch system instead of ~arch ?
Comment 18 Thierry Carrez (RETIRED) gentoo-dev 2006-02-26 04:04:50 UTC
Still no coordinated release date, pending more affected source code analysis.
Comment 19 Tobias Scherbaum (RETIRED) gentoo-dev 2006-02-26 12:01:50 UTC
ppc stable
Comment 20 Gustavo Zacarias (RETIRED) gentoo-dev 2006-02-27 07:14:08 UTC
sparc done for.
Comment 21 Thierry Carrez (RETIRED) gentoo-dev 2006-03-03 10:28:16 UTC
Looks public :

Ready for GLSA
Comment 22 Tavis Ormandy (RETIRED) gentoo-dev 2006-03-06 02:20:42 UTC
here's the patch ubuntu are using if any architectures need to patch rather than bump
Comment 23 Thierry Carrez (RETIRED) gentoo-dev 2006-03-10 13:20:19 UTC
GLSA 200603-07
ppc-macos and mips should mark stable to benefit from GLSA
Comment 24 Fabian Groffen gentoo-dev 2006-03-11 02:10:53 UTC
ppc-macos stable
Comment 25 Jasper Bryant-Greene 2006-03-27 17:03:53 UTC
*** Bug 127800 has been marked as a duplicate of this bug. ***