Bugzilla Bug 122940

Chris Moore <dooglus@gmail.com> reports flex may generate exploitable code,
making a potentially large number of packages vulnerable, or at least needing
to regerate parsers with a non-vulnerable flex.

generated parser.c:
--------------------- snip --------------------
/* Size of default input buffer. */
#ifndef YY_BUF_SIZE
#define YY_BUF_SIZE 16384
#endif

[...]
            (yy_state_buf) = (yy_state_type *)yyalloc(YY_BUF_SIZE + 2  );
[...]
                (yy_state_ptr) = (yy_state_buf);
                *(yy_state_ptr)++ = yy_current_state;

yy_match:
                do
                        {
                        register YY_CHAR yy_c = yy_ec[YY_SC_TO_UI(*yy_cp)];
                        while ( yy_chk[yy_base[yy_current_state] + yy_c] !=
yy_current_state
)
                                {
                                yy_current_state = (int)
yy_def[yy_current_state];
                                if ( yy_current_state >= 711 )
                                        yy_c = yy_meta[(unsigned int) yy_c];
                                }
                        yy_current_state = yy_nxt[yy_base[yy_current_state] +
(unsigned int)
yy_c];
                        *(yy_state_ptr)++ = yy_current_state; <===== OVERFLOW
                        ++yy_cp;
                        }
                while ( yy_base[yy_current_state] != 4394 );
--------------------- snip --------------------

The code ends up in a loop which runs for as long as the input file
contains the right characters; i. e. the amount of memory overwritten
by the loop is entirely dependant on user input. If the input file
contains more than 4096 characters in a certain token, the buffer will
overflow.

any chance of getting some examples here ?  like what packages has this been
seen in, what version of flex are we talking about, what is the source .y used
to generate that parser.c, etc...

Created an attachment (id=79904) [details]
xboard file to generate flex parser bug


More details from martin pitt:

"At least 2.5.31, but I guess it affects more versions.

However, it doesn't affect all generated parsers, but only those which
are generated by grammars which use either REJECT, or rules with a
'variable trailing context', e. g. a*1 (i. e. an arbitrary number of
a's only if they are followed by an 1). In these rules, the parser
has to keep all backtracking paths"

Attached is an xboard file to reproduce this, also from Martin Pitt.

update: upstream has been informed and are investigating the issue.

From: John Millaway <johnmillaway@yahoo.com>
Subject: Re: [vendor-sec] Buffer overflow in generated flex parsers

Just a caution here. Correctly identifing the list of affected packages is
not possible by a script, but you can identify the packages which are
absolutely NOT affected. To do so,  grep the .l file for  REJECT. If you do
NOT find REJECT, then the scanner is immune to this bug. (However, if you do
find REJECT in the .l file, the scanner is not necessarily affected.)

   if ! grep -qf REJECT scanner.l
   then
       echo 'Scanner not affected.'
   fi

-John

.

Maybe we should do some automated source code audit to determine which packages
in Portage are affected. Tavis ? 

this is fixed in flex-2.5.33

2.5.33 now in portage

Security liaisons, pls test and mark stable. Added flameeyes for blubb and
spb/agriffis for kloeri, because they are away visiting fosdem.

stable on ppc64

Going to test now... a jump from 2.5.4 to 2.5.33 is likely to create problems.

Okay just to explain a bit. Flex 2.5.30 and later has a lot of changes that
makes many syntaxes previously accepted no more supported.
Unfortunately, quite a few packages used to rely on those syntaxes. I had a bad
experience about that on Gentoo/FreeBSD. Although ~arch seems to be all safe,
I'm not sure what is the status on 'arch' versions.

As I said, I wouldn't be surprised to know that some stable packages died
because of that.

Also note that flex had LOTS of internal changes, and flex-2.5.33 ebuild still
misses a sys-devel/m4 RDEPEND (GNU m4 is used internally for parsers
generation)...

Alpha done.

Okay emerge -e world and emerge of a few misc packages depending on flex seems
fine.

Let's hope there aren't packages missing that dep that are going to break :)

amd64 done.

x86 done

> Okay just to explain a bit. Flex 2.5.30 and later has a lot of changes that
> makes many syntaxes previously accepted no more supported.

except that we did a hell of a lot of testing with 2.5.31 before it left
package.mask ... for example, i ran 2.5.31 on all my systems for ~8 months
before removing it from package.mask

i make no claims on 2.5.33 though :P

You did them also on arch system instead of ~arch ?

CVE-2006-0459
Still no coordinated release date, pending more affected source code analysis.

ppc stable

sparc done for.

Looks public : 
http://secunia.com/advisories/19071/

Ready for GLSA

here's the patch ubuntu are using if any architectures need to patch rather
than bump http://patches.ubuntu.com/patches/flex.CVE-2006-0459.diff

GLSA 200603-07
ppc-macos and mips should mark stable to benefit from GLSA

ppc-macos stable

*** Bug 127800 has been marked as a duplicate of this bug. ***