Bugzilla Bug 97921

Hi,
  it starts to be a problem that flex-2.5.31 and above are not available on Gentoo. Ooops, it is in portage but somebody relucts to make it available. It has some 'reentrancy' feature, use -R. I know nothing about it but google around.

  I gues it is not available as an ebuild because flex-2.4 seems to be the "stable" release. That's why I guess 2.5.23 disappeared from portage etc. ;)

Although maybe true that it is unstable, we need flex-2.5.31 for upcoming mod_python-3.2.0 for example. I did a bit research on this and asked some people and the best one can do is to use Debian patches from flex-2.5.31. It seems flex is somewhat unmaintained these days.

Paul Eggert says:
You might just suggest people to use Debian stable.
You can point people to <http://packages.debian.org/stable/devel/flex>
and give them instructions about applying the patches.  It's not
ideal, but it's better than nothing.

It seems the sourceforge page is also unattended. Several bugreports and also patches are laying there. Will someone assemble that? ;-)
http://sourceforge.net/tracker/?group_id=72099&atid=533377

The best I can come up is to install flex as /usr/bin/flex-2.5.31 and the mod_python.ebuild guys will call mod_python/configure --with-flex=/usr/bin/flex-2.5.31.

So, I see flex-2.5.31-r1.ebuild but ask for all patches available through Debian
and ... it seems it could even replace 2.4.

snippet from
http://ftp.debian.org/debian/pool/main/f/flex/flex_2.5.31-31.diff.gz:

+Manoj Srivastava <srivasta@debian.org>
+arch-tag: d4250e44-a0e0-4ee0-adb9-2bd74f6eeb27
--- flex-2.5.31.orig/debian/NEWS.Debian
+++ flex-2.5.31/debian/NEWS.Debian
@@ -0,0 +1,41 @@
+flex (2.5.31-17) unstable; urgency=low
+
+ * This version of Flex is a major upgrade from previous versions. There
+   have been extensive changes. These changes require source changes in
+   lex input files.  The new direction flex is taking is to incorporate
+   new functionality at the expense of POSIX or backward
+   compatibility. This can also be considered part of the gcc migration
+   process; flex has been updated (the buggy, rickety set of patches
+   required to make it work with gcc was dumped in favour of a well
+   engineered upstream migration).
+ * Flex scanners are now reenterant, you may have multiple scanners in
+   the same program with differing sets of defaults, and they play
+   nicer with modern C and C++ compilers. C++ scanners are compatible
+   with recent c++ compilers (conform to ANSI C++, gcc 3.2), supports
+   bison variables yylval and yylloc. Some variables have been
+   renamed. Flex generates C99 defs now, see YY_TRADITIONAL_FUNC_DEFS.
+   yylineno is present in all scanners. yylineno is per-buffer in
+   reentrant scanners. Flex tries its best to output only the relevant
+   portions of the skeleton when generating a scanner, thus avoiding as
+   much conditional compilation as possible
+ * The signature of all functions has changed. flex has new command line
+   options, and option parsing has changed (now also supports POSIX
+   conventions optionally). Handles POSIXLY_CORRECT environment
+   variable.  Various i18n translations are included in the
+   distribution. Flex now works with recent bison versions.  The new
+   scanners do not polllute the global name space, and thus macros that
+   used to be available to user code are no longer present.
+ * Flex has gained an extensive new test suite run at build time to test
+   for regressions.
+ * The flip side is that Flex no longer conforms to the POSIX lex
+   behaviour, and the scanners require conforming implementations when
+   flex is used in ANSI C mode. Flex has broken backwards
+   compatibility. This is not a bug, but done deliberately, by
+   design. 
+ * The package flex-old provides the same behaviour as version 2.5.4a of
+   Flex.
+ * Please make sure you are prepared for these changes in Flex
+   before continuing with its upgrade.
+

flex-2.5.31 is in portage and has been for a long time

it's just hard masked because we've found it to break a ton of packages and
generally be kind of crappy

Debian has marked that as "stable" and has 31 patches collected for it
(2.5.31-31). I think it's worth the effort. How about talking with the Debian's
maintainer?

ive added 2.5.31 back into portage (it's still package.masked) with the Debian
patchset

it's going to need a lot of testing before i'll unmask it so inorder to help,
you should run `emerge world -e` and see if anything fails *because of flex*

Created an attachment (id=63076) [edit]
New flex with debian patch 32 ebuild.

Hi, I reported this in bug #33477
and flex 2.5.31 with debian patch 31 still failed in some programs.
And, flex is in debian stable now, yes, but gentoo compiles all programs while
not many of the debian users do that ;), so gentoo needs a lot of testing.
In fact, the last time I tried with debian patch 31 I only had a problem with
flex, but the problem was because of the program (amiwm) not flex, so perhaps in
a month or so 2.5.31 will be unmasked.

I added the latest ebuild I made for flex.

Please note that the ebuild uses two patches, the debian patch that downloads,
and another patch that it's in portage actually (in sys-devel/flex/files).

(From update of attachment 63076 [edit])
this ebuild is already in portage

Ok, sorry, I though the ebuild in portage had revision 31 of the debian patch
:-)

i looked through old bugs in bugzilla and the only package that jumped out at
me
as still being broken is unixODBC ...

the latest version is 2.2.11 and it is still broken ... however, i went through
their mailing lists and a dev mentioned that the issue is finally resolved in
cvs and the 2.2.12 release will have the fix

To correct myself in the original post, mod_pyhton-3.2.2b at least does not
absolutely require flex-2.5.31 and above to configure/compile:

checking for --with-flex... no
checking for flex... /usr/bin/flex
found /usr/bin/flex, we'll use this. Use --with-flex to specify another.
checking flex version... configure: WARNING: Flex version 2.5.4 found.
    Version 2.5.31 or greater is required.  You can generally ignore this
    warning unless you need to regenerate psp_parser.c from psp_parse.l.
    If you do need regenerate psp_parser.c, use --with-flex to specify the
    location of the correct flex version. See the README for more information.
configure: creating ./config.status

the only thing holding up flex 2.5.31 going into unstable is unixodbc

we either have to wait for a new version to be released, or we pester the
unixodbc gentoo maintainer to backport the required fixes to unixodbc 2.2.11

another package wich won't build with felx-2.5.31 is doxygen 1.4.5 with
doxywizard. fails with this error:

flex -PconfigYY -t config.l >config.cpp
gmake[1]: Entering directory
`/var/tmp/portage/doxygen-1.4.5/work/doxygen-1.4.5/addon/doxywizard'
g++ -c -pipe -Wall -W -fno-exceptions -g -I../../src -I/usr/qt/3/include -o
obj/config.o config.cpp
<stdout>: In function `void configYY_init_buffer(yy_buffer_state*, FILE*)':
<stdout>:2064: error: `isatty' undeclared (first use this function)
<stdout>:2064: error: (Each undeclared identifier is reported only once for
each function it appears in.)
make: *** [doxygen] Error 2

(In reply to comment #13)
Try changing the DEB_VER value in the ebuild from 34 to 36, digest, and
reemerge flex.

Vapier: Please, change this, because I think it could be useful for solving
some bugs. Concerning bug that blocks this it appears that rphillips (unixodbc
mantainer) has been with computer troubles since some time and it appears noone
it's mantaining the ebuild, so bug #110167 can not be solved Any chances on
this?
And, concerning bug #110169 nobody from desktop-wm replied (bug were opened two
months ago)

Thanks in advance.

tried it with DEB_VER 36 same error

well, it's easy enough to push the issues ... just unmask 2.5.31 ;)

updated deb ver to 36

doxygen-1.4.5 emerges just fine for me with 2.5.31

isatty() comes from unistd.h, not from flex

bigmichi1 portage # emerge -pv doxygen

These are the packages that I would merge, in order:

Calculating dependencies ...done!
[ebuild   R   ] app-doc/doxygen-1.4.5  USE="doc qt tetex unicode" 0 kB

Total size of downloads: 0 kB



with sys-devel/flex-2.5.4a-r6 doxygen emerges fine and with 2.5.31 it won't.
befor the flex command is run by make there are no references to isatty, so i
think this must be a problem with flex

please file a new bug about that

ive unmasked 2.5.31 now, thanks everyone for your help