Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 705800 - sys-devel/flex: cross compilation fails, stage1flex segfault - pointer truncation by implicit declaration
Summary: sys-devel/flex: cross compilation fails, stage1flex segfault - pointer trunca...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo's Team for Core System packages
URL: https://github.com/westes/flex/issues...
Whiteboard:
Keywords: PATCH
Depends on:
Blocks: clang-16-porting
  Show dependency tree
 
Reported: 2020-01-18 23:47 UTC by Andrew Aladjev
Modified: 2022-10-28 02:02 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
rpl malloc and realloc declarations (rpl_malloc_and_realloc_declarations.flex.patch,1.28 KB, patch)
2020-01-18 23:47 UTC, Andrew Aladjev
Details | Diff
build.log (original, reuploaded) (file_705800.txt,33.03 KB, text/plain)
2022-06-17 02:11 UTC, Sam James
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Andrew Aladjev 2020-01-18 23:47:06 UTC
Hello. I am working on cross compilation using musl toolchains. I've failed to cross compile sys-devel/flex. Please see build log here https://gist.github.com/andrew-aladev/d16e4c39c2f7940b86d08fbccfecdb17.

The reason of this segfault was well defined by Florian Weimer in the following post. https://developers.redhat.com/blog/2019/04/22/implicit-function-declarations-flexs-use-of-reallocarray/

I've investigated this bug and found that it is not related to flex, it is related to autotools itself. Autotools just adds "#define malloc rpl_malloc" and "#define realloc rpl_realloc" into config. It looks like nobody knows why and nobody will fix it https://lists.gnu.org/archive/html/automake/2003-05/msg00017.html. Autotools are abandoned and broken by design.

This issue can be fixed properly by core flex developer only by moving from autotools to other build system. But we need some fix for now. I will attach my proposed workaround here.

Thank you.

Reproducible: Always
Comment 1 Andrew Aladjev 2020-01-18 23:47:46 UTC
Created attachment 603698 [details, diff]
rpl malloc and realloc declarations
Comment 2 tt_1 2020-09-01 17:46:45 UTC
thank you so much for the patch, it really saved the day! 

if you want to, please try to push the patched ebuild to the overlay for the time being, thanks
Comment 3 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-06-17 02:10:55 UTC
1. In future, please attach build.logs here - not external sites, as they may expire.

2. I was wondering why it goes rogue given the fix for bug 628744, but the answer is there:
```
checking for GNU libc compatible malloc... no
configure: WARNING: result no guessed because of cross compilation
checking for stdlib.h... (cached) yes
checking for GNU libc compatible realloc... no
configure: WARNING: result no guessed because of cross compilation
```

I'd prefer to just give the autoconf tests the correct result (force it) than patch. I'll do that.
Comment 4 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-06-17 02:11:14 UTC
Created attachment 785822 [details]
build.log (original, reuploaded)
Comment 5 Larry the Git Cow gentoo-dev 2022-06-17 02:44:03 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/proj/crossdev.git/commit/?id=51dfd93c5122df670d2753070807e5a172727ded

commit 51dfd93c5122df670d2753070807e5a172727ded
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-06-17 02:39:46 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-06-17 02:43:42 +0000

    site: add linux-musl
    
    Follow-up to 8b1b79e02cc36bc6f2b5583db6aeea333fb8d59b.
    
    Sets
    ```
    ac_cv_func_malloc_0_nonnull=yes
    ac_cv_func_realloc_0_nonnull=yes
    ```
    ... to sort out flex and a bunch of other packages, like procps.
    
    I was wondering why this never seemed to come up when doing cross
    for other "standard" targets which we know work and are used often;
    the answer is that we never had the -musl suffix CHOSTs covered.
    
    (e.g. linux-gnu already has this set.)
    
    Bug: https://bugs.gentoo.org/554032
    Closes: https://bugs.gentoo.org/675368
    Closes: https://bugs.gentoo.org/705800
    See: 8b1b79e02cc36bc6f2b5583db6aeea333fb8d59b
    Signed-off-by: Sam James <sam@gentoo.org>

 wrappers/site/linux-musl | 4 ++++
 1 file changed, 4 insertions(+)
Comment 6 Larry the Git Cow gentoo-dev 2022-06-17 12:11:28 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=4d77781e2db7bb8bef7f390c43efc04ba58ff7bb

commit 4d77781e2db7bb8bef7f390c43efc04ba58ff7bb
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-06-17 02:23:18 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-06-17 12:10:06 +0000

    sys-devel/flex: fix segfault when cross-compiling
    
    Temporary workaround until new crossdev is tagged, stabled,
    and propagates.
    
    Closes: https://bugs.gentoo.org/705800
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-devel/flex/flex-2.6.4-r2.ebuild | 94 +++++++++++++++++++++++++++++++++++++
 1 file changed, 94 insertions(+)
Comment 7 Larry the Git Cow gentoo-dev 2022-06-17 18:44:43 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=8a1d005f7decdb50955cd653762f1b171a4e878e

commit 8a1d005f7decdb50955cd653762f1b171a4e878e
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-06-17 18:34:17 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-06-17 18:44:02 +0000

    sys-devel/crossdev: add 20220617
    
    Bug: https://bugs.gentoo.org/675368
    Bug: https://bugs.gentoo.org/554032
    Bug: https://bugs.gentoo.org/705800
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-devel/crossdev/Manifest                 |  1 +
 sys-devel/crossdev/crossdev-20220617.ebuild | 35 +++++++++++++++++++++++++++++
 sys-devel/crossdev/crossdev-99999999.ebuild |  2 +-
 3 files changed, 37 insertions(+), 1 deletion(-)
Comment 8 Larry the Git Cow gentoo-dev 2022-10-25 11:42:39 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=bec6a7be57fc3493dccef497b5d4b513c315f5f1

commit bec6a7be57fc3493dccef497b5d4b513c315f5f1
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-10-25 11:41:32 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-10-25 11:42:25 +0000

    sys-devel/flex: backport "config.h" fix too
    
    We already have the patch to config.h/configure.ac backported
    but we don't have the fix to actually *use* config.h everywhere,
    and clearly from bug #705800 it wasn't sufficient to just backport
    it (for reasons covered in the bug & upstream bug).
    
    Bug: https://bugs.gentoo.org/628744
    Bug: https://bugs.gentoo.org/705800
    Signed-off-by: Sam James <sam@gentoo.org>

 .../flex/files/flex-2.6.4-implicit-func-decl.patch | 50 ++++++++++++
 sys-devel/flex/flex-2.6.4-r3.ebuild                | 95 ++++++++++++++++++++++
 2 files changed, 145 insertions(+)
Comment 9 Larry the Git Cow gentoo-dev 2022-10-28 01:54:56 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=af4a94a1b5b778c0d983fbe6370b5ded4e6b54e1

commit af4a94a1b5b778c0d983fbe6370b5ded4e6b54e1
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-10-28 00:35:41 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-10-28 01:54:19 +0000

    sys-devel/flex: backport further macro fixes
    
    The gist is that the original glibc-2.26 patch was incomplete
    and didn't help with cross-compilation or non-glibc environments
    fully.
    
    There were a succession of further upstream commits
    now backported here. Noticed when doing some more Clang 16 work.
    
    This complements bec6a7be57fc3493dccef497b5d4b513c315f5f1 in that
    more effort is taken to ensure all necessary macros are defined
    so needed functions are available.
    
    Bug: https://bugs.gentoo.org/628744
    Bug: https://bugs.gentoo.org/705800
    See: bec6a7be57fc3493dccef497b5d4b513c315f5f1
    See: 4d77781e2db7bb8bef7f390c43efc04ba58ff7bb
    See: aa6c077d00c3a21b75ae646421aefadcad527f80
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-devel/flex/Manifest                            |   2 +-
 .../flex-2.6.4-fix-build-with-glibc2.26.patch      | 219 +++++++++++++++++++++
 .../{flex-2.6.4-r3.ebuild => flex-2.6.4-r4.ebuild} |  20 +-
 3 files changed, 233 insertions(+), 8 deletions(-)
Comment 10 Larry the Git Cow gentoo-dev 2022-10-28 02:02:46 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=812e4a387f47eac3a832690655cbceffa51aab18

commit 812e4a387f47eac3a832690655cbceffa51aab18
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-10-28 01:58:07 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-10-28 01:58:19 +0000

    sys-process/procps: drop obsolete cross malloc workaround
    
    We set the right bits in crossdev for musl targets now and, in theory, while
    unlikely, it is possible that some cross targets don't have the right behaviour
    anyway (not that i'm aware of any in use).
    
    Bug: https://bugs.gentoo.org/705800
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-process/procps/procps-3.3.17-r1.ebuild | 13 ++-----------
 1 file changed, 2 insertions(+), 11 deletions(-)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=51ce26ce6dd264cab35669505333ce7bea95b545

commit 51ce26ce6dd264cab35669505333ce7bea95b545
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-10-28 01:57:42 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-10-28 01:57:42 +0000

    sys-devel/flex: drop obsolete cross malloc workaround
    
    We set the right bits in crossdev for musl targets now and, in theory, while
    unlikely, it is possible that some cross targets don't have the right behaviour
    anyway (not that i'm aware of any in use).
    
    Bug: https://bugs.gentoo.org/705800
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-devel/flex/flex-2.6.4-r4.ebuild | 9 ---------
 1 file changed, 9 deletions(-)