Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 471804 - dev-lang/python: cross compiling: target CFLAGS used when building host python
Summary: dev-lang/python: cross compiling: target CFLAGS used when building host python
Status: RESOLVED NEEDINFO
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Development (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Python Gentoo Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-05-30 13:48 UTC by Bertrand Jacquin
Modified: 2018-04-21 18:18 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge --info (info.log,10.10 KB, text/x-log)
2013-05-30 13:49 UTC, Bertrand Jacquin
Details
build.log (dev-lang:python-3.2.3-r2:20130530-133442.log,52.37 KB, text/x-log)
2013-05-30 13:50 UTC, Bertrand Jacquin
Details
build.log-with-PATCH (build.log,182.70 KB, text/x-log)
2013-05-30 15:16 UTC, Bertrand Jacquin
Details
/data/tmp/python-3.2.5-r1.ebuild-cross.diff (python-3.2.5-r1.ebuild-cross.diff,581 bytes, patch)
2013-07-07 22:38 UTC, Bertrand Jacquin
Details | Diff
Fixes wrong emake calls for CBUILD and CHOST directories, additionally fixes $(PGEN) calls for $CHOST (python-3.3.2-r1_crosscompile.patch,423 bytes, patch)
2013-08-11 09:13 UTC, Rafal Lalik
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Bertrand Jacquin 2013-05-30 13:48:29 UTC
src_configure() have a special case when cross compiling that call configure with any empty CC env :

   if tc-is-cross-compiler; then
                (
                multijob_child_init
                cd "${WORKDIR}"/${CBUILD} >/dev/null
                OPT="-O1" CFLAGS="" CPPFLAGS="" LDFLAGS="" CC="$(tc-getCC)" \
                "${S}"/configure \
                        --{build,host}=${CBUILD}

This make the build fail as following attachement

Reproducible: Always
Comment 1 Bertrand Jacquin 2013-05-30 13:49:53 UTC
Created attachment 349644 [details]
emerge --info
Comment 2 Bertrand Jacquin 2013-05-30 13:50:25 UTC
Created attachment 349646 [details]
build.log
Comment 3 Bertrand Jacquin 2013-05-30 13:56:41 UTC
Sorry, revious portion of src_configure was wrong, and was mine, but the log is correct, emptying CC make use the host compiler.

As fast patch would be :

   if tc-is-cross-compiler; then
                (
                multijob_child_init
                cd "${WORKDIR}"/${CBUILD} >/dev/null
-               OPT="-O1" CFLAGS="" CPPFLAGS="" LDFLAGS="" CC="" \
+               OPT="-O1" CFLAGS="" CPPFLAGS="" LDFLAGS="" CC="$(tc-getCC)" \
                "${S}"/configure \
                        --{build,host}=${CBUILD}
Comment 4 Bertrand Jacquin 2013-05-30 15:16:19 UTC
Created attachment 349660 [details]
build.log-with-PATCH

> As fast patch would be :
> 
>    if tc-is-cross-compiler; then
>                 (
>                 multijob_child_init
>                 cd "${WORKDIR}"/${CBUILD} >/dev/null
> -               OPT="-O1" CFLAGS="" CPPFLAGS="" LDFLAGS="" CC="" \
> +               OPT="-O1" CFLAGS="" CPPFLAGS="" LDFLAGS="" CC="$(tc-getCC)" \
>                 "${S}"/configure \
>                         --{build,host}=${CBUILD}

The following patch only delay the build error. Maybe linked to my build environnement. But in case, here is the log
Comment 5 SpanKY gentoo-dev 2013-05-30 17:45:29 UTC
(In reply to Bertrand Jacquin from comment #0)

tc-getCC is the wrong compiler.  that is the target, not the build.  that is why we clear CC so that configure can find the right one based on --build.

it looks like CFLAGS is being taken from the env.  i thought someone already reported this, but i can't seem to find it.
Comment 6 Bertrand Jacquin 2013-06-11 21:05:17 UTC
(In reply to SpanKY from comment #5)
> (In reply to Bertrand Jacquin from comment #0)
> 
> tc-getCC is the wrong compiler.  that is the target, not the build.  that is
> why we clear CC so that configure can find the right one based on --build.
> 
> it looks like CFLAGS is being taken from the env.  i thought someone already
> reported this, but i can't seem to find it.

Do you have any recommendation on how to fix this properly ?
Comment 7 SpanKY gentoo-dev 2013-06-18 21:30:23 UTC
(In reply to Bertrand Jacquin from comment #6)

python is cross-compiling fine for me as is, so i don't know what it'll take to fix things for you.  the configure invocation clears out the target settings and should write the results to the Makefile which is then used when `emake` is run later on.

looks to me like perhaps the build flags aren't being saved to the Makefile but instead area always pulled from the env.  you should check that, and perhaps try running in the tc-is-cross-compiler section of src_compile:
 CFLAGS=${BUILD_CFLAGS} emake ...
Comment 8 Bertrand Jacquin 2013-07-07 22:24:21 UTC
(In reply to SpanKY from comment #7)
> (In reply to Bertrand Jacquin from comment #6)
> 
> python is cross-compiling fine for me as is, so i don't know what it'll take
> to fix things for you.  the configure invocation clears out the target
> settings and should write the results to the Makefile which is then used
> when `emake` is run later on.
> 
> looks to me like perhaps the build flags aren't being saved to the Makefile
> but instead are always pulled from the env.  you should check that, and
> perhaps try running in the tc-is-cross-compiler section of src_compile:
>  CFLAGS=${BUILD_CFLAGS} emake ...

This is better is this case. The build fail but this is another topic (ac_cv_computed_gotos to be defined in my env as defaulted to no when cross-compiling and fail for i586).
Comment 9 Bertrand Jacquin 2013-07-07 22:38:29 UTC
Created attachment 352812 [details, diff]
/data/tmp/python-3.2.5-r1.ebuild-cross.diff

Fix that add recommendation from spanky and force ac_cv_computed_gotos=yes as configure set it to "no" by default when cross compiling which lead to error like :

i586-pc-linux-gnu-gcc -c  -DNDEBUG  -march=geode -Os -mmmx -m3dnow -fno-align-jumps -fno-align-functions -fno-align-labels -fno-align-loops -pipe -fomit-frame-pointer -fwrapv    -I. -IInclude -I/var/tmp/i586-pc-linux-gnu/portage/dev-lang/python-3.2.5-r1/work/Python-3.2.5/Include   -fPIC -DPy_BUILD_CORE -o Python/codecs.o /var/tmp/i586-pc-linux-gnu/portage/dev-lang/python-3.2.5-r1/work/Python-3.2.5/Python/codecs.c
/var/tmp/i586-pc-linux-gnu/portage/dev-lang/python-3.2.5-r1/work/Python-3.2.5/Python/ceval.c: In function 'PyEval_EvalFrameEx':
/var/tmp/i586-pc-linux-gnu/portage/dev-lang/python-3.2.5-r1/work/Python-3.2.5/Python/ceval.c:875:6: error: #error "Computed gotos are not supported on this compiler."
make: *** [Python/ceval.o] Error 1

This due to configure.ac :

AC_MSG_CHECKING(whether $CC supports computed gotos)
AC_CACHE_VAL(ac_cv_computed_gotos,
...
[ac_cv_computed_gotos=yes],
[ac_cv_computed_gotos=no],
[ac_cv_computed_gotos=no]))
Comment 10 Rafal Lalik 2013-08-11 09:12:11 UTC
(In reply to SpanKY from comment #7)
> python is cross-compiling fine for me as is, so i don't know what it'll take

Maybe I can explain this problem since I try to fix it since yesterday and I have partial solution.

In my case I cross-complie python for arm with following (simplified) variables:
CBUILD=x86_64 CHOST=arm CTARGET=arm
My PORTAGE_CONFIGROOT=/var/diskless/arm/image/ points to directory which keeps gentoo image for CHOST=arm, thus also flags is etc/make.conf are related to arm-gcc not x86_64-gcc

Problem is in src_compile() not in src_configure() (like it was suggested before). In this code:

src_compile() {
        if tc-is-cross-compiler; then
                cd "${WORKDIR}"/${CBUILD}
                # Disable as many modules as possible -- but we need a few to install.
                PYTHON_DISABLE_MODULES=$(
                        sed -n "/Extension('/{s:^.*Extension('::;s:'.*::;p}" "${S}"/setup.py | \
                                egrep -v '(unicodedata|time|cStringIO|_struct|binascii)'
                ) \
                PTHON_DISABLE_SSL="1" \
                SYSROOT= \
                emake || die "cross-make failed"
                # See comment in src_configure about these.
                ln python ../${CHOST}/hostpython || die
                ln Parser/pgen ../${CHOST}/Parser/hostpgen || die
                ln -s ../${CBUILD}/build/lib.*/ ../${CHOST}/hostpythonpath || die
        fi

        cd "${WORKDIR}"/${CHOST}
        emake CPPFLAGS="" CFLAGS="" LDFLAGS="" || die "emake failed"
#...
}


problem is with 'SYSROOT= emake || die "cross-make failed"' call, which is executed in ${WORKDIR}${SBUILD} dir whereas it takes CFLAGS from the wrong PORTAGE_CONFIGROOT (in this case for arm which is $CTARGET). So at this step there is high risk that you will pass wrong CFLAGS (for example -march=armv6zk -mfpu=vfp -mfloat-abi=hard) what will lead to fail on $CBUILD-gcc. What more, in the line emake CPPFLAGS="" CFLAGS="" LDFLAGS="" || die "emake failed" where you run the cross-compilator arm-gcc you actually clear flags which should be passed there. So this two lines should be fixed.

I proposed some patch (attached). This patch additionally fixes problem with wrong pgen call. After make starts, pgen is executed to generate headers, but it always executes pgen from Python/pgen which in ${WORKDIR}/${CHOST}/ for CHOST=arm is wrong of course (wrong binary). Thus sed replaces calls of $(PGEN) to ${WORKDIR}/${CBUILD}/$(PGEN) in all Makefiles, which is fine.

It is important to notice that all this problems will not appear when you build python for toolchain: CBUILD==CHOST, it arise only when you build binaries for target host.

I based my patch on python-3.3.2-r1.ebuild
Comment 11 Rafal Lalik 2013-08-11 09:13:20 UTC
Created attachment 355658 [details, diff]
Fixes wrong emake calls for CBUILD and CHOST directories, additionally fixes $(PGEN) calls for $CHOST
Comment 12 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2018-04-21 18:18:35 UTC
Is this still relevant today?