Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 705970 - dev-lang/python: cross compilation using different libc is broken
Summary: dev-lang/python: cross compilation using different libc is broken
Status: UNCONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal minor (vote)
Assignee: Cross compilation support
URL: https://bugs.python.org/issue39399
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-01-20 19:24 UTC by Andrew Aladjev
Modified: 2020-01-20 19:47 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
epython for build patch (epython_for_build.patch,2.60 KB, patch)
2020-01-20 19:27 UTC, Andrew Aladjev
Details | Diff
x86_64-pc-linux-musl-python3.6 (x86_64-pc-linux-musl-python3.6,597 bytes, application/x-shellscript)
2020-01-20 19:29 UTC, Andrew Aladjev
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Andrew Aladjev 2020-01-20 19:24:48 UTC
Hello. I am implementing python cross compilation using "x86_64-pc-linux-musl" toolchain: "x86_64-pc-linux-musl-emerge -v1 python:3.6". Please see the following build log https://gist.github.com/andrew-aladev/e10fa5a8151ffb3c5782edd64ae08b28.

We can see the following part:

Traceback (most recent call last):
  File "/usr/x86_64-pc-linux-musl/tmp/portage/dev-lang/python-3.6.9/image//usr/lib/python3.6/compileall.py", line 17, in <module>
    import struct
  File "/usr/x86_64-pc-linux-musl/tmp/portage/dev-lang/python-3.6.9/image/usr/lib/python3.6/struct.py", line 13, in <module>
    from _struct import *
ImportError: libc.so: cannot open shared object file: No such file or directory

It means that cross compilation of python is not reliable today by design. Python is trying to use PYTHON_FOR_BUILD for loading cross compiled modules. It is not possible in general case. I will create a bug in upstream bugtracker.

I am not sure that this bug will be fixed in the nearest 5 years, so we need a good reliable workaround for today. I want to introduce double recompilation method:

1. x86_64-pc-linux-musl-emerge -v1 python:3.6
2. EPYTHON_FOR_BUILD="x86_64-pc-linux-musl-python3.6" x86_64-pc-linux-musl-emerge -v1 python:3.6

First step is to cross compile python with default broken "PYTHON_FOR_BUILD". Second step is to provide cross compiled python as "EPYTHON_FOR_BUILD". This method works perfect with both native execution (i686, amd64) and qemu user emulation (arm, mips, etc).

Unfortunately today python doesn't support "EPYTHON_FOR_BUILD". I will attach patch and "x86_64-pc-linux-musl-python3.6" file here.

PS the success of double recompilation method depends on "elf_mismatch_is_not_fatal.glibc-2.29.patch". It is provided here https://sourceware.org/bugzilla/show_bug.cgi?id=25341.

Please see here too https://github.com/andrew-aladev/test-images/blob/master/native/x86_64-pc-linux-musl/0-crossdev/buildah.sh. This image is not complete today, because it depends on other musl issues. But you can use it as complete example.


Reproducible: Always
Comment 1 Andrew Aladjev 2020-01-20 19:27:33 UTC
Created attachment 603822 [details, diff]
epython for build patch
Comment 2 Andrew Aladjev 2020-01-20 19:29:11 UTC
Created attachment 603824 [details]
x86_64-pc-linux-musl-python3.6