I have a /usr/arm-unknown-linux-gnueabihf cross compilation environment on CBUILD=x86_64-pc-linux-gnu. Building python extensions using arm-unknown-linux-gnueabihf-emerge wrapper just messes up. Problems: 1) As invoking CBUILD's python (do not officially support cross compilation between ARCHes from my research), It installes packages into /usr/${CTARGET}/usr/lib64 because CBUILD has a 64-bit arch. The CTARGET environment will fail importing new installed packages since their sys.path just do not have lib64 directories (32-bit arch). 2) For packages containing extension modules written in C/C++: a) INC dir is wrong making #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)." b) LIB dir is wrong making linking complains. c) Both a) and b) can be somehow easily dealed with CFLAGS/LDFLAGS. But on python3, C modules' file name is <module>.<python_impl>.so and python_impl contains arch, system, abi info. Those info is taken from CBUILD env, leading to make a xattr.cpython-35m-x86_64-linux-gnu.so named file an ARM architecture ELF. I suggest dening cross compilation by checking CBUILD == CHOST in distutils-r1.eclass as they are not working now and adding supports is complex (some package uses python-config to retrive *FLAGS)
Chewi used to try to get some cross support in but I don't think he has anything working with current eclasses at the moment.
*** Bug 653908 has been marked as a duplicate of this bug. ***
*** Bug 653910 has been marked as a duplicate of this bug. ***
I have managed to mostly work around #1 before as discussed in bug #503874 but my methods broke following changes to eclasses. I have spent time on #2 but never found a way forwards. Python just makes this extremely difficult for us. I'll revisit this stuff once we've got EAPI 7 out of the way.
(In reply to James Le Cuirot from comment #4) > I have managed to mostly work around #1 before as discussed in bug #503874 > but my methods broke following changes to eclasses. > > I have spent time on #2 but never found a way forwards. Python just makes > this extremely difficult for us. > > I'll revisit this stuff once we've got EAPI 7 out of the way. James, I am trying to cross compile a Gentoo ARM system and am running into this problem as well. In my case, problem 2a) is breaking my compile for pyblake an pyxattr...both dependencies of portage. After much struggle, many patches and bug reports, I have a booting system. But at this point I can't build portage because of breaking pyxattr and pyblake. Without portage, I can't install new binpkgs and instead have to rebuild an image any time I actually successfully get some new package to cross compile. I am starting to realize this is an excercise in futility, at least until this issue is resolved. Even if I can hack my way into getting pyblake, pyxattr and portage installed, I expect I will face similar problems getting gentoolkit and eix installed. Do you have any idea what the time frame is for you (or or anyone else) revisiting this issue?
(In reply to Joe Harvell from comment #5) > I am trying to cross compile a Gentoo ARM system and am running into this > problem as well. In my case, problem 2a) is breaking my compile for pyblake > an pyxattr...both dependencies of portage. Yes, I had forgotten about the new problem that is pyblake. pyxattr can be avoided with USE=-xattr but pyblake is not optional. Even if we can't fix Python generally, we need to at least have some solution for this case. Timescales are difficult with me but this is quite high on the list. Hopefully I can get to it sometime in May.
pyblake2 is needed only for python < 3.6.
In case this helps with your efforts on the distutils-r1.... I followed the procedure on this web page (http://whatschrisdoing.com/blog/2009/10/16/cross-compiling-python-extensions/) to manually build a specific python library. For the setup.py install I did "python2.7 setup.py install --install-lib=/usr/lib/python2.7/site-packages --root=<some-local-root-dir>". I then copied from <some-local-root-dir> to the running system and it worked. In my case the python application using the library only supports Python 2.7. Like you said, I noticed the architecture names for the build arch in the libraries produced by setup.py build for Python 3.5. Seems like something has to go upstream to make this work long term.
To clarify, these are the exact steps I took to build: joey@akita adafruit-beaglebone-io-python$ export CC="armv7a-pip-linux-gnueabi-gcc" joey@akita adafruit-beaglebone-io-python$ export CFLAGS="-march=armv7-a -mtune=cortex-a8 -mfpu=neon-vfpv3 -mfloat-abi=hard -Os -pipe -fomit-frame-pointer -ftree-vectorize -flto=10 -I/usr/armv7a-pip-linux-gnueabi/usr/include/python3.5m" joey@akita adafruit-beaglebone-io-python$ export LDSHARED="armv7a-pip-linux-gnueabi-gcc -shared" joey@akita adafruit-beaglebone-io-python$ export LDFLAGS="-flto=10 -fuse-ld=gold -fuse-linker-plugin -L/usr/armv7a-pip-linux-gnueabi/usr/lib -L/usr/armv7a-pip-linux-gnueabi/usr/lib/python3.5m -L/usr/armv7a-pip-linux-gnueabi/lib" joey@akita adafruit-beaglebone-io-python$ python3.5 setup.py build --parallel 10 [build successful] joey@akita adafruit-beaglebone-io-python$ export CFLAGS="-march=armv7-a -mtune=cortex-a8 -mfpu=neon-vfpv3 -mfloat-abi=hard -Os -pipe -fomit-frame-pointer -ftree-vectorize -flto=10 -I/usr/armv7a-pip-linux-gnueabi/usr/include/python2.7" joey@akita adafruit-beaglebone-io-python$ python2.7 setup.py build [build successful] joey@akita adafruit-beaglebone-io-python$ python2.7 setup.py install --install-lib=/usr/lib/python2.7/site-packages --root=~/pip/hack/root-overlay [install successful] For your reference: joey@akita adafruit-beaglebone-io-python$ find build build build/temp.linux-x86_64-2.7 build/temp.linux-x86_64-2.7/source build/temp.linux-x86_64-2.7/source/c_pinmux.o build/temp.linux-x86_64-2.7/source/event_gpio.o build/temp.linux-x86_64-2.7/source/py_gpio.o build/temp.linux-x86_64-2.7/source/c_adc.o build/temp.linux-x86_64-2.7/source/py_pwm.o build/temp.linux-x86_64-2.7/source/constants.o build/temp.linux-x86_64-2.7/source/py_uart.o build/temp.linux-x86_64-2.7/source/spimodule.o build/temp.linux-x86_64-2.7/source/py_adc.o build/temp.linux-x86_64-2.7/source/c_uart.o build/temp.linux-x86_64-2.7/source/c_pwm.o build/temp.linux-x86_64-2.7/source/common.o build/temp.linux-x86_64-3.5 build/temp.linux-x86_64-3.5/source build/temp.linux-x86_64-3.5/source/c_pinmux.o build/temp.linux-x86_64-3.5/source/event_gpio.o build/temp.linux-x86_64-3.5/source/py_gpio.o build/temp.linux-x86_64-3.5/source/c_adc.o build/temp.linux-x86_64-3.5/source/py_pwm.o build/temp.linux-x86_64-3.5/source/constants.o build/temp.linux-x86_64-3.5/source/py_uart.o build/temp.linux-x86_64-3.5/source/spimodule.o build/temp.linux-x86_64-3.5/source/py_adc.o build/temp.linux-x86_64-3.5/source/c_uart.o build/temp.linux-x86_64-3.5/source/c_pwm.o build/temp.linux-x86_64-3.5/source/common.o build/lib.linux-x86_64-2.7 build/lib.linux-x86_64-2.7/Adafruit_I2C.py build/lib.linux-x86_64-2.7/overlays build/lib.linux-x86_64-2.7/overlays/__init__.py build/lib.linux-x86_64-2.7/overlays/builder.py build/lib.linux-x86_64-2.7/Adafruit_BBIO build/lib.linux-x86_64-2.7/Adafruit_BBIO/Encoder.py build/lib.linux-x86_64-2.7/Adafruit_BBIO/__init__.py build/lib.linux-x86_64-2.7/Adafruit_BBIO/SPI.so build/lib.linux-x86_64-2.7/Adafruit_BBIO/UART.so build/lib.linux-x86_64-2.7/Adafruit_BBIO/ADC.so build/lib.linux-x86_64-2.7/Adafruit_BBIO/sysfs.py build/lib.linux-x86_64-2.7/Adafruit_BBIO/GPIO.so build/lib.linux-x86_64-2.7/Adafruit_BBIO/PWM.so build/lib.linux-x86_64-3.5 build/lib.linux-x86_64-3.5/Adafruit_I2C.py build/lib.linux-x86_64-3.5/overlays build/lib.linux-x86_64-3.5/overlays/__init__.py build/lib.linux-x86_64-3.5/overlays/builder.py build/lib.linux-x86_64-3.5/Adafruit_BBIO build/lib.linux-x86_64-3.5/Adafruit_BBIO/Encoder.py build/lib.linux-x86_64-3.5/Adafruit_BBIO/UART.cpython-35m-x86_64-linux-gnu.so build/lib.linux-x86_64-3.5/Adafruit_BBIO/__init__.py build/lib.linux-x86_64-3.5/Adafruit_BBIO/GPIO.cpython-35m-x86_64-linux-gnu.so build/lib.linux-x86_64-3.5/Adafruit_BBIO/SPI.cpython-35m-x86_64-linux-gnu.so build/lib.linux-x86_64-3.5/Adafruit_BBIO/PWM.cpython-35m-x86_64-linux-gnu.so build/lib.linux-x86_64-3.5/Adafruit_BBIO/ADC.cpython-35m-x86_64-linux-gnu.so build/lib.linux-x86_64-3.5/Adafruit_BBIO/sysfs.py joey@akita adafruit-beaglebone-io-python$ find ~/pip/hack/root-overlay /home/joey/pip/hack/root-overlay /home/joey/pip/hack/root-overlay/usr /home/joey/pip/hack/root-overlay/usr/lib /home/joey/pip/hack/root-overlay/usr/lib/python2.7 /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/Adafruit_I2C.py /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/Adafruit_I2C.pyc /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/overlays /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/overlays/__init__.pyc /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/overlays/__init__.py /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/overlays/builder.pyc /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/overlays/builder.py /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/Adafruit_BBIO /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/Adafruit_BBIO/__init__.pyc /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/Adafruit_BBIO/Encoder.py /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/Adafruit_BBIO/__init__.py /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/Adafruit_BBIO/SPI.so /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/Adafruit_BBIO/UART.so /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/Adafruit_BBIO/sysfs.pyc /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/Adafruit_BBIO/ADC.so /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/Adafruit_BBIO/sysfs.py /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/Adafruit_BBIO/GPIO.so /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/Adafruit_BBIO/PWM.so /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/Adafruit_BBIO/Encoder.pyc /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/Adafruit_BBIO-1.0.10-py2.7.egg-info /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/Adafruit_BBIO-1.0.10-py2.7.egg-info/dependency_links.txt /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/Adafruit_BBIO-1.0.10-py2.7.egg-info/PKG-INFO /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/Adafruit_BBIO-1.0.10-py2.7.egg-info/SOURCES.txt /home/joey/pip/hack/root-overlay/usr/lib/python2.7/site-packages/Adafruit_BBIO-1.0.10-py2.7.egg-info/top_level.txt
Some good news. I've been working heavily on this and have made very good progress. A few tweaks to the eclass have allowed me to cross-build most distutils-based packages for Python 2 and 3, usually with no changes. Tested packages with native extensions include: dev-python/lxml dev-python/markupsafe dev-python/msgpack dev-python/netifaces dev-python/numpy dev-python/psutil dev-python/pyalsa dev-python/pyblake2 dev-python/pycurl dev-python/regex dev-python/html5-parser (needs fix to respect pkg-config) dev-python/pillow (needs slight patching) sys-apps/portage Things fell apart when I tried dev-python/dbus-python, which uses autotools rather than distutils. This partly stems from it respecting the PYTHON variable rather than finding python on the PATH. python-utils-r1 sets up wrappers in the PATH and it is now crucial that these wrappers are used. The PYTHON variable bypasses this. I'll have a chat with mgorny as soon as possible to work this out.
This has basically been fixed for a while now. Better still, my earlier monstrosity wasn't needed. :)
