python cross compilation aware ebuilds Reproducible: Always Steps to Reproduce: 1. emerge-cross python 2. emerge-cross dev-lang/<any-package> Actual Results: python doesn't cross compile (2.6) or python can't cross compile any c extension (python distutils bugs) Expected Results: python should cross compile, and successfully cross compile any others modules
Created attachment 190575 [details, diff] python 2.5.2-r8 ebuild patch
Created attachment 190577 [details, diff] python 2.5.4-r2 ebuild patch
Created attachment 190579 [details, diff] python 2.6.2 ebuild patch
Created attachment 190580 [details, diff] python 2.5.2 cross enable ctypes Compilation of the ctypes extension fails for archs that do not have FFI_CLOSURES. This fixes the compilation and makes ctypes also available to those archs.
Created attachment 190582 [details, diff] python 2.5.2 cross setup sysroot Change setup.py to respect the SYSROOT environment variable
Created attachment 190583 [details, diff] python 2.5.2 cross Makefile.pre.in patch Add a libdirname to enable python cross-compiling and add a user specifyable libdir
Created attachment 190585 [details, diff] python 2.5.2 cross distutils Make distutils cross-compile aware (1/3)
Created attachment 190587 [details, diff] python 2.5.2 cross distuils sysconfig Make distutils cross-compile aware (2/3)
Created attachment 190588 [details, diff] python 2.5.2 crpss distutils install patch Make distutils cross-compile aware (3/3)
Created attachment 190590 [details, diff] python 2.6 cross setup sysroot patch like "python 2.5.2 cross setup sysroot", but reworked for 2.6
Created attachment 190591 [details, diff] python 2.6 enable ctypes cross build like "python 2.5.2 cross enable ctypes", but reworked for 2.6
Created attachment 190592 [details, diff] python 2.6 cross distutils sysconfig like "python 2.5.2 cross distuils sysconfig", but reworked for 2.6
Mainly all this patches created by sleipnir(OdinHorse), I just reworked them for python 2.6. All this 3 python versions was tested by Gentoo Openmoko, and work well, plz say us, if patches should be reworked before go upstream. If you have any questions, plz found me (max_posedon) or sleipnir(OdinHorse) in #gentoo-openmoko@freenode.net. (I also will be on #gentoo-python for any discussions)
python-2.6 cross-compiled fine for me (ignoring the normal module issues that all versions of python have)
Created attachment 190639 [details] python 2.6.2 build log cross compilation fails (In reply to comment #14) > python-2.6 cross-compiled fine for me (ignoring the normal module issues that > all versions of python have) > fails for me during install, with (from build.log) Compiling /mnt/floppy/data/gentoo/tmp/portage/dev-lang/python-2.6.2/image//usr/lib/python2.6/json/tests/test_unicode.py ... SyntaxError: ("(unicode error) \\N escapes not supported (can't load unicodedata module)", ('/usr/lib/python2.6/json/tests/test_unicode.py', 8, None, "u = u'\\N{GREEK SMALL LETTER ALPHA}\\N{GREEK CAPITAL LETTER OMEGA}'\n")) Which is fixed by, # Ugly fix, hostpython require few modules to pass test_unicode.py test cp -v /usr/lib/python2.6/lib-dynload/unicodedata.so Modules/. cp -v /usr/lib/python2.6/lib-dynload/time.so Modules/. cp -v /usr/lib/python2.6/lib-dynload/math.so Modules/. but of course I'll be lucky know another way to fix it. One more point, now I understand that I shouldn't create such big ticket, it should be splitted in many, because of many different topics in python should be fixed. And I don't agree that module issue is "normal"...
Hello, Maksim, are you having this issues with py2.5 as well? or just 2.6?. Can you post your emerge --info and tell me what locale are you using? Can somebody else from embedded confirm this bug? I have heard solar sometime ago telling us python2.6 was cross compiling without any issue. Thanks and best regards,
Jesus, the problem is _not_ to cross-compile python itself (in that way the topic is misleading). The problem addressed here is to cross-compile the modules and extensions. I (as one of the patch authors) will spilt this bug up into nice topic chunks and open separate bugs. Should I link them here, to have a single place tracking this? And btw. this problem exists for 2.5 and 2.6 as well. We tested this now for quite some time in the openmoko overlay and it works well. There is one open issue I discussed with solar, but I will mention it on the right ticket.
The ctypes enable patches now have their own bug #269098 as it is a completely different story. Please mark them obsolete here.
Created attachment 190742 [details] null
The patches to make the python modules cross-compile now have their own bug #269111. Please mark them obsolete here (python-*-cross-setup-sysroot).
Please report it to Python upstream [1] and we will apply the version of the patch accepted by upstream. [1] http://bugs.python.org/
Created attachment 192741 [details] remove obsolete patches, because of separate tickets for them bug #269111
This stuff should go upstream first, closing to that effect.
djc, Please do not close bugs which you are not a member of that team that are marked with an importance of High. A lot of valuable time has been spent on these patches and have made attempts to work with upstream. This place holder should remain open till the python and it's submodules correctly work @ gentoo.
Ah, sorry.
Created attachment 311657 [details] python 2.7.3-r2 build log Just not to create duplicate bug: I'm trying to cross-compile python 2.7 and 3.2 for armv6z target and it fails. emerge-armv6z-hardfloat-linux-gnueabi --info: Portage 2.2.0_alpha102_p5 (default/linux/arm/10.0/armv6j, gcc-4.5.3, unavailable, 3.3.2-gentoo x86_64) ================================================================= System uname: Linux-3.3.2-gentoo-x86_64-Intel-R-_Pentium-R-_CPU_G630T_@_2.30GHz-with-gentoo-2.1 Timestamp of tree: Sun, 13 May 2012 17:45:01 +0000 ccache version 3.1.7 [disabled] app-shells/bash: 4.2_p24 dev-java/java-config: 2.1.11-r3 dev-lang/python: 2.6.7-r2, 2.7.2-r3, 3.1.4-r3, 3.2.2 dev-util/ccache: 3.1.7 dev-util/cmake: 2.8.7-r5 dev-util/pkgconfig: 0.26 sys-apps/baselayout: 2.1 sys-apps/openrc: 0.9.9.3 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.68 sys-devel/automake: 1.9.6-r3, 1.11.3 sys-devel/binutils: 2.22-r1 sys-devel/gcc: 4.4.4-r2, 4.5.3-r2 sys-devel/gcc-config: 1.5.1-r1 sys-devel/libtool: 2.4.2 sys-devel/make: 3.82-r3 sys-kernel/linux-headers: 3.2-r1 (virtual/os-headers) sys-libs/glibc: 2.14.1-r2 Repositories: gentoo bircoph alexxy betagarden piczu xhub rion slyfox-gentoo nrndda Local Installed sets: ACCEPT_KEYWORDS="arm ~arm" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-Os -pipe -fomit-frame-pointer -I/usr/armv6z-hardfloat-linux-gnueabi/usr/include/ -I/usr/armv6z-hardfloat-linux-gnueabi/include/" CHOST="armv6z-hardfloat-linux-gnueabi" CONFIG_PROTECT="/etc /var/bind /var/lib/redmine/config" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/apache2-php5.4/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cgi-php5.4/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/php/cli-php5.4/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /var/lib/redmine/config/locales /var/lib/redmine/config/settings.yml" CXXFLAGS="-Os -pipe -fomit-frame-pointer -I/usr/armv6z-hardfloat-linux-gnueabi/usr/include/ -I/usr/armv6z-hardfloat-linux-gnueabi/include/" DISTDIR="/usr/portage/distfiles" FEATURES="assume-digests binpkg-logs buildpkg distlocks ebuild-locks fixlafiles news nodoc noinfo noman parallel-fetch parse-eapi-ebuild-head preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch" FFLAGS="" GENTOO_MIRRORS="http://distfiles.gentoo.org http://trumpetti.atm.tut.fi/gentoo/ ftp://trumpetti.atm.tut.fi/gentoo/ ftp://ftp.tu-clausthal.de/pub/linux/gentoo/ ftp://files.gentoo.org http://files.gentoo.org " LANG="ru_RU.UTF-8" LDFLAGS="-L/usr/armv6z-hardfloat-linux-gnueabi/lib -L/usr/armv6z-hardfloat-linux-gnueabi/usr/lib" MAKEOPTS="-j5" PKGDIR="/usr/armv6z-hardfloat-linux-gnueabi/packages/" PORTAGE_CONFIGROOT="/usr/armv6z-hardfloat-linux-gnueabi/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/usr/armv6z-hardfloat-linux-gnueabi/tmp/" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/var/lib/layman/bircoph /var/lib/layman/alexxy /var/lib/layman/betagarden /var/lib/layman/piczu /var/lib/layman/xhub /var/lib/layman/rion /var/lib/layman/slyfox /usr/local/nrndda-overlay /usr/local/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="arm bzip2 cli cracklib crypt cups cxx fortran gdbm gpm iconv ipv6 modules mudflap ncurses nls nptl openmp pam pcre pppd readline session ssl tcpd unicode xorg zlib" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="fbdev omapfb dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
Created attachment 311659 [details] python 3.2.2-r1 build log Build log for python-3.2.3-r1.
I've managed to get past first errors with python-2.7.3 with ugly hack: sed -i 's#\(\#endif\)$#\1\n\#define PY_FORMAT_LONG_LONG#' "${S}"/pyconfig.h Then I'm got next couple of errors: accidently -I/usr/include is passed for armv6z-hardfloat-linux-gnueabi-gcc. I've worked it around with another ugly hack (for fgrep <...>; do sed <...>), then I've got another error - hostpython segfaults, but runs flawlessly under gdb, coredump is unreadable.
(In reply to comment #28) > I've managed to get past first errors with python-2.7.3 with ugly hack: > sed -i 's#\(\#endif\)$#\1\n\#define PY_FORMAT_LONG_LONG#' > "${S}"/pyconfig.h This is an easier way around it... ac_cv_have_long_long_format=yes emerge python
So, nobody bothered to report latest SIGSEGV upstream, right?
(In reply to comment #30) this is about making python cross-compile, not it crashing
Is there a relation to bug #413863?
*** Bug 413863 has been marked as a duplicate of this bug. ***
*** Bug 511476 has been marked as a duplicate of this bug. ***
It's worth pointing out that this is almost resolved now. The main thing missing is one or two entries like ac_cv_file__dev_ptmx being added to crossdev. Recent version of both Python 2 and 3 cross-compile out of the box. The other remaining issue is a trickier one. A copy of Python is needed on /. If pretty much any version worked then we could just imply the dependency through @system. Unfortunately it's a little fussier than that. Python 3.4 needs at least 3.3, for example. The check is made very early in the configure script. It doesn't matter which version you have eselected. This is tricky because it's a job for HDEPEND, which is going nowhere fast in bug #317337.
*** Bug 534706 has been marked as a duplicate of this bug. ***
Apart from a new issue in 3.7.0 that I will address separately, I believe this has been resolved for some time now.
