Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 269111 - Make python modules cross-compile
Summary: Make python modules cross-compile
Status: RESOLVED UPSTREAM
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Development (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Python Gentoo Team
URL:
Whiteboard:
Keywords:
: 279765 300484 (view as bug list)
Depends on:
Blocks:
 
Reported: 2009-05-09 11:25 UTC by Sven 'sleipnir' Rebhan
Modified: 2012-02-16 13:19 UTC (History)
7 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Respect a sysroot env variable in setup.py (python-2.5.2-cross-setup-sysroot.patch,11.11 KB, patch)
2009-05-09 11:27 UTC, Sven 'sleipnir' Rebhan
Details | Diff
Respect a sysroot env variable in setup.py (python-2.6-cross-setup-sysroot.patch,10.51 KB, patch)
2009-05-09 11:28 UTC, Sven 'sleipnir' Rebhan
Details | Diff
Patch for the 2.5.4-r2 ebuild. (python-2.5.4-r2.ebuild.patch,576 bytes, patch)
2009-05-09 11:29 UTC, Sven 'sleipnir' Rebhan
Details | Diff
Patch for the 2.6.2 ebuild. (python-2.6.2.ebuild.patch,560 bytes, patch)
2009-05-09 11:30 UTC, Sven 'sleipnir' Rebhan
Details | Diff
fixed patch for python-2.6.5-r3 (python-2.6-cross-setup-sysroot.patch,7.19 KB, patch)
2010-07-16 22:32 UTC, NickNill
Details | Diff
altinstall build log fail (build.log,506.09 KB, text/plain)
2010-11-10 03:08 UTC, Leho Kraav (:macmaN @lkraav)
Details
got past altinstall fail (build.log,658.55 KB, text/plain)
2010-11-18 18:15 UTC, Leho Kraav (:macmaN @lkraav)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sven 'sleipnir' Rebhan 2009-05-09 11:25:33 UTC
Python itself can be cross-compiled, but its modules can not. This is due to the fact that in setup.py the host includes and libs are used. By applying the following patches we use a newly introduced environment variable called PYTHON_SYSROOT to find the cross-includes and cross-libs. All credits for the patch of 2.6.2 go to Maksim 'max_posedon' Melnikau.

This is part of bug #268887 which should be splitted up here, so please do not mark this bug as duplicate!

Reproducible: Always
Comment 1 Sven 'sleipnir' Rebhan 2009-05-09 11:27:35 UTC
Created attachment 190749 [details, diff]
Respect a sysroot env variable in setup.py

This patch enables the cross-compilation of the build-in python modules. Tested quite some time in the openmoko overlay.
Comment 2 Sven 'sleipnir' Rebhan 2009-05-09 11:28:12 UTC
Created attachment 190750 [details, diff]
Respect a sysroot env variable in setup.py 

This patch enables the cross-compilation of the build-in python modules. Tested quite some time in the openmoko overlay.
Comment 3 Sven 'sleipnir' Rebhan 2009-05-09 11:29:21 UTC
Created attachment 190751 [details, diff]
Patch for the 2.5.4-r2 ebuild.

Applies the patch for 2.5.4-r2 if cross-compiling.
Comment 4 Sven 'sleipnir' Rebhan 2009-05-09 11:30:04 UTC
Created attachment 190753 [details, diff]
Patch for the 2.6.2 ebuild. 

Applies the patch for 2.6.2 if cross-compiling.
Comment 5 solar (RETIRED) gentoo-dev 2009-05-09 14:54:08 UTC
Reassigning to python for review.
Comment 6 Robert Piasek (RETIRED) gentoo-dev 2009-05-27 11:58:34 UTC
Hi,

Sorry to chase, but have any of you had a chance to look at it?


Thanks,
Rob
Comment 7 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2009-05-28 15:25:08 UTC
Please report it to Python upstream [1] and we will apply the version of the patch accepted by upstream.

[1] http://bugs.python.org/
Comment 8 SpanKY gentoo-dev 2009-08-14 07:31:07 UTC
*** Bug 279765 has been marked as a duplicate of this bug. ***
Comment 9 Christopher Friedt 2009-09-03 12:19:59 UTC
I'm glad this is being investigated - I thought I was going crazy when 'emerge' would bail after I cross compiled python-2.6.2-r1. 

C
Comment 10 NickNill 2010-07-16 22:32:56 UTC
Created attachment 239099 [details, diff]
fixed patch for python-2.6.5-r3
Comment 11 Dirkjan Ochtman gentoo-dev 2010-10-27 12:17:09 UTC
*** Bug 300484 has been marked as a duplicate of this bug. ***
Comment 12 Leho Kraav (:macmaN @lkraav) 2010-11-10 02:56:39 UTC
is this bug why i can't emerge python-2.6.5-r3? none of the patches help. have also tried -j(1-4).

\/ server:2537 /usr/i686-gentoo-linux-uclibc $ i686-gentoo-linux-uclibc-emerge --info
Portage 2.1.8.3 (uclibc/x86, gcc-4.5.1, unavailable, 2.6.34-tuxonice-r4 i686)
=================================================================
System uname: Linux-2.6.34-tuxonice-r4-i686-Intel-R-_Core-TM-_i5_CPU_750_@_2.67GHz-with-gentoo-2.0.1
Timestamp of tree: Mon, 08 Nov 2010 02:00:01 +0000
distcc 3.1 i686-pc-linux-gnu [disabled]
ccache version 2.4 [disabled]
app-shells/bash:     4.1_p7
dev-java/java-config: 2.1.11
dev-lang/python:     2.4.6, 2.5.4-r2, 2.6.4
dev-util/ccache:     2.4-r7
dev-util/cmake:      2.6.4
sys-apps/baselayout: 2.0.1
sys-apps/openrc:     0.6.3
sys-apps/sandbox:    1.6-r2
sys-devel/autoconf:  2.13, 2.63
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.2, 1.11.1
sys-devel/binutils:  2.20.1-r1
sys-devel/gcc:       4.3.2-r3
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6b
sys-devel/make:      3.81
virtual/os-headers:  2.6.34
ACCEPT_KEYWORDS="x86 ~x86"
ACCEPT_LICENSE="* -@EULA"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-Os -pipe -mtune=i686 -fomit-frame-pointer -I/usr/i686-gentoo-linux-uclibc/usr/include/ -I/usr/i686-gentoo-linux-uclibc/include/"
CHOST="i686-gentoo-linux-uclibc"
CONFIG_PROTECT="/etc /opt/openfire/resources/security/"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/eselect/postgresql /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php
/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d
 /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-Os -pipe -mtune=i686 -fomit-frame-pointer -I/usr/i686-gentoo-linux-uclibc/usr/include/ -I/usr/i686-gentoo-linux-uclibc/include/"
DISTDIR="/mnt/datapool/gentoo/distfiles"
FEATURES="assume-digests autoconfig buildpkg distlocks fixpackages news nodoc noinfo noman parallel-fetch protect-owned sandbox sfperms strict strip unmerge-log
s unmerge-orphans userfetch"
GENTOO_MIRRORS="http://trumpetti.atm.tut.fi/gentoo/ ftp://trumpetti.atm.tut.fi/gentoo/"
LANG="et_EE.utf8"
LC_ALL="et_EE.utf8"
LDFLAGS="-L/usr/i686-gentoo-linux-uclibc/lib -L/usr/i686-gentoo-linux-uclibc/usr/lib"
MAKEOPTS="-j4"
PKGDIR="/usr/i686-gentoo-linux-uclibc/packages/"
PORTAGE_CONFIGROOT="/usr/i686-gentoo-linux-uclibc/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclu
de=/local --exclude=/packages"
PORTAGE_TMPDIR="/usr/i686-gentoo-linux-uclibc/tmp/"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/mnt/datapool/gentoo/overlays/layman/leho /var/lib/layman/x11 /var/lib/layman/sunrise /var/lib/layman/arcon /var/lib/layman/thelinux /var/lib/l
ayman/jokey /var/lib/layman/vmware /var/lib/layman/dev-zero /var/lib/layman/oforge /var/lib/layman/doki_pen /var/lib/layman/leho /var/lib/layman/ohnobinki_overl
ay /var/lib/layman/belak.gentoo /var/lib/layman/performous /var/lib/layman/gnr /var/lib/layman/sci"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="cli crypt cxx dri minimal modules mudflap ncurses openmp pcre perl python readline session ssl tcpd uclibc unicode x86 xorg zlib" ALSA_PCM_PLUGINS="adpcm a
law asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol"
 APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_ow
ner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logi
o mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" COLLECTD_PLUGINS="df interface irq load memory
rrdtool swap syslog" ELIBC="uclibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver
oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz
 cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" PHP_TARGETS="php5-2" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="dummy fbdev v4l" XTABLES_ADDONS
="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"

Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 13 Leho Kraav (:macmaN @lkraav) 2010-11-10 03:08:56 UTC
Created attachment 253831 [details]
altinstall build log fail

i also tried emake -i in the ebuild. inside chroot /usr/i686-gentoo-linux-uclibc:

/ # emerge

/usr/bin/python2.6: symbol '__ctype_b_loc': can't resolve symbol in lib '/usr/lib/python2.6/lib-dynload/strop.so'.

/usr/bin/python2.6: symbol '__fprintf_chk': can't resolve symbol in lib '/usr/lib/python2.6/lib-dynload/_collections.so'.

/usr/bin/python2.6: symbol '__memcpy_chk': can't resolve symbol in lib '/usr/lib/python2.6/lib-dynload/fcntl.so'.

/usr/bin/python2.6: symbol '__strcat_chk': can't resolve symbol in lib '/usr/lib/python2.6/lib-dynload/cPickle.so'.

/usr/bin/python2.6: symbol '__ctype_b_loc': can't resolve symbol in lib '/usr/lib/python2.6/lib-dynload/_struct.so'.


!!! Failed to complete python imports. These are internal modules for
!!! python and failure here indicates that you have a problem with python
!!! itself and thus portage is not able to continue processing.

!!! You might consider starting python with verbose flags to see what has
!!! gone wrong. Here is the information we got for this exception:
    unknown dlopen() error


/usr/bin/python2.6: symbol '__memcpy_chk': can't resolve symbol in lib '/usr/lib/python2.6/lib-dynload/fcntl.so'.

/usr/bin/python2.6: symbol '__strcat_chk': can't resolve symbol in lib '/usr/lib/python2.6/lib-dynload/cPickle.so'.

/usr/bin/python2.6: symbol '__ctype_b_loc': can't resolve symbol in lib '/usr/lib/python2.6/lib-dynload/_struct.so'.


!!! Failed to complete python imports. These are internal modules for
!!! python and failure here indicates that you have a problem with python
!!! itself and thus portage is not able to continue processing.

!!! You might consider starting python with verbose flags to see what has
!!! gone wrong. Here is the information we got for this exception:
    unknown dlopen() error

Traceback (most recent call last):
  File "/usr/bin/emerge", line 37, in <module>
    from _emerge.main import emerge_main
  File "/usr/lib/portage/pym/_emerge/main.py", line 16, in <module>
    import portage
  File "/usr/lib/portage/pym/portage/__init__.py", line 23, in <module>
    import pickle
  File "/usr/lib/python2.6/pickle.py", line 34, in <module>
    import struct
  File "/usr/lib/python2.6/struct.py", line 1, in <module>
    from _struct import *
ImportError: unknown dlopen() error
Comment 14 Leho Kraav (:macmaN @lkraav) 2010-11-18 17:53:04 UTC
my case might be something to with uclibc + locale instead. http://bugs.python.org/issue1669349, manually enabling unicodedata in Modules/Setup.dist gives me:

i686-pc-linux-gnu-gcc -pthread -fno-strict-aliasing  -DNDEBUG -O1  -I. -IInclude -I./Include   -DPy_BUILD_CORE  -c ./Modules/unicodedata.c -o Modules/unicodedata.o
Parser/tokenizer_pgen.o: In function `PyTokenizer_Get':
tokenizer_pgen.c:(.text+0xc0a): undefined reference to `__ctype_b'
tokenizer_pgen.c:(.text+0xc82): undefined reference to `__ctype_b'
tokenizer_pgen.c:(.text+0xd1b): undefined reference to `__ctype_b'
tokenizer_pgen.c:(.text+0xd99): undefined reference to `__ctype_b'
tokenizer_pgen.c:(.text+0xe2d): undefined reference to `__ctype_b'
Parser/tokenizer_pgen.o:tokenizer_pgen.c:(.text+0xe87): more undefined references to `__ctype_b' follow
Parser/pgenmain.o: In function `getgrammar':
pgenmain.c:(.text+0x14b): undefined reference to `__fputc_unlocked'
collect2: ld returned 1 exit status
make: *** [Parser/pgen] Error 1
make: *** Waiting for unfinished jobs....
Comment 15 Leho Kraav (:macmaN @lkraav) 2010-11-18 18:12:42 UTC
ok and now also applying this cross-setup-sysroot patch in my overlay took me one step further.

5789 Compiling /var/tmp/portage/dev-lang/python-2.6.5-r3/image//usr/lib/python2.6/zipfile.py ...
5790 PYTHONPATH=/var/tmp/portage/dev-lang/python-2.6.5-r3/image//usr/lib/python2.6  LD_LIBRARY_PATH=/var/tmp/portage/dev-lang/python-2.6.5-r3/work/Python-2.6.5: \
5791         ././hostpython -Wi -t /var/tmp/portage/dev-lang/python-2.6.5-r3/image//usr/lib/python2.6/compileall.py \
5792         -d /usr/lib/python2.6/site-packages -f \
5793         -x badsyntax /var/tmp/portage/dev-lang/python-2.6.5-r3/image//usr/lib/python2.6/site-packages
5794 Listing /var/tmp/portage/dev-lang/python-2.6.5-r3/image//usr/lib/python2.6/site-packages ...
5795 PYTHONPATH=/var/tmp/portage/dev-lang/python-2.6.5-r3/image//usr/lib/python2.6 LD_LIBRARY_PATH=/var/tmp/portage/dev-lang/python-2.6.5-r3/work/Python-2.6.5: \
5796         ././hostpython -Wi -t -O /var/tmp/portage/dev-lang/python-2.6.5-r3/image//usr/lib/python2.6/compileall.py \
5797         -d /usr/lib/python2.6/site-packages -f \
5798         -x badsyntax /var/tmp/portage/dev-lang/python-2.6.5-r3/image//usr/lib/python2.6/site-packages
5799 Listing /var/tmp/portage/dev-lang/python-2.6.5-r3/image//usr/lib/python2.6/site-packages ...
5800 PYTHONPATH=/var/tmp/portage/dev-lang/python-2.6.5-r3/image//usr/lib/python2.6 LD_LIBRARY_PATH=/var/tmp/portage/dev-lang/python-2.6.5-r3/work/Python-2.6.5: \
5801         ././hostpython -Wi -t -c "import lib2to3.pygram, lib2to3.patcomp;lib2to3.patcomp.PatternCompiler()"
5802 Traceback (most recent call last):
5803   File "<string>", line 1, in <module>
5804   File "/var/tmp/portage/dev-lang/python-2.6.5-r3/image/usr/lib/python2.6/lib2to3/pygram.py", line 11, in <module>
5805     from .pgen2 import driver
5806   File "/var/tmp/portage/dev-lang/python-2.6.5-r3/image/usr/lib/python2.6/lib2to3/pgen2/driver.py", line 21, in <module>
5807     import logging
5808   File "/var/tmp/portage/dev-lang/python-2.6.5-r3/image/usr/lib/python2.6/logging/__init__.py", line 34, in <module>
5809     import sys, os, types, time, string, cStringIO, traceback
5810 ImportError: libdl.so.0: cannot open shared object file: No such file or directory
5811 make: [libinstall] Error 1 (ignored)
5812 rmdir: failed to remove `/var/tmp/portage/dev-lang/python-2.6.5-r3/image//usr/lib/python2.6/lib-old': No such file or directory
5813 >>> Completed installing python-2.6.5-r3 into /var/tmp/portage/dev-lang/python-2.6.5-r3/image/

any comments on whether that error affects anything or not.
Comment 16 Leho Kraav (:macmaN @lkraav) 2010-11-18 18:15:16 UTC
Created attachment 254741 [details]
got past altinstall fail
Comment 17 Dirkjan Ochtman gentoo-dev 2012-02-16 13:19:56 UTC
This stuff should go upstream first, closing to that effect.