| Summary: | dev-lang/python-3.3.2 - error: [...]/work/Python-3.3.2/Modules/_ctypes/libffi: No such file or directory | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Matt Turner <mattst88> |
| Component: | New packages | Assignee: | Python Gentoo Team <python> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | bruno, dogshu, mips, rich0 |
| Priority: | High | Keywords: | InOverlay, InVCS |
| Version: | unspecified | ||
| Hardware: | MIPS | ||
| OS: | Linux | ||
| See Also: | https://bugs.gentoo.org/show_bug.cgi?id=504644 | ||
| Whiteboard: | python-overlay | ||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | |||
| Bug Blocks: | 380025, 348650 | ||
| Attachments: |
build.log showing failure
environment patch to add lib32 directories to setup.py build.log |
||
|
Description
Matt Turner
2011-02-14 14:56:44 UTC
Created attachment 262479 [details]
build.log showing failure
Created attachment 262481 [details]
environment
Obvious question: do you have libffi installed at the point you build python? (In reply to comment #3) > Obvious question: do you have libffi installed at the point you build python? Yes, =dev-libs/libffi-3.0.10_rc4 and virtual/libffi are installed. then my bet is that setup.py does some inventing and lame assuming for your system, as it does for Darwin boxes (which was the cause, last time I saw an incarnation of this problem) Arfrever suggested I try 2.7.1 (I had it package.mask'd before, for some reason I've now forgotten), but it fails too in the exact same way. So the problem is that lib_dirs (printed in detect_ctypes()) is detected as ['/usr/local/lib', '/lib64', '/usr/lib64', '/lib', '/usr/lib'] but for N32 MIPS, the libdir is lib32. On MIPS, we've got a clusterfuck of ABIs: lib is for o32 binaries lib32 is for n32 binaries lib64 is for n64 binaries Looks like we need to add lib32 directories to lib_dirs. Created attachment 262513 [details, diff]
patch to add lib32 directories to setup.py
This patch fixes the problem on 2.7.1, and I can't see any reason why it wouldn't for 2.6.6-r1 and 3.1.x.
Does this look reasonable?
I guess adding this in the ebuild (like we do in Prefix) should work for you then too:
append-flags -L${EPREFIX}/$(get_libdir)
append-flags -L${EPREFIX}/usr/$(get_libdir)
Could we please commit one fix or the other? lib32 isn't always a symlink and should be checked just like lib and lib64. Fabian's change affects python's installed Makefile, but my setup.py patch does not. (In reply to comment #11) > Fabian's change affects python's installed Makefile, but my setup.py patch does > not. Hm, also, Fabian's change doesn't fix the problem I'm seeing. marienz helped me on irc and discovered that if we use Fabian's change, but use append-ldflags instead, and modify setup.py:431 to unconditionally execute that block then it fixes the bug.
append-flags -L${EPREFIX}/$(get_libdir)
append-flags -L${EPREFIX}/usr/$(get_libdir)
I'm really tired of this bug.
Hmm, the append-ldflags change sounds pretty much right to me. I'll have to investigate why it work(s/ed) for me.
> I'm really tired of this bug.
Since this bug got a Prefix flavour now, you can expect it to be botted/ignored/thwarted for the forseeable future.
I think that the safest solution would be to apply patch from comment #8 only on mips (42_mips_libdirs.patch). I will include this patch in next snapshots of Python. I have implemented a better solution. Please test Python 2.7.3_pre20110612, 3.2.1_pre20110612 and 3.3_pre20110612 from python overlay. I don't really know what the fix is, but it works. Tested with python-2.7.3_pre20110612. Arfrever, can you please point to the diff or patch that solved this issue? The fix is ??_all_libdir.patch. http://overlays.gentoo.org/proj/python/browser/overlays/python/dev-lang/python/files/2.7-20110612/04_all_libdir.patch Fixed also in 2.6.7, 2.7.2 and 3.1.4. Not yet fixed in 3.2 slot. Release of Python 3.2.1 is delayed... (In reply to comment #20) > Fixed also in 2.6.7, 2.7.2 and 3.1.4. I just tried 2.7.2, and it didn't work. Same error as before, which I don't understand because when I tested the patch in the overlay (python-2.7.3_pre20110612) it worked. Thanks for your help, Arfrever. Sorry for the noise. Ignore that last report. Something was wrong on my end. It works in my stage builds. Thanks a lot for your work, Arfrever! So, python team, this has been fixed for everything but -3.2. Any ETA? (In reply to comment #24) > So, python team, this has been fixed for everything but -3.2. Any ETA? OK... well, I'll just package.mask >=python-3.2 on mips until something changes. I noticed that 3.2.2 has sed GENTOO_LIBDIR like the other ebuilds, so I thought it might have been fixed, but no. error: /var/tmp/portage/dev-lang/python-3.2.2/work/Python-3.2.2/Modules/_ctypes/libffi: No such file or directory Since they retired Arfrever, no one on the python team has acknowledged this bug... You're looking at 3.2.2, right? It could easily be I copied something with the sed without paying much attention. If you point me in the right direction with what you need for mips to work with python-3.2.2, I'd be happy to include it in the next revision. The 04_all_libdir.patch patch and sed @@GENTOO_LIBDIR@@ are the fix. I'm guessing that there are some places that 04_all_libdir.patch should have patched in 3.2 that were missed, but this is just a guess. FYI, dev-lang/python-3.2.2 builds fine for me on my multilib mips system with gcc 4.6.2. I was also able to uninstall python-3.1 and run python-updater without any problems. I think that the python-3.2 mips mask can be removed. flummox ~ # equery list python * Searching for python ... [IP-] [ ] dev-lang/python-2.7.2-r3:2.7 [IP-] [M ] dev-lang/python-3.2.2:3.2 flummox ~ # python3 Python 3.2.2 (default, Mar 6 2012, 15:52:28) [GCC 4.6.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> flummox ~ # emerge --info Portage 2.1.10.49 (default/linux/mips/10.0/multilib/n32, gcc-4.6.2, glibc-2.13-r4, 3.2.9 mips64) ================================================================= System uname: Linux-3.2.9-mips64-R5000_V2.1_FPU_V1.0-with-gentoo-2.1 Timestamp of tree: Wed, 07 Mar 2012 14:15:01 +0000 app-shells/bash: 4.2_p20 dev-lang/python: 2.7.2-r3, 3.2.2 dev-util/pkgconfig: 0.26 sys-apps/baselayout: 2.1 sys-apps/openrc: 0.9.9.2 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.68 sys-devel/automake: 1.11.3 sys-devel/binutils: 2.22-r1 sys-devel/gcc: 4.6.2 sys-devel/gcc-config: 1.5.1 sys-devel/libtool: 2.4.2 sys-devel/make: 3.82-r3 sys-kernel/linux-headers: 3.2-r1 (virtual/os-headers) sys-libs/glibc: 2.13-r4 Repositories: gentoo ACCEPT_KEYWORDS="mips ~mips" ACCEPT_LICENSE="* -@EULA" CBUILD="mips64-unknown-linux-gnu" CFLAGS="-Os -march=r5000 -fomit-frame-pointer" CHOST="mips64-unknown-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-Os -march=r5000 -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="assume-digests binpkg-logs distlocks ebuild-locks fixlafiles news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch" FFLAGS="-Os -march=r5000 -fomit-frame-pointer" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j1" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="" SYNC="rsync://jove.eng.yale.edu/gentoo-portage" USE="berkdb bzip2 cli cracklib crypt cups cxx device-mapper gdbm gpm iconv ipv6 memlimit mips modules mudflap multilib ncurses nls nptl nptlonly offensive pam pcre pppd readline session ssl sysfs tcpd unicode xorg zlib" ALSA_CARDS="au1x00" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="fbdev impact newport dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS flummox ~ # Thanks Jim. I just unpacked my mips box and confirm that 3.2.3 builds fine here. I'm not sure the reason and I'm not sure I care enough to find out. :) I'll remove the mask. Rather disappointed in the python team... This bug was fixed in this commit: *python-3.1.5 (26 Apr 2012) *python-2.7.3-r1 (26 Apr 2012) *python-3.2.3 (26 Apr 2012) *python-2.6.8 (26 Apr 2012) 26 Apr 2012; Mike Gilbert <floppym@gentoo.org> +python-2.6.8.ebuild, +python-2.7.3-r1.ebuild, +python-3.1.5.ebuild, +python-3.2.3.ebuild: Version bumps for security bug 396329. Ebuilds and patchsets based on work by Arfrever in Progress overlay. (In reply to comment #31) > This bug was fixed in this commit In 3.2 slot. Other slots were fixed earlier. Created attachment 352222 [details] build.log python-3.3.2 fails again python-3.2.5 works ok Portage 2.1.12.9 (default/linux/mips/13.0/n32, gcc-4.6.3, glibc-2.17, 3.4.27-rt37-Cavium-Octeon mips64) ================================================================= KiB Mem: 2044516 total, 180900 free KiB Swap: 1048572 total, 1038136 free Timestamp of tree: Fri, 28 Jun 2013 07:45:01 +0000 ld GNU ld (GNU Binutils) 2.23.1 app-shells/bash: 4.2_p45 dev-lang/python: 2.7.5, 3.2.5 dev-util/pkgconfig: 0.28 sys-apps/baselayout: 2.2 sys-apps/openrc: 0.11.8 sys-apps/sandbox: 2.6-r1 sys-devel/autoconf: 2.69 sys-devel/automake: 1.12.4, 1.13.2 sys-devel/binutils: 2.23.1 sys-devel/gcc: 4.6.3, 4.7.3 sys-devel/gcc-config: 1.8 sys-devel/libtool: 2.4.2 sys-devel/make: 3.82-r4 sys-kernel/linux-headers: 3.9 (virtual/os-headers) sys-libs/glibc: 2.17 Repositories: gentoo ACCEPT_KEYWORDS="mips ~mips" ACCEPT_LICENSE="* -@EULA" CBUILD="mips64-unknown-linux-gnu" CFLAGS="-O2 -march=mips64 -mplt -pipe" CHOST="mips64-unknown-linux-gnu" CONFIG_PROTECT="/etc" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -march=mips64 -mplt -pipe" DISTDIR="/usr/portage/distfiles" FCFLAGS="" FEATURES="assume-digests binpkg-logs buildpkg config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch" FFLAGS="" GENTOO_MIRRORS="ftp://ftp.ucsb.edu/pub/mirrors/linux/gentoo/" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j7" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="" SYNC="rsync://mirror.bytemark.co.uk/gentoo-portage" USE="acl berkdb bzip2 cli cracklib crypt cxx gdbm gpm iconv ipv6 mips modules mudflap ncurses nls nptl pam pcre readline session ssl tcpd unicode zlib" ALSA_CARDS="au1x00" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-3" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" RUBY_TARGETS="ruby19 ruby18" USERLAND="GNU" VIDEO_CARDS="fbdev impact newport dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON I had a quick look at this, but I don't see what changed here either in the ebuild or in Python. I wish I remembered if I did anything particularly clever when debugging this before. I think I'd have to add a bunch of logging to detect_ctypes in setup.py and/or look at an strace of setup.py to see where it goes off the rails. (In reply to Marien Zwart from comment #34) > I had a quick look at this, but I don't see what changed here either in the > ebuild or in Python. I wish I remembered if I did anything particularly > clever when debugging this before. I think I'd have to add a bunch of > logging to detect_ctypes in setup.py and/or look at an strace of setup.py to > see where it goes off the rails. Thanks for looking at it. Well we used to have this problem in the patch, then you applied a magic patch that fixed it but somehow it seems more changes are needed in recent python versions. This kinda blocks up2date stages for MIPS64 multilib. @MIPS Any objection to mask this python version for mips? *** Bug 471228 has been marked as a duplicate of this bug. *** Arfrever says this is a different problem than the one originally reported. Could someone please file a new bug and close this one. Based on a conversation between Arfrever and hwoarang in #gentoo-python, it seems that the ebuild in Progress overlay works find. There is probably some difference between our patchsets. *** Bug 452758 has been marked as a duplicate of this bug. *** (In reply to Mike Gilbert from comment #39) > Based on a conversation between Arfrever and hwoarang in #gentoo-python, it > seems that the ebuild in Progress overlay works find. > > There is probably some difference between our patchsets. ping? anyone willing to copy/backport Arfrever's patch properly? I'll put it on my weekend to-do list if nobody gets it first. +*python-3.3.2-r2 (18 Aug 2013) + + 18 Aug 2013; Mike Gilbert <floppym@gentoo.org> + +files/CVE-2013-4073_py33.patch, +python-3.3.2-r2.ebuild: + Use Arfrever's patchset, bug 354877. Apply fix for CVS-2013-4238, bug 480856. Please confirm that this resolves the issue. Markos reports that this is working now. 2.7.6, 3.2.5-r3, and 3.3.4 fail to build on my system. Was this patch dropped? Default 64bit multilib here. |