Relevant error: error: /var/tmp/portage/dev-lang/python-2.6.6-r1/work/Python-2.6.6/Modules/_ctypes/libffi: No such file or directory The only relevant hit I see on google is a gentoo/prefix mailing list post, http://archives.gentoo.org/gentoo-alt/msg_0e371693bff5e4835422d0250da7726e.xml CC'ing them to see if they know what's going on.
Created attachment 262479 [details] build.log showing failure
Created attachment 262481 [details] environment
Obvious question: do you have libffi installed at the point you build python?
(In reply to comment #3) > Obvious question: do you have libffi installed at the point you build python? Yes, =dev-libs/libffi-3.0.10_rc4 and virtual/libffi are installed.
then my bet is that setup.py does some inventing and lame assuming for your system, as it does for Darwin boxes (which was the cause, last time I saw an incarnation of this problem)
Arfrever suggested I try 2.7.1 (I had it package.mask'd before, for some reason I've now forgotten), but it fails too in the exact same way.
So the problem is that lib_dirs (printed in detect_ctypes()) is detected as ['/usr/local/lib', '/lib64', '/usr/lib64', '/lib', '/usr/lib'] but for N32 MIPS, the libdir is lib32. On MIPS, we've got a clusterfuck of ABIs: lib is for o32 binaries lib32 is for n32 binaries lib64 is for n64 binaries Looks like we need to add lib32 directories to lib_dirs.
Created attachment 262513 [details, diff] patch to add lib32 directories to setup.py This patch fixes the problem on 2.7.1, and I can't see any reason why it wouldn't for 2.6.6-r1 and 3.1.x. Does this look reasonable?
I guess adding this in the ebuild (like we do in Prefix) should work for you then too: append-flags -L${EPREFIX}/$(get_libdir) append-flags -L${EPREFIX}/usr/$(get_libdir)
Could we please commit one fix or the other? lib32 isn't always a symlink and should be checked just like lib and lib64.
Fabian's change affects python's installed Makefile, but my setup.py patch does not.
(In reply to comment #11) > Fabian's change affects python's installed Makefile, but my setup.py patch does > not. Hm, also, Fabian's change doesn't fix the problem I'm seeing.
marienz helped me on irc and discovered that if we use Fabian's change, but use append-ldflags instead, and modify setup.py:431 to unconditionally execute that block then it fixes the bug. append-flags -L${EPREFIX}/$(get_libdir) append-flags -L${EPREFIX}/usr/$(get_libdir) I'm really tired of this bug.
Hmm, the append-ldflags change sounds pretty much right to me. I'll have to investigate why it work(s/ed) for me. > I'm really tired of this bug. Since this bug got a Prefix flavour now, you can expect it to be botted/ignored/thwarted for the forseeable future.
I think that the safest solution would be to apply patch from comment #8 only on mips (42_mips_libdirs.patch). I will include this patch in next snapshots of Python.
I have implemented a better solution. Please test Python 2.7.3_pre20110612, 3.2.1_pre20110612 and 3.3_pre20110612 from python overlay.
I don't really know what the fix is, but it works. Tested with python-2.7.3_pre20110612.
Arfrever, can you please point to the diff or patch that solved this issue?
The fix is ??_all_libdir.patch. http://overlays.gentoo.org/proj/python/browser/overlays/python/dev-lang/python/files/2.7-20110612/04_all_libdir.patch
Fixed also in 2.6.7, 2.7.2 and 3.1.4.
Not yet fixed in 3.2 slot. Release of Python 3.2.1 is delayed...
(In reply to comment #20) > Fixed also in 2.6.7, 2.7.2 and 3.1.4. I just tried 2.7.2, and it didn't work. Same error as before, which I don't understand because when I tested the patch in the overlay (python-2.7.3_pre20110612) it worked. Thanks for your help, Arfrever.
Sorry for the noise. Ignore that last report. Something was wrong on my end. It works in my stage builds. Thanks a lot for your work, Arfrever!
So, python team, this has been fixed for everything but -3.2. Any ETA?
(In reply to comment #24) > So, python team, this has been fixed for everything but -3.2. Any ETA? OK... well, I'll just package.mask >=python-3.2 on mips until something changes.
I noticed that 3.2.2 has sed GENTOO_LIBDIR like the other ebuilds, so I thought it might have been fixed, but no. error: /var/tmp/portage/dev-lang/python-3.2.2/work/Python-3.2.2/Modules/_ctypes/libffi: No such file or directory Since they retired Arfrever, no one on the python team has acknowledged this bug...
You're looking at 3.2.2, right? It could easily be I copied something with the sed without paying much attention. If you point me in the right direction with what you need for mips to work with python-3.2.2, I'd be happy to include it in the next revision.
The 04_all_libdir.patch patch and sed @@GENTOO_LIBDIR@@ are the fix. I'm guessing that there are some places that 04_all_libdir.patch should have patched in 3.2 that were missed, but this is just a guess.
FYI, dev-lang/python-3.2.2 builds fine for me on my multilib mips system with gcc 4.6.2. I was also able to uninstall python-3.1 and run python-updater without any problems. I think that the python-3.2 mips mask can be removed. flummox ~ # equery list python * Searching for python ... [IP-] [ ] dev-lang/python-2.7.2-r3:2.7 [IP-] [M ] dev-lang/python-3.2.2:3.2 flummox ~ # python3 Python 3.2.2 (default, Mar 6 2012, 15:52:28) [GCC 4.6.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> flummox ~ # emerge --info Portage 2.1.10.49 (default/linux/mips/10.0/multilib/n32, gcc-4.6.2, glibc-2.13-r4, 3.2.9 mips64) ================================================================= System uname: Linux-3.2.9-mips64-R5000_V2.1_FPU_V1.0-with-gentoo-2.1 Timestamp of tree: Wed, 07 Mar 2012 14:15:01 +0000 app-shells/bash: 4.2_p20 dev-lang/python: 2.7.2-r3, 3.2.2 dev-util/pkgconfig: 0.26 sys-apps/baselayout: 2.1 sys-apps/openrc: 0.9.9.2 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.68 sys-devel/automake: 1.11.3 sys-devel/binutils: 2.22-r1 sys-devel/gcc: 4.6.2 sys-devel/gcc-config: 1.5.1 sys-devel/libtool: 2.4.2 sys-devel/make: 3.82-r3 sys-kernel/linux-headers: 3.2-r1 (virtual/os-headers) sys-libs/glibc: 2.13-r4 Repositories: gentoo ACCEPT_KEYWORDS="mips ~mips" ACCEPT_LICENSE="* -@EULA" CBUILD="mips64-unknown-linux-gnu" CFLAGS="-Os -march=r5000 -fomit-frame-pointer" CHOST="mips64-unknown-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-Os -march=r5000 -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="assume-digests binpkg-logs distlocks ebuild-locks fixlafiles news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch" FFLAGS="-Os -march=r5000 -fomit-frame-pointer" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j1" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="" SYNC="rsync://jove.eng.yale.edu/gentoo-portage" USE="berkdb bzip2 cli cracklib crypt cups cxx device-mapper gdbm gpm iconv ipv6 memlimit mips modules mudflap multilib ncurses nls nptl nptlonly offensive pam pcre pppd readline session ssl sysfs tcpd unicode xorg zlib" ALSA_CARDS="au1x00" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="fbdev impact newport dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS flummox ~ #
Thanks Jim. I just unpacked my mips box and confirm that 3.2.3 builds fine here. I'm not sure the reason and I'm not sure I care enough to find out. :) I'll remove the mask. Rather disappointed in the python team...
This bug was fixed in this commit: *python-3.1.5 (26 Apr 2012) *python-2.7.3-r1 (26 Apr 2012) *python-3.2.3 (26 Apr 2012) *python-2.6.8 (26 Apr 2012) 26 Apr 2012; Mike Gilbert <floppym@gentoo.org> +python-2.6.8.ebuild, +python-2.7.3-r1.ebuild, +python-3.1.5.ebuild, +python-3.2.3.ebuild: Version bumps for security bug 396329. Ebuilds and patchsets based on work by Arfrever in Progress overlay.
(In reply to comment #31) > This bug was fixed in this commit In 3.2 slot. Other slots were fixed earlier.
Created attachment 352222 [details] build.log python-3.3.2 fails again python-3.2.5 works ok Portage 2.1.12.9 (default/linux/mips/13.0/n32, gcc-4.6.3, glibc-2.17, 3.4.27-rt37-Cavium-Octeon mips64) ================================================================= KiB Mem: 2044516 total, 180900 free KiB Swap: 1048572 total, 1038136 free Timestamp of tree: Fri, 28 Jun 2013 07:45:01 +0000 ld GNU ld (GNU Binutils) 2.23.1 app-shells/bash: 4.2_p45 dev-lang/python: 2.7.5, 3.2.5 dev-util/pkgconfig: 0.28 sys-apps/baselayout: 2.2 sys-apps/openrc: 0.11.8 sys-apps/sandbox: 2.6-r1 sys-devel/autoconf: 2.69 sys-devel/automake: 1.12.4, 1.13.2 sys-devel/binutils: 2.23.1 sys-devel/gcc: 4.6.3, 4.7.3 sys-devel/gcc-config: 1.8 sys-devel/libtool: 2.4.2 sys-devel/make: 3.82-r4 sys-kernel/linux-headers: 3.9 (virtual/os-headers) sys-libs/glibc: 2.17 Repositories: gentoo ACCEPT_KEYWORDS="mips ~mips" ACCEPT_LICENSE="* -@EULA" CBUILD="mips64-unknown-linux-gnu" CFLAGS="-O2 -march=mips64 -mplt -pipe" CHOST="mips64-unknown-linux-gnu" CONFIG_PROTECT="/etc" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -march=mips64 -mplt -pipe" DISTDIR="/usr/portage/distfiles" FCFLAGS="" FEATURES="assume-digests binpkg-logs buildpkg config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch" FFLAGS="" GENTOO_MIRRORS="ftp://ftp.ucsb.edu/pub/mirrors/linux/gentoo/" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j7" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="" SYNC="rsync://mirror.bytemark.co.uk/gentoo-portage" USE="acl berkdb bzip2 cli cracklib crypt cxx gdbm gpm iconv ipv6 mips modules mudflap ncurses nls nptl pam pcre readline session ssl tcpd unicode zlib" ALSA_CARDS="au1x00" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-3" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" RUBY_TARGETS="ruby19 ruby18" USERLAND="GNU" VIDEO_CARDS="fbdev impact newport dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
I had a quick look at this, but I don't see what changed here either in the ebuild or in Python. I wish I remembered if I did anything particularly clever when debugging this before. I think I'd have to add a bunch of logging to detect_ctypes in setup.py and/or look at an strace of setup.py to see where it goes off the rails.
(In reply to Marien Zwart from comment #34) > I had a quick look at this, but I don't see what changed here either in the > ebuild or in Python. I wish I remembered if I did anything particularly > clever when debugging this before. I think I'd have to add a bunch of > logging to detect_ctypes in setup.py and/or look at an strace of setup.py to > see where it goes off the rails. Thanks for looking at it. Well we used to have this problem in the patch, then you applied a magic patch that fixed it but somehow it seems more changes are needed in recent python versions. This kinda blocks up2date stages for MIPS64 multilib.
@MIPS Any objection to mask this python version for mips?
*** Bug 471228 has been marked as a duplicate of this bug. ***
Arfrever says this is a different problem than the one originally reported. Could someone please file a new bug and close this one.
Based on a conversation between Arfrever and hwoarang in #gentoo-python, it seems that the ebuild in Progress overlay works find. There is probably some difference between our patchsets.
*** Bug 452758 has been marked as a duplicate of this bug. ***
(In reply to Mike Gilbert from comment #39) > Based on a conversation between Arfrever and hwoarang in #gentoo-python, it > seems that the ebuild in Progress overlay works find. > > There is probably some difference between our patchsets. ping? anyone willing to copy/backport Arfrever's patch properly?
I'll put it on my weekend to-do list if nobody gets it first.
+*python-3.3.2-r2 (18 Aug 2013) + + 18 Aug 2013; Mike Gilbert <floppym@gentoo.org> + +files/CVE-2013-4073_py33.patch, +python-3.3.2-r2.ebuild: + Use Arfrever's patchset, bug 354877. Apply fix for CVS-2013-4238, bug 480856. Please confirm that this resolves the issue.
Markos reports that this is working now.
2.7.6, 3.2.5-r3, and 3.3.4 fail to build on my system. Was this patch dropped? Default 64bit multilib here.
