Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 320669 - dev-python/numpy lapack support is broken
Summary: dev-python/numpy lapack support is broken
Status: RESOLVED TEST-REQUEST
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Gentoo Science Related Packages
URL:
Whiteboard:
Keywords:
: 320673 (view as bug list)
Depends on:
Blocks:
 
Reported: 2010-05-20 02:15 UTC by François Bissey
Modified: 2012-12-07 08:29 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description François Bissey 2010-05-20 02:15:03 UTC
On a ~x86 system I have dev-python/numpy-1.4.1 installed and matplotlib
doesn't build at all it fails with the following:

>>> Emerging (1 of 1) dev-python/matplotlib-0.99.1.1-r1
 * matplotlib-0.99.1.1.tar.gz RMD160 SHA1 SHA256 size ;-) ...                                                                [ ok ]
 * checking ebuild checksums ;-) ...                                                                                         [ ok ]
 * checking auxfile checksums ;-) ...                                                                                        [ ok ]
 * checking miscfile checksums ;-) ...                                                                                       [ ok ]
 * CPV:  dev-python/matplotlib-0.99.1.1-r1
 * REPO: gentoo
 * USE:  cairo elibc_glibc gtk kernel_linux latex qt4 tk userland_GNU x86
>>> Unpacking source...
>>> Unpacking matplotlib-0.99.1.1.tar.gz to /var/tmp/portage/dev-python/matplotlib-0.99.1.1-r1/work
>>> Source unpacked in /var/tmp/portage/dev-python/matplotlib-0.99.1.1-r1/work
>>> Preparing source in /var/tmp/portage/dev-python/matplotlib-0.99.1.1-r1/work/matplotlib-0.99.1.1 ...
 * Applying matplotlib-0.98.5.2-no-xv.patch ...                                                                               [ ok ]
 * Applying matplotlib-0.99.1.1-prefix.patch ...                                                                              [ ok ]
 * Applying matplotlib-0.99.0-freebsd7+.patch ...                                                                             [ ok ]
 * Applying matplotlib-0.99.1.1-libpng14.patch ...                                                                            [ ok ]
>>> Source prepared.
>>> Configuring source in /var/tmp/portage/dev-python/matplotlib-0.99.1.1-r1/work/matplotlib-0.99.1.1 ...
>>> Source configured.
>>> Compiling source in /var/tmp/portage/dev-python/matplotlib-0.99.1.1-r1/work/matplotlib-0.99.1.1 ...
 * Building of dev-python/matplotlib-0.99.1.1-r1 with CPython 2.6...
python2.6 setup.py build -b build-2.6 
============================================================================
BUILDING MATPLOTLIB
            matplotlib: 0.99.1.1
                python: 2.6.5 (r265:79063, May  4 2010, 15:16:09)  [GCC
                        4.4.3]
              platform: linux2

REQUIRED DEPENDENCIES
                 numpy: no
                        * You must install numpy 1.1 or later to build
                        * matplotlib.
 * ERROR: dev-python/matplotlib-0.99.1.1-r1 failed:
===========
I tried with stable numpy but no lick either.
Here my emerge --info:

emerge --info =dev-python/matplotlib-0.99.1.1-r1
Portage 2.2_rc67 (default/linux/x86/10.0, gcc-4.4.3, glibc-2.11.1-r0, 2.6.33-gentoo-r1 i686)
=================================================================
                        System Settings
=================================================================
System uname: Linux-2.6.33-gentoo-r1-i686-Intel-R-_Celeron-R-_CPU_2.80GHz-with-gentoo-2.0.1
Timestamp of tree: Tue, 11 May 2010 03:15:02 +0000
app-shells/bash:     4.1_p5
dev-java/java-config: 2.1.11
dev-lang/python:     2.6.5-r99
dev-python/pycrypto: 2.1.0
dev-util/cmake:      2.8.1-r1
sys-apps/baselayout: 2.0.1
sys-apps/openrc:     0.6.1-r1
sys-apps/sandbox:    2.2
sys-devel/autoconf:  2.13, 2.65
sys-devel/automake:  1.7.9-r1, 1.8.5-r3, 1.9.6-r3, 1.10.3, 1.11.1
sys-devel/binutils:  2.20.1-r1
sys-devel/gcc:       4.3.4, 4.4.3-r2
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6b
virtual/os-headers:  2.6.33
ACCEPT_KEYWORDS="x86 ~x86"
ACCEPT_LICENSE="* -@EULA"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=native -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/X11/xkb /usr/share/config /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -march=native -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests collision-protect distlocks fixpackages news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unmerge-logs unmerge-orphans userfetch"
GENTOO_MIRRORS="ftp://mirror.ihug.co.nz/gentoo http://distfiles.gentoo.org http://www.ibiblio.org/pub/Linux/distributions/gentoo http://gentoo.eliteitminds.com"
LDFLAGS="-Wl,--as-needed"
LINGUAS="en_GB fr"
MAKEOPTS="-j3 VERBOSE=1"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/var/lib/layman/sage-on-gentoo /var/lib/layman/kde /home/fbissey/Work/overlay"
SYNC="rsync://rsync.au.gentoo.org/gentoo-portage"
USE="3dnow 3dnowext X a52 aac aalib acl acpi alsa amr ao apm asf audiofile berkdb bidi bootsplash branding bzip2 cairo cddb cdf cdparanoia cli composite consolekit cracklib crypt cups curl cxx dbus device-mapper dga dmi dri dts dv dvd dvdr dvdread encode exif expat ffmpeg fftw flac fontconfig foomaticdb fortran fpx fts3 gdbm geoip gif gimpprint glib glitz glut gmp graphviz gs gstreamer gtk gtk2 hal iconv id3tag idn imagemagick imlib imlib2 ipv6 ipython java jbig joystick jpeg jpeg2k kde kdehiddenvisibility kipi kpathsea ladspa lame lapack latex lcms libffi libsamplerate lzo mad mathml matroska mikmod mime mmx mmx2 mmxext mng modules motif mp3 mp4 mpeg mplayer mudflap musicbrainz ncurses netcdf network nls nptl nptlonly ogg openal openexr opengl openmp pcre pdf perl png policykit portaudio posix ppds pppd python qt3support qt4 quicktime readline reflection reiserfs sdl secure-delete session sndfile sox spell spl sqlite3 sse sse2 ssl startup-notification svg svga sysfs szip t1lib tcl tcltk tcpd tetex tga theora tiff tk truetype unicode usb vcd vnc vorbis wavpack webkit wmf x264 x86 xanim xattr xcb xcomposite xine xinerama xml xorg xpm xsl xv xvid xvmc zeroconf zlib" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="evdev keyboard mouse joystick" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en_GB fr" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="vesa" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" 
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LANG, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

=================================================================
                        Package Settings
=================================================================

dev-python/matplotlib-0.99.1.1-r1 was built with the following:
USE="cairo gtk latex qt4 tk -doc -examples -excel -fltk -traits -wxwidgets"
Comment 1 François Bissey 2010-05-20 05:15:15 UTC
*** Bug 320673 has been marked as a duplicate of this bug. ***
Comment 2 Sébastien Fabbro (RETIRED) gentoo-dev 2010-05-31 20:29:30 UTC
Hi Francois,

Could not reproduce it here. Could it be that your numpy was built for another python version and it needs to builds for your python-2.6.5-r99? 

Comment 3 François Bissey 2010-05-31 22:12:47 UTC
(In reply to comment #2)
> Hi Francois,
> 
> Could not reproduce it here. Could it be that your numpy was built for another
> python version and it needs to builds for your python-2.6.5-r99? 
> 
I will check, but I pretty much tried to rebuild everything including numpy
with the same version of python.
I have since revised python-2.6.5-r99 to match the latest 2.6.5-r2 (this 
version of python has been quite bumpy) - the only difference is the pickle
patch needed for sage.
I will try again when I have access to the machine.
That's not the only issue I have on this testing machine (~x86) so it
may be a symptom from a deeper problem.
Comment 4 Thomas Kahle (RETIRED) gentoo-dev 2010-06-28 20:49:50 UTC
Hi, I ran into this when rebuilding during the libpng upgrade.
I can confirm that this bug is present with any combination of 
python-2.6.5-r2 or python-2.6.5-r99 (sage-overlay)
and 
matplotlib-0.99.1.1-r1 or matplotlib-0.99.3

I'm on mostly stable x86:

$ emerge --info
Portage 2.1.8.3 (default/linux/x86/10.0/desktop, gcc-4.4.3, glibc-2.11.2-r0, 2.6.34-gentoo i686)
=================================================================
System uname: Linux-2.6.34-gentoo-i686-Intel-R-_Core-TM-2_Duo_CPU_L7500_@_1.60GHz-with-gentoo-2.0.1
Timestamp of tree: Mon, 28 Jun 2010 12:30:01 +0000
ccache version 2.4 [enabled]
app-shells/bash:     4.0_p37
dev-java/java-config: 2.1.10
dev-lang/python:     2.6.5-r2, 3.1.2-r3
dev-util/ccache:     2.4-r7
dev-util/cmake:      2.6.4-r3
sys-apps/baselayout: 2.0.1
sys-apps/openrc:     0.6.0-r1
sys-apps/sandbox:    1.6-r2
sys-devel/autoconf:  2.13, 2.65
sys-devel/automake:  1.5, 1.9.6-r3, 1.10.3, 1.11.1
sys-devel/binutils:  2.20.1-r1
sys-devel/gcc:       4.4.3-r2
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6b
virtual/os-headers:  2.6.30-r1
ACCEPT_KEYWORDS="x86"
ACCEPT_LICENSE="*"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=core2 -O2 -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/X11/xkb /usr/share/config /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/eselect/postgresql /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-march=core2 -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests ccache collision-protect distlocks fixpackages news parallel-fetch protect-owned sandbox sfperms splitdebug strict unmerge-logs unmerge-orphans userfetch"
GENTOO_MIRRORS="http://de-mirror.org/distro/gentoo/ http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/ http://ftp.uni-erlangen.de/pub/mirrors/gentoo"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="en en_US de ja es fr it"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_EXTRA_OPTS="--timeout=300"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage/layman/sunrise /usr/local/portage/tom-overlay /home/tom/gentoo/sci /home/tom/gentoo/sage-on-gentoo"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="X a52 aac acl acpi alsa anthy apache2 avahi bash-completion berkdb bluetooth branding bzip2 cairo cddb cdparanoia cdr cjk cli consolekit cracklib crypt ctype cups curl cxx daap dbus djvu dri dts dvd dvdr emacs encode exif expat fam fbcon ffmpeg firefox flac fortran ftp gd gdbm gif gimp glut gmp gnome gnome-keyring gnutls gpm graphviz gstreamer gtk guile hal hdaps iconv ieee1394 imagemagick imap imlib ipod ipv6 java javascript jpeg kde latex lcms ldap leim libnotify lm_sensors m17n-lib mad migemo mikmod mime mmx mng modules mp3 mp4 mpeg mplayer mudflap mule musicbrainz mysql mysqli ncurses nls nptl nptlonly nsplugin obex ocaml ogg oggvorbis openal opengl openmp pam pango pcmcia pcre pdf perl php plasma plotutils png policykit ppds pppd python qt3support qt4 readline reflection samba sasl sdl semantic-desktop session slang smartcard spell spl sqlite sqlite3 sse ssl startup-notification svg sysfs tcpd texlive theora threads tiff tk truetype uim unicode usb v4l v4l2 visualization vorbis wicd wifi win32codecs wxwindows x264 x86 xcb xcomposite xft xine xinerama xml xorg xulrunner xv xvid zeroconf zlib zsh-completion" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" DVB_CARDS="usb-dib0700" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en en_US de ja es fr it" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="intel" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" 
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS
Comment 5 Thomas Kahle (RETIRED) gentoo-dev 2010-06-30 08:07:32 UTC
Hi there. the reason for this bug is a non-functional nympy due to broken lapack. If you are having this issue just try to run python and "import numpy" on the shell. 

To work around: 
emerge numpy with USE="-lapack"
emerge matplotlib

or of course fix lapack! I'm ccing sci now.
Comment 6 François Bissey 2010-06-30 08:47:58 UTC
that could explain a lot of things since I suffer from a broken atlas 
as well. I will try switching to a working lapack and see what happens.
Comment 7 Sébastien Fabbro (RETIRED) gentoo-dev 2010-06-30 17:06:36 UTC
(In reply to comment #5)
> Hi there. the reason for this bug is a non-functional nympy due to broken
> lapack. If you are having this issue just try to run python and "import numpy"
> on the shell. 

Which blas, cblas, and lapack? What is the result of eselect {blas,cblas,lapack} show?
Comment 8 Thomas Kahle (RETIRED) gentoo-dev 2010-06-30 19:44:11 UTC
(In reply to comment #7)
> (In reply to comment #5)
> > Hi there. the reason for this bug is a non-functional nympy due to broken
> > lapack. If you are having this issue just try to run python and "import numpy"
> > on the shell. 
> 
> Which blas, cblas, and lapack? What is the result of eselect
> {blas,cblas,lapack} show?

Ok, cblas was a good hint. It depends on the selected cblas implementation!

# eselect cblas set gsl
# python               
Python 2.6.5 (release26-maint, Jun 29 2010, 11:37:24) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/site-packages/numpy/__init__.py", line 132, in <module>
    import add_newdocs
  File "/usr/lib/python2.6/site-packages/numpy/add_newdocs.py", line 9, in <module>
    from lib import add_newdoc
  File "/usr/lib/python2.6/site-packages/numpy/lib/__init__.py", line 13, in <module>
    from polynomial import *
  File "/usr/lib/python2.6/site-packages/numpy/lib/polynomial.py", line 17, in <module>
    from numpy.linalg import eigvals, lstsq
  File "/usr/lib/python2.6/site-packages/numpy/linalg/__init__.py", line 47, in <module>
    from linalg import *
  File "/usr/lib/python2.6/site-packages/numpy/linalg/linalg.py", line 22, in <module>
    from numpy.linalg import lapack_lite
ImportError: /usr/lib/liblapack.so.0: undefined symbol: clapack_ilaenv
>>> 
# eselect cblas set atlas 
# python import numpy
/usr/bin/python2.6: can't open file 'import': [Errno 2] No such file or directory
denkmatte files # python           
Python 2.6.5 (release26-maint, Jun 29 2010, 11:37:24) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy


Is this expected? (I bet cblas eselects gsl upon merge of gsl?)
Comment 9 François Bissey 2010-06-30 19:53:01 UTC
Actually your trace points to the fact that you have lapack-atlas eselected
while using gsl cblas. Try switching to lapack-reference - you will almost
certainly have to rebuild.
Comment 10 Thomas Kahle (RETIRED) gentoo-dev 2010-06-30 19:58:38 UTC
(In reply to comment #9)
> Actually your trace points to the fact that you have lapack-atlas eselected
> while using gsl cblas. Try switching to lapack-reference - you will almost
> certainly have to rebuild.

You are right, lapack-reference and cblas-gsl together also work. 
Should we resolve this by adding some einfo to the numpy ebuild, or is the user expected to know that she should have all atlas, or the reference/gsl combination?

Comment 11 Sébastien Fabbro (RETIRED) gentoo-dev 2010-06-30 20:02:42 UTC
numpy is tricky when it comes to lapack implementation. If you want a full featured numpy, you should do the following:

1. Make sure all your blas,cblas,lapack are set to atlas (or atlas-threads if you are on a multicore box)
2. install numpy and reverse dependencies (matplotlib, scipy,...)
3. if it does not compile, go back to 1., but select the reference implementations.

You could also install gsl at step 2.
it is very hacky, but the eselect lapack need work. So stay tuned.
Comment 12 Justin Lecher (RETIRED) gentoo-dev 2012-12-07 08:29:20 UTC
Please reopen if this still exists with a correctly setuped blas/lapck isntallation and the rebuild procedure Sebastien described.