| Summary: | scipy can't import submodules because of clapack issues | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | webb.sprague |
| Component: | Current packages | Assignee: | Gentoo Science Related Packages <sci> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | sci, suhlhorn |
| Priority: | High | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Other | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
Hi, In my experience, scipy is picky about blas/lapack implementations. (You will notice that the dependecies in the ebuild are blas-atlas and lapack-atlas, and not the virtual implementations, but that your system uses blas-reference and lapack-reference.) I suggest you try the following: Set your blas profile to f77-ATLAS/c-atlas, set your lapack profile to f77-ATLAS, and reinstall scipy. (In reply to comment #1) > I suggest you try the following: Set your blas profile to f77-ATLAS/c-atlas, > set your lapack profile to f77-ATLAS, and reinstall scipy. That works, thanks. Closing this for now, as there is no way to force a specific blas/lapack profile to be used. (Reopen and correct me if I am wrong.) Is there no way to check which implementation is in use during src_unpack()? If not, the capability should be added to the eselect module. It should also be easy enough to change implementations at build time and have a very loud warning afterwards. (In reply to comment #4) > Is there no way to check which implementation is in use during src_unpack()? That should be pkg_setup(), I suppose. ;-) Well, we should be able to, but it does not seem to work. When I run 'eselect blas/lapack show' from inside an ebuild, it reports the profiles as '(none)'. Does it work for anyone? > It should also be easy enough to change implementations at build time and > have a very loud warning afterwards. I dislike changing the user's configuration without asking. It could break other applications that were compiled against different BLAS/LAPACK profiles, and might cause confusion. Better warn beforehand. I used the old-fashioned {blas,lapack}-config instead of the eselect modules, and added checks for ATLAS in scipy-0.4.8-r1.
(In reply to comment #6) scipy-0.4.2-r1 fails to build when my blas profile is set to 'threaded-ATLAS:' Calculating dependencies... done! [ebuild R ] sci-libs/scipy-0.4.8-r1 USE="fftw -debug" 0 kB Total size of downloads: 0 kB Would you like to merge these packages? [Yes/No] >>> Emerging (1 of 1) sci-libs/scipy-0.4.8-r1 to / >>> checking ebuild checksums >>> checking auxfile checksums >>> checking miscfile checksums >>> checking distfiles checksums * Checking active BLAS implementations for ATLAS. Current profiles: F77 BLAS: /usr/lib/blas/f77-threaded-ATLAS C BLAS: /usr/lib/blas/c-threaded-ATLAS * Your F77 BLAS profile is not set to the ATLAS implementation, * which is required by scipy to compile and run properly. * Use: 'blas-config -f ATLAS' to activate ATLAS. * Your C BLAS profile is not set to the ATLAS implementation, * Which is required by scipy to compile and run properly. * Use: 'blas-config -c ATLAS' to activate ATLAS. * Checking active LAPACK implementation for ATLAS. Current profile: F77 LAPACK: /usr/lib/lapack/f77-ATLAS !!! ERROR: sci-libs/scipy-0.4.8-r1 failed. Call stack: ebuild.sh, line 1541: Called dyn_setup ebuild.sh, line 657: Called pkg_setup scipy-0.4.8-r1.ebuild, line 72: Called die !!! Active BLAS/LAPACK implementations are not ATLAS. !!! If you need support, post the topmost build error, and the call stack if relevant. When I change the profile to 'ATLAS', it builds without error. Does this mean my version of scipy is not using the threaded libs, or is this just a problem with the ATLAS check in the ebuild? -stephen You could try changing each instance like f77-ATLAS to 'f77.*ATLAS' and that might fix it up. Not sure whether scipy will handle adding -lpthread, however. Also find it very very strange that it only works w/ atlas. I've reverted this because it seems to work OK with other implems as long as you're consistent about it, and it breaks in an annoying way with newer eselect modules (see bug #136023 e.g.) |
My installation of scipy cannot import scipy.stats (and a number of other modules) due to an undefined symbol clapack_sgesv in clapack.so. The output from scipy.test(), the stack trace from ">>> import scipy.stats", my "emerge info" are below. Let me know if I can help more. Arch = ~86 scipy.__version__ == 0.4.8 cowboy ~ # blas-config -p Current profiles: F77 BLAS: /usr/lib/blas/f77-reference C BLAS: /usr/lib/blas/c-reference cowboy ~ # lapack-config -p Current profile: F77 LAPACK: /usr/lib/lapack/f77-reference In [13]: scipy.test() import signal -> failed: /usr/lib/python2.4/site-packages/scipy/lib/lapack/clapack.so: undefined symbol: clapack_sgesv import lib.lapack -> failed: /usr/lib/python2.4/site-packages/scipy/lib/lapack/clapack.so: undefined symbol: clapack_sgesv import cluster -> failed: /usr/lib/python2.4/site-packages/scipy/lib/lapack/clapack.so: undefined symbol: clapack_sgesv import integrate -> failed: /usr/lib/python2.4/site-packages/scipy/lib/lapack/clapack.so: undefined symbol: clapack_sgesv import special -> failed: /usr/lib/python2.4/site-packages/scipy/lib/lapack/clapack.so: undefined symbol: clapack_sgesv import linalg -> failed: /usr/lib/python2.4/site-packages/scipy/lib/lapack/clapack.so: undefined symbol: clapack_sgesv import maxentropy -> failed: /usr/lib/python2.4/site-packages/scipy/lib/lapack/clapack.so: undefined symbol: clapack_sgesv import stats -> failed: /usr/lib/python2.4/site-packages/scipy/lib/lapack/clapack.so: undefined symbol: clapack_sgesv Found 4 tests for scipy.io.array_import Found 397 tests for scipy.ndimage Found 89 tests for scipy.sparse.sparse Found 20 tests for scipy.fftpack.pseudo_diffs Found 6 tests for scipy.optimize.optimize Found 5 tests for scipy.interpolate.fitpack Found 12 tests for scipy.io.mmio Found 18 tests for scipy.fftpack.basic Found 1 tests for scipy.optimize.zeros Found 4 tests for scipy.fftpack.helper Warning: FAILURE importing tests for <module 'scipy.special.basic' from '...ckages/scipy/special/basic.pyc'> /usr/lib/python2.4/site-packages/scipy/lib/lapack/__init__.py:16: ImportError: /usr/lib/python2.4/site-packages/scipy/lib/lapack/clapack.so: undefined symbol: clapack_sgesv (in ?) Found 128 tests for scipy.lib.blas.fblas Found 1 tests for scipy.optimize.cobyla Found 16 tests for scipy.lib.blas Found 0 tests for __main__ Don't worry about a warning regarding the number of bytes read. Warning: 1000000 bytes requested, 20 bytes read. ...E............................................................................................................................................................................................................................................................................................................................................................................................................. 2 3 1 2 2 3 2 1 3 3 3 3 1 3 3 . 3 3 1 3 3 ...........Use minimum degree ordering on A'+A. .....................Use minimum degree ordering on A'+A. .....................Use minimum degree ordering on A'+A. .......................Use minimum degree ordering on A'+A. .........................................................................TESTING CONVERGENCE zero should be 1 function f2 cc.bisect : 1.0000000000001952 cc.ridder : 1.0000000000004658 cc.brenth : 0.9999999999999997 cc.brentq : 0.9999999999999577 function f3 cc.bisect : 1.0000000000001952 cc.ridder : 1.0000000000000000 cc.brenth : 1.0000000000000009 cc.brentq : 1.0000000000000011 function f4 cc.bisect : 1.0000000000001952 cc.ridder : 1.0000000000001452 cc.brenth : 0.9999999999993339 cc.brentq : 0.9999999999993339 function f5 cc.bisect : 1.0000000000001952 cc.ridder : 1.0000000000004574 cc.brenth : 0.9999999999991442 cc.brentq : 0.9999999999991442 function f6 cc.bisect : 1.0000000000001952 cc.ridder : 0.9999999999998919 cc.brenth : 1.0000000000014924 cc.brentq : 1.0000000000007443 ........caxpy:n=4 ..caxpy:n=3 ....ccopy:n=4 ..ccopy:n=3 .............cscal:n=4 ....cswap:n=4 ..cswap:n=3 .....daxpy:n=4 ..daxpy:n=3 ....dcopy:n=4 ..dcopy:n=3 .............dscal:n=4 ....dswap:n=4 ..dswap:n=3 .....saxpy:n=4 ..saxpy:n=3 ....scopy:n=4 ..scopy:n=3 .............sscal:n=4 ....sswap:n=4 ..sswap:n=3 .....zaxpy:n=4 ..zaxpy:n=3 ....zcopy:n=4 ..zcopy:n=3 .............zscal:n=4 ....zswap:n=4 ..zswap:n=3 ..Result: [ 4.957975 0.64690335] (exact result = 4.955356249106168, 0.666666666666666) ................. ====================================================================== ERROR: check_integer (scipy.io.tests.test_array_import.test_read_array) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.4/site-packages/scipy/io/tests/test_array_import.py", line 55, in check_integer from scipy import stats File "/usr/lib/python2.4/site-packages/scipy/stats/__init__.py", line 7, in ? from stats import * File "/usr/lib/python2.4/site-packages/scipy/stats/stats.py", line 193, in ? import scipy.special as special File "/usr/lib/python2.4/site-packages/scipy/special/__init__.py", line 10, in ? import orthogonal File "/usr/lib/python2.4/site-packages/scipy/special/orthogonal.py", line 66, in ? from scipy.linalg import eig File "/usr/lib/python2.4/site-packages/scipy/linalg/__init__.py", line 8, in ? from basic import * File "/usr/lib/python2.4/site-packages/scipy/linalg/basic.py", line 18, in ? from scipy.lib.lapack import get_lapack_funcs File "/usr/lib/python2.4/site-packages/scipy/lib/lapack/__init__.py", line 16, in ? import clapack ImportError: /usr/lib/python2.4/site-packages/scipy/lib/lapack/clapack.so: undefined symbol: clapack_sgesv ---------------------------------------------------------------------- Ran 701 tests in 22.651s FAILED (errors=1) In [15]: import scipy.stats --------------------------------------------------------------------------- exceptions.ImportError Traceback (most recent call last) /root/<ipython console> /usr/lib/python2.4/site-packages/scipy/stats/__init__.py 5 from info import __doc__ 6 ----> 7 from stats import * 8 from distributions import * 9 from rv import * /usr/lib/python2.4/site-packages/scipy/stats/stats.py 191 import numpy.core.umath as math 192 from numpy.core.umath import * --> 193 import scipy.special as special 194 import scipy.linalg as linalg 195 /usr/lib/python2.4/site-packages/scipy/special/__init__.py 8 from basic import * 9 import specfun ---> 10 import orthogonal 11 from orthogonal import legendre, chebyt, chebyu, chebyc, chebys, \ 12 jacobi, laguerre, genlaguerre, hermite, hermitenorm, gegenbauer, \ /usr/lib/python2.4/site-packages/scipy/special/orthogonal.py 64 import _cephes as cephes 65 _gam = cephes.gamma ---> 66 from scipy.linalg import eig 67 68 def poch(z,m): /usr/lib/python2.4/site-packages/scipy/linalg/__init__.py 6 from linalg_version import linalg_version as __version__ 7 ----> 8 from basic import * 9 from decomp import * 10 from matfuncs import * /usr/lib/python2.4/site-packages/scipy/linalg/basic.py 16 #from lapack import get_lapack_funcs 17 from flinalg import get_flinalg_funcs ---> 18 from scipy.lib.lapack import get_lapack_funcs 19 from numpy import asarray,zeros,sum,newaxis,greater_equal,subtract,arange,\ 20 conjugate,ravel,r_,mgrid,take,ones,dot,transpose,sqrt,add,real /usr/lib/python2.4/site-packages/scipy/lib/lapack/__init__.py 14 15 import flapack ---> 16 import clapack 17 18 _use_force_clapack = 1 ImportError: /usr/lib/python2.4/site-packages/scipy/lib/lapack/clapack.so: undefined symbol: clapack_sgesv EMERGE INFO: ------------ cowboy ~ # emerge info *** Deprecated use of action 'info', use '--info' instead Portage 2.1_pre7-r5 (default-linux/x86/2005.0, gcc-3.3.6, glibc-2.3.6-r3, 2.6.11-gentoo-r9 i686) ================================================================= System uname: 2.6.11-gentoo-r9 i686 AMD Athlon(tm) Processor Gentoo Base System version 1.12.0_pre17 distcc 2.18.3 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled] ccache version 2.4 [enabled] dev-lang/python: 2.4.2-r1 sys-apps/sandbox: 1.2.17 sys-devel/autoconf: 2.13, 2.59-r7 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2 sys-devel/binutils: 2.16.1-r2 sys-devel/libtool: 1.5.22 virtual/os-headers: 2.6.11-r3 ACCEPT_KEYWORDS="x86 ~x86" AUTOCLEAN="yes" CBUILD="i686-pc-linux-gnu" CFLAGS=" -march=athlon -O2 -pipe -fomit-frame-pointer" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/eselect/compiler /etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c /etc/env.d" CXXFLAGS=" -march=athlon -O2 -pipe -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig ccache distlocks metadata-transfer sandbox sfperms" GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="x86 X alsa apache2 apm arts avi berkdb bitmap-fonts blas crypt cups dba dri eds emacs emboss encode f77 fftw foomaticdb fortran gdbm gif gnome gpm gstreamer gtk gtk2 imlib ipv6 isdnlog jpeg kde libg++ libwww mad mikmod mime mmap motif mp3 mpeg ncurses nls objc ogg oggvorbis opengl oss pam pdflib perl png postgres pppd python qt quicktime readline sdl spell ssl svg tcltk tcpd tidy truetype truetype-fonts type1-fonts unicode vorbis xml2 xmms xv zlib elibc_glibc kernel_linux userland_GNU" Unset: ASFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS, MAKEOPTS