After a system upgrade, I get the following error when using scipy: from scipy.linalg import clapack ImportError: /usr/lib64/python2.4/site-packages/scipy/linalg/clapack.so: undefined symbol: clapack_sgesv This is with MKL. I tried re-ermerging MKL and scipy, and making sure that all the relevant eselects were done, but to no avail. I tried other lapack implementations, but that didn't help. Reproducible: Always Portage 2.1.3.9 (default-linux/amd64/2007.0/desktop, gcc-4.1.2, glibc-2.5-r4, 2.6.22-gentoo-r2 x86_64) ================================================================= System uname: 2.6.22-gentoo-r2 x86_64 Intel(R) Core(TM)2 CPU 6400 @ 2.13GHz Timestamp of tree: Fri, 12 Oct 2007 01:31:01 +0000 app-shells/bash: 3.2_p17 dev-java/java-config: 1.3.7, 2.0.33-r1 dev-lang/python: 2.4.4-r5 dev-python/pycrypto: 2.0.1-r6 sys-apps/baselayout: 1.12.9-r2 sys-apps/sandbox: 1.2.17 sys-devel/autoconf: 2.13, 2.61-r1 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10 sys-devel/binutils: 2.17-r1 sys-devel/gcc-config: 1.3.16 sys-devel/libtool: 1.5.24 virtual/os-headers: 2.6.21 ACCEPT_KEYWORDS="amd64" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=nocona -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config" CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c" CXXFLAGS="-O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="distlocks metadata-transfer sandbox sfperms strict unmerge-orphans userfetch" GENTOO_MIRRORS="ftp://ftp.belnet.be/mirror/rsync.gentoo.org/gentoo/ ftp://distro.ibiblio.org/pub/linux/distributions/gentoo/" MAKEOPTS="-j3" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --filter=H_**/files/digest-*" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="X aalib acl acpi alsa amd64 arts atlas avi bash-completion berkdb bitmap-fonts cairo cdr cli cracklib crypt cups cvs dbus dri dvd dvdr dvdread eds emboss encode esd evo f77 fam firefox flac fortran gdbm gif gphoto2 gpm gstreamer gtk hal iconv imap imlib ipv6 isdnlog jpeg kde kerberos ldap mad midi mikmod mmx mp3 mpeg mudflap ncurses nls nptl nptlonly nsplugin ntplonly nvidia ogg oggvorbis opengl openmp oss pam pcre pda pdf pdflib perl pic png pppd prelink python qt3 qt3support qt4 qtmt quicktime readline reflection samba sdl session smime snmp spell spl sse sse2 ssl subversion svg tcltk tcpd tetex tiff tk truetype truetype-fonts type1-fonts unicode vorbis xml xorg xv" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" ELIBC="glibc" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="nvidia" Unset: CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY
Hi Peter, Haven't really test it with MKL yet. I thought mkl had clapack bindings. I will look into it asap. Sébastien
I could not reproduce it with a similar system. The only thing I could think is the eselect bug again. Remove /etc/env.d/{blas,cblas,lapack}/lib*/config, and re-eselect mkl to all, making sure all libs points to then re-emerge scipy.
I tried your suggestion, but to no avail... I can now reproduce this on a second (non 64-bit) system as well. One thing I noticed, though: if I try importing scipy a second time from the Python prompt, I get a different error: >>> from scipy import * Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/lib/python2.4/site-packages/scipy/linalg/__init__.py", line 8, in ? from basic import * File "/usr/lib/python2.4/site-packages/scipy/linalg/basic.py", line 17, in ? from lapack import get_lapack_funcs File "/usr/lib/python2.4/site-packages/scipy/linalg/lapack.py", line 17, in ? from scipy.linalg import flapack ImportError: cannot import name flapack Not sure if this is relevant, though, could be a leftover from the first failed import.
I can reproduce this on an amd64 box with lapack-reference-3.1.1-r1. I'll have a look at it. Markus
The main problem here is the fact the scipy names the lapack symbols improperly. It seems that setup.py is trying to do a lot of autodetection of various blas/lapack packages. On my box it eventually also picks generic_clapack.pyf which names various symbols as clapack_foo. This of course won't fly since at least lapack-reference and lapack-atlas export them as foo_ (not sure about MKL). I'll post a patch that fixes the issues for me. Please give it a spin and see if it works for you. cheers, Markus
Created attachment 133332 [details, diff] patch to fix improperly named symbols
Success! (at least on 1 of my machines, the other is at work) Thanks for the quick fix, Peter
I've committed -r2 which contains this patch. Thanks, Markus
Looks like this one is back with scipy-0.6.0-r3 and mkl-9.1.023... Portage 2.1.3.19 (default-linux/x86/2007.0/desktop, gcc-4.1.2, glibc-2.6.1-r0, 2.6.22-gentoo-r9 i686) ================================================================= System uname: 2.6.22-gentoo-r9 i686 Intel(R) Pentium(R) M processor 2.00GHz Timestamp of tree: Fri, 18 Jan 2008 02:30:01 +0000 distcc 2.18.3 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled] app-shells/bash: 3.2_p17-r1 dev-java/java-config: 1.3.7, 2.0.33-r1 dev-lang/python: 2.4.4-r6 dev-python/pycrypto: 2.0.1-r6 sys-apps/baselayout: 1.12.10-r5 sys-apps/sandbox: 1.2.18.1-r2 sys-devel/autoconf: 2.13, 2.61-r1 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10 sys-devel/binutils: 2.18-r1 sys-devel/gcc-config: 1.4.0-r4 sys-devel/libtool: 1.5.24 virtual/os-headers: 2.6.23-r3 ACCEPT_KEYWORDS="x86" CBUILD="i686-pc-linux-gnu" CFLAGS="-march=pentium-m -O3 -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/lib/mozilla/defaults/pref /usr/share/config" CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c /etc/udev/rules.d" CXXFLAGS="-O2 -mcpu=i686 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="cvs distlocks metadata-transfer sandbox sfperms strict unmerge-orphans userfetch" GENTOO_MIRRORS="http://cesium.di.uminho.pt/pub/gentoo/ http://fido.online.kz/gentoo ftp://mirror.isp.net.au/pub/gentoo/" LANG="en_US.utf8" LC_ALL="en_US.utf8" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --filter=H_**/files/digest-*" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="X a52 aac accessibility acl acpi alsa amarok amd arts asf automount bash-completion batch berkdb bidi bitmap-fonts bittorrent bl cairo cdparanoia cdr cgi cli cracklib crypt css cups curl dbus dpms dri dvd dvdr dvdread eds emboss encode esd ethereal evo exif fam fasttrack fat ffmpeg fftw firefox flac font-server fortran gdb gdbm gif gnutella gphoto2 gpm gsm gstreamer gtk gtk2 hal hdf5 iconv imap immqt-bc innodb ipv6 isdnlog ivtv jpeg kde kdepim kerberos kqemu latex ldap lirc lm_sensors logrotate mad magic midi mikmod mjpeg mmx mp3 mp4live mpeg mpeg2 mplayer msn mudflap musepack musicbrainz mysql mythtv ncurses nls nptl nptlonly nsplugin nvidia ofx ogg opengl openmp openssh openssl oss pam pcre pda pdf perl png pppd python qt3 qt3support qt4 quicktime readline real reflection samba sdl server session spell spl sqlite sqlite3 sse sse-filters sse2 ssl subversion svg svga tcl tcpd threads tiff tk truetype truetype-fonts type1-fonts udev unicode v4l v4l2 vcd vfat videos vidix visualization vorbis webdav wifi win32codecs wma x86 xinerama xml xorg xv xvid xvmc zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIRC_DEVICES="pctv" USERLAND="GNU" VIDEO_CARDS="nvidia" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LDFLAGS, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Hi Peter, Unfortunately Markus patch could not be applied (see bug #197527). So I tried several blas/lapack combination, could reproduce the behaviour and updated scipy and numpy. The import clapack command should now work. However note that of all lapack implementations, only lapack-atlas bring clapack_* routines, and is incomplete relative to the f77 implementation. So when you import scipy.linalg.clapack with scipy built on mkl, it will not crash but give you an empty module. Only building scipy on lapack-atlas will produce the clapack module. This is the intended behaviour upstream. I am not sure it is worth a use flag though. Let me know if you see any issue with it.
> relative to the f77 implementation. So when you import scipy.linalg.clapack > with scipy built on mkl, it will not crash but give you an empty module. Only > building scipy on lapack-atlas will produce the clapack module. Since it is the upstream intended behaviour, I think this one could be closed.
