Running scipy.test(verbose=2) results in test_singular (test_linsolve.TestLinsolve) ... Segmentation fault This happens in both Python2.7 and Python3.1. I have those installed as scipy dependencies: sci-libs/blas-atlas-3.9.23-r4 sci-libs/lapack-atlas-3.9.23-r3 dev-python/numpy-1.5.1 sci-libs/superlu-4.1-r1 sci-libs/arpack-96-r2 media-libs/qhull-2010.1-r2 Reproducible: Always Portage 2.1.9.42 (default/linux/amd64/10.0/desktop, gcc-4.5.2, glibc-2.11.3-r0, 2.6.36-gentoo-r8-loc2 x86_64) ================================================================= System uname: Linux-2.6.36-gentoo-r8-loc2-x86_64-Intel-R-_Core-TM-2_Quad_CPU_Q9550_@_2.83GHz-with-gentoo-1.12.14 Timestamp of tree: Fri, 15 Apr 2011 14:45:01 +0000 app-shells/bash: 4.1_p9 dev-lang/python: 2.7.1-r1, 3.1.3-r1 dev-util/cmake: 2.8.4 sys-apps/baselayout: 1.12.14-r1 sys-apps/sandbox: 2.4 sys-devel/autoconf: 2.13, 2.65-r1 sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.1 sys-devel/binutils: 2.20.1-r1 sys-devel/gcc: 4.5.2 sys-devel/gcc-config: 1.4.1 sys-devel/libtool: 2.2.10 sys-devel/make: 3.81-r2 sys-kernel/linux-headers: 2.6.36.1 virtual/os-headers: 0 ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA dlj-1.1" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -mtune=native -msse4.1 -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=native -mtune=native -msse4.1 -O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="assume-digests binpkg-logs buildsyspkg distlocks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch" FFLAGS="" GENTOO_MIRRORS="http://ftp.ucsb.edu/pub/mirrors/linux/gentoo/ http://gentoo.osuosl.org/" LANG="en_US" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LINGUAS="en zh zh_CN zh_TW" MAKEOPTS="-j4" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/var/lib/layman/emacs /var/lib/layman/science /var/lib/layman/python /var/lib/layman/sunrise" SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage" USE="X acpi amd64 bash-completion berkdb bzip2 cairo cdr cjk cleartype consolekit corefonts cpudetection cracklib crypt cups cxx dbm dbus dbx dga djvu dri dvd dvdr emacs encode enscript exif fftw firefox fontconfig foomaticdb fortran fortran95 gdu gif git gnutls graphicsmagick graphviz gsl gtk hdf5 ieee1394 imap java6 javascript jpeg jpeg2k kpathsea lapack latex lcms libcaca libnotify libwww lm_sensors logrotate lzo mad matroska md5sum mime mmap mmx mmxext mng modules mono mozilla mp3 mp4 mpeg mplayer msn multilib musepack nas ncurses nls nptl nptlonly nsplugin ntl objc ocaml ocamlopt ogg openal opengl openmp oss pam pango pch pcre pdf perl plotutils png policykit posix ppds pppd profile python qt3support rar readline ropemacs rtc ruby sasl schroedinger semantic session sharedmem smp soap sockets socks5 spell sqlite sqlite3 sse sse2 sse3 ssl ssse3 startup-notification svg symlink sysfs syslog sysvipc szip tcl tcpd theora threads tidy tiff tk trace truetype udev unicode usb vcd vdpau vim-syntax vorbis webkit wmf wxwindows x264 xattr xcb xft xine xinerama xml xorg xpm xprint xscreensaver xulrunner xv xvid xvmc zip zlib zsh-completion" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="dav dav_fs dav_lock" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en zh zh_CN zh_TW" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Attach full build log of scipy an perhaps a gdb backtrace of the crash too.
Please attach the entire build log.
Created attachment 270563 [details] build log
Created attachment 270567 [details] traceback
Could you please try to remove scipy and test again. It seems the test takes some files from the old installation. test_non_square (test_linsolve.TestLinsolve) ... ok test_singular (test_linsolve.TestLinsolve) ... Program received signal SIGSEGV, Segmentation fault. 0x00007fffdc5cc58e in dpivotL () from /usr/lib64/libsuperlu.so.4 (gdb) bt #0 0x00007fffdc5cc58e in dpivotL () from /usr/lib64/libsuperlu.so.4 #1 0x00007fffdc5c8586 in dgstrf () from /usr/lib64/libsuperlu.so.4 #2 0x00007fffdc5c39dc in dgssv () from /usr/lib64/libsuperlu.so.4 #3 0x00007fffdc80f78d in Py_gssv () from /usr/lib64/python2.7/site-packages/scipy/sparse/linalg/dsolve/_superlu.so #4 0x00007ffff7b0163a in PyEval_EvalFrameEx () from /usr/lib64/libpython2.7.so.1.0 #5 0x00007ffff7b02d49 in PyEval_EvalCodeEx () from /usr/lib64/libpython2.7.so.1.0 #6 0x00007ffff7b014cd in PyEval_EvalFrameEx () from /usr/lib64/libpython2.7.so.1.0
okay I got it wrong So your are running the tests outside the ebuild, don't you?
Yes, I am running scipy's tests outside ebuild.
Scipy upstream here --- this is to our knowledge a SuperLU bug, which is patched in the version bundled with Scipy. I'd in general suggest not linking Scipy against the system SuperLU. This particular library just does abort(); or exit(); if it fails to decompose a matrix. This behavior can be modified at SuperLU build time by supplying the USER_* #defines to it, and in Scipy we alter the default behavior so that instead of exiting, appropriate Python exceptions are raised. I don't see how this could be done without rebuilding SuperLU when Scipy is built --- and so we bundle it.
Can't we get your SuperLU patches included by us or SuperLU upstream?
Here's a diff: https://github.com/pv/scipy-work/blob/enh/superlu-update/scipy/sparse/linalg/dsolve/SuperLU/scipychanges.patch The behavior on errors in the unpatched library is not so nice for Scipy --- it just aborts the program, e.g. when encountering including singular matrices in *gsitrf. The Scipy patch redirects aborting error paths to longjmp out, but this really is possible only at compile time, so there are not really good alternatives to building Scipy using the bundled library.
OK so your own patch cannot be just dropped in superLU separately. What about getting superLU to behave better for scipy without making it depend on python stuff from scipy? It then could be forwarded to superLU and you wouldn't have the burden of caring for it. Right now you are just giving yourself the perpetual burden of fixing superLU (until you decide to stop using it).
*** Bug 405075 has been marked as a duplicate of this bug. ***
Perhaps. In the meantime, you probably should drop the crashing patch from the ebuild.
The scipy ChangeLog mentions this bug here along with a notice that qhull isn't unbundled either. Are there any known problems with qhull as well? (In reply to comment #11) > What about getting superLU to behave better for scipy without making it > depend on python stuff from scipy? I suggest the following: create global function pointers superLU_exit and superLU_abort. Patch superlu to export those symbols, to initialize those variables to the normal exit and abort functions, and to use those pointers whenever one of the functions in question would otherwise be called. That way, any application or library could change those pointers, in order to handle problems in a nicer way. scipy could change those handlers for as long as its longjmp target is valid. Using such variables is not perfect, as it might lead to problems when two or more users attempt to change those in a way which isn't sufficiently localized to avoid conflicts. But it beats compile-time configuration any day. Has anybody checked the sources to see whether the superLU library cleans up properly before calling error or abort? Otherwise, turning a terminating error into a non-terminating exception is just asking for memory leaks and inconsistent data. Haven't looked at the code yet, though.
I worked a bit on the scipy patch for superlu. It now compiles but the tests still fail due to not finding the scipy_slu_config.h header. Maybe the patch helps someone to get further.
Created attachment 343326 [details, diff] scipy patch updated for use in superlu. Tests still fail.
Please see whether this is still valid for scipy 0.14/0.15.
