Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 469116 - dev-python/numpy fails to detect ATLAS
Summary: dev-python/numpy fails to detect ATLAS
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All All
: Normal normal (vote)
Assignee: Gentoo Science Related Packages
Keywords: PATCH
Depends on:
Reported: 2013-05-08 21:54 UTC by brandsmeier
Modified: 2019-06-23 15:38 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---

patch/workaround (numpy_fix.patch,1.56 KB, text/plain)
2013-05-08 21:55 UTC, brandsmeier
original error message when emerging scipy (error_log.txt,7.72 KB, text/plain)
2013-05-08 21:56 UTC, brandsmeier
emerge --info (emerge_info.txt,5.74 KB, text/plain)
2013-05-08 21:56 UTC, brandsmeier
numpy build log (build.log,218.45 KB, text/x-log)
2014-08-12 11:52 UTC, Tolga Dalman
numpy site.cfg (site.cfg,185 bytes, text/plain)
2014-08-12 11:53 UTC, Tolga Dalman

Note You need to log in before you can comment on or make changes to this bug.
Description brandsmeier 2013-05-08 21:54:35 UTC
Building scipy fails because no blas/cblas/lapack libraries are found. I have atlas installed and want scipy to use it. This seems to be a problem of numpy instead, as scipy relies on numpy to detect it.

I debugged this to be because `/usr/lib64/python2.7/site-packages/numpy/distutils/` does not search for ATLAS in the gentoo style. I attached a patch that succesfully works to detect ATLAS on my gentoo system and also has a working scipy.
Comment 1 brandsmeier 2013-05-08 21:55:27 UTC
Created attachment 347778 [details]
Comment 2 brandsmeier 2013-05-08 21:56:09 UTC
Created attachment 347780 [details]
original error message when emerging scipy
Comment 3 brandsmeier 2013-05-08 21:56:31 UTC
Created attachment 347782 [details]
emerge --info
Comment 4 brandsmeier 2013-05-08 22:14:12 UTC
Currently installed numpy:

[ebuild   R   ~] dev-python/numpy-1.7.1  USE="lapack -doc {-test}" PYTHON_TARGETS="python2_7 python3_2 -python2_5 -python2_6 -python3_1 (-python3_3)" 0 kB
Comment 5 Justin Lecher gentoo-dev 2013-05-09 09:47:45 UTC
which atlas are you using? from sci overlay?
Comment 6 brandsmeier 2013-05-09 22:57:13 UTC
Yes, apparently I am using atlas from the science overlay. But I didn't pay good attention here, I had atlas-3.8 installed. The current version in the science overlay is atlas-3.10. Unfortunately, at the moment I am unable to update atlas to the 3.10 version. 

Now I installed ATLAS from source, and that actuall compiles without problems. It actually also contains files such as libf77blas.a that scipy was searching for an my 3.8 version didn't contain. But I still fail to get it to work with numpy and scipy from gentoo (it is installed in /usr/local/atlas).

The failure of the atlas-3.10 version in the science overlay is:

/usr/bin/gcc -o ATL_dtrsmRUNU.o -c -DL2SIZE=4194304 -I/var/tmp/portage/sci-libs/atlas-3.10.1-r1/work/ATLAS_shared/include -I/var/tmp/portage/sci-libs/atlas-3.10.1-r1/work/ATLAS//include -I/var/tmp/portage/sci-libs/atlas-3.10.1-r1/work/ATLAS//include/contrib -DAdd_ -DF77_INTEGER=int -DStringSunStyle -DATL_OS_Linux -DATL_ARCH_Corei2 -DATL_CPUMHZ=2401 -DATL_AVX -DATL_SSE3 -DATL_SSE2 -DATL_SSE1 -DATL_USE64BITS -DATL_GAS_x8664 -m64 -DWALL -DATL_FULL_LAPACK -DATL_NCPU=8 -march=nocona -O3 -pipe -fomit-frame-pointer -fPIC -m64 -DDREAL -DUpper_ \
         -DNotranspose_ -DUnitDiag_ /var/tmp/portage/sci-libs/atlas-3.10.1-r1/work/ATLAS//src/blas/level3/kernel/ATL_trsmR.c
/var/tmp/portage/sci-libs/atlas-3.10.1-r1/work/ATLAS//src/blas/level3/kernel/ATL_trsmKL_rk4.c: In function ‘ATL_rk4’:
/var/tmp/portage/sci-libs/atlas-3.10.1-r1/work/ATLAS//src/blas/level3/kernel/ATL_trsmKL_rk4.c:31:4: error: unknown type name ‘__m256d’
/var/tmp/portage/sci-libs/atlas-3.10.1-r1/work/ATLAS//src/blas/level3/kernel/ATL_trsmKL_rk4.c:32:4: error: unknown type name ‘__m256d’
Comment 7 Martin Luessi 2013-05-16 18:19:45 UTC
The proposed patch works for me with atlas 3.10 from the science overlay and numpy 1.7.1.
Comment 8 Justin Lecher gentoo-dev 2013-05-16 18:24:29 UTC
(In reply to comment #7)
> The proposed patch works for me with atlas 3.10 from the science overlay and
> numpy 1.7.1.

The patch works because you still have leftovers from the old installation. what happens if you do

rm /usr/lib{,64}/lib{blas,lapack}.{a,so*}

and try again without the patch?
Comment 9 Tolga Dalman 2014-08-12 08:42:02 UTC
Just hit the same error with numpy-1.8.1. I just don't understand how this can happen. Also I haven't any atlas libary installed (not as such; is it part of BLAS ?).
Comment 10 François Bissey 2014-08-12 09:11:32 UTC
The error is relatively generic in case of failure to find a suitable blas/lapack, ATLAS or otherwise.
So before anything else you should provide the following info
eselect blas list
eselect cblas list
eselect lapack list
Comment 11 Tolga Dalman 2014-08-12 11:31:47 UTC
# eselect blas list
Installed BLAS for library directory lib64
  [1]   reference *
# eselect cblas list
Installed CBLAS for library directory lib64
  [1]   gsl
  [2]   reference *
# eselect lapack list
Installed LAPACK for library directory lib64
  [1]   reference *
Comment 12 François Bissey 2014-08-12 11:36:00 UTC
Can you attach your own build log and site.cfg that should be in ${S}.
Comment 13 Tolga Dalman 2014-08-12 11:52:00 UTC
Created attachment 382738 [details]
numpy build log
Comment 14 Tolga Dalman 2014-08-12 11:53:17 UTC
Created attachment 382740 [details]
numpy site.cfg
Comment 15 François Bissey 2014-08-12 18:21:13 UTC
I think you have a different problem. blas/lapack is detected at the beginning and something fails later. You should have a build log for python 2.7 and one for 3.3 I would need them as they are mangled in this log and it makes it difficult to stop what happened exactly.
Comment 16 Tolga Dalman 2014-08-12 19:38:53 UTC
Indeed. I found the issue - numpy doesn't work (anymore?) with -flto. Disabling that flag resolves the issue. Thanks for looking into it.
Comment 17 François Bissey 2014-08-12 21:40:04 UTC
Interesting. I am not using it myself but I have a hard time thinking numpy does something specific to prevent lto. If it really doesn't work it should be filtered.
Comment 18 Roger 2015-10-05 19:36:31 UTC
dev-python/numpy seems to not handle compiling well using distcc and/or distcc-pump.

The following worked for me:

# FEATURES="-distcc -distcc-pump" emerge dev-python/nump
Comment 19 Benda Xu gentoo-dev 2019-06-23 12:50:07 UTC
Hi, does the fix of Bug 688392 solve this puzzle, too?
Comment 20 Roger 2019-06-23 15:38:35 UTC
Comment #19:  I'm currently not using DistCC at this current time.