Bug 556576 – dev-python/numpy-1.9.2[lapack] Intel MKL fatal error: Cannot load libmkl_avx2.so

• Product: Gentoo Linux
• Status: RESOLVED TEST-REQUEST

[Description Tiziano Müller (RETIRED) 2015-08-03 11:53:44 UTC]

I built numpy with blas/cblas/blacs/lapack set to mkl64-gfortran/gcc-openmp (with the virtuals and eselect module from the sci-overlay). 

When trying to run the following minimal example:

  from numpy.linalg import _umath_linalg
  from numpy import array
  _umath_linalg.inv(array([[1,2],[4,0]]))

python crashes with the following error message:

  Intel MKL FATAL ERROR: Cannot load libmkl_avx2.so or libmkl_def.so.

Reading through https://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl or http://www.scipy.org/scipylib/building/linux.html#intel-mkl-11-0-updated-dec-2012 makes me think that numpy should handle mkl separately, rather than generic via the pkg-config wrappers from the sci-overlay.

[Comment 1 François Bissey 2015-08-03 21:35:59 UTC]

I don't have MKL but it looks to me like `ld.so.conf` would need to be updated with the proper folder. Barring that, setting `LD_RUN_PATH` is a solution that would work with gnu and intel compilers (at least by the look of the page you linked).

The problem with `LD_RUN_PATH` is that it will be discarded if there are any explicit stuff `-Wl,-rpath` at linking time. IBM xl fortran compilers do that to make sure you get the right runtime libraries by default making `LD_RUN_PATH` useless in that particular example.

So I think you should have the right paths added in a file in `/etc/ld.so.conf.d/` have `ldconfig` run properly and it should work.

[Comment 2 Tiziano Müller (RETIRED) 2015-08-04 11:46:49 UTC]

... and when selecting mkl64-dynamic-openmp for lapack/blas, it seems that the installed lapack library is completely ignored since none of the shared libraries link to mkl_rt/iomp as they should.

[Comment 3 Tiziano Müller (RETIRED) 2015-08-04 11:50:18 UTC]

(In reply to Francois Bissey from comment #1)
> I don't have MKL but it looks to me like `ld.so.conf` would need to be
> updated with the proper folder. Barring that, setting `LD_RUN_PATH` is a
> solution that would work with gnu and intel compilers (at least by the look
> of the page you linked).

 $ grep -i intel /etc/ld.so.conf
/opt/intel/composerxe-2015_update3.3.187/compiler/lib/intel64
/opt/intel/composerxe-2015_update3.3.187/mpirt/lib/intel64
/opt/intel/composerxe-2015_update3.3.187/compiler/lib/ia32
/opt/intel/composerxe-2015_update3.3.187/mpirt/lib/ia32
/opt/intel/composerxe-2015.3.187/mkl/lib/intel64
/opt/intel/composerxe-2015.3.187/mkl/lib/ia32


> 
> The problem with `LD_RUN_PATH` is that it will be discarded if there are any
> explicit stuff `-Wl,-rpath` at linking time. IBM xl fortran compilers do
> that to make sure you get the right runtime libraries by default making
> `LD_RUN_PATH` useless in that particular example.
> 
> So I think you should have the right paths added in a file in
> `/etc/ld.so.conf.d/` have `ldconfig` run properly and it should work.

it is well possible that this is a problem with the Intel libraries themselves since a `strace` shows that the mentioned libraries are being successfully opened.

[Comment 4 François Bissey 2015-08-05 01:17:22 UTC]

I talked about /etc/ld.conf because it is a typical message for something not configured properly in there.

Now I would be curious about that bit of strace output. What feels likely to me is that the version of the library found is not suitable (32bits instead of 64 for example) and is later rejected and the right one is not searched for somehow.