Bug 928506 – sci-libs/lapack-3.12.0-r1[eselect-ldso] and sci-libs/openblas-0.3.26[eselect-ldso] do not install properly breaking "Blas-lapack-switch"

• Status: UNCONFIRMED
• Product: Gentoo Linux

[Description ktoupt 2024-04-03 09:22:00 UTC]

Basically, this is about configuring dynamic linker to load a specific
implementation of a library. Quick summary of the way I think it is
supposed to work:

1. `eselect {blas,lapack}` adds configuration in /etc/ld.so.conf.d for
the dynamic linker to find the desired implementation. For example,
/usr/lib/{blas,lapack}/openblas.

2. At run-time, the linker looks at directories, starting at the default
directories, like /usr/lib, does not find the desired library, then
looks in the custom directories.

3. Finally, it loads the unversioned name, which is a symlink to the
versioned name.

Refer to <https://wiki.gentoo.org/wiki/Blas-lapack-switch> for more
details.


But, there are a couple of issues with the way
sci-libs/lapack-3.12.0-r1[eselect-ldso] and
sci-libs/openblas-0.3.26[eselect-ldso] are installed:

sci-libs/lapack, the reference implementation, is installing directly
into /usr/lib, then creating symlinks in
/usr/lib/{blas,lapack}/reference back to /usr/lib. /usr/lib is the first
place the linker looks at, so the reference implementation will always
be used regardless of the other paths configured in
/etc/ld.so.conf.d(?).

sci-libs/openblas also installs directly into /usr/lib, but,
specifically as libopenblas (instead of just lib{blas,lapack}, so this
is not a problem. However, in /usr/lib/{lapack/openblas}/openblas, the
unversioned filenames are symlinks to the reference ones installed in
/usr/lib! So, even if the linker started looking in here instead of
/usr/lib, it would still load the reference implementation!


Basically, what I think should be done is the following:

sci-libs/lapack[eselect-ldso] should not install anything in /usr/lib;
it should only install in /usr/lib/{blas,lapack}/reference.

And, sci-libs/openblas[eselect-ldso] should just fix the symlinks to be
pointing to itself instead of back at /usr/lib, so, e.g.

	dosym -r /usr/$(get_libdir)/libblas.so.3 /usr/$(get_libdir)/blas/openblas/libblas.so

should be changed to

	dosym -r /usr/$(get_libdir)/blas/openblas/libblas.so /usr/$(get_libdir)/blas/openblas/libblas.so


I haven't checked the other providers, like sci-libs/mkl or
sci-libs/blis.


Also, somewhat related, for some reason the
/etc/env.d/{blas,lapack}/lib/openblas files were not generated for me
initially, making it impossible to use eselect to switch. I manually
changed the "config" in there to openblas, which really does nothing,
then rebuilt sci-libs/lapack and sci-libs/openblas (with no other
changes) and somehow that generated the correct openblas file. Not sure
why it wasn't generated the first time or where it is being generated
from...

[Comment 1 Sam James 2024-04-03 09:23:30 UTC]

I think we have bug 923998 for openblas.

[Comment 2 Sam James 2024-04-03 09:24:12 UTC]

Would you mind making one bug per issue? openblas paths being wrong is bug 923998 I think. We can use this one for one specific thing with lapack.

Please file new ones for others and cross-link them.

[Comment 3 ktoupt 2024-04-03 11:27:44 UTC]

Yes, bug 923998 is correct for openblas. I don't know how I missed that.


As for the sci-libs/lapack, I am starting to think the way it is
installed is correct. It is desired to link to reference if the program
does not rely on extensions provided by another implementation. So
having the reference implementation in /usr/lib sounds good. See
<https://wiki.gentoo.org/wiki/BLAS_and_LAPACK_Providers#Potential_Hazards>

Instead, I think the real bug has to do with the dynamic linker (and
possibly merged-usr and musl). See bug 928509, which I just opened.


Finally, for the missing /etc/env.d/{blas,lapack}/lib/openblas files, I
am not sure if that is worth opening a bug report for, as I could be
misremembering something or did something wrong. Although, it would be
interesting to find what generates those files.

[Comment 4 Benda Xu 2024-05-24 04:28:57 UTC]

> sci-libs/openblas also installs directly into /usr/lib, but,
specifically as libopenblas (instead of just lib{blas,lapack}, so this
is not a problem.

Sorry I don't understand what you are referring to. I have sci-libs/openblas[eselect-ldso]:

/opt/gentoo/usr/lib64
/opt/gentoo/usr/lib64/blas
/opt/gentoo/usr/lib64/blas/openblas
/opt/gentoo/usr/lib64/blas/openblas/libblas.so -> libblas.so.3
/opt/gentoo/usr/lib64/blas/openblas/libblas.so.3
/opt/gentoo/usr/lib64/blas/openblas/libcblas.so -> libcblas.so.3
/opt/gentoo/usr/lib64/blas/openblas/libcblas.so.3
/opt/gentoo/usr/lib64/cmake
/opt/gentoo/usr/lib64/cmake/openblas
/opt/gentoo/usr/lib64/cmake/openblas/OpenBLASConfig.cmake
/opt/gentoo/usr/lib64/cmake/openblas/OpenBLASConfigVersion.cmake
/opt/gentoo/usr/lib64/lapack
/opt/gentoo/usr/lib64/lapack/openblas
/opt/gentoo/usr/lib64/lapack/openblas/liblapack.so -> liblapack.so.3
/opt/gentoo/usr/lib64/lapack/openblas/liblapack.so.3
/opt/gentoo/usr/lib64/lapack/openblas/liblapacke.so -> liblapacke.so.3
/opt/gentoo/usr/lib64/lapack/openblas/liblapacke.so.3
/opt/gentoo/usr/lib64/libopenblas.so -> libopenblas_zen-r0.3.23.so
/opt/gentoo/usr/lib64/libopenblas.so.0 -> libopenblas_zen-r0.3.23.so
/opt/gentoo/usr/lib64/libopenblas_zen-r0.3.23.so
/opt/gentoo/usr/lib64/pkgconfig
/opt/gentoo/usr/lib64/pkgconfig/openblas.pc

and there is no files in /usr/lib, how can I reproduce your issue?

[Comment 5 Sam James 2024-05-24 04:31:16 UTC]

We fixed that one in bug 923998. The remaining issue here is the musl thing.