Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 688392 - dev-python/numpy Numpy cannot detect CBLAS while compiling numpy.core.multiarray.
Summary: dev-python/numpy Numpy cannot detect CBLAS while compiling numpy.core.multiar...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: AMD64 Linux
: Normal normal (vote)
Assignee: Gentoo Linux bug wranglers
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-06-20 14:20 UTC by ZongyuZ
Modified: 2019-06-23 12:47 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
the output of emerge --info (emerge--info.txt,12.04 KB, text/plain)
2019-06-20 14:20 UTC, ZongyuZ
Details
the patch created by me. It should solve the problem. (numpy-1.14.5.ebuild.patch,614 bytes, patch)
2019-06-20 14:21 UTC, ZongyuZ
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description ZongyuZ 2019-06-20 14:20:19 UTC
Created attachment 580220 [details]
the output of emerge --info

Numpy cannot detect CBLAS while compiling numpy.core.multiarray.

Gentoo parses cblas.pc while the host is trying to emerge dev-python/numpy with USE:lapack enabled.

The ebuild uses `pkg-config --cflags-only-I cblas` to detect the 'include_dirs' to be added in 'site.cfg' of numpy, which causes the problem.

This command will print nothing, if I'm using cblas-reference as my cblas provider, because -I/usr/include is enabled by default.

But numpy detects cblas interface by:
"obj = c.compile([src], output_dir=tmpdir,
    include_dirs=self.get_include_dirs())",
where 'self.get_include_dirs()' will return None, since site.cfg defines 'include_dirs' with nothing.
(https://github.com/numpy/numpy/blob/master/numpy/distutils/system_info.py#L1893)

The variable 'src' is a source file which contains a cblas function.
The compilation will fail, since the compiler cannot find <cblas.h>.

Then numpy will think there is no CBLAS while compiling numpy.core.multiarray, then fallback to its own implementation.

As far as I know, it will affect numpy.matmul(), and make numpy.matmul() very slow.

Whether the multiarray module is compiled with CBLAS can be checked out by:
```
ldd /usr/lib64/python3.6/site-packages/numpy/core/multiarray.cpython-36m-x86_64-linux-gnu.so
```

So the soluton is to add a valid include_dir, then the CBLAS-detection will not fail, and numpy.matmul() will use cblas instead of numpy's implementation.

I'll provide a numpy-1.14.5.patch, which seems to solve the problem, and my system infomation. Hope they will help.
Comment 1 ZongyuZ 2019-06-20 14:21:37 UTC
Created attachment 580222 [details, diff]
the patch created by me. It should solve the problem.
Comment 2 Larry the Git Cow gentoo-dev 2019-06-23 12:47:48 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=f5c71a4aa9615c8c14caa8e7076519ab2a4b9824

commit f5c71a4aa9615c8c14caa8e7076519ab2a4b9824
Author:     Benda Xu <heroxbd@gentoo.org>
AuthorDate: 2019-06-23 12:46:21 +0000
Commit:     Benda Xu <heroxbd@gentoo.org>
CommitDate: 2019-06-23 12:47:37 +0000

    dev-python/numpy: fix cblas detection.
    
      Drop old ebuilds versionator eclass.
      Drop live ebuild.  It has not been taken care of since numpy-1.9.
    
    Suggested-By: Zongyu Zhang <zzy2529420793@gmail.com>
    Closes: https://bugs.gentoo.org/647072
    Closes: https://bugs.gentoo.org/688392
    Package-Manager: Portage-2.3.52, Repoman-2.3.12
    Signed-off-by: Benda Xu <heroxbd@gentoo.org>

 dev-python/numpy/Manifest                          |  16 --
 .../files/numpy-1.10.2-no-hardcode-blas.patch      |  64 --------
 .../files/numpy-1.12.1-no-hardcode-blas.patch      |  72 ---------
 .../numpy/files/numpy-1.8.0-no-hardcode-blas.patch |  25 ---
 .../numpy/files/numpy-1.9.2-no-hardcode-blas.patch |  35 -----
 dev-python/numpy/numpy-1.10.4.ebuild               | 154 -------------------
 dev-python/numpy/numpy-1.13.3.ebuild               | 167 ---------------------
 dev-python/numpy/numpy-1.14.5.ebuild               |   3 +-
 dev-python/numpy/numpy-1.15.4.ebuild               |   3 +-
 dev-python/numpy/numpy-1.16.1.ebuild               |   3 +-
 dev-python/numpy/numpy-1.8.2.ebuild                | 145 ------------------
 dev-python/numpy/numpy-1.9.2.ebuild                | 146 ------------------
 dev-python/numpy/numpy-9999.ebuild                 | 134 -----------------
 13 files changed, 6 insertions(+), 961 deletions(-)