Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 201921 - numpy-1.0.4 broken: libgomp.so.1: shared object cannot be dlopen()e
Summary: numpy-1.0.4 broken: libgomp.so.1: shared object cannot be dlopen()e
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All Linux
: Normal enhancement (vote)
Assignee: Gentoo Science Related Packages
URL:
Whiteboard:
Keywords:
Depends on: 258219
Blocks:
  Show dependency tree
 
Reported: 2007-12-11 08:00 UTC by Donnie Berkholz (RETIRED)
Modified: 2010-06-23 08:41 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Donnie Berkholz (RETIRED) gentoo-dev 2007-12-11 08:00:41 UTC
I couldn't compile python-biggles against numpy because of this error:

donnie@supernova $ python
Python 2.5.1 (r251:54863, Nov 29 2007, 02:26:13)
[GCC 4.2.2 (Gentoo 4.2.2 p1.0)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
imTraceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/site-packages/numpy/__init__.py", line 43, in <modul
e>
    import linalg
  File "/usr/lib/python2.5/site-packages/numpy/linalg/__init__.py", line 4, in
<module>
    from linalg import *
  File "/usr/lib/python2.5/site-packages/numpy/linalg/linalg.py", line 25, in <
module>
    from numpy.linalg import lapack_lite
ImportError: libgomp.so.1: shared object cannot be dlopen()ed
>>>
Comment 1 Markus Dittrich (RETIRED) gentoo-dev 2007-12-11 11:19:54 UTC
Hi Donnie,

If you have USE='-openmp' for gcc you may have had
this turned on in the past and some lib probably
linked against libgomp in the process and needs to be
rebuild now. Does revdep-rebuild pick up anything?

cheers,
Markus
Comment 2 Donnie Berkholz (RETIRED) gentoo-dev 2007-12-12 07:49:36 UTC
I have openmp on right now (and have always had it on) and that library exists on my system at /usr/lib/gcc/i686-pc-linux-gnu/4.2.2/libgomp.so.1.
Comment 3 Markus Dittrich (RETIRED) gentoo-dev 2007-12-12 11:28:08 UTC
I've got it too and numpy works just fine. I don't think numpy has any
OpenMP code in it hence I am confused as to what would try pulling libgomp
in. I've checked all *.so in my numpy install and there's no libgomp
dependency to be found particularly none in lapack_lite.so. I presume
recompiling doesn't fix it?
Comment 4 Sébastien Fabbro (RETIRED) gentoo-dev 2007-12-12 11:35:08 UTC
works fine here with gcc-4.2.2 and as-needed on both amd64 and x86.

What are the blas/cblas/lapack you are with, and which FFLAGS did you use to compile those? Any -fopenmp in the gfortran flags?
Comment 5 Donnie Berkholz (RETIRED) gentoo-dev 2007-12-13 06:55:03 UTC
This was a fresh compile... a workaround is building numpy -lapack, so it appears to be a problem related to the external libs. Here's some clues that might help:

donnie@supernova $ eselect lapack show
lib: acml-gfortran-openmp
donnie@supernova $ eselect blas show
lib: goto
donnie@supernova $ eselect cblas show
lib: mkl-threads
Comment 6 Sébastien Fabbro (RETIRED) gentoo-dev 2007-12-13 08:49:01 UTC
> donnie@supernova $ eselect lapack show
> lib: acml-gfortran-openmp
> donnie@supernova $ eselect blas show
> lib: goto
> donnie@supernova $ eselect cblas show
> lib: mkl-threads

Were you trying your best to break the system :) ? 
As commented in our recent blas/lapack docs at
http://www.gentoo.org/proj/en/science/blas-lapack.xml
this is not really safe.
Though we have no way to prevent such combinations. Probably the best will be for the eselect-{blas,cblas,lapack} to only allow working ones.

Meanwhile I will try to reproduce your breakage to see if I missed some linking options in the pkg-config files.

Comment 7 Donnie Berkholz (RETIRED) gentoo-dev 2007-12-13 08:58:47 UTC
Well, I am interested in performance. =) All I see in that guide that seems close is something that says you should use gcc to avoid trouble. Could you point me to the part you're talking about?
Comment 8 Sébastien Fabbro (RETIRED) gentoo-dev 2007-12-13 09:10:23 UTC
(In reply to comment #7)
> Well, I am interested in performance. =) All I see in that guide that seems
> close is something that says you should use gcc to avoid trouble. Could you
> point me to the part you're talking about?
> 

You're right:"Try to be consistent" was only in the compiler section. I will update it to also have it in the switching libraries section.
Performance is mostly due to the blas kernels. Depending on your hardware I would recommend:
- intel based: mkl for blas,cblas,lapack
- amd based: amcl for blas,lapack + cblas-reference for cblas
- all: blas-atlas for blas,cblas and lapack-atlas for lapack
- all: blas-goto for blas, cblas-reference for cblas, lapack-reference for lapack
But I haven't done the benchmarking really.
Comment 9 Sébastien Fabbro (RETIRED) gentoo-dev 2007-12-19 13:54:20 UTC
Hi,

numpy with acml for blas,lapack and reference for cblas won't compile as well, with the same crash that Donnie faced.
I tracked it down on acml (which I updated) and it looks a as-needed problem when you link anything with acml compiled with gfortran. It seems that the folks at AMD forgot to link with gfortran before packaging. I submitted the problem upstream.

As far as this bug is concerned Donnie, could you check the numpy failure with the new acml and with/without the as-needed LDFLAGS?
Comment 10 Donnie Berkholz (RETIRED) gentoo-dev 2007-12-19 18:53:01 UTC
I can't test acml 4.x because I don't have an amd64, so even if AMD fixes the bug, it'll probably never work for me in 3.x. Do you want me to install acml or numpy or both without as-needed?
Comment 11 Sébastien Fabbro (RETIRED) gentoo-dev 2007-12-19 19:34:14 UTC
I just committed the fixes also for the older versions.
On x86, you can use the 3.6* series.
Comment 12 Sébastien Fabbro (RETIRED) gentoo-dev 2008-03-31 14:54:08 UTC
Update to reproduce:
$ emerge acml
$ eselect blas set acml-gfortran-openmp
$ eselect lapack set acml-gfortran-openmp
$ emerge numpy   # with lapack flag
$ python -c "import numpy"
[...]
 "libgomp.so.1: shared object cannot be dlopen()e"

Although acml still does not work with as-needed, it seems it is a gcc-4.2 bug: see http://gcc.gnu.org/bugzilla/show_bug.cgi?id=28482
Fixed in gcc-4.3 and hopefully will be backported to gcc-4.2.4.
Meanwhile, I updated the pkg-config files of acml to force -Wl,--no-as-needed.
Comment 13 Sébastien Fabbro (RETIRED) gentoo-dev 2008-07-08 10:31:53 UTC
(In reply to comment #12)

> Although acml still does not work with as-needed, it seems it is a gcc-4.2 bug:
> see http://gcc.gnu.org/bugzilla/show_bug.cgi?id=28482
> Fixed in gcc-4.3 and hopefully will be backported to gcc-4.2.4.

It seems that no fix in gcc-4.2.4, and acml is incompatible with gcc-4.3.
Comment 15 Justin Lecher (RETIRED) gentoo-dev 2010-06-23 08:41:34 UTC
I understand this bug as fixed. If not, please reopen or comment.