| Summary: | sci-libs/scipy-0.10.1: automagic dependency on libqhull.so | ||
|---|---|---|---|
| Product: | Portage Development | Reporter: | Mathieu Z <mobiusstripper> |
| Component: | Unclassified | Assignee: | Gentoo Science Related Packages <sci> |
| Status: | RESOLVED TEST-REQUEST | ||
| Severity: | normal | CC: | Martin.vGagern |
| Priority: | Normal | ||
| Version: | 2.2 | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: | Rename and always use bundled libqhull | ||
|
Description
Mathieu Z
2012-04-15 22:25:04 UTC
(In reply to comment #0) > At the end of a successful emerge @preserved-rebuild, the following > preserved-lib message persists: > > emerge @preserved-rebuild > ... ... ... ... ... ... ... > !!! existing preserved libs: > >>> package: dev-libs/libffi-3.0.11 > * - /usr/lib64/libffi.so.5 > * - /usr/lib64/libffi.so.5.0.10 > * used by /usr/lib64/ghc-7.4.1/libHSrts-ghc7.4.1.so > (dev-lang/ghc-7.4.1) > * used by /usr/lib64/ghc-7.4.1/libHSrts_debug-ghc7.4.1.so > (dev-lang/ghc-7.4.1) > * used by /usr/lib64/ghc-7.4.1/libHSrts_thr-ghc7.4.1.so > (dev-lang/ghc-7.4.1) > * used by /usr/lib64/ghc-7.4.1/libHSrts_thr_debug-ghc7.4.1.so > (dev-lang/ghc-7.4.1) This one is bug 411789 and bug 411925. > >>> package: media-libs/qhull-2010.1-r2 > * - /usr/lib64/libqhull.so > * used by /usr/lib64/python2.7/site-packages/scipy/spatial/qhull.so > (sci-libs/scipy-0.10.1) > * used by > /usr/lib64/python3.2/site-packages/scipy/spatial/qhull.cpython-32.so > (sci-libs/scipy-0.10.1) > Use emerge @preserved-rebuild to rebuild packages using these libraries Post build.log of scipy here. First from "emerge -1 scipy" and then from after "emerge -C scipy qhull" and "emerge -1 scipy" (In reply to comment #1) > (In reply to comment #0) > > At the end of a successful emerge @preserved-rebuild, the following > > preserved-lib message persists: > > > > emerge @preserved-rebuild > > ... ... ... ... ... ... ... > > !!! existing preserved libs: > > >>> package: dev-libs/libffi-3.0.11 > > * - /usr/lib64/libffi.so.5 > > * - /usr/lib64/libffi.so.5.0.10 > > * used by /usr/lib64/ghc-7.4.1/libHSrts-ghc7.4.1.so > > (dev-lang/ghc-7.4.1) > > * used by /usr/lib64/ghc-7.4.1/libHSrts_debug-ghc7.4.1.so > > (dev-lang/ghc-7.4.1) > > * used by /usr/lib64/ghc-7.4.1/libHSrts_thr-ghc7.4.1.so > > (dev-lang/ghc-7.4.1) > > * used by /usr/lib64/ghc-7.4.1/libHSrts_thr_debug-ghc7.4.1.so > > (dev-lang/ghc-7.4.1) > > This one is bug 411789 and bug 411925. > > > >>> package: media-libs/qhull-2010.1-r2 > > * - /usr/lib64/libqhull.so > > * used by /usr/lib64/python2.7/site-packages/scipy/spatial/qhull.so > > (sci-libs/scipy-0.10.1) > > * used by > > /usr/lib64/python3.2/site-packages/scipy/spatial/qhull.cpython-32.so > > (sci-libs/scipy-0.10.1) > > Use emerge @preserved-rebuild to rebuild packages using these libraries > > Post build.log of scipy here. > First from "emerge -1 scipy" and then from after "emerge -C scipy qhull" and > "emerge -1 scipy" libffi.so.5 and libqhull.so are both orphaned files. Their corresponding packages are removed during an depclean but these are preserved by scipy and ghc. Apparently, re-emerging scipy and ghc do not remove them. However, emerge -C scipy and ghc DO remove them and subsequent re-emerge did not bring them back. This is a valid one. the preserved_libs feature keeps them on the system and scipy chooses them automatically for linking. We need to stop this. Looking at the build log, I noticed these: ar: adding 16 object files to build-2.7/temp.linux-x86_64-2.7/libqhull.a […] /usr/bin/x86_64-pc-linux-gnu-gfortran -Wall -Wl,--as-needed -shared build-2.7/temp.linux-x86_64-2.7/scipy/spatial/qhull.o -L/usr/lib64 -L/usr/lib64 -Lbuild-2.7/temp.linux-x86_64-2.7 -lqhull -llapack -lblas -latlas -lcblas -lcblas -lblas -latlas -lpython2.7 -lgfortran -o build-2.7/lib.linux-x86_64-2.7/scipy/spatial/qhull.so This looks to me as if scipy were building its own libqhull.a static library, and attempt to link that into qhull.so. But when there is a libqhull.so available on the system, the build will use that instead, resulting in the observed library dependency. Looking at the source, the libqhull sources bundled with scipy appear to be from 2010. There have been some changes since then, but nothing radical. So I guess the most gentooish way would be depending on system libqhull and getting rid of the bundled one. However, there is this entry in the scipy ChangeLog: 06 Mar 2012; Sébastien Fabbro <bicatali@gentoo.org> +scipy-0.10.1.ebuild: Version bump. Unfortunately abandoning our effort unbundling superlu and qhull until upstream fix their issues (bug #364083) There were versions which depended on libqhull, but apparently there was a reason not to do so at the moment. The bug in question mostly talks about superlu, so I'm not sure how much it applies to qhull as well. If there is enough progress on those unbundling efforts, we might simply wait for a solution there, with workarounds along the lines of comment #2. If no solution to the unbundling approach is expected soon, then it would be better to make sure never to use the system libqhull. That could be achieved either by renaming the library (e.g. from "libqhull.a" to "libscipy_qhull.a") or by changing the compiler arguments to explicitely name the file (i.e. from "-Lbuild-2.7/temp.linux-x86_64-2.7 -lqhull" to "build-2.7/temp.linux-x86_64-2.7/libqhull.a"). Not sure where the latter should be done, and how it affects portability, so I'd go for the former. I'm a bit surprised that I found no unbundling efforts on Debian or Fedora. Apparently there are no ready-to-use patches there yet. But if we go for certain bundling instead of unbundling, then they won't have a problem as long as they build their binary packages inside a chroot with no libqhull installed. Created attachment 313965 [details, diff]
Rename and always use bundled libqhull
This patch renames the bundled libqhull. Now the bundled version will always be used, ignoring any version which might be installed on the system in the regular way, preserved by portage, or otherwise accessible in some fashion. Compiles for me, and gets rid of the library dependency according to ldd.
I think the patch is fine. It avoid library confusion. We should probably give superLU the same treatment. I am not sure we should do that bit of hard work for upstream, unless we are sure they will adopt it. Please see whether this is still valid for scipy 0.14/0.15. qlist scipy | grep so$ | xargs scanelf -n | grep qhull ET_DYN libptlapack.so.3,libatlcblas.so.3,libf77blas.so.3,libpython2.7.so.1.0,libm.so.6,libc.so.6 /usr/lib64/python2.7/site-packages/scipy/spatial/qhull.so ET_DYN libptlapack.so.3,libatlcblas.so.3,libf77blas.so.3,libpython3.3.so.1.0,libm.so.6,libc.so.6 /usr/lib64/python3.3/site-packages/scipy/spatial/qhull.cpython-33.so ET_DYN libptlapack.so.3,libatlcblas.so.3,libf77blas.so.3,libpython3.4.so.1.0,libm.so.6,libc.so.6 /usr/lib64/python3.4/site-packages/scipy/spatial/qhull.cpython-34.so |
