At the end of a successful emerge @preserved-rebuild, the following preserved-lib message persists: emerge @preserved-rebuild ... ... ... ... ... ... ... !!! existing preserved libs: >>> package: dev-libs/libffi-3.0.11 * - /usr/lib64/libffi.so.5 * - /usr/lib64/libffi.so.5.0.10 * used by /usr/lib64/ghc-7.4.1/libHSrts-ghc7.4.1.so (dev-lang/ghc-7.4.1) * used by /usr/lib64/ghc-7.4.1/libHSrts_debug-ghc7.4.1.so (dev-lang/ghc-7.4.1) * used by /usr/lib64/ghc-7.4.1/libHSrts_thr-ghc7.4.1.so (dev-lang/ghc-7.4.1) * used by /usr/lib64/ghc-7.4.1/libHSrts_thr_debug-ghc7.4.1.so (dev-lang/ghc-7.4.1) >>> package: media-libs/qhull-2010.1-r2 * - /usr/lib64/libqhull.so * used by /usr/lib64/python2.7/site-packages/scipy/spatial/qhull.so (sci-libs/scipy-0.10.1) * used by /usr/lib64/python3.2/site-packages/scipy/spatial/qhull.cpython-32.so (sci-libs/scipy-0.10.1) Use emerge @preserved-rebuild to rebuild packages using these libraries Reproducible: Always Steps to Reproduce: 1. emerge @preserved-rebuild 2. 3. # emerge --info Portage 2.2.0_alpha100 (default/linux/amd64/10.0, gcc-4.5.3, glibc-2.14.1-r3, 3.2.6-ck x86_64) ================================================================= System uname: Linux-3.2.6-ck-x86_64-Intel-R-_Core-TM-_i7_CPU_L_620_@_2.00GHz-with-gentoo-2.1 Timestamp of tree: Sat, 14 Apr 2012 06:00:01 +0000 ccache version 3.1.7 [enabled] app-shells/bash: 4.2_p24 dev-java/java-config: 2.1.11-r3 dev-lang/python: 2.7.3, 3.2.2-r1 dev-util/ccache: 3.1.7 dev-util/cmake: 2.8.7-r5 dev-util/pkgconfig: 0.26 sys-apps/baselayout: 2.1 sys-apps/openrc: 0.9.9.3 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.68 sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.5 sys-devel/binutils: 2.22-r1 sys-devel/gcc: 4.5.3-r2 sys-devel/gcc-config: 1.7 sys-devel/libtool: 2.4.2 sys-devel/make: 3.82-r3 sys-kernel/linux-headers: 3.3 (virtual/os-headers) sys-libs/glibc: 2.14.1-r3 Repositories: x11 lisp science desktop-effects gentoo-zh gentoo-haskell java-overlay cj-overlay sunrise mz-overlay gentoo Installed sets: ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -fomit-frame-pointer -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/openvpn/easy-rsa" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=native -fomit-frame-pointer -O2 -pipe" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--keep-going --quiet-build n" FEATURES="assume-digests binpkg-logs ccache distlocks ebuild-locks fail-clean fixlafiles metadata-transfer news parallel-fetch preserve-libs protect-owned sandbox sfperms sign split-elog split-log strict unknown-features-warn unmerge-logs unmerge-orphans userfetch xattr" FFLAGS="" GENTOO_MIRRORS="http://distro.ibiblio.org/pub/linux/distributions/gentoo/ http://gentoo.osuosl.org/ http://ftp.ucsb.edu/pub/mirrors/linux/gentoo/" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-O2 -Wl,--hash-style=both" LINGUAS="en ja zh zh_CN zh_TW en_US en_GB" MAKEOPTS="-j4 --load-average" PKGDIR="/usr/portage//packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/var/lib/layman/x11 /var/lib/layman/lisp /var/lib/layman/science /var/lib/layman/desktop-effects /var/lib/layman/gentoo-zh /var/lib/layman/haskell /var/lib/layman/java-overlay /var/lib/layman/cj-overlay /var/lib/layman/sunrise /usr/local/portage /usr/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="3dnow 3dnowext X a52 aac aalib acl acpi aim alsa amd64 apache2 audiofile avahi bash-completion bcmath berkdb bidi bittorrent blas bluetooth branding btrfs bzip2 c++0x cairo calendar caps ccache cdaudio cdda chm cjk cleartype cli clisp consolekit cracklib crypt cscope css ctype cups curl curlwrappers cxx dbus dbx dga dhcpcd directfb djvu dri dts dv dvd dvdread dvi ebook eds emacs emacs-w3 encode enscript evo exif expat faac fam fastcgi fbcon ffmpeg fftw flac flash fontconfig foomaticdb fortran freetds ftp fuse gallium gcl gconf gd gdbm ginac git glib glitz glut gmp gnome gnome-keyring gnuplot gnutls gpg gphoto2 gpm gps gs gsl gstreamer gtk gtk3 gtkhtml gzip hdaps highlight httpd iconv icq icu idn ieee1394 imagemagick imap imlib innodb inotify introspection ipod ipv6 jabber java javascript joystick jpeg jpeg2k kpathsea lame lapack laptop latex latex3 lcms libcaca libnotify llvm lm_sensors lua luatex lzma m17n-lib mad matroska mercurial mime mmx mmxext mng modplug modules mp3 mp4 mpeg mpi mplayer msn mtp mudflap multilib musepack musicbrainz mysql mysqli nautilus ncurses netboot networkmanager nis nls nntp nptl nptlonly nsplugin nss ntp odbc ogg openal openexr opengl openmp openntpd openssl osc oscar pae pam pch pcre pda pdf plotutils png policykit posix postgres ppds pppd pulseaudio qq qt3support qt4 quicktime rar rcs readline recode resolvconf rss rtmp samba sasl scanner sdl session simplexml slang sndfile snmp sockets socks5 speex spell sql sqlite sqlite3 sse sse2 sse3 ssl ssse3 subversion svg sysfs system-sqlite t1lib taglib tcpd theora threads tidy tiff timidity tk truetype tta udev unicode upnp usb v4l v4l2 vaapi vcd vdpau vhosts vim vim-syntax vorbis wav wavepack wicd wifi wma wmf wmp wxwidgets wxwindows x264 x86emu xattr xcb xcomposite xetex xforms xfs xft xine xinerama xml xmmp xorg xosd xrandr xsl xv xvid yahoo zip zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias proxy_http proxy" APACHE2_MPMS="worker" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DRACUT_MODULES="lvm crypt syslog ssh-client caps" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev joystick keyboard mouse synaptics wacom" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en ja zh zh_CN zh_TW en_US en_GB" NETBEANS_MODULES="cnd php mobility websvccommon apisupport java ide harness nb dlight enterprise profiler" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby19 ruby18" USERLAND="GNU" VIDEO_CARDS="intel i965 vesa vga fbdev v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" USE_PYTHON="2.7 3.2" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
(In reply to comment #0) > At the end of a successful emerge @preserved-rebuild, the following > preserved-lib message persists: > > emerge @preserved-rebuild > ... ... ... ... ... ... ... > !!! existing preserved libs: > >>> package: dev-libs/libffi-3.0.11 > * - /usr/lib64/libffi.so.5 > * - /usr/lib64/libffi.so.5.0.10 > * used by /usr/lib64/ghc-7.4.1/libHSrts-ghc7.4.1.so > (dev-lang/ghc-7.4.1) > * used by /usr/lib64/ghc-7.4.1/libHSrts_debug-ghc7.4.1.so > (dev-lang/ghc-7.4.1) > * used by /usr/lib64/ghc-7.4.1/libHSrts_thr-ghc7.4.1.so > (dev-lang/ghc-7.4.1) > * used by /usr/lib64/ghc-7.4.1/libHSrts_thr_debug-ghc7.4.1.so > (dev-lang/ghc-7.4.1) This one is bug 411789 and bug 411925. > >>> package: media-libs/qhull-2010.1-r2 > * - /usr/lib64/libqhull.so > * used by /usr/lib64/python2.7/site-packages/scipy/spatial/qhull.so > (sci-libs/scipy-0.10.1) > * used by > /usr/lib64/python3.2/site-packages/scipy/spatial/qhull.cpython-32.so > (sci-libs/scipy-0.10.1) > Use emerge @preserved-rebuild to rebuild packages using these libraries Post build.log of scipy here. First from "emerge -1 scipy" and then from after "emerge -C scipy qhull" and "emerge -1 scipy"
(In reply to comment #1) > (In reply to comment #0) > > At the end of a successful emerge @preserved-rebuild, the following > > preserved-lib message persists: > > > > emerge @preserved-rebuild > > ... ... ... ... ... ... ... > > !!! existing preserved libs: > > >>> package: dev-libs/libffi-3.0.11 > > * - /usr/lib64/libffi.so.5 > > * - /usr/lib64/libffi.so.5.0.10 > > * used by /usr/lib64/ghc-7.4.1/libHSrts-ghc7.4.1.so > > (dev-lang/ghc-7.4.1) > > * used by /usr/lib64/ghc-7.4.1/libHSrts_debug-ghc7.4.1.so > > (dev-lang/ghc-7.4.1) > > * used by /usr/lib64/ghc-7.4.1/libHSrts_thr-ghc7.4.1.so > > (dev-lang/ghc-7.4.1) > > * used by /usr/lib64/ghc-7.4.1/libHSrts_thr_debug-ghc7.4.1.so > > (dev-lang/ghc-7.4.1) > > This one is bug 411789 and bug 411925. > > > >>> package: media-libs/qhull-2010.1-r2 > > * - /usr/lib64/libqhull.so > > * used by /usr/lib64/python2.7/site-packages/scipy/spatial/qhull.so > > (sci-libs/scipy-0.10.1) > > * used by > > /usr/lib64/python3.2/site-packages/scipy/spatial/qhull.cpython-32.so > > (sci-libs/scipy-0.10.1) > > Use emerge @preserved-rebuild to rebuild packages using these libraries > > Post build.log of scipy here. > First from "emerge -1 scipy" and then from after "emerge -C scipy qhull" and > "emerge -1 scipy" libffi.so.5 and libqhull.so are both orphaned files. Their corresponding packages are removed during an depclean but these are preserved by scipy and ghc. Apparently, re-emerging scipy and ghc do not remove them. However, emerge -C scipy and ghc DO remove them and subsequent re-emerge did not bring them back.
This is a valid one. the preserved_libs feature keeps them on the system and scipy chooses them automatically for linking. We need to stop this.
Looking at the build log, I noticed these: ar: adding 16 object files to build-2.7/temp.linux-x86_64-2.7/libqhull.a […] /usr/bin/x86_64-pc-linux-gnu-gfortran -Wall -Wl,--as-needed -shared build-2.7/temp.linux-x86_64-2.7/scipy/spatial/qhull.o -L/usr/lib64 -L/usr/lib64 -Lbuild-2.7/temp.linux-x86_64-2.7 -lqhull -llapack -lblas -latlas -lcblas -lcblas -lblas -latlas -lpython2.7 -lgfortran -o build-2.7/lib.linux-x86_64-2.7/scipy/spatial/qhull.so This looks to me as if scipy were building its own libqhull.a static library, and attempt to link that into qhull.so. But when there is a libqhull.so available on the system, the build will use that instead, resulting in the observed library dependency. Looking at the source, the libqhull sources bundled with scipy appear to be from 2010. There have been some changes since then, but nothing radical. So I guess the most gentooish way would be depending on system libqhull and getting rid of the bundled one. However, there is this entry in the scipy ChangeLog: 06 Mar 2012; Sébastien Fabbro <bicatali@gentoo.org> +scipy-0.10.1.ebuild: Version bump. Unfortunately abandoning our effort unbundling superlu and qhull until upstream fix their issues (bug #364083) There were versions which depended on libqhull, but apparently there was a reason not to do so at the moment. The bug in question mostly talks about superlu, so I'm not sure how much it applies to qhull as well. If there is enough progress on those unbundling efforts, we might simply wait for a solution there, with workarounds along the lines of comment #2. If no solution to the unbundling approach is expected soon, then it would be better to make sure never to use the system libqhull. That could be achieved either by renaming the library (e.g. from "libqhull.a" to "libscipy_qhull.a") or by changing the compiler arguments to explicitely name the file (i.e. from "-Lbuild-2.7/temp.linux-x86_64-2.7 -lqhull" to "build-2.7/temp.linux-x86_64-2.7/libqhull.a"). Not sure where the latter should be done, and how it affects portability, so I'd go for the former. I'm a bit surprised that I found no unbundling efforts on Debian or Fedora. Apparently there are no ready-to-use patches there yet. But if we go for certain bundling instead of unbundling, then they won't have a problem as long as they build their binary packages inside a chroot with no libqhull installed.
Created attachment 313965 [details, diff] Rename and always use bundled libqhull This patch renames the bundled libqhull. Now the bundled version will always be used, ignoring any version which might be installed on the system in the regular way, preserved by portage, or otherwise accessible in some fashion. Compiles for me, and gets rid of the library dependency according to ldd.
I think the patch is fine. It avoid library confusion. We should probably give superLU the same treatment. I am not sure we should do that bit of hard work for upstream, unless we are sure they will adopt it.
Please see whether this is still valid for scipy 0.14/0.15. qlist scipy | grep so$ | xargs scanelf -n | grep qhull ET_DYN libptlapack.so.3,libatlcblas.so.3,libf77blas.so.3,libpython2.7.so.1.0,libm.so.6,libc.so.6 /usr/lib64/python2.7/site-packages/scipy/spatial/qhull.so ET_DYN libptlapack.so.3,libatlcblas.so.3,libf77blas.so.3,libpython3.3.so.1.0,libm.so.6,libc.so.6 /usr/lib64/python3.3/site-packages/scipy/spatial/qhull.cpython-33.so ET_DYN libptlapack.so.3,libatlcblas.so.3,libf77blas.so.3,libpython3.4.so.1.0,libm.so.6,libc.so.6 /usr/lib64/python3.4/site-packages/scipy/spatial/qhull.cpython-34.so
