lddtree is a very neat tool, which I use in Arch Linux a lot. As a bugwrangler it is convenient to tell people reporting an issue with some package not being able to load a shared library, "use lddtree to figure out which lib is actually problematic". For example, every time gcc is updated, we get complains that numpy doesn't work, because people use their own self-compiled blas/lapack packages which need to be rebuilt... But it doesn't work: $ lddtree /usr/lib/python3.6/site-packages/numpy/core/multiarray.cpython-36m-x86_64-linux-gnu.so /usr/lib/python3.6/site-packages/numpy/core/multiarray.cpython-36m-x86_64-linux-gnu.so (interpreter => None) libcblas.so.3 => None libm.so.6 => None libpython3.6m.so.1.0 => None libpthread.so.0 => None libc.so.6 => None ld-linux-x86-64.so.2 => None Using -x, the following additional info is reported: $ lddtree -x /usr/lib/python3.6/site-packages/numpy/core/multiarray.cpython-36m-x86_64-linux-gnu.so root = / ParseLdSoConf(//etc/ld.so.conf) glob: /etc/ld.so.conf.d/*.conf ParseLdSoConf(/etc/ld.so.conf.d/fakechroot.conf) ParseLdSoConf(/etc/ld.so.conf.d/fakeroot.conf) ldpaths[conf] = ['/usr/lib/libfakeroot/fakechroot', '/usr/lib/libfakeroot'] ldpaths[env] = [] argv[x] = /usr/lib/python3.6/site-packages/numpy/core/multiarray.cpython-36m-x86_64-linux-gnu.so +auto-root = /usr/lib/python3.6/site-packages/numpy/core/multiarray.cpython-36m-x86_64-linux-gnu.so globbed = /usr/lib/python3.6/site-packages/numpy/core/multiarray.cpython-36m-x86_64-linux-gnu.so ParseELF(/usr/lib/python3.6/site-packages/numpy/core/multiarray.cpython-36m-x86_64-linux-gnu.so) ldpaths[rpath] = [] ldpaths[runpath] = [] FindLib(libcblas.so.3) checking: /usr/lib/libfakeroot/fakechroot/libcblas.so.3 checking: /usr/lib/libfakeroot/libcblas.so.3 FindLib(libm.so.6) checking: /usr/lib/libfakeroot/fakechroot/libm.so.6 checking: /usr/lib/libfakeroot/libm.so.6 FindLib(libpython3.6m.so.1.0) checking: /usr/lib/libfakeroot/fakechroot/libpython3.6m.so.1.0 checking: /usr/lib/libfakeroot/libpython3.6m.so.1.0 FindLib(libpthread.so.0) checking: /usr/lib/libfakeroot/fakechroot/libpthread.so.0 checking: /usr/lib/libfakeroot/libpthread.so.0 FindLib(libc.so.6) checking: /usr/lib/libfakeroot/fakechroot/libc.so.6 checking: /usr/lib/libfakeroot/libc.so.6 FindLib(ld-linux-x86-64.so.2) checking: /usr/lib/libfakeroot/fakechroot/ld-linux-x86-64.so.2 checking: /usr/lib/libfakeroot/ld-linux-x86-64.so.2 ... The issue seems to be it cannot detect the python interpreter, and we don't add the default /usr/lib(64) to our ld.so.conf, so lddtree.py never looks for libs in /usr/lib, even though the python interpreter *does* pull in /lib64/ld-linux-x86-64.so.2 as its own interpreter. I'm not sure if the correct solution is to just fall back on ld.so as the interpreter, or somehow detect/special-case python, or simply seed the ldpaths with common defaults. Reproducible: Always
JUst to clarify: do I understand correctly this is the output of an arch system and not gentoo?
I think presence of python interpreter is unrelated. shared libraries don't have intepreter set in ELF and it's fine. They don't have to be selfcontained binaries and in theory any dynamic loader (with it's own defaults) can load them. What seems to differ on your system is lack of path to /lib* in /etc/ld.so.conf* files. Gentoo has: $ cat /etc/ld.so.conf /lib64 /lib32 /lib <and many others> You are correct it does not perfectly match search order of default dynamic loader. glibc's ld has a few more defaults set: $ LD_DEBUG=all /lib64/ld-linux-x86-64.so.2 --inhibit-cache --list /bin/ls 2>&1 | fgrep 'search path' 1270: search path=/lib64/tls/x86_64/x86_64:/lib64/tls/x86_64:/lib64/tls/x86_64:/lib64/tls:/lib64/x86_64/x86_64:/lib64/x86_64:/lib64/x86_64:/lib64:/usr/lib64/tls/x86_64/x86_64:/usr/lib64/tls/x86_64:/usr/lib64/tls/x86_64:/usr/lib64/tls:/usr/lib64/x86_64/x86_64:/usr/lib64/x86_64:/usr/lib64/x86_64:/usr/lib64 (system search path) Default configuration can vary from system to system. For example on prefixed system there is no /lib* or /usr/lib* paths at all: $ LD_DEBUG=all ./gentoo/g821/lib64/ld-linux-x86-64.so.2 --inhibit-cache --list /bin/ls 2>&1 | fgrep 'search path' 1445: search path=/home/slyfox/gentoo/g821/lib64/tls/x86_64/x86_64:/home/slyfox/gentoo/g821/lib64/tls/x86_64:/home/slyfox/gentoo/g821/lib64/tls/x86_64:/home/slyfox/gentoo/g821/lib64/tls:/home/slyfox/gentoo/g821/lib64/x86_64/x86_64:/home/slyfox/gentoo/g821/lib64/x86_64:/home/slyfox/gentoo/g821/lib64/x86_64:/home/slyfox/gentoo/g821/lib64:/home/slyfox/gentoo/g821/usr/lib64/tls/x86_64/x86_64:/home/slyfox/gentoo/g821/usr/lib64/tls/x86_64:/home/slyfox/gentoo/g821/usr/lib64/tls/x86_64:/home/slyfox/gentoo/g821/usr/lib64/tls:/home/slyfox/gentoo/g821/usr/lib64/x86_64/x86_64:/home/slyfox/gentoo/g821/usr/lib64/x86_64:/home/slyfox/gentoo/g821/usr/lib64/x86_64:/home/slyfox/gentoo/g821/usr/lib64 (system search path) I think simplest workarounds for you would be to: - populate /etc/ld.so.conf (or equivalent) - or inject ld's defaults into ParseLdSoConf function
> shared libraries don't have intepreter set in ELF and it's fine Except when conceptually you're using it even though technically you don't use it directly at all. Though I suppose it is true you don't usually need to check the search path of a library, since it is usually used directly by some other binary, whereas python (and other scripting languages) needs you to import it. > - or inject ld's defaults into ParseLdSoConf function Yes, it worked fine when changing > paths = [] to > paths = ['/usr/lib'] But I'd prefer something that didn't require patching. Would it be possible to build this during configure? Maybe ParseELF() should default to the system ld.so for interp, if none is detected in the ELF object itself?
(In reply to Eli Schwartz from comment #3) > > - or inject ld's defaults into ParseLdSoConf function > > Yes, it worked fine when changing > > > paths = [] > > to > > > paths = ['/usr/lib'] > > But I'd prefer something that didn't require patching. Would it be possible > to build this during configure? Maybe ParseELF() should default to the > system ld.so for interp, if none is detected in the ELF object itself? Having a configure option to inject defaults sounds fine.
defaulting to any set of paths at configure time is incorrect. lddtree.py is, by design, agnostic of the active runtime environment. if you hardcode e.g. /usr/lib64 because CHOST is x86_64-linux-gnu, you break lddtree.py usage for cross targets. this is precisely why lddtree.py uses the interp binary to try and figure out a default. however, if ld.so.conf doesn't contain the system paths, and you're probing an ELF that doesn't have an interp set (like a shared lib), then yes lddtree won't be able to discover the right default paths. we could fallback to probing the interp from $ROOT/bin/sh to recover. that should hopefully handle the majority of real world systems.
*** Bug 800221 has been marked as a duplicate of this bug. ***
