Please do not mark this as a duplicate of bug 4411 and close it; or if you do so, please re-open bug 4411 as the problem is still occurring. Whilst in most cases linker scripts in /usr/lib suffice for libraries in /lib, they do not in cases of interpreted languages (e.g. using ghci from dev-lang/ghc, or even Template Haskell when compiling with GHC proper, as TH acts as an interpreter). This is even a problem with C programs using dlopen(), as I'll show in the next entry with an attachment. Linker scripts _do_ break dlopen: if you try and dlopen("libpcre.so", _) then it will fail. The problem is that whilst /lib is searched before /usr/lib, there is no libpcre.so in /lib, just libpcre.so.0{,.0.1} (which people don't know to look for when writing their code). So the first "valid" file the dynamic linker finds when you dlopen() is the linker script in /usr/lib, and since that isn't a valid ELF file dlopen() fails. If we're going to keep with these linker scripts in /usr/lib, can we at least consider having symlinks in /lib for libpcre, etc. like there are for most other libraries? Otherwise we'll keep having problems with interpreted languages that need to open the libs at runtime rather than using the static linker. As an example of where this is a problem, dev-haskell/pcre-light (this and the following packages are in the Haskell overlay) dynamically links to libpcre, and then dev-haskell/highlighting-kate uses it; but when we try to actually use highliting-kate in app-text/pandoc, it fails to build due to not being able to find libpcre. However, if we manually create symlink /lib/libpcre.so -> /lib/libpcre.so.0.0.1, then it will successfully build.
Created attachment 208605 [details] Sample C file that breaks dlopen() This is an example based upon the one from dlopen's manpage using libpcre instead of libm. Compile this using "$ gcc -rdynamic -o breakDlopen breakDlopen.c -ldl" and then try to run it; it will fail. However, if you edit the source to explicitly ask for "libpcre.so.0.1" or "libpcre.so.0" then it will work. Thus, usage of linking scripts is still a problem. Please either include foo.so symlinks in /lib or replace linker scripts with symlinks.
That means it's a bug in dev-libs/libpcre, not to create the /lib/libpcre.so symlink, right?
(In reply to comment #2) > That means it's a bug in dev-libs/libpcre, not to create the /lib/libpcre.so > symlink, right? > Yes, and for all other libs that have linking scripts.
Uhm. No. dlopen() should be used with a soname just like the linker does, since you're *assuming* a given ABI (and thus a given soname). To accept *any* soname means that you're loading stuff without knowing the ABI it has. See http://blog.flameeyes.eu/2009/10/27/a-shared-library-by-any-other-name about the details of symlinks.
(In reply to comment #4) > dlopen() should be used with a soname just like the linker does, since you're > *assuming* a given ABI (and thus a given soname). OK, that's a fair point for "installed" packages and we should refer to the sonames rather than just the library name. But how do we get from a "source" package (which just specifies that it needs pcre; see http://hackage.haskell.org/packages/archive/pcre-light/0.3.1/pcre-light.cabal for example) to resolve the exact soname that we can use for package registration (using ghc-pkg in our case)? We can't use ld since we're not generating .so files (more akin to special bytecode files like a java JAR). For a more "manual" example, consider doing something like this: ghci Foo.hs -lpcre (where ghci is the GHC interpreter). We have here the source package of Foo.hs, and we want to pick the current pcre version (same as if we did "gcc foo.c -lpcre"). If this is successfully compiled to bytecode and loaded, then we know the ABI is right for the "current" version of the library; but there's no obvious way of getting the exact soname corresponding to the ABI that i'ts compiled against that we can use.
*** Bug 294093 has been marked as a duplicate of this bug. ***
Assigning to QA now. Should something happen here? If not please close.
At best it's stuff for toolchain; dlopen("libfoo.so") is IMHO always wrong anyway.
not only is dlopen("libfoo.so") broken for the reasons Diego cites, it also simply wont work on a lot of distributions out there that dont include the debug packages by default. they only include the SONAMEs because that is the only thing packages link against. so in one way, the current behavior is a good thing as it prevents people from doing stupid stuff. do you have any other examples for us to consider ?
I'm trying to reconcile the need (i.e., link against the current package) in Ivan's comment #5 with the current situation. If linking against libpcre.so is not good practice, then what is? How do we find out what the current soname is? Is there some way to use ld to get us what we want? (Note: for ghc I have a patch that reads the linker script and uses a regular expression to find the GROUP ( ... ) command. Is this generalizable? Is it the right way to go?)
you use the SONAME for whatever ABI you're blindly encoding into your code. or avoid the pointless dlopen-ing and simply link against it in the first place.
(In reply to comment #11) > you use the SONAME for whatever ABI you're blindly encoding into your code. or > avoid the pointless dlopen-ing and simply link against it in the first place. I think we may be thinking of different use cases. ghc's interactive version is an interpreter, so it needs to be able to link dynamically. (I wonder how Python does it. I'll investigate.) When you talk about blindly encoding an ABI, isn't that what you do in a C program? I don't call a function with an soname in my C code. I expect the linker to find it for me. Please correct me if I'm wrong.
no, that isnt what you do when compiling & linking. you include the header, compile it, and then link against the same library version (thus encoding the SONAME). if you did something stupid like packaged up the .o objects for someone else to link against, that would be blindly encoding the ABI without the explicit SONAME linkage. whether you are banging on the library from compiled code or interpreted code, the result is the same -- you're making assumptions about the ABI which really are only valid for 1 specific SONAME. and again, any interpreted script that bangs directly on "libfoo.so" is going to break on every distribution out there (if only for different reasons).
(In reply to comment #13) > whether you are banging on the library from compiled code or interpreted code, > the result is the same -- you're making assumptions about the ABI which really > are only valid for 1 specific SONAME. and again, any interpreted script that (almost irrelevant to haskell problems) So all static .a(==bunch of .o files) libraries gentoo builds - break that assumptions about ABI, because noone can be sure what headers they were built against (those .a libs can be dependant on another libs/headers), right? How do I know what -lpcre version I need for, say, /usr/lib64/libglib-2.0.a ? U pcre_compile2 U pcre_config U pcre_dfa_exec U pcre_exec U pcre_fullinfo U pcre_get_stringnumber U pcre_get_stringtable_entries U pcre_study
Created attachment 225265 [details, diff] zlib-1.2.4.ebuild.patch Is there any reason why the symlinks from /usr/lib to /lib would not work? Attached patch does not meet QA (it uses hand move instead of do* as I couldn't find move replacement) % cat test.c #include <zlib.h> int main() { z_stream str; deflateInit(&str, 6); } % ls -l /usr/lib/libz.* -rw-r--r-- 1 root root 354530 Mar 25 23:29 /usr/lib/libz.a lrwxrwxrwx 1 root root 16 Mar 25 23:29 /usr/lib/libz.so -> /lib64/libz.so.1 % gcc -lz test.c % ldd a.out linux-vdso.so.1 => (0x00007fff61a30000) libz.so.1 => /lib/libz.so.1 (0x00007f183cf9b000) libc.so.6 => /lib/libc.so.6 (0x00007f183cc42000) /lib64/ld-linux-x86-64.so.2 (0x00007f183d1b3000) Package information: app-admin/eselect-compiler: (none) app-shells/bash: 4.1_p2-r1 dev-java/java-config: 2.1.10 dev-lang/python: 2.6.5 3.1.2 dev-python/pycrypto: (none) dev-util/ccache: (none) dev-util/cmake: 2.8.0-r2 dev-util/confcache: (none) sys-apps/baselayout: 2.0.1 sys-apps/openrc: 0.6.1-r1 sys-apps/sandbox: 2.2 sys-devel/autoconf: 2.13 2.65 sys-devel/automake: 1.10.3 1.11.1 1.9.6-r3 sys-devel/binutils: 2.20.1 sys-devel/gcc: 4.4.3 sys-devel/gcc-config: 1.4.1 sys-devel/libtool: 2.2.6b virtual/os-headers: 2.6.33 (for sys-kernel/linux-headers::installed) % ghci -package zlib GHCi, version 6.12.1: http://www.haskell.org/ghc/ :? for help Loading package ghc-prim ... linking ... done. Loading package integer-gmp ... linking ... done. Loading package base ... linking ... done. Loading package bytestring-0.9.1.5 ... linking ... done. Loading package zlib-0.5.2.0 ... linking ... done. Loading package ffi-1.0 ... linking ... done. Prelude>
i missed comment #14 ... the assumption is that the headers and the static library are from the same version. if they arent, that's a broken package manager, or someone screwed up their system. but yes, if you were to take a static library and deploy it with different headers, you could possibly be breaking things and this isnt a "bug". i still havent seen anything worth changing, so ...
*** Bug 346095 has been marked as a duplicate of this bug. ***
At least for GHC-7.0, the linker script patch (GHC ticket #2615) I mentioned in comment #10 is finally available. This resolves Ivan's original need. GHC 6.12.3 is now stable. The current version of GHC 6.12.3 doesn't have the linker script patch, but if you want it, please contact the Gentoo Haskell project at haskell@gentoo.org and request it.
