Variants B and C: Using Import Files to create shared libraries on AIX with runtime linking

aixrtl-importfiles (text/plain), 7.52 KB, created by Michael Haubenwallner (RETIRED) on 2011-01-21 14:09:56 UTC

Another variant to get some 'soname' feature would be with 'Import Files',
but there are corner cases where they do not work.

Import Files on AIX basically contain a symbol list, and a filename referring
to the shared object providing the named symbols at runtime. This looks like
the best way to emulate elf-soname support, but it has two drawbacks, shown
below the two variants I have tried:

Variant B:
 *) Create the shared object libNAME.so.X.Y.Z with '-G' linker flag.
 *) Create the import file libNAME.so.X.imp, referring to 'libNAME.so.X'
 *) Pack the import file into libNAME.so.X.Y.Z.imp.a
 *) Create symlink libNAME.so.X -> libNAME.so.X.Y.Z
 *) Create symlink libNAME.so -> libNAME.so.X.Y.Z.imp.a
 *) Create libNAME.a from static objects.
Install results in filesystem:
   import library:   libNAME.so.X.Y.Z.imp.a
   shared library:   libNAME.so.X.Y.Z
   used by loader:   libNAME.so.X -> libNAME.so.X.Y.Z
   used by linker:   libNAME.so -> libNAME.so.X.Y.Z.imp.a
   static library:   libNAME.a
Useability results:
 ++ Most obvious from filesystem point of view.
 ++ Using dlopen("libNAME.so.X") works similar to ELF.
 ++ 'Static library' is found when runtime linking (-brtl) is not enabled,
    so there is no way to have unsatisfied symbols at runtime.
 ++ When linking with /immediate/path/libNAME.a, only static linking occurs.
 ++ When linking with /immediate/path/libNAME.so, 'libNAME.so.X' is encoded
    in to the resulting binary without '/immediate/path'.
 ++ Is automatically loaded at runtime, even when not *referenced* at linktime.
 ++ Improved link time, linker does not need to read the whole shared object.
 -- There is an additional 'import library'.
 -- The symlinks point to different files.
 -- Using dlopen("libNAME.so") does not work.
 == The two drawbacks described below.

Variant B1:
 *) Create the shared object libNAME.so.X.Y.Z with '-G -bernotok' linker flags.
 *) Create the import file libNAME.so, referring to 'libNAME.so.X'
 *) Create symlink libNAME.so.X -> libNAME.so.X.Y.Z
 *) Create libNAME.a from static objects.
Install results in filesystem:
   shared library:   libNAME.so.X.Y.Z
   used by loader:   libNAME.so.X -> libNAME.so.X.Y.Z
   import file:      libNAME.so
   static library:   libNAME.a
Useability results:
 ++ Filesystem layout is most similar to ELF.
 ++ Using dlopen("libNAME.so.X") works similar to ELF.
 ++ 'Static library' is found when runtime linking (-brtl) is not enabled.
 ++ When linking with /immediate/path/libNAME.a, only static linking occurs.
 ++ When linking with /immediate/path/libNAME.so, 'libNAME.so.X' is encoded
    in to the resulting binary without '/immediate/path'.
 ++ Is automatically loaded at runtime, even when not *referenced* at linktime.
 ++ Improved link time, linker does not need to read the whole shared object.
 -- collect2 of <gcc-4.4(.2?) sometimes doesn't like standalone import files.
 -- There is an additional 'import file', not obviously named 'libNAME.so'.
 -- The unversioned filename does not point to the real shared lib.
 -- Using dlopen("libNAME.so") does not work.
 == The two drawbacks described below.
 
Variant B2:
 *) Create the shared object libNAME.so.X with '-G -bernotok' linker flags.
 *) Create the import file libNAME.imp, referring to 'libNAME.so.X'
 *) Pack the import file _first_ into libNAME.so.X.Y.Z.a
 *) Pack the shared object _second_ into libNAME.so.X.Y.Z.a for scripting convenience
 *) Create symlink libNAME.so -> libNAME.so.X.Y.Z.a
 *) Create libNAME.a from static objects.
Install results in filesystem:
   import archive: libNAME.so.X.Y.Z.a
   shared object:  libNAME.so.X
   used by linker: libNAME.so -> libNAME.so.X.Y.Z.a
   static library: libNAME.a
Useability results:
 ++ Filesystem layout is somewhat obvious.
 ++ Filesystem layout is somehow similar to ELF.
 ++ Using dlopen("libNAME.so.X") works similar to ELF.
 ++ 'Static library' is found when runtime linking (-brtl) is not enabled.
 ++ When linking with /immediate/path/libNAME.a, only static linking occurs.
 ++ When linking with /immediate/path/libNAME.so, 'libNAME.so.X' is encoded
    in to the resulting binary without '/immediate/path'.
 ?? Is automatically loaded at runtime, even when not *referenced* at linktime.
 ++ Improved link time, linker does not need to read the whole shared object.
 ++ Works with collect2 of <gcc-4.4 too.
 -- The shared object is stored on disk twice when convenienced.
 -- The unversioned filename does not point to the real shared lib.
 -- Using dlopen("libNAME.so") does not work,
    however dlopen("libNAME.so(libNAME.so.X)") can work.
 == The two drawbacks described below.
 
Variant C:
 *) Create the shared object 'NAME.so' with '-G' linker flag.
 *) Create the import file NAME.imp, referring to 'libNAME.so.X(NAME.so)'
 *) Pack NAME.imp _first_ into libNAME.so.X.Y.Z
 *) Pack NAME.so _second_ into libNAME.so.X.Y.Z
 *) Create symlink libNAME.so.X -> libNAME.so.X.Y.Z
 *) Create symlink libNAME.so -> libNAME.so.X.Y.Z
 *) Create libNAME.a from static objects
Install results in filesystem:
   all-in-one file:   libNAME.so.X.Y.Z
   used by  loader:   libNAME.so.X -> libNAME.so.X.Y.Z
   used by  linker:   libNAME.so -> libNAME.so.X.Y.Z
   static  library:   libNAME.a
Useability results:
 ++ Filesystem layout similar to ELF.
 ++ 'Static library' is found when runtime linking (-brtl) is not enabled,
    so there is no way to have unsatisfied symbols at runtime.
 ++ When linking with /immediate/path/libNAME.a, only static linking occurs.
 ++ When linking with /immediate/path/libNAME.so, 'libNAME.so.X(NAME.so)' is
    encoded in to the resulting binary without '/immediate/path', because the
	import file is found before the shared object.
 ++ Using dlopen() works identically on both symlinks and the real file.
 ++ Is automatically loaded at runtime, even when not referenced at linktime.
 -- Not obvious from filesystem point of view. One could name the all-in-one
    file 'libNAME.so.X.Y.Z.a', which would be slightly more obvious.
 -- Using dlopen() requires special coded object-name along with RTLD_MEMBER,
    either dlopen("libNAME.so.X(NAME.so)", RTLD_MEMBER)
        or dlopen("libNAME.so(NAME.so)", RTLD_MEMBER).
 == The two drawbacks described below.


Drawback 1:
It is impossible to specify undefined symbols in an import file for the
shared object to allow for subsequent link steps keeping those symbols in the
created shared object or executables, to be resolved by runtime linking when
the executable is run.

OTOH, this even does not work with shared archive members without Import
Files, so it looks like something like GNU-ld flag '--as-needed' does not work
as expected in some circumstances.

But as shared libraries with unresolved symbols are broken "by design" anyway,
and real runtime dependency management isn't going to work with broken shared
libraries, this becomes a non-problem as more and more packages get this right.

However, using '# autoload' allows to always reference shared libraries linked
against, even when they are archive members. This somehow is like the GNU-ld
flag '-no-as-needed'.



Drawback 2:
AIX can statically link shared objects. But this does not work with Import
Files. When runtime linking is in enabled, libNAME.so is searched before
libNAME.a, even when the linker is told to statically link shared objects
using the '-bnso' linker flag. One could use '-bstatic' before each -lNAME
to avoid searching libNAME.so, but switching back afterwards with '-bdynamic'
also kills the effect of '-bnso'.
Eventually, we should not use '-bnso' any more, but '-bstatic' and '-bdynamic'
instead. This also allows to link statically in general, while libc continues
to be linked dynamically.