Bugzilla Bug 75420

Ok, so I've decided to setup a tracker bug to deal with multilib related issues
in portage and toolchain... it's mostly portage related, but toolchain is on CC
so they can follow and make comments.

To sumarize my last email to dev-portage and toolchain, I hope to alter the
multilib setup in portage to use a list of supported ABIs, and each ABI has
CFLAGS, LIBDIR, etc assigned which are required to build for that ABI.  See
default-linux/sparc/sparc64-multilib/dev/make.defaults for an example.

I have committed a change to eutils.eclass to make use of this new approach (if
it's not sparc64-multilib, then the old approach is used), and I am testing a
glibc ebuild which makes use of this functionality to emerge both 32bit and
64bit versions at the same time.

Things I've noticed since my last mail to dev-portage and toolchain:
It would be best if the ${CC} exported to ebuild.sh was "${CHOST}-gcc $(eval
echo \${CFLAGS_${ABI}})" rather than "${CHOST}-gcc" (and likewise for ${CXX}) 
This is because the build process of many packages (like glibc) use ${CC} but
ignore ${CFLAGS} for some things (like "$(CC) --print-file-name=crtbegin.o" for
instance).  The abi's cflags (eg: -m32) need to be passed in to get the right
crtbegin.o.

I've also updated some portage/bin scripts that use CONF_LIBDIR to use
LIBDIR_${ABI} if it is defined.  I'll attach the patch.

Created an attachment (id=46698) [edit]
patch of portage/bin to use LIBDIR_${ABI} instead of CONF_LIBDIR if it is
defined

Here's the patch I mentioned which applies to current cvs to use LIBDIR_${ABI}
if it defined over CONF_LIBDIR in the portage/bin scripts (econf, dolib, and
preplib)

Created an attachment (id=46769) [edit]
gcc-wrapper-multilib.patch

Here's one for toolchain (specifically lv when he gets back):

I modified the gcc wrapper to honor the ${ABI} and ${CFLAGS_${ABI}} environment
variables.  Also, I rolled support for (${CHOST}-)?g{cc,++}{32,64} into
wrapper.c as I saw that was on the todo list.

To show that it works:

(01:17:26 Fri Dec 24 2004 root@aeris sparc64)
~ $ export ABI="sparc64" && gcc a.c -o a.64 && file a.64
a.64: ELF 64-bit MSB executable, SPARC V9, version 1 (SYSV), for GNU/Linux
2.0.0, dynamically linked (uses shared libs), not stripped

(01:17:34 Fri Dec 24 2004 root@aeris sparc64)
~ $ export ABI="sparc32" && gcc a.c -o a.32  && file a.32
a.32: ELF 32-bit MSB executable, SPARC32PLUS, V8+ Required, version 1 (SYSV),
for GNU/Linux 2.0.0, dynamically linked (uses shared libs), not stripped

(01:17:45 Fri Dec 24 2004 root@aeris sparc64)
~ $ echo $CFLAGS_sparc32
-m32

(01:17:52 Fri Dec 24 2004 root@aeris sparc64)
~ $ echo $CFLAGS_sparc64
-m64

Created an attachment (id=46850) [edit]
FEATURES=distcc patch to ebuild.sh for CFLAGS_${ABI}

here's a patch to ebuild.sh to honor CFLAGS_${ABI} if it is set for calling
distcc

umm, wouldnt it be a lot easier to have portage just append the appropriate
CFLAGS_ABI onto CFLAGS ?

vapier: Please see my initial comment... In theory, yes it would be prettier to
just append ${CFLAGS_<ABI>}, and that's what I originally tried, but that would
be good if GCC always used CFLAGS.  It doesn't.  the problem is many packages
ignore CFLAGS for simple things (and some do entirely).  As an example, glibc
required a 10K patch (23 hunks) to add in the $CFLAGS_<ABI> where it ignored
CFLAGS.  It took me 2 days to track down all the problems.  As an alternative,
I just tried CC="$(tc-getCC) ${CGLAGS_ABI}" and all was well.  This approach
makes the toolchain always honor $CFLAGS_<ABI>, it's straightforward, and it
avoids manhours in tracking down instances where packages don't honor CFLAGS
and forcing them to.

(From update of attachment 46769 [edit])
I've talked to lv in #-dev today and this has been committed to gcc-config.

(From update of attachment 46850 [edit])
Setting this to obsolete.  I think the prefered option is to patch distcc like
the gcc wrapper was patched.

Ok, I've got a distcc patch all made up, and I included it in portage, but it's
commented out of the ebuild pending lisa's approval.  Lisa, could you check it
out, and if it looks good uncomment the epatch line in the ebuild?  The code is
pretty much identical to what went into the gcc wrapper.

And here's proof it works:
(01:06:20 Tue Dec 28 2004 jeremy@cid x86_64)
~/tmp $ export CFLAGS_x86="-m32"

(01:06:27 Tue Dec 28 2004 jeremy@cid x86_64)
~/tmp $ export CFLAGS_amd64="-m64"

(01:06:30 Tue Dec 28 2004 jeremy@cid x86_64)
~/tmp $ export ABI="x86" && distcc gcc bad.c && file a.out
a.out: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux
2.6.5, dynamically linked (uses shared libs), not stripped

(01:06:39 Tue Dec 28 2004 jeremy@cid x86_64)
~/tmp $ export ABI="amd64" && distcc gcc bad.c && file a.out
a.out: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV), for GNU/Linux
2.6.5, dynamically linked (uses shared libs), not stripped

(01:06:47 Tue Dec 28 2004 jeremy@cid x86_64)
~/tmp $ export ABI="x86" && /usr/lib/distcc/bin/gcc bad.c && file a.out
a.out: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux
2.6.5, dynamically linked (uses shared libs), not stripped

(01:07:33 Tue Dec 28 2004 jeremy@cid x86_64)
~/tmp $ export ABI="amd64" && /usr/lib/distcc/bin/gcc bad.c && file a.out
a.out: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV), for GNU/Linux
2.6.5, dynamically linked (uses shared libs), not stripped

ive taken the ABI changes out of 1.3.8 and put them into 1.3.9

i want 1.3.8 to go stable and it seems fine thus far ... introducing ABI stuff into 1.3.8 would prevent a new clean stable tree

Created an attachment (id=47272) [edit]
updated libdir patch for current portage/bin

Here's an updated patch which doesn't fork off an 'eval echo'

committed

Added to portage_2_0.

Created an attachment (id=47436) [edit]
FEATURES=multilib-pkg patch (RFC)

Ok, this isn't meant to be included in portage yet, but I've reached a
milestone, so I'd like comments from anyone who has any thoughts regarding this
.  Also, I intentionally left out some indents in dyn_unpack() to make the
patch more readable... I'll of course add them in in a final patch.

This patch adds a new FEATURE to portage called 'multilib-pkg'	With it
enabled, some packages in the tree (currently none) will install libs for all
the ABIs in ${MULTILIB_ABIS}.  Additionally, the ebuild can control which ABIs
are allowed using the ${ABI_ALLOW} and ${ABI_DENY} variables which have the
expected logic.  These can be set in pkg_setup() for things like
'USE=win32codecs emerge mplayer' on amd64.

And to show it off, I tested it out on zlib:

(23:47:31 Sun Jan 02 2005 root@aeris sparc64)
~ $ file /lib/libz.so.1.2.2 
/lib/libz.so.1.2.2: ELF 32-bit MSB shared object, SPARC32PLUS, V8+ Required,
version 1 (SYSV), stripped

(23:47:57 Sun Jan 02 2005 root@aeris sparc64)
~ $ file /lib64/libz.so.1.2.2 
/lib64/libz.so.1.2.2: ELF 64-bit MSB shared object, SPARC V9, version 1 (SYSV),
stripped

You need to add RESTRICT="multilib-pkg" to the zlib ebuild to get it to emerge
both libs using this patch.

I'm not really keen on RESTRICT="multilib-pkg" as it's not really a RESTRICTion
... but I don't think we need variable bloat either to add another flag to
ebuilds, so any suggestions on that are more than welcome.

I'll also be adding a debugging FEATURE 'multilib-strict' which will cause the
emerge to die if a binary of an invalid ABI ends up in /lib, /usr/lib, or
/usr/X11R6/lib

Created an attachment (id=47440) [edit]
FEATURES=multilib-pkg patch (RFC)

Fixed a couple bugs:
Missed escaping a new line
reset DEST and umask for each run through src_install...

Created an attachment (id=47558) [edit]
updated FEATURES=multilib-pkg patch

Yet another step closer...

This should make it easier on packages which do things like 'AR="$(tc-getAR)
rc"' in src_compile.  With the previous patch, subsequent calls to src_compile
would result in AR="${CHOST}-ar rc rc"	Now, we save/restore the environment
for each ABI separately.  It's not quite optimized yet, but the logic is
there...

Created an attachment (id=47684) [edit]
FEATURES=multilib-strict patch

Here's a patch to help finding packages which don't honor libdir properly. 
Users can set FEATURES=multilib-strict and report back when packages try
installing libs into the wrong directory.  Profiles can set the following to
take advantage of this:

MULTILIB_STRICT_DIRS="/lib /usr/lib /usr/kde/*/lib /usr/qt/*/lib
/usr/X11R6/lib"
MULTILIB_STRICT_DENY="64-bit"

Note, this can be used in current profiles and doesn't require the new multilib
setup, so current users can start helping us by using this.  The
MULTILIB_STRICT_DENY is simply passed to egrep on the result of 'file <file>'
for everything in the MULTILIB_STRICT_DIRS.  If the egrep passes, we die.

Created an attachment (id=47826) [edit]
FEATURES=multilib-pkg patch

Cleaned up a bit...

optimized the unset portion of the env-switching to just unset everything then
source the old env with the option of setting FEATURES=multilib-debug to use
the old behavior and see what is being unset.

Handle aborted src_{unpack,compile,test,install} better.

Move 'true' to the end of dyn_clean as that was overlooked in a recent patch
(jstubbs, you probably want to fix that part before -r9...)

Applies cleanly against current cvs again...

(From update of attachment 47272 [edit])
marking obsolete as it's in a release now

(From update of attachment 47684 [edit])
marking obsolete as it's in a release now

Created an attachment (id=48489) [edit]
distcc-gentoo-multilib.patch.patch

The distcc multilib patch needs a little update. It uses unitialized variables
which lead to breakage, at least for me. If I emerge distcc with
-fomit-frame-pointer in CFLAGS, executing /usr/lib/distcc/bin/gcc will result
in exit code 105 i.e. "Out of memory."

newflagsCount is the only one used unitialized, all others have an assignment
later in the code.

(From update of attachment 48489 [edit])
incvs, thanks.

Created an attachment (id=49143) [edit]
portage-multilib-strict-exempt.patch

Please apply:
Applies against current cvs (portage_2_0)

This adds another option to multilib-strict (and sets a default for it) which
allows exemptions.  This is neccessary for perl modules and gcc which take care
of their own abi separation using subdirs (/usr/lib/perl5/5.8.6/x86_64-linux
for example).

in portage-2.0.51-r16 $(get_libdir) returns lib, but it should return lib64
(amd64-2005.0 profile

lanius: Can youu show me where it's doing this?  I just threw 'einfo
$(get_libdir)' into src_unpack of zlib's ebuild to test it, and it printed out
lib64.  Please come talk to me on IRC.

eradicator: try this in bash:

source /sbin/functions.sh; echo $(get_libdir)

lanius.  ok, I'll take a look at that later, but what actually uses that?

init script may use it top find out the correct lib directory, i'm currently
working on motif-config that needs it, don't know about java-config or
gcc-config, or similar tools

I know gcc-config and opengl-update don't use it.

The libdir needs to be configured into the init script.  Using get_libdir() in /sbin/functions.sh assumes that on any given system, the same libdir is always used.  That is an incorrect assumption as get_libdir() will change based on the ${ABI}.  So, we need to go through the tree and figure out who is using that when they shouldn't be.  Do you have a list of scripts you know who use it?

I added it to functions.sh some months ago to get {blas,lapack}-config working
on amd64-multilib. Might be that it's not quite up to date with current
multilib...

it's not a matter of being up to date or not... it's simply that that it can't
work.  You're essentially assuming that get_libdir will always be the same for
every package on the system, but that is not the case.  You can have a 32bit
version of mplayer installed (get_libdir=lib) and a 64bit version of xine
installed (get_libdir=lib64).  Granted these aren't the best examples, but I'm
trying to illustrate that the correct logic would be to do something like this
in the ebuild:

dosed "s:@@LIBDIR@@:$(get_libdir):" /etc/conf.d/myconf

Sounds reasonable. So get_libdir should be removed from functions.sh completly,
right?

yep.

So Danny, please fix {blas,lapack}-config

Ok, looks like I've hit a snag in some multilib stuff thanks in part to gcc and
in part to libtool.  Using the standarg auto*, liibtool build chain, libtool
searches through the directories listed in `gcc -print-search-dirs`'s libraries
section.  

To see this, just search for '$CC -print-search-dirs' in a configure script. 
The problem is this always lists the base directory rather than the `gcc
-print-multi-directory` subdirectory.  Thus if sparc32 is default, libtool
needs to search through
/usr/lib/gcc/sparc64-unknown-linux-gnu/3.4.3-20050110/32, but
/usr/lib/gcc/sparc64-unknown-linux-gnu/3.4.3-20050110 is listed instead.  Thus,
it notices the 64bit libstdc++.la instead of the 32bit one.  Deleting the .la
files is a temporary fix as it just passes the -lstdc++ through to the linker,
but I'd rather find a better solution to this problem...

gcc -m32 -print-search-dirs

did a bit of maintenance in the ebuild, please move to distcc-2.18.3-r5. ;)

Jeremy, if you get a chance could you dump something of an update to this bug
regarding current support?

http://ramblings.hudscabin.com/blogs/index.php/2005/05/02/multilib_and_toolchain_thoughts

That's probably the best update I have for you. No patches for it yet... and I think it's best to shy away from the multilib-pkg approach... it's just an ugly hack...

but I think CHOST aliases might be in orded to make things easy... so if the user sets 'CHOST=amd64 emerge blah' portage/the profile should know to change it to CHOST=x86_64-pc-linux-gnu... but that's something for later...

I'm just following the instructions while emerging 'distcc' (doesn't look like 
many did just from scanning the posts).

The patch worked fine.

closing this as multilib-pkg isn't the way to go, so it should be closed...
blah.

Started to emerge distcc, got this message:
 * Applying distcc-gentoo-multilib-r1.patch ...
 * Please report to bug #75420 success or failure of this patch.
Patched with no problems, I think. (Didn't error, at least)

(In reply to comment #39)
> closing this as multilib-pkg isn't the way to go, so it should be closed...
> blah.
> 

Can I remove distcc-2.18.3-r8 which had the multilib patch then?