Bugzilla Bug 284957

Although klibc has been used for ARCH=arm for quite some time (in other
distros), the keyword has been missing from the Gentoo ebuild. Klibc is used,
among other things, for boot-splash. I haven't yet tested splash-utils for arm
yet, but Spock says that it should work for generic framebuffer devices. 

This bug includes patches for the klibc-1.5.15 ebuild, enabling ARM (and EABI)
support. It also cleans up some of the cross-specific routines for a cleaner
appearance. 

Some other incidentals include patches borrowed from OpenEmbedded / Angstrom.
See the attachments for details. 

Reproducible: Always

Created an attachment (id=204076) [details]
dev-libs/klibc-1.5.15-ebuild-with-arm-support.diff

Created an attachment (id=204078) [details]
dev-libs/klibc-1.5.15.ebuild

Created an attachment (id=204079) [details]
dev-libs/klibc/files/oe/dash_readopt.patch

Created an attachment (id=204080) [details]
dev-libs/klibc/files/oe/klibc_kexecsyscall.patch

Created an attachment (id=204084) [details]
dev-libs/klibc/files/oe/losetup.patch

Created an attachment (id=204086) [details]
dev-libs/klibc/files/oe/klibc-1.5.15/fstype-sane-vfat-and-jffs2-for-1.5.patch

Created an attachment (id=204087) [details]
dev-libs/klibc/files/oe/klibc-1.5.15/modprobe.patch

Created an attachment (id=204089) [details]
dev-libs/klibc/files/oe/klibc-1.5.15/wc.patch

Created an attachment (id=204130) [details]
dev-libs/klibc/klibc-1.5.15.ebuild

removed a line of code that was creating conflicting files (symlinks) in /bin
instead of /usr/lib/klibc/bin

I'm not going to accept it in this form. Please split it up like this:
1. Patch series to add savedconfig support.
2. Patch series to add new features (really should be sent upstream, not merged
here).

I'm doing seperate work on it for the ARM keyword, which should really be quite
minimal. Should be done in a couple of days, and you can respin thereafter (I
suggest taking the feature additions to upstream in the meantime).

Basic ARM support is now implemented as I promised.

Please respin off that, with two separate sets of changes.

Created an attachment (id=204289) [details]
ebuild-move-fns-up.patch

This patch just moves kernel_asm_arch() and kernel_defconfig() above
src_unpack(). It's necessary because the savedconfig routines should be
performed in src_unpack(). The require_savedconfig() function (see next patch
file) uses kernel_asm_arch(). The kernel_defconfig() function requires
savedconfig_dst() (see next patch file).

Created an attachment (id=204291) [details]
ebuild-with-savedconfig.patch

first, apply ebulid-move-fns-up.patch, and then this one

Questions:

1) exactly where and when ABI or ARCH interfere with any part of the build
process?

I was using armv5tel-softfloat-linux-gnueabi-emerge and had no difficulties
whatsoever when ABI or ARCH were left intact for kernel / klibc, src_compile /
src_install. I never unset either of those variables. 

2) where and how is it that the T variable is used by klibc ?

3) In src_install(), can you please clarify the following ?

==============================================================================
        if tc-is-cross-compiler ; then
                klibc_prefix=$("${S}/klcc/${KLIBCARCH}-klcc"
-print-klibc-prefix)
        else
                klibc_prefix=$("${S}/klcc/klcc" -print-klibc-prefix)
        fi
...

        for x in gunzip zcat ; do
                rm -f "${D}/${klibc_prefix}/bin/${x}"
                dosym gzip "${klibc_prefix}/bin/${x}"
        done
==============================================================================

As far as I've seen, when cross-compiling klibc, you do not end up with klcc as
a cross-compiler, so how is ${KLIBCARCH}-klcc expected to run natively? 

With something like, armv5tel-softfloat-linux-gnueabi-emerge klibc, the
'klibc_prefix' variable is empty (because ${KLIBCARCH}-klcc is not a runnable
native binary - e.g. x86 / arm), and therefore installs conflicting files into
${ROOT}/bin . 

Is this a nuance specifically for ppc64 -> ppc, or x86_64 -> x86 ? 

Please fix it appropriately.

Comments:

1) The ebuild can be made much 'cleaner' looking if most of the ARCH / ABI
related things are removed (see attachment #204130 [details]). Unsetting ARCH / ABI is
unnecessary IMHO ... unless it's for a strange ppc64->ppc or x86_64->x86
nuance.

PS: I do like your additional  AOABI / AEABI checks - good work!

(In reply to comment #14)
> 1) exactly where and when ABI or ARCH interfere with any part of the build
> process?
The kernel build system uses ARCH. The tc-* functions need BOTH of them to be
set for Gentoo usage. If you have ARCH unset, and ABI set, you may get
undefined behavior, so we unset both of them together.

When the userland bitcount and the kernel bitcount are different, and the
userland could be one of two possible options, then you need to be very
careful. This is most commonly seen on PPC64-32UL. (It's rarer on sparc, as
they always have a 64-bit kernel, and afaik no 64-bit userland).

Additionally, ARCH is normally populated with the Gentoo name for an
architecture. Which doesn't match the kernel name always, and also doesn't
match the klibc name.

ARM:
gentoo:arm, kernel:arm, klibc:arm, defconfig:${MACHINE}_defconfig

Opteron:
gentoo:amd64, kernel:x86, klibc:x86_64, defconfig:defconfig

Athlon:
gentoo:x86 kernel:x86, klibc: i386, defconfig:defconfig

PPC64-32ul (Apple G5):
gentoo:ppc, kernel:powerpc, klibc:ppc, defconfig:ppc64_defconfig

PPC64-64ul (Apple G5, not supported in the ebuild right now)
gentoo:ppc64, kernel:powerpc, klibc:ppc, defconfig:ppc64_defconfig

PPC32 (Efika)
gentoo:ppc, kernel:powerpc, klibc:ppc, defconfig:pmac32_defconfig

If you happen to have $ARCH set to a value that the kernel doesn't like, well,
it falls over and dies, badly.

> 2) where and how is it that the T variable is used by klibc ?
It was used in older version, I don't think it's used anymore. Probably safe to
remove now.

> 3) In src_install(), can you please clarify the following ?
> ==============================================================================
>         if tc-is-cross-compiler ; then
>                 klibc_prefix=$("${S}/klcc/${KLIBCARCH}-klcc"
> -print-klibc-prefix)
>         else
>                 klibc_prefix=$("${S}/klcc/klcc" -print-klibc-prefix)
>         fi
> ...
> 
>         for x in gunzip zcat ; do
>                 rm -f "${D}/${klibc_prefix}/bin/${x}"
>                 dosym gzip "${klibc_prefix}/bin/${x}"
>         done
> ==============================================================================
> As far as I've seen, when cross-compiling klibc, you do not end up with klcc as
> a cross-compiler, so how is ${KLIBCARCH}-klcc expected to run natively? 
> 
> With something like, armv5tel-softfloat-linux-gnueabi-emerge klibc, the
> 'klibc_prefix' variable is empty (because ${KLIBCARCH}-klcc is not a runnable
> native binary - e.g. x86 / arm), and therefore installs conflicting files into
> ${ROOT}/bin . 
${KLIBCARCH}-klcc should be a cross-compiler last I checked, but that was a
long time ago. I tend to do native builds (yes, even on ARM), rather than
cross-compiles. So it is possible that part of the codebase is wrong. If
${KLIBCARCH}-klcc is a cross-compiler, it should be runnable on the local
machine, and target the ${KLIBCARCH}.

> Comments:
> 1) The ebuild can be made much 'cleaner' looking if most of the ARCH / ABI
> related things are removed (see attachment #204130 [details] [edit]). Unsetting ARCH / ABI is
> unnecessary IMHO ... unless it's for a strange ppc64->ppc or x86_64->x86
> nuance.
Yes it's required.

(In reply to comment #13)
> Created an attachment (id=204291) [edit] [details]
> ebuild-with-savedconfig.patch
> first, apply ebulid-move-fns-up.patch, and then this one
Thanks for a respin, it made the review a LOT easier.

0.
I've merged the ebuild-move-fns-up.patch, it's of clear benefit.

1.
======
require_savedconfig
die "USE=\"savedconfig\" is required for ARCH=${ARCH}"
======
ARM does NOT require any specific defconfig. The entire point of the defconfig
code is so that you get the correct headers for the architecture and ABI. The
ebuild modifications I did work perfectly on my DB-MV78100-BP board. The
architecture is common for all of ARM afaik, and the ABI part we take care of
with the CHOST check. If you've got a native ARM build that builds klibc such
that it doesn't work, then I'm interested in it.

2.
A lot of this appears to be a poor imitation of the savedconfig eclass:
======
SAVED_DEFCONFIG_SRC="${PORTAGE_CONFIGROOT}etc/portage/savedconfig/${CATEGORY}/${PF}"
======
please see the eclass, a lot more possible locations are applicable, esp.
important are those with CHOST/CTARGET/nil for cross-compiling.

3.
======
# do first to avoid wasting time or resources
======
You can avoid duplicating a lot of logic in testing, by just moving the
restore_config way further up (into pkg_setup even), with the destination as a
tempfile in ${T}, and then moving that file to the desired location when the
kernel sources are available.

4.
If we're going to use the the savedconfig for a kernel config, we're likely to
end up S.O.L. when we want to use it for klibc's config :-(. This may require
reworking the eclass.

5.
======
+IUSE="debug n32 savedconfig"
======
The eclass adds it for you.

6.
If we're going to do this, I don't see any reason to limit it to ARM. MIPS and
embedded PPC stand to benefit as well. MIPS really matters heavily, as the
kernel config controls the ABI depending on which ones the hardware is capable
of (some hardware cannot do certain ABIs).

7.
Did you take the rest of the feature additions upstream?

(From update of attachment 204079 [details])
Send patch to upstream.

(From update of attachment 204080 [details])
Send patch to upstream.

(From update of attachment 204084 [details])
Send patch to upstream.

(From update of attachment 204086 [details])
Send patch to upstream.

(From update of attachment 204087 [details])
Send patch to upstream.

(From update of attachment 204089 [details])
Send patch to upstream.

(From update of attachment 204289 [details])
Merged.

(In reply to comment #17)
> 6.
> If we're going to do this, I don't see any reason to limit it to ARM. MIPS and
> embedded PPC stand to benefit as well. MIPS really matters heavily, as the
> kernel config controls the ABI depending on which ones the hardware is capable
> of (some hardware cannot do certain ABIs).

Actually, if klibc just does a default ARMv4 target (i think that's the case -
omitting thumb, because it differs slightly with v4T, v5T, and v6T2), then
maybe it would be possible to get away with not using savedconfig at all for
ARM. ARM kernels also use .config for CONFIG_AEABI and CONFIG_OABI_COMPAT, but
it should be easy enough to replace that with a sed expression on defconfig.
Still, if PPC and MIPS would benefit from this, then savedconfig is a good
thing. What ARM ISA did your ARM board use? ... All ARM ISA's, even Cortex-A8,
are backward-compatible with ARMv4, I believe. Klibc is also specifically
intended to be light-weight, not necessarily optimized, so if we can find a
suitable ARMv4 defconfig, then that should be good enough for all ARM.

I'll change the ebuild by move the savedconfig file to ${T} and then to
${KS}/arch/${KLIBCASMARCH}/configs/savedconfig_defconfig, as you suggested.
That's a good idea, and then I can remove those nasty error checks. 

However, with all of the various arches and their differing ABI rules and
compatibilities, it might be nice to factor that out into a abi_fixup()
function based on ARCH. 

Note, that the move-fn patch only moves kernel_asm_arch() and klibc_arch(), but
not kernel_defconfig() . 

Also, I thought I'd point out ... the "proper" way to cross-compile both klibc
and the kernel are by using CROSS_COMPILE="${CHOST}-" (and
ARCH="$(kernel_asm_arch)" for the kernel), not by setting CC. Simply setting CC
will eventually fail (either on linux or klibc) because the proper LD is not
being used. Setting HOSTCC=$(tc-getBUILD_CC) is probably unnecessary...
normally the kernel would choose the correct CBUILD compiler.

I'm going to submit another patch for the CROSS_COMPILE arguments (i.e.
kernel_xargs, klibc_xargs), but I might not get around to doing everything else
for a few days yet. I'm trying to finish up a major project for a customer
today, documentation & all. I'm also right in the middle of 1 1/2 months of
exams (doing a MSc on the while working f/t!), so I've got a lot on the go at
the moment. 

Since I 'borrowed' the incidental patches from OE, there might be a chance that
they've already submitted them to upstream, but I haven't yet checked, again
due to "crunch time".

(In reply to comment #25)
> Actually, if klibc just does a default ARMv4 target (i think that's the case -
> omitting thumb, because it differs slightly with v4T, v5T, and v6T2), then
> maybe it would be possible to get away with not using savedconfig at all for
> ARM. ARM kernels also use .config for CONFIG_AEABI and CONFIG_OABI_COMPAT, but
> it should be easy enough to replace that with a sed expression on defconfig.
> Still, if PPC and MIPS would benefit from this, then savedconfig is a good
> thing. What ARM ISA did your ARM board use? ... All ARM ISA's, even Cortex-A8,
> are backward-compatible with ARMv4, I believe. Klibc is also specifically
> intended to be light-weight, not necessarily optimized, so if we can find a
> suitable ARMv4 defconfig, then that should be good enough for all ARM.
the versatile_defconfig that 'make defconfig' does for ARM should suffice I
think :-). Feroceon here, v5 ISA.

> Also, I thought I'd point out ... the "proper" way to cross-compile both klibc
> and the kernel are by using CROSS_COMPILE="${CHOST}-" (and
> ARCH="$(kernel_asm_arch)" for the kernel), not by setting CC. Simply setting CC
> will eventually fail (either on linux or klibc) because the proper LD is not
> being used. Setting HOSTCC=$(tc-getBUILD_CC) is probably unnecessary...
> normally the kernel would choose the correct CBUILD compiler.
wrong: CROSS_COMPILE="${CHOST}-" 
right: CROSS_COMPILE="${CTARGET}-"

We aren't cross-compiling the kernel at all, just using it's headers.

arm/embedded: Can anybody confirm if the current klibc ebuilds work/fail for
cross-compiles? (check that the ${TARGET}-klcc ELFs are for the correct
architecture too please.

> I'm going to submit another patch for the CROSS_COMPILE arguments (i.e.
> kernel_xargs, klibc_xargs), but I might not get around to doing everything else
> for a few days yet. I'm trying to finish up a major project for a customer
> today, documentation & all. I'm also right in the middle of 1 1/2 months of
> exams (doing a MSc on the while working f/t!), so I've got a lot on the go at
> the moment. 
Thats fine, I don't see this as high priority.

> Since I 'borrowed' the incidental patches from OE, there might be a chance that
> they've already submitted them to upstream, but I haven't yet checked, again
> due to "crunch time".
I'll shoot HPA an email to ask, since I'm not sure if it's in any VCS (but
check git.k.o first).

(In reply to comment #26)
> wrong: CROSS_COMPILE="${CHOST}-" 
> right: CROSS_COMPILE="${CTARGET}-"
> 
> We aren't cross-compiling the kernel at all, just using it's headers.

Oh right - I was still assuming that klcc was _not_ a cross compiler... which
is (oddly) not the case. 

This is a bit of a frictional subject, especially with the embedded people. 

I would really suggest that the klibc cross-toolchain be lumped in with
crossdev, so that cross-compiling klibc for ARCH actually results in a klcc
that runs on ARCH. Then one could easily run build ${CHOST-}klcc with crossdev
and use it for package like splashutils that require a cross-klcc. However, if
I do recall correctly, somebody once suggested that crossdev should support
klibc and Mike swiftly decapitated them, so it's likely not going to happen
anytime soon. 

Still... it would be nice to have an ARM klcc when I cross-compile klcc for
ARM, because compiling it natively on some machines is pretty darn slow.

(In reply to comment #27)
> (In reply to comment #26)
> > wrong: CROSS_COMPILE="${CHOST}-" 
> > right: CROSS_COMPILE="${CTARGET}-"

Actually, as it turns out emerge does not set CTARGET unless it's building a
cross-compiler (i.e. crossdev), so CHOST it is!

> > wrong: CROSS_COMPILE="${CTARGET}-" 
> > right: CROSS_COMPILE="${CHOST}-"

typically embedded is concerned with uClibc as it is the only serious C
library.  the other alternatives are niche implementations and as such, only
one or two people actually watch over/care about it.  as for who those people
are wrt klibc, i have no idea.

as for emerge not setting CTARGET, that's no excuse.  look at glibc/uClibc
ebuilds to see how to handle CTARGET initialization.

crossdev is no requirement for proper cross-compiling support.  it is merely a
frontend to automate the steps that can easily be done by hand.  and in order
to be added to crossdev, you need a sane tuple.  i havent seen any tuples that
indicate "this is a klibc target".

(In reply to comment #29)
> as for emerge not setting CTARGET, that's no excuse.  look at glibc/uClibc
> ebuilds to see how to handle CTARGET initialization.

I'm actually of the opinion that regular ebuilds (generally, when
cross-compiling too) should not use CTARGET, but that should be reserved for
creating cross-toolchains. That's exactly what the GNU folks intended it to be
used for. 

As for klibc / crossdev... it's pretty far-fetched. Aside from RedHat (Debian?
Ubuntu?) using it for the initramfs, and splashutils using it, I don't think
very many of the embedded users would actually create a crossdev-style
toolchain from it, and so it's not really worth maintaining in crossdev.
Although, I suppose a reasonable tuple would be something like
armv5tel-softfloat-linux-klibceabi . It would always be a softfloat toolchain,
of course. 

there is no "ctarget should be reserved".  either target makes sense, or it
doesnt.  if you're building klibc to be used with a cross-toolchain, then use
ctarget.  if you're cross-compiling klibc for use with a native toolchain, then
use chost.