Created attachment 516502 [details] build log Seems musl incorrectly generates the /lib/ld-musl-${arch}.so.1 symlink: ./tools/install.sh -D -l /usr/lib/libc.so /tmp/arm-unknown-linux-musleabi/portage/sys-libs/musl-1.1.18-r1/image//lib/ld-musl-arm.so. 1 || true This causes the build to fail with this cryptic message: * ERROR: sys-libs/musl-1.1.18-r1::gentoo failed (install phase): * (no error message) * * Call stack: * ebuild.sh, line 124: Called src_install * environment, line 2330: Called die * The specific snippet of code: * [[ -e "${D}"/lib/ld-musl-${arch}.so.1 ]] || die; * * If you need support, post the output of `emerge --info '=sys-libs/musl-1.1.18-r1::gentoo'`, * the complete build log and the output of `emerge -pqv '=sys-libs/musl-1.1.18-r1::gentoo'`. * The complete build log is located at '/tmp/arm-unknown-linux-musleabi/portage/sys-libs/musl-1.1.18-r1/temp/build.log'. * The ebuild environment file is located at '/tmp/arm-unknown-linux-musleabi/portage/sys-libs/musl-1.1.18-r1/temp/environment'. * Working directory: '/tmp/arm-unknown-linux-musleabi/portage/sys-libs/musl-1.1.18-r1/work/musl-1.1.18' * S: '/tmp/arm-unknown-linux-musleabi/portage/sys-libs/musl-1.1.18-r1/work/musl-1.1.18' The link should be relative; the command should read: ./tools/install.sh -D -l ../usr/lib/libc.so /tmp/arm-unknown-linux-musleabi/portage/sys-libs/musl-1.1.18-r1/image//lib/ld-musl-arm.so. 1 || true Reproduction steps: 1. layman -a musl 2. crossdev -t arm-unknown-linux-musleabi # This will fail when it gets to gcc. 3. Go to the overlay generated by crossdev, replace the symlink for `gcc` that currently points to /usr/portage, to the equivalent in the musl overlay. 4. emerge cross-arm-unknown-linux-musleabi/gcc # You may need to play with USE-flag settings to get it to build 5. Set up /usr/cross-arm-unknown-linux-musleabi/etc/portage/make.conf with ELIBC="musl" and other options 6. arm-unknown-linux-musleabi-emerge sys-libs/musl
Further digging, it seems ${arch} is not set… Modified ebuild: if [[ ${CATEGORY} != cross-* ]] ; then local arch=$("${D}"usr/lib/libc.so 2>&1 | sed -n '1s/^musl libc (\(.*\))$/\1/p') [[ -e "${D}"/lib/ld-musl-${arch}.so.1 ]] || die "Broken ld-musl-${arch}.so.1" cp "${FILESDIR}"/ldconfig.in "${T}" || die sed -e "s|@@ARCH@@|${arch}|" "${T}"/ldconfig.in > "${T}"/ldconfig || die into / dosbin "${T}"/ldconfig into /usr dobin "${T}"/getconf dobin "${T}"/getent dobin "${T}"/iconv echo 'LDPATH="include ld.so.conf.d/*.conf"' > "${T}"/00musl || die doenvd "${T}"/00musl || die fi Result: * ERROR: sys-libs/musl-1.1.18-r1::gentoo failed (install phase): * Broken ld-musl-.so.1
Created attachment 516524 [details] proposed ebuild Okay, fixed the problem. The problem is two-fold. 1. The Makefile symlinks /usr/lib/libc.so to the target, which is wrong because when installed in a subdirectory, the actual file we want will be in ${ROOT}/usr/lib/libc.so. I fix this by patching the Makefile in src_prepare: src_prepare() { # Patch Makefile to fix ld.so symlink sed -i -e '/\$(INSTALL) -D -l \$(libdir)\/libc.so/ { s:\$(libdir)/libc.so:$(shell realpath --relative-to="$(@D)" $(DESTDIR)&): }' "${S}/Makefile" || die "patch failed" eapply_user } 2. The ebuild script tries to set ${arch} by executing ld.so. An x86-64 host is *never* going to natively execute an armv5te binary. The solution, good ol'e `strings`: local arch=$( $(tc-getPROG STRINGS strings) "${D}"usr/lib/libc.so \ | sed -n '/^musl libc / { s/^.*(\(.*\))$/\1/; p }') toolchain-funcs lacks a tc-getSTRINGS, but I was able to reverse engineer how the others worked to come up with that.
(In reply to Stuart Longland from comment #2) > Created attachment 516524 [details] > proposed ebuild > > Okay, fixed the problem. The problem is two-fold. > > 1. The Makefile symlinks /usr/lib/libc.so to the target, which is wrong > because > when installed in a subdirectory, the actual file we want will be in > ${ROOT}/usr/lib/libc.so. > > I fix this by patching the Makefile in src_prepare: > > src_prepare() { > # Patch Makefile to fix ld.so symlink > sed -i -e '/\$(INSTALL) -D -l \$(libdir)\/libc.so/ { > s:\$(libdir)/libc.so:$(shell realpath --relative-to="$(@D)" $(DESTDIR)&): > }' "${S}/Makefile" || die "patch failed" > eapply_user > } > > 2. The ebuild script tries to set ${arch} by executing ld.so. An x86-64 host > is *never* going to natively execute an armv5te binary. The solution, > good > ol'e `strings`: > > local arch=$( $(tc-getPROG STRINGS strings) "${D}"usr/lib/libc.so \ > | sed -n '/^musl libc / { s/^.*(\(.*\))$/\1/; p }') > > toolchain-funcs lacks a tc-getSTRINGS, but I was able to reverse engineer > how the others worked to come up with that. Have you tested any of these changes on a native compile? To be honest, I'm not interested in cross compiling. I'm willing to support it but if and only if it doesn't break how things are being done natively. I suspect change 1 above will cause problems in a catalyst run. Some suggestions: 1) get tc-getSTRINGS into toolchian.eclass so that its discussed by a broader group of people who can asses your approach. 2) test on a native comile to make sure nothing breaks and protect anything that changes stuff behind `is_crosscompile`.
(In reply to Anthony Basile from comment #3) > Have you tested any of these changes on a native compile? To be honest, I'm > not interested in cross compiling. I'm willing to support it but if and only > if it doesn't break how things are being done natively. I suspect change 1 > above will cause problems in a catalyst run. Not as yet, the steps I'm doing are: 0. Build a arm-unknown-linux-musleabi toolchain. 1. Get a basic system going in /usr/arm-unknown-linux-musleabi that I can make into a chroot environment on the target device. 2. Throw this on an ARMv5 system, do an `emerge -e @system` 3. Install catalyst 4. Tar up my root as a seed stage and throw that at catalyst. Not much different to what I did for mips64eln64. At the moment, I'm stuck at (1)… specifically, compiling sys-devel/gcc to run on the target. Soon as I'm over that hump though, I will be trying this out with Catalyst. I'll probably also do this all over again for arm-unknown-linux-musleabihf… as I have a Raspberry Pi here (ARMv6-based) that I'd like to try musl on. > Some suggestions: > > 1) get tc-getSTRINGS into toolchian.eclass so that its discussed by a > broader group of people who can asses your approach. > > 2) test on a native comile to make sure nothing breaks and protect anything > that changes stuff behind `is_crosscompile`. Yep, will do.
(In reply to Stuart Longland from comment #4) > (In reply to Anthony Basile from comment #3) > > Have you tested any of these changes on a native compile? To be honest, I'm > > not interested in cross compiling. I'm willing to support it but if and only > > if it doesn't break how things are being done natively. I suspect change 1 > > above will cause problems in a catalyst run. > > Not as yet, the steps I'm doing are: > 0. Build a arm-unknown-linux-musleabi toolchain. > 1. Get a basic system going in /usr/arm-unknown-linux-musleabi that I can > make into a chroot environment on the target device. > 2. Throw this on an ARMv5 system, do an `emerge -e @system` > 3. Install catalyst > 4. Tar up my root as a seed stage and throw that at catalyst. > > Not much different to what I did for mips64eln64. At the moment, I'm stuck > at (1)… specifically, compiling sys-devel/gcc to run on the target. Soon as > I'm over that hump though, I will be trying this out with Catalyst. you don't have to go to exotic systems. you can test with amd64 and i686 tarballs on the mirrors. > > I'll probably also do this all over again for arm-unknown-linux-musleabihf… > as I have a Raspberry Pi here (ARMv6-based) that I'd like to try musl on. > > > Some suggestions: > > > > 1) get tc-getSTRINGS into toolchian.eclass so that its discussed by a > > broader group of people who can asses your approach. > > > > 2) test on a native comile to make sure nothing breaks and protect anything > > that changes stuff behind `is_crosscompile`. > > Yep, will do.
Yeah, I realise that… actually my i[456]86 and AMD64 musl builds are out of date. Been meaning to fix that for a while, so this gives me an excuse.
I'm in the process of testing this with a native build within Catalyst… I am presently battling this little gem: copying chroot-functions.sh to /var/tmp/catalyst/tmp/musl/stage1-amd64-20180126//tmp Ensure the file has the executable bit set Running stage1-chroot.sh in chroot /var/tmp/catalyst/tmp/musl/stage1-amd64-20180126/ Adding USE=" build" to make.conf for portage build emerge --quiet --usepkg --buildpkg --newuse --oneshot --update --newuse sys-apps/portage Skipping seed stage update... emerge --quiet --usepkg --buildpkg --newuse --oneshot --nodeps sys-apps/baselayout emerge --quiet --usepkg --buildpkg --newuse --oneshot app-shells/bash:0 virtual/pkgconfig sys-devel/bison dev-python/pyxattr sys-devel/binutils sys-apps/diffutils app-arch/gzip sys-apps/gawk sys-apps/coreutils sys-apps/less sys-apps/baselayout sys-devel/gcc sys-apps/findutils app-arch/bzip2 virtual/libc app-arch/xz-utils net-misc/wget sys-apps/makedev sys-devel/libtool app-arch/tar sys-devel/patch virtual/package-manager virtual/editor sys-apps/grep sys-apps/file sys-apps/sandbox dev-util/pkgconf net-misc/rsync sys-devel/flex sys-devel/autoconf sys-apps/net-tools virtual/shadow sys-apps/which sys-apps/sed virtual/os-headers sys-devel/gnuconfig sys-devel/gettext sys-apps/attr sys-devel/make sys-devel/automake !!! The ebuild selected to satisfy ">=app-text/asciidoc-8.6.3" has unmet requirements. - app-text/asciidoc-8.6.10::gentoo USE="-examples -graphviz -highlight -test" PYTHON_SINGLE_TARGET="(-pypy) -python2_7" PYTHON_TARGETS="python2_7 (-pypy)" The following REQUIRED_USE flag constraints are unsatisfied: exactly-one-of ( python_single_target_pypy python_single_target_python2_7 ) The above constraints are a subset of the following complete expression: exactly-one-of ( python_single_target_pypy python_single_target_python2_7 ) python_single_target_pypy? ( python_targets_pypy ) python_single_target_python2_7? ( python_targets_python2_7 ) (dependency required by "sys-apps/paludis-2.6.0::gentoo" [ebuild]) (dependency required by "app-admin/perl-cleaner-2.25::gentoo" [ebuild]) (dependency required by "dev-lang/perl-5.24.3::gentoo" [ebuild]) (dependency required by "sys-devel/autoconf-2.69-r4::gentoo" [ebuild]) (dependency required by "sys-devel/libtool-2.4.6-r3::gentoo" [ebuild]) (dependency required by "sys-devel/libtool" [argument]) The problem affects glibc too… basically unless I manually break into the chroot and tack portage-utils onto that list, perl-cleaner tries to drag in paludis. Soon as I've worked around that issue, I should be in a better position to exercise these ebuild changes.
(In reply to Stuart Longland from comment #7) > I'm in the process of testing this with a native build within Catalyst… > > I am presently battling this little gem: > upgrade your system to portage-2.3.20. make sure it also gets into the catalyst tarballs. i just marked it stable for amd64 and x86 in my local portage tree. its a really nasty bug in 2.3.19-r1. also, use catalyst-2.0.18-r4.
> make sure it also gets into the > catalyst tarballs. let me explain, when you create the catalyst tarballs using `catalyst -s current` then portage gets in there. you need to make sure that portage-2.3.20 is marked stable for amd64 and x86 BEFORE you run `catalyst -s current`.
Ahh, right… wondered if it was something like that. It'll be a tomorrow job methinks. Desktop is hibernated so I can get some sleep (a quiet room-mate a Phenom II X6 loaded with fans does not make!), but it sounds like the best course will be for me to unpack my seed tarballs, update portage on those, then re-pack them and try it all again. If I wait until daylight hours, that should give the mirrors plenty of time to propagate those changes. :-)
Okay, so after some rejigging /usr/portage to make portage-2.3.20 stable and patching musl… and an own-goal nuking my /dev tree (catalyst didn't unmount the bind mounts properly before I did an rm -fr) … I got as far as stage 3: emerge --quiet --usepkg --buildpkg --newuse -e @system emerge: there are no ebuilds to satisfy "x11-misc/shared-mime-info". (dependency required by "dev-libs/glib-2.52.3::musl[mime]" [ebuild]) (dependency required by "dev-util/pkgconfig-0.29.2::gentoo[-internal-glib]" [ebuild]) (dependency required by "virtual/pkgconfig-0-r1::gentoo" [ebuild]) (dependency required by "sys-apps/kbd-2.0.4::gentoo" [ebuild]) (dependency required by "@system" [argument]) !!! catalyst: run script failed. The fact that it successfully completed stage 2 suggests to me that native building of musl with this ebuild works.
(In reply to Anthony Basile from comment #3) > 1) get tc-getSTRINGS into toolchian.eclass so that its discussed by a > broader group of people who can asses your approach. Okay, so I floated the idea in bug #645888 and it was suggested that `strings` is `strings` no matter what you call it. So that earlier change can be replaced with: local arch=$( strings "${D}"usr/lib/libc.so \ | sed -n '/^musl libc / { s/^.*(\(.*\))$/\1/; p }') I think SpanKY raises a good point though; why can't we derive ${arch} from the ARCH or CTARGET variables? I tried to find a way to make it work, without really understanding why the code is there. Perhaps we need to re-consider this? > 2) test on a native comile to make sure nothing breaks and protect anything > that changes stuff behind `is_crosscompile`. After resolving an unrelated issue with my portage snapshot, my desktop coughed up these: http://repo.longlandclan.id.au/gentoo/stages/musl/x86/amd64/ So evidently, it works within Catalyst, at least for AMD64. I've still got to get some things straightened out with my i[456]86 stages.
Okay, think I've found the mystery of why x86 is broken: RC=0 stuartl@rikishi /tmp $ cat /usr/portage/profiles/hardened/linux/musl/x86/parent ../../../../default/linux/uclibc/x86 .. RC=0 stuartl@rikishi /tmp $ cat /usr/portage/profiles/hardened/linux/musl/amd64/parent ../../../../default/linux/musl/amd64 ..
(In reply to Stuart Longland from comment #13) > Okay, think I've found the mystery of why x86 is broken: > > RC=0 stuartl@rikishi /tmp $ cat > /usr/portage/profiles/hardened/linux/musl/x86/parent > ../../../../default/linux/uclibc/x86 > .. > RC=0 stuartl@rikishi /tmp $ cat > /usr/portage/profiles/hardened/linux/musl/amd64/parent > ../../../../default/linux/musl/amd64 > .. i was just made aware in bug #645926. there are new profiles coming down the pipeline, so i'm not going to fix this now, just replace it with the new profiles.
Right, so having fixed up the profile reference in my local tree, I now have successful i486 and i686 builds: http://repo.longlandclan.id.au/gentoo/stages/musl/x86/i486/ (uploading at time of writing) http://repo.longlandclan.id.au/gentoo/stages/musl/x86/i686/ It'd be nice to try the changes out on non-x86 as well, but I'm confident it shouldn't break anything there.
This bug is mainly my fault. Deriving the ldso name from CTARGET should be possible, but would involve a lot of case work, and would need to be updated each time we add a new arch. (The name can depend on whether hard or soft float is used and on endianness.) Alpine[1] abuses the Makefile to get the ldso name. Alternatively, the ldso name is the concatenation of ARCH and SUBARCH from config.mak. [1]: https://git.alpinelinux.org/cgit/aports/plain/main/musl/APKBUILD
(In reply to Felix Janda from comment #16) > This bug is mainly my fault. Hey, I'm not criticising. :-) > Deriving the ldso name from CTARGET should be possible, but would > involve a lot of case work, and would need to be updated each time we > add a new arch. (The name can depend on whether hard or soft float is > used and on endianness.) Just a thought, could we do something like this: case ${CTARGET} in arm-*-linux-musleabi) arch="arm" ;; x86_64-*-linux-musl) arch="x86_64" ;; # … etc *) arch=$( strings "${D}"usr/lib/libc.so \ | sed -n '/^musl libc / { s/^.*(\(.*\))$/\1/; p }') ;; esac Then, we only fall through to using `strings` if other methods fail. We might put an `ewarn` message there to say to the effect of "Unrecognised target ${CTARGET} maps to ${arch}, please file a bug to get this recorded". The build will still work then, and we get to learn what the different combinations are. Alternatively, do we just rely on the fact that there's going to be a symlink matching the pattern ld-musl-*.so, and extract ${arch} that way? > Alpine[1] abuses the Makefile to get the ldso name. Alternatively, > the ldso name is the concatenation of ARCH and SUBARCH from config.mak. > > [1]: https://git.alpinelinux.org/cgit/aports/plain/main/musl/APKBUILD That sounds simple enough to extract.
hardcoding all possible dynamic linker names looks like a lot of work. Note that they depend on softfloat/hardfloat and the endianess. For the second problem raised in Comment 2, try if the following works: diff --git a/sys-libs/musl/musl-1.1.18.ebuild b/sys-libs/musl/musl-1.1.18.ebuild index e36e219945..85ef92c6db 100644 --- a/sys-libs/musl/musl-1.1.18.ebuild +++ b/sys-libs/musl/musl-1.1.18.ebuild @@ -92,7 +92,7 @@ src_install() { dosym ${sysroot}/lib/${ldso} ${sysroot}/usr/bin/ldd if [[ ${CATEGORY} != cross-* ]] ; then - local arch=$("${D}"usr/lib/libc.so 2>&1 | sed -n '1s/^musl libc (\(.*\))$/\1/p') + local arch=$(sed -n 's/^ARCH = \(.*\)$/\1/p' config.mak)$(sed -n 's/^SUBARCH = \(.*\)$/\1/p' config.mak) [[ -e "${D}"/lib/ld-musl-${arch}.so.1 ]] || die cp "${FILESDIR}"/ldconfig.in "${T}" || die sed -e "s|@@ARCH@@|${arch}|" "${T}"/ldconfig.in > "${T}"/ldconfig || die I don't quite understand the first problem in Commment 2. The symlink for the dynamic linker looks totally fine to me. Note that the dynamic linker becomes only useful after deploying the crosscompiled system or after chrooting into it (using something like qemu userland emulation).
Not to be a nag, but I just ran into this today myself when using crossdev to emerge @system into a sdcard for my asus c201 chromebook I'm doing u-boot dev on/for/with and was wondering if there was any progress towards a resolution. Quite a lot works well using cross-emerge --root /mnt --sysroot /mnt @system, but without a libc its kinda pointless.
*** Bug 696236 has been marked as a duplicate of this bug. ***
Hello. I want to say that everyone are trying to find the universal fix, but there is much easier solution: find "/usr/portage/sys-libs/musl" -maxdepth 1 -name musl-*.ebuild \ -exec sed -i "s/local arch=.*$/local arch=\"arm\"/g" "{}" \; \ -exec ebuild "{}" manifest \; Why? You will have to include arm or mips specific fixes in your image build anyway. There are no universal fixes (in musl too) for other issues. So I think it is better to maintain separate build directories for arm, armeb, aarch64, aarch64_be, etc. All fixes in these directories don't need to be universal.
Created attachment 603636 [details, diff] relative libc symink
Created attachment 639906 [details, diff] Fix cross compile for main arch This still needs to be tested with catalyst, I have tested it with native as well as cross compiles. Feel free to test and report your findings.
(In reply to Jory A. Pratt from comment #23) > Created attachment 639906 [details, diff] [details, diff] > Fix cross compile for main arch > > This still needs to be tested with catalyst, I have tested it with native as > well as cross compiles. Feel free to test and report your findings. While this will work for just about all arch it will not work for 32bit arm, or x86. I will have to add some magic to get them set properly.
Created attachment 640190 [details, diff] fix cross compilation for all supported targets I have not had a chance to test it, the sooner we can get basic testing out the way sooner I will push to get catalyst builds done for testing.
Created attachment 640240 [details, diff] Fix cross compile issue V2 Fix space I introduced.
Given that musl does quite a bit of ${CHOST} handling on it's own in ./configure phase I'd suggest using it's final values as is as they are used in: Makefile:LDSO_PATHNAME = $(syslibdir)/ld-musl-$(ARCH)$(SUBARCH).so.1 Something like the following does the right thing for a few cases I tried: --- a/sys-libs/musl/musl-9999.ebuild +++ b/sys-libs/musl/musl-9999.ebuild @@ -98,7 +98,11 @@ src_install() { dosym ${sysroot}/lib/${ldso} ${sysroot}/usr/bin/ldd if [[ ${CATEGORY} != cross-* ]] ; then - local arch=$("${D}"usr/lib/libc.so 2>&1 | sed -n '1s/^musl libc (\(.*\))$/\1/p') + # Fish out of config: + # ARCH = ... + # SUBARCH = ... + # and print $(ARCH)$(SUBARCH). + local arch=$(awk '{ k[$1] = $3 } END { printf("%s%s", k["ARCH"], k["SUBARCH"]); }' config.mak) [[ -e "${D}"/lib/ld-musl-${arch}.so.1 ]] || die cp "${FILESDIR}"/ldconfig.in "${T}" || die sed -e "s|@@ARCH@@|${arch}|" "${T}"/ldconfig.in > "${T}"/ldconfig || die
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=a9b1dd3546c9849e5e342d246bdddea95c00dd2a commit a9b1dd3546c9849e5e342d246bdddea95c00dd2a Author: Sergei Trofimovich <slyfox@gentoo.org> AuthorDate: 2020-05-20 22:06:40 +0000 Commit: Jory Pratt <anarchy@gentoo.org> CommitDate: 2020-05-20 22:12:03 +0000 sys-libs/musl: extract $(ARCH)$(SUBARCH) from config.mak Closes: https://bugs.gentoo.org/642612 Closes: https://bugs.gentoo.org/645626 Package-Manager: Portage-2.3.99, Repoman-2.3.22 Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org> Signed-off-by: Jory Pratt <anarchy@gentoo.org> sys-libs/musl/musl-1.1.24.ebuild | 6 +++++- sys-libs/musl/musl-1.2.0.ebuild | 6 +++++- sys-libs/musl/musl-9999.ebuild | 6 +++++- 3 files changed, 15 insertions(+), 3 deletions(-)