| Summary: | /usr/bin/armv7a-unknown-linux-gnueabihf-strip fails to find various link sections while striping statically linked files during src_install (affected packages: libcxx-7.0.1, libcxxabi-7.0.1, compiler-rt-7.0.1, compiler-rt-plugins-7.0.1) | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | tt_1 <herrtimson> |
| Component: | Current packages | Assignee: | LLVM support project <llvm> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | arm, mgorny, toolchain |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| URL: | https://reviews.llvm.org/D48155 | ||
| See Also: |
https://sourceware.org/bugzilla/show_bug.cgi?id=23817 https://bugs.gentoo.org/show_bug.cgi?id=603594 |
||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | |||
| Bug Blocks: | 595834, 674068 | ||
| Attachments: |
compressed build logs
output of emerge --info compressed build logs compressed build log from compiler-rt-7.0.0 build log from app-arch/bzip2 static lib, readelf output and build log of compiler-rt gentoo-fnoaddrsig.patch (for clang) patch backported to clang-7.0.1 |
||
Created attachment 549486 [details]
output of emerge --info
my emerge --info
(from qemu-static chroot)
since /usr/bin/armv7a-unknown-linux-gnueabihf-strip is part of binutils - I would like to raise attention to the fact that I upgraded binutils (and binutils-libs) just before doing the upgrade of llvm/clang from 6.0.1 to 7.0.0
I'm fairly certain that this behaviour didn't happen with the stable binutils and llvm/clang-6.0.1
I can offer to reproduce the combination of binutils-2.30-r4 and clang/llvm-6.0.1 with a fresh stage upon request (it still takes 3 hrs within a qemu-static chroot)
Note to myself: downgrading binutils to 2.30-r3 from -r4 did not make this vanish. Didn't yet do a rebuild of the whole llvm/clang toolchain due to a lack of spare cycles. Grabbed a fresh armv7-stage3 and reproduced this with the stable binutils-2.30-r2 Also ruled out: this has nothing to do with USE="static-analyzer"; flipping the flag has no meaning for the linking.
[ebuild R *] sys-devel/clang-7.0.0:7::gentoo USE="static-analyzer -debug -default-compiler-rt -default-libcxx -doc {-test} -xml (-z3)" LLVM_TARGETS="(ARM) -AArch64 -AMDGPU -BPF -Hexagon -Lanai -MSP430 -Mips -NVPTX -PowerPC -Sparc -SystemZ -X86 -XCore" PYTHON_TARGETS="python2_7" 0 KiB
So this seems to be related to the statically linked files (don't know if these are binaries or libraries) in all of the four packages. It doesn't matter if gcc is used or clang itself for compiling. Todo: test llvm/clang-6 from a clean stage3 llvm/clang-6.0.1 is clean of the linking failures. @jer - Since I flipped stable and unstable binutils with llvm/clang-7.0.0, without any different behavior, would you mind to assing this to the llvm team? Could you test if using lld to link makes any difference? Sure, it's scheduled for tomorrow.
I'm going to set via package.env:
LDFLAGS="${LDFLAGS},-fuse-ld=lld"
And build with gcc, unless there's a clang useflag, for all llvm/clang related packages.
I don't think gcc supports that. Created attachment 551336 [details]
compressed build logs
Ok, so I did only libcxx-7.0.0, libcxxabi-7.0.0, compiler-rt-7.0.0 and compiler-rt-plugins-7.0.0 with:
CC="clang"
CXX="clang++"
LDFLAGS="${LDFLAGS},-fuse-ld=lld"
is this what you intended?
Nope. '-fuse-ld=lld' must go to clang, not the linker, so without '-Wl,'. Sorry, but I don't understand how to enable that, package.env wise.
LDFLAGS="${LDFLAGS},-fuse-ld=lld" is wrong, got that. You want me to use '-fuse-ld=lld' like a C/CPP flag?
Just put space instead of ','. Created attachment 551404 [details]
compressed build log from compiler-rt-7.0.0
Ok, got it. Now there is four times -fuse-ld=lld in the flags?
And there are still the same warnings about link sections not being found.
Injected values from package.env are:
LDFLAGS="${LDFLAGS} -fuse-ld=lld"
Ok, I'm sorry, I was really stupid and missed the fact that compiler-rt is just static libs, so it doesn't use link editor at all. @arm, can you reproduce something like it? Can it be related to the errors from qemu? I'm almost certain that I hit this the first time when compiling llvm/clang nativly, but to be sure I did set up a chroot: Calculating dependencies... done! [ebuild N ~] sys-devel/llvm-common-7.0.0::gentoo 0 KiB [ebuild N *] sys-libs/libomp-7.0.0::gentoo USE="(-cuda) -hwloc -offload -ompt -test" 890 KiB [ebuild N ~] sys-devel/llvm-7.0.0:7::gentoo USE="gold libffi ncurses -debug -doc (-exegesis) -libedit -test -xar -xml" LLVM_TARGETS="(ARM) -AArch64 -AMDGPU -BPF -Hexagon -Lanai -MSP430 -Mips -NVPTX -PowerPC -Sparc -SystemZ -X86 -XCore" 0 KiB [ebuild N ~] sys-devel/llvmgold-7::gentoo 0 KiB [ebuild N *] sys-libs/llvm-libunwind-7.0.0::gentoo USE="static-libs -debug -test" 78 KiB [ebuild N *] sys-devel/lld-7.0.0::gentoo USE="-test" 895 KiB [ebuild N *] sys-libs/libcxxabi-7.0.0::gentoo USE="libunwind static-libs -test" 0 KiB [ebuild N *] sys-libs/libcxx-7.0.0::gentoo USE="libcxxabi libunwind static-libs -libcxxrt -test" 0 KiB [ebuild N *] sys-devel/clang-7.0.0:7::gentoo USE="-debug -default-compiler-rt -default-libcxx -doc -static-analyzer -test -xml (-z3)" LLVM_TARGETS="(ARM) -AArch64 -AMDGPU -BPF -Hexagon -Lanai -MSP430 -Mips -NVPTX -PowerPC -Sparc -SystemZ -X86 -XCore" PYTHON_TARGETS="python2_7" 882 KiB [ebuild N *] sys-devel/clang-common-7.0.0::gentoo 0 KiB [ebuild N *] sys-libs/compiler-rt-7.0.0:7.0.0::gentoo USE="clang -test" 0 KiB [ebuild N *] sys-libs/compiler-rt-sanitizers-7.0.0:7.0.0::gentoo USE="clang -test" 0 KiB [ebuild N *] sys-devel/clang-runtime-7.0.0:7.0.0::gentoo USE="compiler-rt libcxx openmp sanitize -crt" 0 KiB Total: 13 packages (13 new), Size of downloads: 2.743 KiB it's going to take a bit more than 24hrs for an answer. I don't think this is related to the qemu chroot. The native compile finished, and apart from the obvious qemu: Unsupported syscall: 383 and 391 there is no difference in the log. 100% the same warnings, same files, same segments. However, I added the patchset from libcxxabi git to make it compile when gcc is used, and in this case the stripping of the static files didn't trigger the warnings. But it's not the patchset solving this, since I applied it anyway using CC/X=clang/++ in the logs of my initial posts which produced the warnings. So, do we agree this happens when stripping static-libs build by clang-7, on arm? Created attachment 551634 [details]
build log from app-arch/bzip2
I made a short test to compile app-arch/bzip USE="static-libs" and clang instead of gcc, and results are as expected:
strip: armv7a-unknown-linux-gnueabihf-strip --strip-unneeded -R .comment -R .GCC.command.line -R .note.gnu.gold-version
lib/libbz2.so.1.0.6
usr/bin/bzip2recover
bin/bzip2
usr/lib/libbz2.a
armv7a-unknown-linux-gnueabihf-strip: /var/tmp/portage/app-arch/bzip2-1.0.6-r10/image/usr/lib/stImUpsE/bzlib.o: Failed to find link section for section 11
armv7a-unknown-linux-gnueabihf-strip: /var/tmp/portage/app-arch/bzip2-1.0.6-r10/image/usr/lib/stImUpsE/bzlib.o: Failed to find link section for section 11
I think I found it.
With clang-7 there is a new flag (cflag?) called -faddrsig, and it seems to be broken on arm. Disabling it with -fno-addrsig resolves the warning during striping of static-libs.
How did I found out? I used readelf to fuzz with the misslinked static-libs and found out that they are linked to .llvm_addrsig. Clang-6.0.1 didn't produce static-libs with that value, and upon greping through the clang source code I found llvm_addrsig being linked to -faddrsig/-fno-addrsig.
Disabled it via package.env:
CFLAGS="${CFLAGS} -fno-addrsig"
problem solved.
@mgorny: Is there a way to mitigate the issue for the portage tree? I mean, it's possible to mask package, use flags, but cflags?
Created attachment 551754 [details]
static lib, readelf output and build log of compiler-rt
Adding a sample (static-lib, build log and output from readelf) from compiler-rt
It seems not all instances of addrsig are affected.
Thank you for the detailed analysis. Could you try to create a small test case for it? I mean, using some small code instead of those big libraries. @toolchain, could you look whether it's specifically an issue with LLVM or with binutils? Does not seem to be qemu-specific. Cross-compiling on amd64 yields the same index mismatch:
$ LANG=C USE=static-libs CC="/usr/lib64/llvm/7/bin/clang -target armv7a-unknown-linux-gnueabihf --sysroot=/usr/armv7a-unknown-linux-gnueabihf" CFLAGS="-march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=hard" armv7a-unknown-linux-gnueabihf-emerge -v1 app-arch/bzip2 --quiet-build=n
...
armv7a-unknown-linux-gnueabihf-strip: /tmp/portage-tmpdir/portage/app-arch/bzip2-1.0.6-r10/image/usr/lib/stSjRW4N/blocksort.o: failed to find link section for section 10
armv7a-unknown-linux-gnueabihf-strip: /tmp/portage-tmpdir/portage/app-arch/bzip2-1.0.6-r10/image/usr/lib/stSjRW4N/blocksort.o: failed to find link section for section 10
...
Minimal reproducer:
$ echo 'void f(void){}' > a.c
$ /usr/lib64/llvm/7/bin/clang -target armv7a-unknown-linux-gnueabihf -c a.c -o a.o
$ LANG=C armv7a-unknown-linux-gnueabihf-strip --strip-unneeded -R .comment -R .GCC.command.line -R .note.gnu.gold-version a.o
armv7a-unknown-linux-gnueabihf-strip: st5EpxDK: failed to find link section for section 6
armv7a-unknown-linux-gnueabihf-strip: st5EpxDK: failed to find link section for section 6
Neither readelf not elflint understands the section type of .llvm_addrsig:
$ LANG=C readelf -a a.o
Section Headers:
[Nr] Name Type Addr Off Size ES Flg Lk Inf Al
[ 0] NULL 00000000 000000 000000 00 0 0 0
[ 1] .strtab STRTAB 00000000 000105 000066 00 0 0 1
[ 2] .text PROGBITS 00000000 000034 000004 00 AX 0 0 4
[ 3] .ARM.exidx ARM_EXIDX 00000000 000038 000008 00 AL 2 0 4
[ 4] .rel.ARM.exidx REL 00000000 0000fc 000008 08 9 3 4
[ 5] .comment PROGBITS 00000000 000040 00002e 01 MS 0 0 1
[ 6] .note.GNU-stack PROGBITS 00000000 00006e 000000 00 0 0 1
[ 7] .ARM.attributes ARM_ATTRIBUTES 00000000 00006e 00003c 00 0 0 1
[ 8] .llvm_addrsig LOOS+0xfff4c03 00000000 000104 000001 00 E 9 0 1
[ 9] .symtab SYMTAB 00000000 0000ac 000050 10 1 4 4
$ eu-elflint a.o
section [ 8] '.llvm_addrsig' has unsupported type 1879002115
section [ 8] '.llvm_addrsig' contains invalid processor-specific flag(s) 0x80000000
It does not seem to be arm-specific either:
$ cat a.c
void g(void){}
$ /usr/lib64/llvm/7/bin/clang -c a.c -o b.o
$ LANG=C strip --strip-unneeded -R .comment -R .GCC.command.line -R .note.gnu.gold-version b.o
strip: stNViIFP: failed to find link section for section 5
strip: stNViIFP: failed to find link section for section 5
$ LANG=C file b.o
b.o: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped
https://reviews.llvm.org/D47744 added new LLVM-specific section header type. I don't see support of it in binutils.
Thus warnings of not being able to process this section are expected.
At a glance I don't understand what exactly those sections contain. It seems that data is symbol indices and not relocations. strip probably breaks that section.
@Sergej: thank you for your effort, if I understand your posting correctly it means that this is a bug in binutils then? It's a good question. I'm not sure. I see a few avenues to explore: 1. On binutils side perhaps binutils should not strip binaries with section types it does not understand and leave them as-is (and emit a warning). I'll file a binutils bug and we'll see how it goes. 2. On user's side I suggest trying STRIP=llvm-strip in cross-make.conf to avoid corrupting binaries. I'm not sure about right syntax. Or FEATURES=nostrip as a last-resort hammer. 3. On portage's side portage's strip could also detect presence of such sections and avoid stripping. 4. On clang's side enabling -faddrsig unconditionally might be too optimistic at least in Gentoo. But it also conveys important information about symbols. (In reply to Sergei Trofimovich from comment #24) > 1. On binutils side perhaps binutils should not strip binaries with section > types it does not understand and leave them as-is (and emit a warning). I'll > file a binutils bug and we'll see how it goes. Filed https://sourceware.org/PR23817 for additional input. > > 1. On binutils side perhaps binutils should not strip binaries with section > > types it does not understand and leave them as-is (and emit a warning). I'll > > file a binutils bug and we'll see how it goes. > > Filed https://sourceware.org/PR23817 for additional input. Alan explained the situation in https://sourceware.org/PR23817#c1. I suggest coming back to llvm upstream and deciding there what should be done about incompatibility. I'm going to see if we cam get it disabled upstream by default. Upstream pointed out to me that it happens only when stripping object files (and therefore static archives) but not executables. Maybe we should consider stopping to strip those instead, as suggested in bug 603594. It seems that premature stripping of objects is not really necessary, and is apt to cause issues. Created attachment 553912 [details, diff]
gentoo-fnoaddrsig.patch (for clang)
Alternatively, please test the attached clang patch. It switches the default for Gentoo systems to -fno-addrsig; though I honestly don't like diverging from upstream here.
The patch from #28 is going to make it into portage-2.3.52, right? And the clang patch is against clang-9999 it seems? (In reply to tt_1 from comment #30) > The patch from #28 is going to make it into portage-2.3.52, right? And the > clang patch is against clang-9999 it seems? Yes and yes. However, it should be rather easy to rebase. Could you push out a portage-2.3.51-r1 with 2404ddca9d5db7992bf6853cbde8ca944224560c, in case 2.3.52 isn't ready when llvm/clang-7.0.1 hits the tree? Created attachment 555474 [details, diff]
patch backported to clang-7.0.1
So, I took a fresh stage and used qemu-static for faster results:
portage-9999 doesn't prevent the beforementioned errors/warnings with static linking.
clang-9999 + patch from #29 does work, which means the warnings/errors are gone from the logs. I'll add some examples and logs by tomorrow.
@mgorny: I tried to backport the patch to 7.0.1_rc2, is it correct?
I still don't have a strong opinion on this but with more targets actually disabling -faddrsig, I suppose Gentoo could do the same. However, I need to also fix tests for it. This is now in trunk. I'll probably backport it into -r1 before it goes stable. fixed in clang-8.0.0, thanks for all the effort! :-) |
Created attachment 549484 [details] compressed build logs followup from #667174 this might be related to stripping of statically linked files only I'm going to attach all four log files, and a list of llvm/clang updates from 6.0.1 to 7.0.0 which I ran today. At first I've been suspicious that something went wrong during the upgrade, but as I uninstalled the whole llvm/clang toolchain and recompiled it from scratch, the emerge produced the very same warnings as before.