| Summary: | =sys-kernel/gentoo-sources-4.14.10-r1 various objtool warnings (-march=native) | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Hadrien Lacour <hadrien.lacour> |
| Component: | Current packages | Assignee: | Gentoo Kernel Bug Wranglers and Kernel Maintainers <kernel> |
| Status: | RESOLVED UPSTREAM | ||
| Severity: | normal | CC: | adamcarter3, alexander, jasmin+gentoo, slyfox, stefantalpalaru, tb, tsmksubc, vityokster |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: |
emerge -pqv
emerge --info build log kernel config arch/x86/events/core.o gcc opts gcc target gentoo-sources-5.7.7-Piledriver-debugging.tar.bz2 |
||
Created attachment 512362 [details]
emerge --info
Created attachment 512364 [details]
build log
Created attachment 512366 [details]
kernel config
I also tried using ld.bfd instead of ld.gold. These are likely caused by 17.0 profile, namely gcc[pie]. Perhaps, objtool needs to be tweaked to understand new code pattern for control flow change. Given that it reports so tiny values, like '.text+0x125' I would guess objtool does not understand some PIE-induced relocations. Can you upload single .o file and a post reported warning for it? Created attachment 512368 [details]
arch/x86/events/core.o
Sure:
arch/x86/events/core.o: warning: objtool: x86_pmu_hw_config()+0x13: can't find jump dest instruction at .text+0x15cd
(In reply to Hadrien Lacour from comment #6) > Created attachment 512368 [details] > arch/x86/events/core.o > > Sure: > > arch/x86/events/core.o: warning: objtool: x86_pmu_hw_config()+0x13: can't > find jump dest instruction at .text+0x15cd $ objdump -d -r core.o 00000000000015a8 <x86_pmu_hw_config>: 15a8: 8b 87 f8 00 00 00 mov 0xf8(%rdi),%eax 15ae: a9 00 80 01 00 test $0x18000,%eax 15b3: 0f 85 ad 00 00 00 jne 1666 <x86_pmu_hw_config+0xbe> 15b9: 8f ea 78 10 c0 0f 02 bextr $0x20f,%eax,%eax 15c0: 00 00 15c2: 83 f8 01 cmp $0x1,%eax 15c5: 7e 13 jle 15da <x86_pmu_hw_config+0x32> 15c7: 0f b6 05 00 00 00 00 movzbl 0x0(%rip),%eax # 15ce <x86_pmu_hw_config+0x26> 15ca: R_X86_64_PC32 x86_pmu+0x165 15ce: 83 e0 0f and $0xf,%eax ... 1666: 0f b6 0d 00 00 00 00 movzbl 0x0(%rip),%ecx # 166d <x86_pmu_hw_config+0xc5> 1669: R_X86_64_PC32 x86_pmu+0x16c 166d: 31 d2 xor %edx,%edx ... At least control flow instructions are not special. It looks like objtool believes instruction starts at x86_pmu_hw_config()+0x13 (0x15bb, looks wrong). (Decoded test incorrectly?) Local reproducer:
$ make allyesconfig
$ make arch/x86/events/core.o
$ tools/objtool/objtool check arch/x86/events/core.o
(In reply to Hadrien Lacour from comment #6) > Created attachment 512368 [details] > arch/x86/events/core.o > > Sure: > > arch/x86/events/core.o: warning: objtool: x86_pmu_hw_config()+0x13: can't > find jump dest instruction at .text+0x15cd Can you also dump your output of: $ LANG=C gcc -Q -O2 -march=native --help=optimizers I'd like to replicate generated instructions by compiler locally. Created attachment 512486 [details]
gcc opts
Here, I'm on piledriver.
(In reply to Hadrien Lacour from comment #10) > Created attachment 512486 [details] > gcc opts > > Here, I'm on piledriver. Thanks! Can you also add target ones? $ LANG=C gcc -Q -O2 -march=native --help=target Created attachment 512488 [details]
gcc target
I've added tiny debug entry to get the idea which instruction length is decoded incorrectly:
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -266,6 +266,7 @@ static int decode_instructions(struct objtool_file *file)
&insn->len, &insn->type,
&insn->immediate,
&insn->stack_op);
+ fprintf (stderr, "offset: 0x%lx; len=%u; ret=%i\n", offset, insn->len, ret);
if (ret)
goto err;
'bextr' instruction length is incorrect (reported 2, actual 9):
15b9: 8f ea 78 10 c0 0f 02 bextr $0x20f,%eax,%eax
15c0: 00 00
offset: 0x15b9; len=2; ret=0
'bextr' is likely coming from -march=native expansion: -march=bdver2 -mbmi (but I failed to reproduce it locally on gcc-7.2.0).
How exactly you are specifying -march=native when building kernel?
Do you run genkernel or cr to /usr/src and run 'make CFLAGS="-O2 -march=native -pipe"' there?
Now that you say it, maybe I don't build it with march=native. I only do `sudo make -j8` inside /usr/src/linux; the CFLAGS I mentionned are in my make.conf. (In reply to Hadrien Lacour from comment #14) > Now that you say it, maybe I don't build it with march=native. I only do > `sudo make -j8` inside /usr/src/linux; the CFLAGS I mentionned are in my > make.conf. I think something leaks out to your environment to propagate -march=native to kernel. Let's check what arguments are passed to the compiler in your case: $ sudo rm arch/x86/events/core.o $ sudo make arch/x86/events/core.o V=1 That should force core.o rebuild and will dump raw gcc command. [ I've finally reproduced 'bextr' generation locally. It requires CFLAGS="-march=bdver2 -mtbm". fun fact: "bextr $number,..." is not a BMI encoding (as I initially thought) but TBM encoding. BMI only defines "register,register,register" form of VEX encoding, TBM is "immediate,register,register" form, only AMD implements it currently. Somewhat explains why kernel disassembler misses it as disassembler is based on intel manual. ] Oh, I think I've found it: https://dev.gentoo.org/~mpagano/genpatches/trunk/4.14/5010_enable-additional-cpu-optimizations-for-gcc.patch It's enabled with USE=experimental and enables -march=native for CONFIG_MNATIVE + cflags-$(CONFIG_MNATIVE) += $(call cc-option,-march=native) You have it set, right? Yeah, I should have mentioned it. (In reply to Sergei Trofimovich from comment #16) > Oh, I think I've found it: > > https://dev.gentoo.org/~mpagano/genpatches/trunk/4.14/5010_enable-additional- > cpu-optimizations-for-gcc.patch > > It's enabled with USE=experimental and enables -march=native for > CONFIG_MNATIVE > + cflags-$(CONFIG_MNATIVE) += $(call cc-option,-march=native) > > You have it set, right? I confirm this bug for sys-kernel/gentoo-sources:4.14.15. with "Native optimizations autodetected by GCC" I have same errors. with "Generic-x86-64" I have successfully built kernel. emerge -pv1 sys-kernel/gentoo-sources These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild R ] sys-kernel/gentoo-sources-4.14.15:4.14.15::gentoo USE="experimental symlink -build" 0 KiB Total: 1 package (1 reinstall), Size of downloads: 0 KiB I confirm this bug for sys-kernel/gentoo-sources-4.14.15 and 4.14.14 with segfault (CONFIG_MPILEDRIVER=y, AMD FX-8320E): mm/.tmp_memory.o: warning: objtool: print_bad_pte()+0x9c: return with modified stack frame mm/.tmp_memory.o: warning: objtool: wp_page_copy()+0x3a6: return with modified stack frame mm/.tmp_memory.o: warning: objtool: __follow_pte_pmd.isra.10()+0x296: return with modified stack frame mm/.tmp_memory.o: warning: objtool: _vm_normal_page()+0x2a: return with modified stack frame mm/.tmp_memory.o: warning: objtool: unmap_page_range()+0xf8: return with modified stack frame mm/.tmp_memory.o: warning: objtool: finish_mkwrite_fault()+0xc7: return with modified stack frame mm/.tmp_memory.o: warning: objtool: do_wp_page()+0xd2: return with modified stack frame mm/.tmp_memory.o: warning: objtool: alloc_set_pte()+0x38e: return with modified stack frame /bin/sh: строка 1: 23359 Ошибка сегментирования ./tools/objtool/objtool check "mm/.tmp_memory.o" make[1]: *** [scripts/Makefile.build:321: mm/memory.o] Ошибка 139 make: *** [Makefile:1028: mm] Ошибка 2 With "Generic-x86-64" I have successfully built kernel. FYI, on gentoo-sources 4.15.0, changing from CONFIG_MPILEDRIVER to CONFIG_MBULLDOZER was enough to make the kernel build. Selecting CONFIG_MBULLDOZER instead of CONFIG_PILEDRIVER solves this bug for me and also bug #645774 starting with kernel 4.14.14 solved upstream as comment Can you tell what's the first fixed version? I tried with 4.14.25 and 4.15.9 with CONFIG_MPILEDRIVER to no avail. Reopening until we confirm the issue was fixed/reported upstream. It's a bit tricky as upstream kernel has no mechanism to build kernel with -march=native today. Please look similar: https://bugs.gentoo.org/726660 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95671 https://github.com/graysky2/kernel_gcc_patch/issues/55 (In reply to Maxim Britov from comment #25) > Please look similar: > https://bugs.gentoo.org/726660 > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95671 > https://github.com/graysky2/kernel_gcc_patch/issues/55 Yes, I think we're OK on these kernels or greater: sys-kernel/gentoo-sources-5.7.7 sys-kernel/gentoo-sources-5.4.50 sys-kernel/gentoo-sources-4.19.131 sys-kernel/gentoo-sources-4.14.187 Anyone experience this issue with USE=experimental and one of these versions (or greater) ? Yes, I'm seeing this issue with gentoo-sources-5.7.7 on an FX-8320E. TBM is probably not the only AMD-specific instruction encoding that objtool does not understand. If you still see the bug please extract a few things: - extract .o file that objtool complains about - extract precise warning message - expand -march=native's target fields as https://bugs.gentoo.org/642924#c11 Created attachment 648694 [details]
gentoo-sources-5.7.7-Piledriver-debugging.tar.bz2
"Make" deletes the intermediary object file, so I had to copy and run the gcc command from the output of `make V=1` to recreate it.
(In reply to Ștefan Talpalaru from comment #29) > Created attachment 648694 [details] > gentoo-sources-5.7.7-Piledriver-debugging.tar.bz2 > > "Make" deletes the intermediary object file, so I had to copy and run the > gcc command from the output of `make V=1` to recreate it. The error is: arch/x86/events/amd/ibs.o: warning: objtool: perf_ibs_init()+0x24: can't find jump dest instruction at .text+0x36 0000000000000000 <perf_ibs_init>: 0: e8 00 00 00 00 callq 5 <perf_ibs_init+0x5> 5: 8b 87 e0 00 00 00 mov 0xe0(%rdi),%eax b: 3b 05 00 00 00 00 cmp 0x0(%rip),%eax # 11 <perf_ibs_init+0x11> 11: 74 64 je 77 <perf_ibs_init+0x77> 13: 3b 05 00 00 00 00 cmp 0x0(%rip),%eax # 19 <perf_ibs_init+0x19> 19: 48 c7 c2 00 00 00 00 mov $0x0,%rdx 20: 74 5c je 7e <perf_ibs_init+0x7e> 22: 8f ea 78 10 97 08 01 bextr $0x20f,0x108(%rdi),%edx 29: 00 00 0f 02 00 00 The problematic instruction is 'bextr $0x20f,0x108(%rdi),%edx'. It's the same TBM encoding issue that objconv can't handle. I don't see -mno-tbm in build command: gcc -Wp,-MD,arch/x86/events/amd/.ibs.o.d -nostdinc -isystem /usr/lib/gcc/x86_64-pc-linux-gnu/10.1.0/include -I./arch/x86/include -I./arch/x86/include/generated -I./include -I./arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/kconfig.h -include ./include/linux/compiler_types.h -D__KERNEL__ -Wall -Wundef -Werror=strict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE -Werror=implicit-function-declaration -Werror=implicit-int -Wno-format-security -std=gnu89 -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -m64 -falign-jumps=1 -falign-loops=1 -mno-80387 -mno-fp-ret-in-387 -mpreferred-stack-boundary=3 -mskip-rax-setup -march=native -mno-red-zone -mcmodel=kernel -Wno-sign-compare -fno-asynchronous-unwind-tables -mindirect-branch=thunk-extern -mindirect-branch-register -fno-jump-tables -fno-delete-null-pointer-checks -Wno-frame-address -Wno-format-truncation -Wno-format-overflow -Wno-address-of-packed-member -O3 -fno-allow-store-data-races -Wframe-larger-than=2048 -fstack-protector-strong -Wno-unused-but-set-variable -Wimplicit-fallthrough -Wno-unused-const-variable -fno-var-tracking-assignments -pg -mrecord-mcount -mfentry -DCC_USING_FENTRY -Wdeclaration-after-statement -Wvla -Wno-pointer-sign -Wno-stringop-truncation -Wno-zero-length-bounds -Wno-array-bounds -Wno-stringop-overflow -Wno-restrict -Wno-maybe-uninitialized -fno-strict-overflow -fno-merge-all-constants -fmerge-constants -fno-stack-check -fconserve-stack -Werror=date-time -Werror=incompatible-pointer-types -Werror=designated-init -fmacro-prefix-map=./= -fcf-protection=none -Wno-packed-not-aligned -DKBUILD_MODFILE='"arch/x86/events/amd/ibs"' -DKBUILD_BASENAME='"ibs"' -DKBUILD_MODNAME='"ibs"' -c -o arch/x86/events/amd/ibs.o arch/x86/events/amd/ibs.c ./tools/objtool/objtool orc generate --no-fp --retpoline arch/x86/events/amd/ibs.o Is this still an issue for anyone here? Yes, the problem persists with kernel 5.13.11, binutils 2.36.1-r2 and gcc 11.2.0. (In reply to Ștefan Talpalaru from comment #32) > Yes, the problem persists with kernel 5.13.11, binutils 2.36.1-r2 and gcc > 11.2.0. Can I have your .config for 5.13.11, please? Kernel config generating objtool warnings on a Piledriver CPU, with "CONFIG_MNATIVE_AMD=y": https://gist.github.com/stefantalpalaru/6da79304b2bd832a01450b141b21a050 Enabling CONFIG_MPILEDRIVER, instead of that, lets me build the kernel without any warning. Closing as their is a workaround, to continue to pursue this, you could take it to the cpu opt upstream |
Created attachment 512360 [details] emerge -pqv These started with 4.14.9 (so I'm staying on 4.14.8-r1). Here are the first ones: arch/x86/events/amd/uncore.o: warning: objtool: amd_uncore_cpu_starting()+0x9b: can't find jump dest instruction at .text+0x125 arch/x86/events/core.o: warning: objtool: x86_pmu_hw_config()+0x13: can't find jump dest instruction at .text+0x1495 kernel/fork.o: warning: objtool: account_kernel_stack()+0xd4: sibling call from callable instruction with modified stack frame kernel/fork.o: warning: objtool: sysctl_max_threads()+0xee: stack state mismatch: cfa1=7+104 cfa2=7+96 arch/x86/entry/vdso/vma.o: warning: objtool: vgetcpu_cpu_init()+0x9e: return with modified stack frame arch/x86/events/amd/ibs.o: warning: objtool: perf_ibs_init()+0x6a: stack state mismatch: cfa1=7+8 cfa2=7+0 ... What I tried: - At first, I thought this was caused by ORC replacing frame_unwinder (I use olddefconfig) but reverting it didn't solve anything. - I also tried with CFLAGS="-O2 -march=native -pipe" (I use -O3, usually). So the only thing remaining I can think of is the 17.0 profile.