When compiling a multilib llvm:11 with CPU_FLAGS_X86=xop (XOP added to AMD 15h gen1, removed after 15h gen4; I have a 15h-gen1 CPU), clang passes [-target-feature +xop] as arguments. The troublesome line of code, plus a little context: switch (BitWidth) { case 64: return APInt(BitWidth, llvm::reverseBits<uint64_t>(U.VAL)); // <-- THE FATAL LINE case 32: return APInt(BitWidth, llvm::reverseBits<uint32_t>(U.VAL)); This appears to be inlined to call __builtin_bitreverse64(U.VAL). This produces the following error when compiling APInt.cpp with -fno-lto (with -flto, the error comes during linking; with -flto=thin a double-free error occurs???): libc++abi: terminating with uncaught exception of type std::length_error: vector [...] clang-10: error: clang frontend command failed due to signal (use -v to see invocation) clang version 10.0.1 Target: i386-pc-linux-gnu Thread model: posix InstalledDir: /usr/lib/llvm/10/bin clang-10: note: diagnostic msg: PLEASE submit a bug report to and include the crash backtrace, preprocessed source, and associated run script. clang-10: note: diagnostic msg: ******************** PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT: Preprocessed source(s) and associated run script(s) are located at: clang-10: note: diagnostic msg: /var/tmp/portage/sys-devel/llvm-11.0.0/temp/APInt-074daf.cpp clang-10: note: diagnostic msg: /var/tmp/portage/sys-devel/llvm-11.0.0/temp/APInt-074daf.sh clang-10: note: diagnostic msg: ******************** Here's where I drilled into this and finally found the problem: https://forums.gentoo.org/viewtopic-p-8541220.html#8541220 I'm not sure how clang decides to use XOP. I'm currently attempting the build llvm:11 without 'xop' in my make.conf. I'm thinking I may need to build clang:10 without it, then build llvm:11... Reproducible: Always Portage 3.0.9 (python 3.8.6-final-0, default/linux/amd64/17.1, gcc-9.3.0, glibc-2.31-r6, 5.8.12-gentoo x86_64) ================================================================= System Settings ================================================================= System uname: Linux-5.8.12-gentoo-x86_64-AMD_FX-tm-8150_Eight-Core_Processor-with-glibc2.2.5 KiB Mem: 16409784 total, 5754772 free KiB Swap: 33554428 total, 33553648 free Timestamp of repository gentoo: Sun, 29 Nov 2020 02:15:01 +0000 Head commit of repository gentoo: 8b0958c0a395b432ca7cba7d8a2c7f8136cb99d0 sh bash 5.0_p18 ld GNU ld (Gentoo 2.33.1 p2) 2.33.1 app-shells/bash: 5.0_p18::gentoo dev-java/java-config: 2.3.1::gentoo dev-lang/perl: 5.30.3::gentoo dev-lang/python: 2.7.18-r2::gentoo, 3.6.11-r2::gentoo, 3.7.8-r2::gentoo, 3.8.6::local dev-util/cmake: 3.16.5::gentoo sys-apps/baselayout: 2.7::gentoo sys-apps/openrc: 0.42.1::gentoo sys-apps/sandbox: 2.18::gentoo sys-devel/autoconf: 2.13-r1::gentoo, 2.69-r5::gentoo sys-devel/automake: 1.16.1-r1::gentoo sys-devel/binutils: 2.33.1-r1::gentoo, 2.34-r2::gentoo sys-devel/gcc: 9.3.0-r1::gentoo sys-devel/gcc-config: 2.3.1::gentoo sys-devel/libtool: 2.4.6-r6::gentoo sys-devel/make: 4.2.1-r4::gentoo sys-kernel/linux-headers: 5.4-r1::gentoo (virtual/os-headers) sys-libs/glibc: 2.31-r6::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://rsync.us.gentoo.org/gentoo-portage priority: -1000 sync-rsync-extra-opts: sync-rsync-verify-jobs: 1 sync-rsync-verify-metamanifest: yes sync-rsync-verify-max-age: 24 local location: /usr/local/portage masters: gentoo libressl location: /var/lib/layman/libressl masters: gentoo priority: 50 steam-overlay location: /var/lib/layman/steam-overlay masters: gentoo priority: 50 ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="@FREE" CBUILD="x86_64-pc-linux-gnu" CC="clang" CFLAGS="-march=native -O3 -pipe -flto=thin" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXX="clang++" CXXFLAGS="-march=native -O3 -pipe -flto=thin" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--with-bdeps y" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-march=native -O3 -pipe -flto=thin" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-march=native -O3 -pipe -flto=thin" GENTOO_MIRRORS="http://gentoo.osuosl.org/" LANG="en_US.utf8" LDFLAGS="-Wl,-O3 -fuse-ld=lld -rtlib=compiler-rt" MAKEOPTS="-j9" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="X acl amd64 berkdb bzip2 cli crypt dri fontconfig iconv icu idn ipv6 libedit libglvnd libressl libtirpc lzma maildir mmx multilib ncurses nls nptl openmp pam pcre pic seccomp split-usr sse sse2 ssl tcpd udev unicode vim-syntax wayland xattr xcb zlib zsh-completion zstd" ABI_X86="64" ADA_TARGET="gnat_2018" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" CAMERAS="canon" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx fma4 mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 xop" CURL_SSL="libressl" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="pc" INPUT_DEVICES="evdev keyboard libinput mouse" KERNEL="linux" L10N="de en fr nb nn no" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LLVM_TARGETS="AMDGPU" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-2 php7-3 php7-4" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_7" PYTHON_TARGETS="python3_6 python3_7 python3_8" RUBY_TARGETS="ruby25 ruby26" USERLAND="GNU" VIDEO_CARDS="amdgpu" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Created attachment 677290 [details] APInt-074daf.cpp.xz
Created attachment 677293 [details] APInt-074daf.sh
So it's not lost, let me repeat from the bug description: Here's where I drilled into this and finally found the problem: https://forums.gentoo.org/viewtopic-p-8541220.html#8541220 I'm not sure how clang decides to use XOP. I'm currently attempting the build llvm:11 without 'xop' in my make.conf. I'm thinking I may need to build clang:10 without it, then build llvm:11...
It isn't enough to disable that flag in CPU_FLAGS_X86. -march=native enables it, seemingly regardless of how clang/llvm were built. % clang '-###' -march=native x.cpp 2>&1 | sed 's/"//g' clang version 10.0.1 Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/lib/llvm/10/bin /usr/lib/llvm/10/bin/clang-10 -cc1 -triple x86_64-pc-linux-gnu -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name x.cpp -mrelocation-model static -mthread-model posix -mframe-pointer=all -fmath-errno -fno-rounding-math -masm-verbose -mconstructor-aliases -munwind-tables -target-cpu bdver1 -target-feature +sse2 -target-feature +cx16 -target-feature +sahf -target-feature -tbm -target-feature -avx512ifma -target-feature -sha -target-feature -gfni -target-feature +fma4 -target-feature -vpclmulqdq -target-feature +prfchw -target-feature -bmi2 -target-feature -cldemote -target-feature -fsgsbase -target-feature -ptwrite -target-feature -xsavec -target-feature +popcnt -target-feature +aes -target-feature -avx512bitalg -target-feature -movdiri -target-feature -xsaves -target-feature -avx512er -target-feature -avx512vnni -target-feature -avx512vpopcntdq -target-feature -pconfig -target-feature -clwb -target-feature -avx512f -target-feature -clzero -target-feature -pku -target-feature +mmx -target-feature +lwp -target-feature -rdpid -target-feature +xop -target-feature -rdseed -target-feature -waitpkg -target-feature -movdir64b -target-feature +sse4a -target-feature -avx512bw -target-feature -clflushopt -target-feature +xsave -target-feature -avx512vbmi2 -target-feature +64bit -target-feature -avx512vl -target-feature -invpcid -target-feature -avx512cd -target-feature +avx -target-feature -vaes -target-feature +cx8 -target-feature -fma -target-feature -rtm -target-feature -bmi -target-feature -enqcmd -target-feature -rdrnd -target-feature -mwaitx -target-feature +sse4.1 -target-feature +sse4.2 -target-feature -avx2 -target-feature +fxsr -target-feature -wbnoinvd -target-feature +sse -target-feature +lzcnt -target-feature +pclmul -target-feature -prefetchwt1 -target-feature -f16c -target-feature +ssse3 -target-feature -sgx -target-feature -shstk -target-feature +cmov -target-feature -avx512vbmi -target-feature -avx512bf16 -target-feature -movbe -target-feature -xsaveopt -target-feature -avx512dq -target-feature -adx -target-feature -avx512pf -target-feature +sse3 -dwarf-column-info -fno-split-dwarf-inlining -debugger-tuning=gdb -resource-dir /usr/lib/llvm/10/bin/../../../../lib/clang/10.0.1 -internal-isystem /usr/include/c++/v1 -internal-isystem /usr/local/include -internal-isystem /usr/lib/llvm/10/bin/../../../../lib/clang/10.0.1/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -fdebug-compilation-dir /root -ferror-limit 19 -fmessage-length 0 -fgnuc-version=4.2.1 -fobjc-runtime=gcc -fcxx-exceptions -fexceptions -fdiagnostics-show-option -o /tmp/x-7c892e.o -x c++ x.cpp It looks like gcc/clang support the option -mno-xop. Including that causes the corresponding -target-feature argument to drop entirely. So I think I can just stuff that into my CFLAGS and everything will work.
I can finally confirm that adding -mno-xop to my CFLAGS (after -march=native, I'm not sure if order matters) allows multilib LLVM 11 to compile on my AMD 15h CPU. If I had to decide the best place to put this hack for all Gentoo users, I'd say it'd be to add -mno-xop in an eclass, since it appears to be a bug in clang (otherwise, what's with the backtrace). But I know that's obtrusive. Maybe instead have the llvm build abort immediately if it detects XOP is enabled with the provided CFLAGS and tells the user to disable it in their make.conf? I'm going to try to file an upstream bug, but I'm not sure what the turnaround time for creating an account with bugs.llvm.org is. Self service account creation is disabled.
Also, here's a minimal test-case: // clang -m32 -march=native -c test.cpp // clang -m32 -march=native -mno-xop -c test.cpp // test.cpp: int main(int argc, char **argv) { return __builtin_bitreverse64(argc); } If the first command fails with 'terminating with uncaught exception of type std::length_error: vector' while the second succeeds, XOP code generation is broken.
(Correction: the first command needs -mxop if you aren't running an amd 15h CPU.)
Filed upstream: https://bugs.llvm.org/show_bug.cgi?id=48451
It looks like it may be fixed in LLVM 12, which isn't finalized yet: https://reviews.llvm.org/rG57c0c4a27575840ae0a48eb9f8455a5ed087c857 https://github.com/llvm/llvm-project/commit/57c0c4a27575840ae0a48eb9f8455a5ed087c857 However, the -mno-xop flag would still be necessary to build LLVM/clang 12 the first time.
upstream bug report says the fix has been backported to llvm-11.0.1?
(In reply to tt_1 from comment #10) > upstream bug report says the fix has been backported to llvm-11.0.1? It looks like it is. The commit (referenced in the upstream bug) is shown on the first page of 11.0.1's commits on GitHub. https://github.com/llvm/llvm-project/commits/llvmorg-11.0.1 https://github.com/llvm/llvm-project/commit/ca8de9ad8895ab1368135f6fc63f29fe92b75c76 I guess that means I can probably get rid of my -mno-xop flag since I'm on 11.1 now...
