I'm getting a bootstrap comparison failure for =sys-devel/gcc-12.2.1_p20230121-r1. This has occurred for multiple attempts at compilation. Logs to be attached below. emerge -pqv '=sys-devel/gcc-12.2.1_p20230121-r1::gentoo' [ebuild NS ] sys-devel/gcc-12.2.1_p20230121-r1 [11.3.0] USE="cet* (cxx) (defraphite (-ieee-long-double) -jit (-libssp) -lto -objc -objc++ -objc-gc (-pch) - emerge --info '=sys-devel/gcc-12.2.1_p20230121-r1::gentoo' Portage 3.0.44 (python 3.10.10-final-0, default/linux/amd64/17.1/hardened, gcc-11.3.0, glibc-2.36-r7, 5.18.19-gentoo-r1 x86_64) ================================================================= System Settings ================================================================= System uname: Linux-5.18.19-gentoo-r1-x86_64-12th_Gen_Intel-R-_Core-TM-_i7-1280P-with-glibc2.36 KiB Mem: 65548932 total, 28934200 free KiB Swap: 75497468 total, 75497468 free Timestamp of repository gentoo: Sun, 16 Apr 2023 21:30:01 +0000 Head commit of repository gentoo: 7631dd545b729a264af76aab1b35940e838430b5 sh bash 5.1_p16-r2 ld GNU ld (Gentoo 2.38 p4) 2.38 ccache version 4.7.4 [disabled] app-misc/pax-utils: 1.3.5::gentoo app-shells/bash: 5.1_p16-r2::gentoo dev-java/java-config: 2.3.1::gentoo dev-lang/perl: 5.36.0-r2::gentoo dev-lang/python: 2.7.18_p16-r1::gentoo, 3.10.10_p3::gentoo, 3.11.2_p2::gentoo dev-lang/rust: 1.66.1::gentoo dev-util/ccache: 4.7.4::gentoo dev-util/cmake: 3.25.3::gentoo dev-util/meson: 1.0.1::gentoo sys-apps/baselayout: 2.13-r1::gentoo sys-apps/openrc: 0.46::gentoo sys-apps/sandbox: 2.29::gentoo sys-devel/autoconf: 2.13-r7::gentoo, 2.71-r5::gentoo sys-devel/automake: 1.16.5::gentoo sys-devel/binutils: 2.38-r2::gentoo, 2.39-r4::gentoo sys-devel/binutils-config: 5.5::gentoo sys-devel/clang: 14.0.6-r1::gentoo, 15.0.7-r1::gentoo sys-devel/gcc: 11.3.0::gentoo sys-devel/gcc-config: 2.10::gentoo sys-devel/libtool: 2.4.7-r1::gentoo sys-devel/lld: 14.0.6-r1::gentoo, 15.0.7::gentoo sys-devel/llvm: 14.0.6-r2::gentoo, 15.0.7::gentoo sys-devel/make: 4.3::gentoo sys-kernel/linux-headers: 6.1::gentoo (virtual/os-headers) sys-libs/glibc: 2.36-r7::gentoo Repositories: gentoo location: /var/db/repos/gentoo sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 volatile: True sync-rsync-verify-metamanifest: yes sync-rsync-verify-max-age: 24 sync-rsync-extra-opts: sync-rsync-verify-jobs: 1 ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -march=native" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /etc/stunnel/stunnel.conf /usr/share/config /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-O2 -pipe -march=native" DISTDIR="/var/cache/distfiles" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" FCFLAGS="-O2 -pipe -march=native" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe -march=native" GENTOO_MIRRORS="ftp://mirrors.rit.edu/gentoo/ ftp://gentoo.cites.uiuc.edu/pub/gentoo/" LANG="en_US.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LEX="flex" LINGUAS="en nb" MAKEOPTS="-j20" PKGDIR="/var/cache/binpkgs" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" SHELL="/bin/bash" USE="a52 aac acl acpi alsa amd64 apm bash-completion bluetooth bzip2 cdda cddb cet cli crypt css cxx dbus djvu dri dvd dvdr elogind eselect-ldso ffmpeg flac fortran gamepad gcj gdbm gif gnuplot gnutls hardened hdf5 heif iconv ipv6 jpeg jpeg2k libglvnd libtirpc mmx mozilla mp3 mp4 mpeg mpi mplayer multilib ncurses nls nptl ocaml ocamlopt offensive ogg opengl openmp pam pcre pdf pie png policykit pulseaudio readline seccomp split-usr sse sse2 ssl ssp svg test-rust theora threads tiff truetype unicode usb vaapi vim-syntax vorbus x264 xattr xtpax xv zlib" ABI_X86="64" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="synaptics evdev" KERNEL="linux" L10N="en nb no" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-4 php8-0" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_10" PYTHON_TARGETS="python3_10 python3_11" RUBY_TARGETS="ruby30" USERLAND="GNU" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS Reproducible: Always
Created attachment 860217 [details] gcc build logs
Created attachment 860218 [details] cfgcleanup stage2
Created attachment 860219 [details] cfgcleanup stage3
Oh lord. That's a very large list of files which differ. Let's start with: gcc/crtprec32.o differs gcc/crtprec64.o differs gcc/crtprec80.o differs gcc/crtfastmath.o differs gcc/32/crtprec32.o differs gcc/32/crtprec64.o differs Can you make a tarball with the stage2 version of all of those, and the stage3 ones in a separate dir? Same tarball ideally.
Created attachment 860222 [details] Stage 2 object files Subset of the stage 2 object files due to size
Created attachment 860223 [details] stage 2 object files Subset of the stage 2 object files due to size
Created attachment 860224 [details] stage 3 object files Subset of the stage 3 object files due to size
In case it helps, gcc-12.2.1_p20230304 also failed to emerge with a bootstrap comparison failure. However, an upgrade to 11.3.1_p20230120-r1 from 11.3.0 worked just fine. The upgrade to 11.3.1 also added the cet use flag, which was not present before (this flag was also new to my system for the 12.2.1_p20230121-r1 build.) Compilation of 12.2.1_p20230121-r1 failed again with the same bootstrap comparison failure when compiling with 11.3.1_p20230120-r1 rather than 11.3.0.
seems that between the stages the detected value for --param=l1-cache-size=32 changed
+1 similar issue here with gcc-12.2.1_p20230304 The list of diffs is really long on my side, too.
I get the same with gcc 13.1.0-r1.
I think we really need a new bug per instance of this because it could easily be wildly different causes.
(In reply to Arsen Arsenović from comment #9) > seems that between the stages the detected value for > --param=l1-cache-size=32 changed Seems like this was a good tip. When I compiled gcc 13.1.0-r1 with -march=alderlake instead of my usual -march=native, the comparison succeeded. So it seems like this is the issue, for me at least.
(In reply to Mads from comment #13) > When I compiled gcc 13.1.0-r1 with -march=alderlake instead of my usual > -march=native, the comparison succeeded. So it seems like this is the issue, > for me at least. I was able to get it working with this tip. Thank you!
Also ran into this issue with sys-devel/gcc-12.3.1_p20230526. This is on a system with an Intel Core i7-1260P. I installed and ran app-misc/resolve-march-native, which gave me: -march=alderlake -mabm -mno-cldemote -mno-kl -mno-pconfig -mno-sgx -mno-widekl -mshstk --param=l1-cache-line-size=64 --param=l1-cache-size=48 --param=l2-cache-size=18432 Replacing -march=native with that list of flags makes the compilation succeed, and should be equivalent as far as I understand?
i can confirm that using the flags from app-misc/resolve-march-native resolves this issue for me both with sys-devel/gcc-12.3.1_p20230825 and sys-devel/gcc-13.2.1_p20230826
As a quick note, this issue continues to persist, for me, on gcc-12.3.1_p20230526
(In reply to Mads from comment #13) > When I compiled gcc 13.1.0-r1 with -march=alderlake instead of my usual > -march=native, the comparison succeeded. So it seems like this is the issue, > for me at least. This tip helped me too, thanks. CPU: i9-13900HX, gcc: 13.2.1_p20231014, `-march=alderlake -O3 -pipe`.
Could someone with a hybrid CPU show me: * cat /proc/cpuinfo * lscpu -p Thanks
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=4fba38fc35fe2966574dd6bfd68ee82cd354976c commit 4fba38fc35fe2966574dd6bfd68ee82cd354976c Author: Sam James <sam@gentoo.org> AuthorDate: 2024-03-23 14:16:38 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2024-03-23 14:48:47 +0000 toolchain.eclass: add workaround for hybrid CPUs Hybrid/big.little/PE CPUs may report an inconsistent cache size across cores which can cause GCC's bootstrapping to fail its self-comparison. When CBUILD is amd64 or x86 and -march=native is in CFLAGS, iterate over all cores and record l1-cache-size. If any differ, use the first one we found. Bug: https://gcc.gnu.org/PR111768 Closes: https://bugs.gentoo.org/904426 Closes: https://bugs.gentoo.org/908523 Closes: https://bugs.gentoo.org/915389 Signed-off-by: Sam James <sam@gentoo.org> eclass/toolchain.eclass | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
Thank you to everybody for helping
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=f64da62e50ad607de3a95b6a1d0d2ed8ba8bd268 commit f64da62e50ad607de3a95b6a1d0d2ed8ba8bd268 Author: Sam James <sam@gentoo.org> AuthorDate: 2024-03-24 14:05:00 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2024-03-24 14:05:31 +0000 toolchain.eclass: improve hybrid check Thanks to stikonas for debugging on IRC. Bug: https://bugs.gentoo.org/904426 Bug: https://bugs.gentoo.org/908523 Bug: https://bugs.gentoo.org/915389 Signed-off-by: Sam James <sam@gentoo.org> eclass/toolchain.eclass | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=e897f91e8e81b8168e7045db2f6d8dd9ebdb9ddf commit e897f91e8e81b8168e7045db2f6d8dd9ebdb9ddf Author: Sam James <sam@gentoo.org> AuthorDate: 2024-03-24 17:19:46 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2024-03-24 17:47:08 +0000 toolchain.eclass: abort if hybrid CPU detected w/ -march=native Unfortunately, the previous approach can't work. --param doesn't fully wipe out the previous value added by -march=native, so we still get a failed comparison. Users hitting this should install app-misc/resolve-march-native, run resolve-march-native, and use that in their *FLAGS instead of -march=native - at least for sys-devel/gcc via package.env, if not in make.conf. Therefore, our only real option is to just abort when we detect a problematic situation and tell users what to do. The only other idea I had was to try taskset in src_compile which feels super brittle and not sure it'd even work at all. Thanks to Andrei for testing and debugging with us on IRC & the bug. Bug: https://bugs.gentoo.org/904426 Bug: https://bugs.gentoo.org/908523 Bug: https://bugs.gentoo.org/915389 Bug: https://bugs.gentoo.org/927688 Thanks-to: Andrei Liavonchykau <andreil499@gmail.com> Signed-off-by: Sam James <sam@gentoo.org> eclass/toolchain.eclass | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
*** Bug 927930 has been marked as a duplicate of this bug. ***
(In reply to Larry the Git Cow from comment #23) > The bug has been referenced in the following commit(s): > > https://gitweb.gentoo.org/repo/gentoo.git/commit/ > ?id=e897f91e8e81b8168e7045db2f6d8dd9ebdb9ddf > > commit e897f91e8e81b8168e7045db2f6d8dd9ebdb9ddf > Author: Sam James <sam@gentoo.org> > AuthorDate: 2024-03-24 17:19:46 +0000 > Commit: Sam James <sam@gentoo.org> > CommitDate: 2024-03-24 17:47:08 +0000 > > toolchain.eclass: abort if hybrid CPU detected w/ -march=native > > Unfortunately, the previous approach can't work. --param doesn't fully > wipe out the previous value added by -march=native, so we still get a > failed > comparison. > > Users hitting this should install app-misc/resolve-march-native, run > resolve-march-native, and use that in their *FLAGS instead of > -march=native - > at least for sys-devel/gcc via package.env, if not in make.conf. > > Therefore, our only real option is to just abort when we detect a > problematic > situation and tell users what to do. > > The only other idea I had was to try taskset in src_compile which feels > super > brittle and not sure it'd even work at all. > > Thanks to Andrei for testing and debugging with us on IRC & the bug. > > Bug: https://bugs.gentoo.org/904426 > Bug: https://bugs.gentoo.org/908523 > Bug: https://bugs.gentoo.org/915389 > Bug: https://bugs.gentoo.org/927688 > Thanks-to: Andrei Liavonchykau <andreil499@gmail.com> > Signed-off-by: Sam James <sam@gentoo.org> > > eclass/toolchain.eclass | 11 +++++++++-- > 1 file changed, 9 insertions(+), 2 deletions(-) Hello, I wanted to notice that the check should also look for CXXFLAGS values and die. Currently, it's possible to update CFLAGS to force anderlake, pass the check, but still get bootstrap comparison failures due to CXXFLAGS still using -march=native. Thanks!
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=d0d87a54bb3bb7b077c75b44ff419177fc03ebb8 commit d0d87a54bb3bb7b077c75b44ff419177fc03ebb8 Author: Sam James <sam@gentoo.org> AuthorDate: 2024-09-30 01:22:34 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2024-09-30 01:51:43 +0000 toolchain.eclass: check CXXFLAGS too in -march=native sanity check Bug: https://bugs.gentoo.org/904426 Thanks-to: Pacho Ramos <pacho@gentoo.org> Signed-off-by: Sam James <sam@gentoo.org> eclass/toolchain.eclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=d6788adfff00dacda8c3f1a19d806dad5ba80e89 commit d6788adfff00dacda8c3f1a19d806dad5ba80e89 Author: Sam James <sam@gentoo.org> AuthorDate: 2025-01-19 01:16:12 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2025-01-19 01:16:12 +0000 toolchain.eclass: add TODO for collecting differing bootstrap objects Add a TODO for bootstrap comparison failures, where we should include the stage2 & stage3 differing objects to avoid having to ask reporters to manually collect... Bug: https://bugs.gentoo.org/904426 Bug: https://bugs.gentoo.org/920719 Bug: https://bugs.gentoo.org/932789 Bug: https://bugs.gentoo.org/947752 Signed-off-by: Sam James <sam@gentoo.org> eclass/toolchain.eclass | 2 ++ 1 file changed, 2 insertions(+)
