emerging dev-lang/python-3.8.0 results in this error: --- checking for --with-libc=STRING... set LIBC="" checking for x64 gcc inline assembler... yes checking whether float word ordering is bigendian... unknown configure: error: Unknown float word ordering. You need to manually preset ax_cv_c_float_words_bigendian=no (or yes) according to your system. --- Reproducible: Always # emerge --info Portage 2.3.79 (python 3.7.5-final-0, default/linux/amd64/17.1/systemd, gcc-9.2.0, glibc-2.30-r2, 5.4.0-rc6 x86_64) ================================================================= System uname: Linux-5.4.0-rc6-x86_64-Intel-R-_Core-TM-_i5-6500_CPU_@_3.20GHz-with-gentoo-2.6 KiB Mem: 16141608 total, 2097108 free KiB Swap: 12337144 total, 12014328 free Timestamp of repository gentoo: Wed, 13 Nov 2019 13:00:01 +0000 Head commit of repository gentoo: e6afd290f01cfdb6233316c85b4a899a8a8890cc sh bash 5.0_p11 ld GNU ld (Gentoo 2.33.1 p1) 2.33.1 ccache version 3.7.5 [enabled] app-shells/bash: 5.0_p11::gentoo dev-java/java-config: 2.2.0-r4::gentoo dev-lang/perl: 5.30.0::gentoo dev-lang/python: 2.7.17::gentoo, 3.7.5-r1::gentoo dev-util/ccache: 3.7.5::gentoo dev-util/cmake: 3.15.5::gentoo dev-util/pkgconfig: 0.29.2::gentoo sys-apps/baselayout: 2.6-r1::gentoo sys-apps/sandbox: 2.18::gentoo sys-devel/autoconf: 2.13-r1::gentoo, 2.69-r4::gentoo sys-devel/automake: 1.16.1-r2::gentoo sys-devel/binutils: 2.33.1::gentoo sys-devel/gcc: 9.2.0-r2::gentoo sys-devel/gcc-config: 2.1::gentoo sys-devel/libtool: 2.4.6-r5::gentoo sys-devel/make: 4.2.1-r4::gentoo sys-kernel/linux-headers: 5.3-r1::gentoo (virtual/os-headers) sys-libs/glibc: 2.30-r2::gentoo Repositories: gentoo location: /var/cache/portage sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 sync-rsync-extra-opts: sync-rsync-verify-max-age: 24 sync-rsync-verify-jobs: 1 sync-rsync-verify-metamanifest: yes x-portage location: /usr/local/portage masters: gentoo priority: 0 ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="* -@EULA Intel-SDP Oracle-BCLA-JavaSE Oracle-EADLA AMD-GPU-PRO-EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -march=native -pipe -fuse-linker-plugin -flto -ftree-vectorize -ftree-slp-vectorize -falign-functions=32 -fgraphite-identity -floop-nest-optimize" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php7.3/ext-active/ /etc/php/cgi-php7.3/ext-active/ /etc/php/cli-php7.3/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -march=native -pipe -fuse-linker-plugin -flto -ftree-vectorize -ftree-slp-vectorize -falign-functions=32 -fgraphite-identity -floop-nest-optimize" DISTDIR="/var/cache/portage/distfiles" EMERGE_DEFAULT_OPTS="--jobs=4 --load-average=4.5" ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs candy ccache cgroup clean-logs collision-protect config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync mount-sandbox multilib-strict network-sandbox news parallel-fetch parallel-install pid-sandbox preserve-libs protect-owned sandbox sfperms splitdebug strict strict-keepdir unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed -flto -Wl,--hash-style=gnu" LINGUAS="en" MAKEOPTS="-j5" PKGDIR="/var/cache/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="X a52 aac aacs acl acpi aiglx alac alsa amd64 amr amrnb amrwb animation apache2 apng argon2 artworkextra asf aspnet2 asyncns authdaemond avahi avi bash-completion bcmath berkdb bitmap-fonts bluetooth bluray bmp bonjour brotli btrfs bzip2 cairo caps cdda cddb cdparanoia cdr celt chm clang cli crypt cryptsetup css cups curl cxx daap dbus device-mapper dga dhclient dirac divx4linux dlloader dns dnssec dovecot-sasl dpms dri dri3 dts dv dvb dvd dvdr dvdread egl enca encode exif ext4 faac fam fame fat fbcondecor fbsplash ffmpeg fftw firefox flac fontconfig foomaticdb fortran freetype ftp fuse gaim gd gdbm geoip gif gimpprint glamor gles gles1 gles2 glib glibc-omitfp glitz glut gost gphoto2 graphite gsm gstreamer gstreamer10 gtk gtk2 gudev hardenedphp hbci howl howl-compat http2 https hwdb iconv icu idn imagemagick imap imlib innodb ipv6 java java5 javafx jbig jce joystick jpeg jpeg2k junit kdbus kdehiddenvisibility kvm lapack lcms libfame libkms libnotify librtmp libtirpc libusb live llvm lm_sensors lvm lz4 lzma lzo mad maildir mdadm mdnsresponder-compat metalink mjpeg mmap mms mng motif mozilla mozsvg mp3 mpeg mpeg2 mplayer mpx multilib musepack mysql mythtv natspec ncurses nemesi network networkmanager nls nptl nptlonly ntfs ocl-icd offensive ofx ogg oggvorbis openal opencl openexr opengl openmp openvg opus pam pango pcre pcre16 pcre32 pcsc-lite pdf pdflib perl pgo phonon pic pie png pnp policykit ppd ppds pulseaudio python quicktime quvi rar readline rtc rtmp schroedinger screensaver sctp sdl seccomp session sftp sharedmem shout smp sna sni sox speex spell split-usr sqlite srt ssh ssl steamruntime suspend2 svg systemd theora threads tiff tls transcode truetype truetype-fonts type-fonts udev udisks unicode unzip upnp ups usb user-session uxa v4l2 vaapi vcd videos virgl vorbis vpx vulkan wayland webdav webp webservices wifi win32codecs wmf wv x264 x265 xanim xattr xcb xcomposite xdg xinerama xml xml2 xpm xprint xrandr xv xvfb xvid xvmc zeroconf zip zlib zstd" ABI_X86="64" ADA_TARGET="gnat_2018" ALSA_CARDS="hda-intel" APACHE2_MODULES="actions alias auth_basic authn_core authz_core authn_socache cache_disk authn_alias authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex brotli cache dav dav_fs dav_lock dconf deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias auth_digest proxy proxy_ajp proxy_connect proxy_http proxy_fcgi reqtimeout cgi cgid unixd socache_shmcb proxy_html xml2enc" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" CURL_SSL="openssl" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64 pc" INPUT_DEVICES="evdev joystick keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LIRC_DEVICES="userspace" NETBEANS_MODULES="apisupport cnd groovy gsf harness ide identity j2ee java mobility nb php profiler soa visualweb webcommon websvccommon xml" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-3" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_7" PYTHON_TARGETS="python2_7 python3_7 python3_8" QEMU_SOFTMMU_TARGETS="arm i386 x86_64" QEMU_USER_TARGETS="arm i386 x86_64" RUBY_TARGETS="ruby25" USERLAND="GNU" VIDEO_CARDS="intel i965 amdgpu radeon radeonsi" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Created attachment 595964 [details] config.log
Created attachment 595966 [details] build.log
My first guess is that your CFLAGS are triggering this. Please start with a simpler set of flags, (eg. -O2 -pipe) and work up from there.
On my system, I had to remove -lto. O3 and graphite seems to be fine.
Confirmed, disabling lto works around this issue.
The relevant autoconf macro is running grep on an object file, which seems like a recipe for disaster. https://github.com/python/cpython/blob/v3.8.0/m4/ax_c_float_words_bigendian.m4#L52
I believe appending -ffat-lto-objects remedies this bug by adding the fat that -flto optimizes out.
Created attachment 596238 [details] python-3.8.0 fat log
media-libs/x264 uses a similar approach and failes with LTO. Adding -ffat-lto-objects solves the issue as stated in comment #7 The thing is that .o files generated with LTO do not contain the normal program code and therefor grep and strings do not work as intended, using fat-lto-objects leads to normal program code in .o files.
Python has a configure option --with-lto. This sets the following flags for compiling with lto: LTOFLAGS="-flto -fuse-linker-plugin -ffat-lto-objects -flto-partition=none"
(In reply to Johannes Hirte from comment #10) Thanks for pointing that out. I'm not sure how best to respect user CFLAGS/LDFLAGS in the case of -flto. Here are a few options: 1. Unconditionally remove -flto from CFLAGS. Add a 'lto' USE flag to dev-lang/python to toggle the --with-lto configure option. 2. Inspect the user CFLAGS, and add -ffat-lto-objects if -flto is present. 2. Inspect the user CFLAGS, and warn if -flto is present but -ffat-lto-objects is missing.
4. Inspect CFLAGS, and pass --with-lto if -lto is present.
(In reply to Michał Górny from comment #12) > 4. Inspect CFLAGS, and pass --with-lto if -lto is present. I generally prefer not to mess with configure flags based on CFLAGS, but that seems reasonable in this case.
Just ran into this on ARM. Are there any changes pending for python 3.8 with LTO?
Option 4 doesn't work; the configure script does not append -ffat-lto-objects to CFLAGS until after the float endian check.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=337c1cbdae45b58405e96f06077fdd29203e0ad0 commit 337c1cbdae45b58405e96f06077fdd29203e0ad0 Author: Mike Gilbert <floppym@gentoo.org> AuthorDate: 2020-01-05 23:18:45 +0000 Commit: Mike Gilbert <floppym@gentoo.org> CommitDate: 2020-01-05 23:19:48 +0000 dev-lang/python: fix build with -flto in CFLAGS Bug: https://bugs.gentoo.org/700012 Package-Manager: Portage-2.3.84_p2, Repoman-2.3.20_p24 Signed-off-by: Mike Gilbert <floppym@gentoo.org> dev-lang/python/python-3.8.1.ebuild | 7 ++++++- dev-lang/python/python-3.9.0_alpha2.ebuild | 5 +++++ 2 files changed, 11 insertions(+), 1 deletion(-)
(In reply to Larry the Git Cow from comment #16) > The bug has been referenced in the following commit(s): > > https://gitweb.gentoo.org/repo/gentoo.git/commit/ > ?id=337c1cbdae45b58405e96f06077fdd29203e0ad0 > > commit 337c1cbdae45b58405e96f06077fdd29203e0ad0 > Author: Mike Gilbert <floppym@gentoo.org> > AuthorDate: 2020-01-05 23:18:45 +0000 > Commit: Mike Gilbert <floppym@gentoo.org> > CommitDate: 2020-01-05 23:19:48 +0000 > > dev-lang/python: fix build with -flto in CFLAGS > > Bug: https://bugs.gentoo.org/700012 > Package-Manager: Portage-2.3.84_p2, Repoman-2.3.20_p24 > Signed-off-by: Mike Gilbert <floppym@gentoo.org> > > dev-lang/python/python-3.8.1.ebuild | 7 ++++++- > dev-lang/python/python-3.9.0_alpha2.ebuild | 5 +++++ > 2 files changed, 11 insertions(+), 1 deletion(-) This only catches '-flto'. If someone is specifying the number of parallel processes like '-flto=8' it doesn't work.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=bc22f76af021e82362cb4be63e8f9ba452ac4906 commit bc22f76af021e82362cb4be63e8f9ba452ac4906 Author: Mike Gilbert <floppym@gentoo.org> AuthorDate: 2020-01-13 17:28:29 +0000 Commit: Mike Gilbert <floppym@gentoo.org> CommitDate: 2020-01-13 17:28:29 +0000 dev-lang/python: also match '-flto=*' Bug: https://bugs.gentoo.org/700012 Package-Manager: Portage-2.3.84_p2, Repoman-2.3.20_p24 Signed-off-by: Mike Gilbert <floppym@gentoo.org> dev-lang/python/python-3.8.1.ebuild | 2 +- dev-lang/python/python-3.9.0_alpha2.ebuild | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
I just hit this when using Clang with -flto=thin. Any chance you can catch that as well?
(In reply to Calum Knight from comment #19) > I just hit this when using Clang with -flto=thin. Any chance you can catch > that as well? Nevermind, It turns out that, unsurprisingly, you can't have "fat" thin LTO objects so this can't be fixed the same way.
I just submitted PGO+LTO patches against bug 61541 I suggest closing this bug in favor of that one.
I agree that -flto=thin should be caught. But I really wish Python had a better test. LLD with thin LTO got a speedup of roughly 36 when linking sys-libs/llvm, which is quite significant and I really like keeping thin LTO around.
Alright so I've just updated clang to 11, which may be relevant, but -ffat-lto-objects is not supported. % clang -march=native -O3 -pipe -flto -ffat-lto-objects blank.c -c clang-11: warning: optimization flag '-ffat-lto-objects' is not supported [-Wignored-optimization-argument] Personally, I was confused why the ebuild wasn't appending the argument even though I had a plain old '-flto' argument. So I guess I'm back to hacking up ebuilds.
reporting that this is still a thing in version 3.9.2. To WA this, I added the performance and lto options as described here: https://docs.python.org/3/using/configure.html#performance-options
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=074a78fe03e77fe3337dae773c3d46d0af47eb95 commit 074a78fe03e77fe3337dae773c3d46d0af47eb95 Author: Sam James <sam@gentoo.org> AuthorDate: 2021-11-17 10:49:17 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2021-11-17 10:49:17 +0000 dev-lang/python: backport PGO, LTO to 3.9.x and 3.10.x Bug: https://bugs.gentoo.org/615412 Bug: https://bugs.gentoo.org/700012 Signed-off-by: Sam James <sam@gentoo.org> dev-lang/python/python-3.10.0_p1.ebuild | 18 +++++++++++++++++- dev-lang/python/python-3.9.9.ebuild | 18 +++++++++++++++++- 2 files changed, 34 insertions(+), 2 deletions(-) https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=b85750dd2326759e7f102d81e5943bd4e5557a8a commit b85750dd2326759e7f102d81e5943bd4e5557a8a Author: Sam James <sam@gentoo.org> AuthorDate: 2021-11-17 10:06:12 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2021-11-17 10:46:28 +0000 dev-lang/python: add LTO Bug: https://bugs.gentoo.org/615412 Bug: https://bugs.gentoo.org/700012 Signed-off-by: Sam James <sam@gentoo.org> Closes: https://github.com/gentoo/gentoo/pull/22853 Signed-off-by: Sam James <sam@gentoo.org> dev-lang/python/metadata.xml | 1 + dev-lang/python/python-3.11.0_alpha2.ebuild | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-)
I'm going to say this is fixed given that we have USE=lto for enabling LTO 'correctly' now.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ff65f5c822417e8347598fcb75a1e575ba35bf58 commit ff65f5c822417e8347598fcb75a1e575ba35bf58 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-08-28 22:40:54 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-08-28 22:40:54 +0000 dev-lang/python: avoid confused LTO configure test w/ cross CBUILD Python No real need to take care here as it gets thrown away, so just bang in -fno-lto. Bug: https://bugs.gentoo.org/700012 Signed-off-by: Sam James <sam@gentoo.org> dev-lang/python/python-3.10.13.ebuild | 8 ++++++-- dev-lang/python/python-3.11.5.ebuild | 8 ++++++-- dev-lang/python/python-3.12.0_beta4_p2.ebuild | 8 ++++++-- dev-lang/python/python-3.12.0_rc1_p5.ebuild | 8 ++++++-- dev-lang/python/python-3.12.0_rc1_p6.ebuild | 8 ++++++-- dev-lang/python/python-3.8.18.ebuild | 8 ++++++-- dev-lang/python/python-3.9.18.ebuild | 8 ++++++-- 7 files changed, 42 insertions(+), 14 deletions(-)
> I'm going to say this is fixed given that we have USE=lto for enabling LTO 'correctly' now. lto USE flag was dropped in the last commit. https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-lang/python/python-3.12.1_p1.ebuild#n32
(In reply to Boris Staletic from comment #28) > > I'm going to say this is fixed given that we have USE=lto for enabling LTO 'correctly' now. > > lto USE flag was dropped in the last commit. > > https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-lang/python/python-3.12. > 1_p1.ebuild#n32 Yes, that's fine. See the commit message which did it and https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-lang/python/python-3.12.1_p1.ebuild#n288.
Python since 3.11 supports choosing between full and thin lto: https://github.com/python/cpython/blob/3.11/configure#L1796 Maybe this could be added to the ebuilds?
