Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 581602 - distcc wiki page suggest unsafe -mno-* CFLAGS, breaks gimp and others
Summary: distcc wiki page suggest unsafe -mno-* CFLAGS, breaks gimp and others
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: AMD64 Linux
: Normal normal (vote)
Assignee: Gentoo Toolchain Maintainers
URL: https://bugzilla.gnome.org/show_bug.c...
Whiteboard:
Keywords:
: 663986 (view as bug list)
Depends on:
Blocks:
 
Reported: 2016-04-30 00:29 UTC by Robin Bankhead
Modified: 2019-11-24 22:17 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
gimp-2.9.2 build.log (gzipped) (gimp-2.9.2-build.log.gz,34.17 KB, application/x-gzip)
2016-06-04 12:36 UTC, Robin Bankhead
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Robin Bankhead 2016-04-30 00:29:00 UTC
Error text:
gimpoperationnormalmode-sse4.c: In function ‘gimp_operation_normal_mode_process_pixels_sse4’:
/usr/lib/gcc/x86_64-pc-linux-gnu/5.3.0/include/smmintrin.h:191:1: error: inlining failed in call to always_inline ‘_mm_blend_ps’: target specific option mismatch
 _mm_blend_ps (__m128 __X, __m128 __Y, const int __M)
 ^
gimpoperationnormalmode-sse4.c:102:25: error: called from here
               out_pixel = _mm_blend_ps (out_pixel, out_alpha, 0x08);
                         ^
In file included from gimpoperationnormalmode-sse4.c:31:0:
/usr/lib/gcc/x86_64-pc-linux-gnu/5.3.0/include/smmintrin.h:191:1: error: inlining failed in call to always_inline ‘_mm_blend_ps’: target specific option mismatch
 _mm_blend_ps (__m128 __X, __m128 __Y, const int __M)
 ^
gimpoperationnormalmode-sse4.c:102:25: error: called from here
               out_pixel = _mm_blend_ps (out_pixel, out_alpha, 0x08);
                         ^

emerge --info
Portage 2.2.28 (python 3.4.3-final-0, default/linux/amd64/13.0/desktop/plasma, gcc-5.3.0, glibc-2.23-r2, 4.5.0-gentoo x86_64)
=================================================================
System uname: Linux-4.5.0-gentoo-x86_64-AMD_E2-1800_APU_with_Radeon-tm-_HD_Graphics-with-gentoo-2.2
KiB Mem:     3622972 total,    164820 free
KiB Swap:    9047036 total,   9043468 free
Timestamp of repository gentoo: Fri, 29 Apr 2016 19:45:02 +0000
sh bash 4.3_p42-r2
ld GNU ld (Gentoo 2.25.1 p1.1) 2.25.1
distcc 3.2rc1 x86_64-pc-linux-gnu [enabled]
ccache version 3.2.5 [enabled]
app-shells/bash:          4.3_p42-r2::gentoo
dev-java/java-config:     2.2.0-r3::gentoo
dev-lang/perl:            5.22.1::gentoo
dev-lang/python:          2.7.11-r2::gentoo, 3.4.3-r7::gentoo
dev-util/ccache:          3.2.5::gentoo
dev-util/cmake:           3.5.2::gentoo
dev-util/pkgconfig:       0.29.1::gentoo
sys-apps/baselayout:      2.2::gentoo
sys-apps/openrc:          0.20.5::gentoo
sys-apps/sandbox:         2.10-r2::gentoo
sys-devel/autoconf:       2.13::gentoo, 2.69-r2::gentoo
sys-devel/automake:       1.11.6-r2::gentoo, 1.14.1-r1::gentoo, 1.15-r2::gentoo
sys-devel/binutils:       2.25.1-r1::gentoo
sys-devel/gcc:            4.9.3::gentoo, 5.3.0::gentoo
sys-devel/gcc-config:     1.8-r1::gentoo
sys-devel/libtool:        2.4.6-r2::gentoo
sys-devel/make:           4.1-r1::gentoo
sys-kernel/linux-headers: 4.5::gentoo (virtual/os-headers)
sys-libs/glibc:           2.23-r2::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://hazel/gentoo-portage
    priority: -1000

local_overlay
    location: /usr/local/portage
    masters: gentoo
    priority: 0

ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -fomit-frame-pointer -march=btver1 -mmmx -mno-3dnow -msse -msse2 -msse3 -mssse3 -msse4a -mcx16 -msahf -mno-movbe -mno-aes -mno-sha -mno-pclmul -mpopcnt -mabm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-bmi2 -mno-tbm -mno-avx -mno-avx2 -mno-sse4.2 -mno-sse4.1 -mlzcnt -mno-rtm -mno-hle -mno-rdrnd -mno-f16c -mno-fsgsbase -mno-rdseed -mprfchw -mno-adx -mfxsr -mno-xsave -mno-xsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 -mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-clwb -mno-pcommit -mno-mwaitx --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=512 -mtune=btver1 -fstack-protector-strong"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/themes/oxygen-gtk/gtk-2.0"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.5/ext-active/ /etc/php/apache2-php5.6/ext-active/ /etc/php/apache2-php7.0/ext-active/ /etc/php/cgi-php5.5/ext-active/ /etc/php/cgi-php5.6/ext-active/ /etc/php/cgi-php7.0/ext-active/ /etc/php/cli-php5.5/ext-active/ /etc/php/cli-php5.6/ext-active/ /etc/php/cli-php7.0/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -pipe -fomit-frame-pointer -march=btver1 -mmmx -mno-3dnow -msse -msse2 -msse3 -mssse3 -msse4a -mcx16 -msahf -mno-movbe -mno-aes -mno-sha -mno-pclmul -mpopcnt -mabm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-bmi2 -mno-tbm -mno-avx -mno-avx2 -mno-sse4.2 -mno-sse4.1 -mlzcnt -mno-rtm -mno-hle -mno-rdrnd -mno-f16c -mno-fsgsbase -mno-rdseed -mprfchw -mno-adx -mfxsr -mno-xsave -mno-xsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 -mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-clwb -mno-pcommit -mno-mwaitx --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=512 -mtune=btver1 -fstack-protector-strong"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--nospinner --verbose-conflicts --quiet-build=n"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs ccache distcc distlocks ebuild-locks fail-clean fixlafiles merge-sync news preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://gentoo.mirrors.ovh.net/gentoo-distfiles/ http://ftp.snt.utwente.nl/pub/os/linux/gentoo http://mirror.leaseweb.com/gentoo/ http://mirrors.linuxant.fr/distfiles.gentoo.org/ http://mirror.netcologne.de/gentoo/"
LANG="en_GB.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j20 -l2.6"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/tmp"
USE="X a52 aac acl acpi alsa amd64 bash-completion berkdb bluetooth branding bzip2 cairo cdda cddb cdr cli consolekit cracklib crypt css cups cxx dbus declarative dga djvu dri dts dvd dvdr emboss encode exif fam fbcon ffmpeg firefox flac fortran ftp gd gdbm geoip gif gimp glamor gmp gnutls gphoto2 gpm graphviz gtk hddtemp iconv imagemagick imap inotify java javascript jit jpeg kde kipi lame latex lcms ldap libkms libnotify lm_sensors lzma lzo mad maildir matroska mbox mmap mmx mmxext mng modules mp3 mp4 mpeg mplayer mtp multilib musicbrainz mysql ncurses nls nptl nsplugin ocamlopt offensive ofx ogg opengl openmp pam pango pcre pda pdf perl phonon php plasma png policykit posix postscript ppds qml qt3support qt4 qt5 quicktime radius raw readline rss rtc samba scanner sdl seccomp session sharedmem smp snmp sockets socks5 sound spell sqlite sqlite3 sse sse2 ssl startup-notification subversion svg syslog tcpd theora threads tidy tiff tk tokenizer touchpad truetype udev udisks ukit unicode upnp upnp-av upower usb v4l vaapi vcd vdpau vhosts vnc vorbis widgets wifi wmf wps wxwidgets x264 xa xattr xcb xcomposite xft xine xinerama xml xpm xscreensaver xv xvid xvmc zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext popcnt sse sse2 sse3 sse4a ssse3" ELIBC="glibc" ENLIGHTENMENT_MODULES="backlight battery clock comp conf-applications conf-dialogs conf-display conf-edgebindings conf-interaction conf-intl conf-keybindings conf-menus conf-paths conf-performance conf-randr conf-shelves conf-theme conf-window-manipulation conf-window-remembers connman cpufreq dropshadow everything fileman fileman-opinfo gadman ibar ibox mixer msgbus notification pager quickaccess shot start syscon systray tasks temperature tiling winlist wizard xkbswitch" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64 multiboot" INPUT_DEVICES="keyboard mouse evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en_GB" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_4" RUBY_TARGETS="ruby23" SANE_BACKENDS="hp net" USERLAND="GNU" VIDEO_CARDS="radeon r600" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
Comment 1 SpanKY gentoo-dev 2016-05-01 23:28:06 UTC
you should always attach the full build log when filing bug reports
Comment 2 Robin Bankhead 2016-06-04 12:36:43 UTC
Created attachment 436424 [details]
gimp-2.9.2 build.log (gzipped)

Sorry, thought I'd attached this already but apparently not. (Possibly failed silently or without my noticing as I don't remember gzipping it.)
Comment 3 Sebastian Pipping gentoo-dev 2016-07-01 11:37:35 UTC
Could you try with GCC 4.9.3 (and 5.4.0) please and report back?  Thanks!
Comment 4 Johannes Hirte 2017-02-01 22:52:42 UTC
Same problem with gimp-2.9.4-r1 and gcc-6.3 here. It's caused by the compiler flags. Mine are (extracted from -march=native)

-O2 -march=amdfam10 -mmmx -m3dnow -msse -msse2 -msse3 -mno-ssse3 -msse4a -mcx16 -msahf -mno-movbe -mno-aes -mno-sha -mno-pclmul -mpopcnt -mabm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-bmi2 -mno-tbm -mno-avx -mno-avx2 -mno-sse4.2 -mno-sse4.1 -mlzcnt -mno-rtm -mno-hle -mno-rdrnd -mno-f16c -mno-fsgsbase -mno-rdseed -mprfchw -mno-adx -mfxsr -mno-xsave -mno-xsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 -mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-clwb -mno-mwaitx -mno-clzero -mno-pku --param l1-cache-size=64 --param l1-cache-line-size=64 --param l2-cache-size=1024 -mtune=amdfam10 -pipe

stripping this to 

-O2 -march=amdfam10 -pipe

makes gimp compile
Comment 5 Mart Raudsepp gentoo-dev 2017-12-17 20:12:20 UTC
Still the same issue with gimp-2.9.8 and gcc 7.2.0.
Comment 6 Mart Raudsepp gentoo-dev 2017-12-17 21:12:29 UTC
It succeeds with -march=native instead of written out -mno-sse4 and co. Probably something similar to bug 610340 in essence
Comment 7 Sebastian Pipping gentoo-dev 2017-12-18 20:56:58 UTC
(In reply to Johannes Hirte from comment #4)
> Mine are (extracted from -march=native)

I'd be curious if you used app-misc/resolve-march-native (https://github.com/hartwork/resolve-march-native) for the stripping.


(In reply to Johannes Hirte from comment #4)
> stripping this to 
> 
> -O2 -march=amdfam10 -pipe
> 
> makes gimp compile

I can re-produce your case myself now:


cd media-gfx/gimp
ebuild gimp-2.9.8.ebuild clean configure
cd /var/tmp/portage/media-gfx/gimp-2.9.8/work/gimp-2.9.8/app/operations/layer-modes/
make CFLAGS='-O2 -mno-sse4.2 -mno-sse4.1 -pipe' gimpoperationnormal-sse4.o


I'm expecting this to be too specific for upstream to want to address it, but we'll see: https://bugzilla.gnome.org/show_bug.cgi?id=791757

I'm unsure if it's worth stripping these CFLAGS in all (non-stable) Gimp ebuilds, if upstream does not address it.  Any thoughts?
Comment 8 Matt Turner gentoo-dev 2017-12-19 00:21:17 UTC
I worked around this kind of idiocy in Mesa a few years ago. There's no reason to specify -mno-* flags. The only way they make a difference is if some other flag turned them on for you (i.e., -march=...), in which case... what are you doing?

Users should be told not to do this, and I don't think we should waste our time supporting this case.
Comment 9 Sebastian Pipping gentoo-dev 2018-01-03 19:16:41 UTC
Following the recommendation to not fix now.  Please feel free to re-open as needed.
Comment 10 Mart Raudsepp gentoo-dev 2018-01-27 04:09:31 UTC
People specify them because that's what -march=native expanded out gives them. E.g. the matching part of
gcc -march=native -E -v - </dev/null 2>&1 | grep cc1 |grep '[-]march.*[-]mtune\S*'

They do that because they either want to use -march=native but also distcc or just don't want the compiler to have to figure this out for each compilation unit, or just someone told them they might want to do this (hopefully also telling to re-do this after major gcc upgrades).
They don't know that the -mno-* flags shouldn't be necessary and might cause problems.

Usually the problems would be coming from the build system adding the -msse4.1 kind of flags itself to the build CFLAGS, assuming they stuck, but the ordering of CFLAGS of build system additions and user CFLAGS puts the -mno to override the enabling that the build system believes to be present.
This is either a CFLAGS ordering problem or just an unfortunate case, which doesn't happen with -march=native itself because that doesn't FORCE it off like it does when it's "expanded" as per above.

I happen to also do this expanding on some machines (presumably I wanted to use or used distcc at some point, though actually don't these days), so I hit this and a similar case in gst-plugins-base personally as well.
For gst-plugins-base I solved it with a filter-flags as per https://bugs.gentoo.org/610340#c18 for the time being, but believe there's also some complex build system issues at play that could be solved upstream.
Maybe something similar is going on here.

Re-opening for consideration with the extra details. Feel free to just WONTFIX again if you so decide.
Comment 11 Robin Bankhead 2018-01-27 23:41:53 UTC
To explain my "idiocy", I was following the Gentoo wiki:

https://wiki.gentoo.org/wiki/Distcc#CFLAGS_and_CXXFLAGS

Now admittedly if I had read the article on mgorny's blog linked there, I would probably have dropped the redundant flags for the sake of log brevity and we wouldn't be here. However, nowhere is it suggested that using the full inlined list will actually cause build failures.

Perhaps it would be helpful to point this out in the article, to nip future bouts of idiocy in the bud.
Comment 12 Sebastian Pipping gentoo-dev 2018-01-28 14:58:06 UTC
I'd want to learn more about why -mno* flags are special, too.
Comment 13 Matt Turner gentoo-dev 2018-01-28 21:06:21 UTC
(In reply to Robin Bankhead from comment #11)
> To explain my "idiocy", I was following the Gentoo wiki:
> 
> https://wiki.gentoo.org/wiki/Distcc#CFLAGS_and_CXXFLAGS
> 
> Now admittedly if I had read the article on mgorny's blog linked there, I
> would probably have dropped the redundant flags for the sake of log brevity
> and we wouldn't be here. However, nowhere is it suggested that using the
> full inlined list will actually cause build failures.
> 
> Perhaps it would be helpful to point this out in the article, to nip future
> bouts of idiocy in the bud.

Thanks, that sheds some light on it. I'll fix those instructions to filter out -mno-* flags.

(In reply to Sebastian Pipping from comment #12)
> I'd want to learn more about why -mno* flags are special, too.

Special? They're not.

Again, the problem is that

 (1) gcc requires instruction sets be enabled in order to generate code for them, even from intrinsic functions

 (2) Lots of software has run-time enabled fast paths and wants to build the fast paths (using intrinsic functions) regardless of whether the host CPU supports executing those fast paths

 (3) Since the build system cannot assume the default CFLAGS will allow gcc to compile those intrinsics, it turns on instruction sets selectively for files containing the fast paths (with -msse4.1 for instance)

 (4) This breaks when users pass -mno-* since it disables the instruction sets the build system explicitly enabled

Again, this is completely unnecessary, since instruction sets are not enabled without some -march=... or -m* flag turning them on.

I've now explained this far too many times in far too many forums, so I'll now take my leave.
Comment 14 Daniel Santos 2018-01-29 04:10:21 UTC
(In reply to Matt Turner from comment #13)
>  (1) gcc requires instruction sets be enabled in order to generate code for
> them, even from intrinsic functions
> 
>  (2) Lots of software has run-time enabled fast paths and wants to build the
> fast paths (using intrinsic functions) regardless of whether the host CPU
> supports executing those fast paths
> 
>  (3) Since the build system cannot assume the default CFLAGS will allow gcc
> to compile those intrinsics, it turns on instruction sets selectively for
> files containing the fast paths (with -msse4.1 for instance)
> 
>  (4) This breaks when users pass -mno-* since it disables the instruction
> sets the build system explicitly enabled
> 
> Again, this is completely unnecessary, since instruction sets are not
> enabled without some -march=... or -m* flag turning them on.
> 
> I've now explained this far too many times in far too many forums, so I'll
> now take my leave.


And the challenge with this is that, based upon my discussions with one of the gcc i386 backend developers, there actually are some processors that are represented with one march value and then subtracting specific ISAs.  So, in theory, omitting -mno-<ISA> can also produce a bad executable (illegal instruction) on some machines.  My understanding is that they are very old processors and that no such case exists with any modern x86 CPUs, so hopefully nobody will run into the problem.  I haven't looked into this for any other arch.

On the bright side, starting with GCC 6.5, 7.3 and 8.0, an important bug ( https://gcc.gnu.org/bugzilla/show_bug.cgi?id=39851 ) has been fixed that allows us to correctly extrapolate the real flags.  I have written a bash script to suck out that value (https://github.com/daniel-santos/distccflags/blob/master/distccflags).  The distcc team said they would integrate it if I convert it to C (presumably by accepting -march=native and dynamically converting it), but anybody else is welcome to do so too. :)

Unfortunately, modifying GCC to do this analysis for us would be a large undertaking due to the non-uniform nature of the various GCC backends ( see also https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81519 ).

I should also add that the cache parameters are embedded in the march value (which maps to a struct processor_costs object), so it turns out that they are unnecessary (example: https://github.com/gcc-mirror/gcc/blob/e03f627d3e6bc1afbb09e9c1445f2aab7d61d52e/gcc/config/i386/x86-tune-costs.h#L258 ).

If you *really* want to know the most succinct value, you can build GCC 7.3 or 8 and run my script (I presume the backport was picked for 7.3, but I haven't tested it).
Comment 15 Daniel Santos 2018-01-29 04:25:10 UTC
Upon further reflection, this sounds like a deficiency in either the ebuild or the upstream configure.ac and/or Makefile.in.  Explicitly enabling an ISA for an entire build can produce bad code since GCC will attempt to vectorize some operations at -O2 and higher.  Just depending upon a historic fact that it doesn't *happen* to produce bad code when using one version of the compiler is not a guarantee that the next version will not vectorize something it hadn't before, attempting to use an ISA the processor doesn't support.  I think this issue should be addressed upstream and additional ISAs should only be enabled on the modules that use them and cannot be vectorized.

At this time, __attribute__((optimize(xxx))) doesn't support machine options, but perhaps it should so that such changes can be made at the function level.
Comment 16 Matt Turner gentoo-dev 2018-01-29 21:35:16 UTC
(In reply to Daniel Santos from comment #15)
> Upon further reflection, this sounds like a deficiency in either the ebuild
> or the upstream configure.ac and/or Makefile.in.  Explicitly enabling an ISA
> for an entire build can produce bad code since GCC will attempt to vectorize
> some operations at -O2 and higher.

No no no no no. That's not what's happening. *AGAIN*, this is about projects enabling select -m* flags on select files! The code is only executed if a run-time check says the CPU supports the instruction set.
Comment 17 Daniel Santos 2018-03-01 17:48:25 UTC
(In reply to Matt Turner from comment #16)
> (In reply to Daniel Santos from comment #15)
> > Upon further reflection, this sounds like a deficiency in either the ebuild
> > or the upstream configure.ac and/or Makefile.in.  Explicitly enabling an ISA
> > for an entire build can produce bad code since GCC will attempt to vectorize
> > some operations at -O2 and higher.
> 
> No no no no no. That's not what's happening. *AGAIN*, this is about projects
> enabling select -m* flags on select files! The code is only executed if a
> run-time check says the CPU supports the instruction set.

Well it looks like this needs to be re-examined.  I knew it was hypothetically possible, but now it is certain: there are in fact situations where stripping all -mno-* flags is wrong and can thus potentially produce bad code.  Upstream should strip any -mno-<isa> flags when they need to add their -m<isa> counterpart.

Please note this particular case: https://github.com/daniel-santos/distccflags/issues/1#issuecomment-369467668  This seems to have come about when Intel added AES and Carry-Less Multiplication Instruction seemingly as soon as they had it ready rather than waiting for their next arch?  Either way, it's plausible for there to be more cases as well.

I tried to get this better managed in GCC, but ISAs are managed by each backend in their own ways.  To *really* clean this up, we need to implement a framework in the middle-end and convert each backend to use it.
Comment 18 Matt Turner gentoo-dev 2018-03-01 18:05:27 UTC
(In reply to Daniel Santos from comment #17)
> (In reply to Matt Turner from comment #16)
> > (In reply to Daniel Santos from comment #15)
> > > Upon further reflection, this sounds like a deficiency in either the ebuild
> > > or the upstream configure.ac and/or Makefile.in.  Explicitly enabling an ISA
> > > for an entire build can produce bad code since GCC will attempt to vectorize
> > > some operations at -O2 and higher.
> > 
> > No no no no no. That's not what's happening. *AGAIN*, this is about projects
> > enabling select -m* flags on select files! The code is only executed if a
> > run-time check says the CPU supports the instruction set.
> 
> Well it looks like this needs to be re-examined.  I knew it was
> hypothetically possible, but now it is certain: there are in fact situations
> where stripping all -mno-* flags is wrong and can thus potentially produce
> bad code.  Upstream should strip any -mno-<isa> flags when they need to add
> their -m<isa> counterpart.

I don't know of any build system that can do that. Autotools certainly cannot.
Comment 19 Daniel Santos 2018-03-05 15:26:44 UTC
I think I know of a way to do this, but tied up at the moment.  Give me a few days and let me see if I can come back with a solution.  If it works and is clean enough, maybe we can standardize it in some way. (An eclass or something?) iirc, the rules for GCC's machine flags are funny, any -mno-<isa> overrides a future -m<isa>, although I don't recall for certain.  I'll re-examine that too.
Comment 20 Sergei Trofimovich (RETIRED) gentoo-dev 2019-01-27 12:41:31 UTC
If we have the documentation that suggests passing '-mno-*' flags explicitly we should fix that. The conservative way to extract effective flags should be something like:
    $ diff -U0 <(gcc -Q --help=target) <(gcc -Q --help=target -march=native) | egrep -v '^\-.*disabled|^@@'
and should not normally contain newly enabled '-mno-*' options.
Comment 21 Sergei Trofimovich (RETIRED) gentoo-dev 2019-01-27 13:42:49 UTC
Daniel, what do you think of packaging distccflags in gentoo?

Before doing that (and suggest it an official way to expand -march=native) I suggest one more addition:

validate that proposed flags by distccflags are equivalent to originally passed flags.

Having the confidence that tool does something we can rely on toolchain@ would also reuse your tool in ICE reporting guide:
    https://wiki.gentoo.org/wiki/Gcc-ICE-reporting-guide#Expand_-march.3Dnative.2C_exact_gcc_version_and_other_system-specific_options
Comment 22 Sebastian Pipping gentoo-dev 2019-01-27 16:39:34 UTC
(In reply to Sergei Trofimovich from comment #21)
> Daniel, what do you think of packaging distccflags in gentoo?

app-misc/resolve-march-native does not do what you need?  if it has bugs, I'd be happy about reports and PRs at https://github.com/hartwork/resolve-march-native
Comment 23 Pacho Ramos gentoo-dev 2019-04-09 16:50:02 UTC
*** Bug 663986 has been marked as a duplicate of this bug. ***
Comment 24 Sergei Trofimovich (RETIRED) gentoo-dev 2019-11-24 22:17:57 UTC
I've removed raw command that produces redundant -mno-* flags and provided primitive diff as a hint:
    $ diff -U0 <(LANG=C gcc -Q -O2 -march=sandybridge --help=target) <(LANG=C gcc -Q -O2 -march=native --help=target)
while leaving links to other resources intact:
    https://wiki.gentoo.org/wiki/Distcc#CFLAGS_and_CXXFLAGS