Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 57223

Summary: compiling gcc: -Os replaced with -O2
Product: Gentoo Linux Reporter: Gavin <colonel_dolphin>
Component: New packagesAssignee: Gentoo Toolchain Maintainers <toolchain>
Status: RESOLVED FIXED    
Severity: enhancement    
Priority: High    
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: appends SAFE_GCC_O_FLAG to "emerge info" results
replace-with-safe-gcc-o-flag() [ 4 lines of code ]

Description Gavin 2004-07-15 13:41:43 UTC
-Os works fine with gcc, but ebuild replaces -Os with -O2.
This is Gentoo .. please give me a choice without making me edit the ebuild for gcc.

Read the docs for gcc's -Os and -O2 options.  For the most part -Os is a subset of -O2, and thus less likely to cause problems than -O2.

Please give me a choice, rather than deleting my choice in compiler optimization flags.  Thanks.

Reproducible: Always
Steps to Reproduce:
1.
2.
3.
Comment 1 Gavin 2004-07-15 13:42:57 UTC
I believe all recent gcc ebuilds and several other unrelated packages make the same mistake.
Comment 2 Martin Schlemmer (RETIRED) gentoo-dev 2004-07-15 13:57:18 UTC
Because:
1) It creates a gcc that compiles faster
2) Some archs creates a _more_ buggy gcc (k6-2 comes to mind, can't remember
   the others)
3) Because of above, we want to keep our sanity
4) You can still edit the ebuild, and keep it in portage-local, so we do
   not really take away your choice.
Comment 3 Travis Tilley (RETIRED) gentoo-dev 2004-07-15 14:59:22 UTC
we cant even make this arch specific, since on x86 there are some weird subarchs that will blow up and others that wont... but i know for sure -Os on amd64 breaks things hardcore. elsewhere as well...

do you have a more elegant solution to this problem that wont increase the number of pointless bug reports we have to close?
Comment 4 Gavin 2004-07-15 20:26:23 UTC
Hmmm .. you both make very good points.

I've been tweaking ebuilds for the last few months to use -Os, but it's really a headache to update them constantly.  I use -Os on both Athlon-XP and Via C3 (same gcc options: -march=i586 -mmmx -m3dnow) with gcc 3.3.3, 2.6.5, and glibc from 20040420.  No problems that I have found.  People using Via C3 systems often use -Os everywhere (due to very tiny CPU cache). Perhaps gcc team has worked some of the bugs out of using "-Os" over the last few months?

Regarding something elegant without pestering everyone with bug reports from clueless users, perhaps a function/macro/variable for use in ebuilds that replaces "dangerous" -O options with a less "dangerous", but configurable option, specified in /etc/make.globals (safely tucked away where most clueless users won't think to tweak)?

For example, add 'SAFE_GCC_O_FLAG="-O2"' to /etc/make.globals, then ebuilds can use this, instead of hardcoding it.  Some users may even want something more safe than "-O2" (e.g. they want an entire system without optimization, and with '-g').
Comment 5 Travis Tilley (RETIRED) gentoo-dev 2004-07-15 21:23:13 UTC
such a thing might be useful to have, especially since there's been recent work on gentoo-embedded, which could probably benefit from using -Os. as for asking if you had an elegant solution, i also meant a solution with patches :)

for general discussion, bugzilla probably isnt the best place. i'd really suggest subscribing to the gentoo-dev mailing list and voicing your opinions there. http://www.gentoo.org/main/en/lists.xml
Comment 6 Gavin 2004-07-17 09:18:09 UTC
Created attachment 35638 [details, diff]
appends SAFE_GCC_O_FLAG to "emerge info" results
Comment 7 Gavin 2004-07-17 09:29:05 UTC
Hi Travis.  How about appending the following to Gentoo's /etc/make.global?
SAFE_GCC_O_FLAG="-O2"

Then, in ebuilds (e.g. for gcc) DO NOT use:
export CFLAGS="$(echo "${CFLAGS}" | sed -e 's|-O[0-9s]\?|-O2|g')"
Instead use this:
CFLAGS="${CFLAGS//-O?} $SAFE_GCC_O_FLAG"

Unlike the gcc ebuild, the simple bash expression above creates no extra processes.

Emerge needs a tiny patch so that "emerge info" meets your requirements for: "[not] increase the number of pointless bug reports we have to close?".

Now, stepping back for moment .. perhaps we should consider a duplicate set of C*FLAGS that are supposed to be "safe", rather than just one extra variable for identifying a "safe" optimization level for the user's architecture/kernel/compiler/library combination?  Oh, well, at least the SAFE_GCC_O_FLAG approach would solve my most immediate frustration with needing to tweak several ebuilds continually with this most trivial change (from -O2 to -Os).
Comment 8 Gavin 2004-07-17 09:55:56 UTC
Also, for those that prefer using "inherit flag-o-matic", they can use:
replace-flags -O? $SAFE_GCC_O_FLAG

However, it really isn't as safe as the direct method I demonstrated above.  The method I used above appends the new, desired optimization flag.  Since gcc only uses the last optimization flag found on the command line, and flag-o-matic tries to directly replace an existing flag ... there is more room for "gotchya's" with replace-flags.  Also, the bash method is clear, concise, simple, and familiar to all who are already familiar with bash without whatever overhead exists in "inherit flag-o-matic".
Comment 9 Martin Schlemmer (RETIRED) gentoo-dev 2004-07-17 14:25:06 UTC
> Then, in ebuilds (e.g. for gcc) DO NOT use:
> export CFLAGS="$(echo "${CFLAGS}" | sed -e 's|-O[0-9s]\?|-O2|g')"
> Instead use this:
> CFLAGS="${CFLAGS//-O?} $SAFE_GCC_O_FLAG"
>
> Unlike the gcc ebuild, the simple bash expression above creates no extra
> processes.

Uhm, no, please do not - that breaks on '-O'.  There is a bug about that
somewhere ...


Comment 10 Gavin 2004-07-18 09:01:10 UTC
> Uhm, no, please do not - that breaks on '-O'.
> There is a bug about that somewhere ...

What version of bash are you using?
Works peachy here (except when -O is the last option).

# foo="-O -frepo -O foobar -whatever -O2 -moreopts -O3"
# echo ${foo//-O?}
-frepo foobar -whatever -moreopts

Even if someone has CFLAGS="-O", 'export CFLAGS="${CFLAGS//-O?} $SAFE_GCC_O_FLAG"' should still work, since gcc only uses the optimization flag that is appended.  For those that really care about making the result look "pretty" for the rare case when '-O' is the last option listed in CFLAGS, without the extra processes, try:
CFLAGS="${CFLAGS%-O}"
export CFLAGS="${CFLAGS//-O?} $SAFE_GCC_O_FLAG"

In the end, I really don't care how its done, just so long as I don't need to maintain a portage overlay with every ebuild I need that forces a package to be built with "-O2".  Those with tight constraints on their CPU cache (even larger servers with high working loads) can sometimes benefit from avoiding -O2, depending on the application, timeslice length, etc.
Comment 11 Martin Schlemmer (RETIRED) gentoo-dev 2004-07-18 10:20:31 UTC
You are doing it wrong:

---
$ (foo="-O blah"; echo ${foo//-O?/-O2};)
-O2blah
---

Which will bork gcc.  Sure, there are other ways do do it with bash
replacement, but existing method is tested, and only run minimal, so not
really an speed issue.

If you really have an issue with it though, this should be the better way:

---
CFLAGS="${CFLAGS//-O?/ } $SAFE_GCC_O_FLAG"
---
Comment 12 Martin Schlemmer (RETIRED) gentoo-dev 2004-07-18 10:44:35 UTC
Actually, that issue is only if you replace -O? with something else.  If
not like you did, it should not be an issue (or think not).  It might still
be the safer option to replace it with a <space> rather then nothing just
in case ...
Comment 13 Gavin 2004-07-18 19:48:02 UTC
Ok!  Sounds good to me Martin.  Lets do it.  

How do we add SAFE_GCC_O_FLAG to /etc/make.global, and pass the word that ebuilds should use:
---
CFLAGS="${CFLAGS//-O?/ } $SAFE_GCC_O_FLAG"
---

instead of something like:
gcc-3.3.4-r1.ebuild:    export CFLAGS="$(echo "${CFLAGS}" | sed -e 's|-O[0-9s]\?|-O2|g')"
Comment 14 Martin Schlemmer (RETIRED) gentoo-dev 2004-07-19 11:11:24 UTC
Will prob have to do something like:

---
[ -z "${SAFE_GCC_O_FLAG}" ] && SAFE_GCC_O_FLAG="-O2"
CFLAGS="${CFLAGS//-O?/ } ${SAFE_GCC_O_FLAG}"
---

As make.global have to be updated portage side ... that is except if it is
done profile side, and I am not on track with the new cascade stuff, so will
rather not be the one to do it.

Comment 15 Gavin 2004-07-20 17:23:46 UTC
Yup.  It might be a long time before we could expect everyone using new ebuilds to have a make.global with SAFE_GCC_O_FLAG defined.

I think we're on the right track, and other people will pick up on the idea and expand it (if needed) by providing different defaults for different profiles.

To get the ball rolling and solve the "lack of over-loadable gcc optimization default" issue for 90+% of the Gentoo user base, I think you're last post is right on the money =)

I wonder if the change would be more appealing to ebuild maintainers, if the change was encapsulated into an update to the flag-o-matic class?  Maybe something simple, like a bash function called "replace-with-safe-gcc-o-flag"?

/usr/portage/eclass/flag-o-matic.eclass:

replace-with-safe-gcc-o-flag() {
[ -z "${SAFE_GCC_O_FLAG}" ] && SAFE_GCC_O_FLAG="-O2"
CFLAGS="${CFLAGS//-O?/ } ${SAFE_GCC_O_FLAG}"
    CFLAGS=" ${CFLAGS} "
    CXXFLAGS=" ${CXXFLAGS} "
    CFLAGS="${CFLAGS// ${1} / ${2} }"
    CXXFLAGS="${CXXFLAGS// ${1} / ${2} }"
    CFLAGS="${CFLAGS:1:${#CFLAGS}-2}"
    CXXFLAGS="${CXXFLAGS:1:${#CXXFLAGS}-2}"
    export CFLAGS CXXFLAGS
    return 0
}

Comment 16 Gavin 2004-07-20 17:33:09 UTC
Blast .. I hit TAB and ENTER (vi habits) .. next thing I know it committed ... anyway, here is what I wanted to say:

replace-with-safe-gcc-o-flag() {
  [ -z "${SAFE_GCC_O_FLAG}" ] && SAFE_GCC_O_FLAG="-O2"
  CFLAGS="${CFLAGS//-O?/ } ${SAFE_GCC_O_FLAG}"
  CXXFLAGS="${CXXFLAGS//-O?/ } ${SAFE_GCC_O_FLAG}"
  export CFLAGS CXXFLAGS
  return 0
}


Comment 17 Travis Tilley (RETIRED) gentoo-dev 2004-07-20 18:08:47 UTC
an addition to flag-o would most certainly be preferred, especially if it makes use of previously existing functions in flag-o.

once again, i'll mention that nothing will ever get done here without discussion on the gentoo-dev mailing list... so hurry up and subscribe, voice your ideas, get feedback, etc. on bugzilla you have just a handfull of people on the toolchain@gentoo.org alias seeing this, and we tend to be distracted easily with other stuff :)
Comment 18 Gavin 2004-10-01 15:46:48 UTC
Created attachment 40892 [details]
replace-with-safe-gcc-o-flag() [ 4 lines of code ]

The solution is sooo simple.  Please apply.
Comment 19 Jeremy Huddleston (RETIRED) gentoo-dev 2005-02-10 02:38:56 UTC
toolchain.eclass does this.