Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 872971 - sys-devel/gcc-config: set CCACHE_COMPILERCHECK in /etc/env.d for dev-util/ccache to not invalidate caches b/t snapshots
Summary: sys-devel/gcc-config: set CCACHE_COMPILERCHECK in /etc/env.d for dev-util/cca...
Status: RESOLVED CANTFIX
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Toolchain Maintainers
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-09-26 00:27 UTC by Sam James
Modified: 2022-10-10 14:18 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-09-26 00:27:20 UTC
From the ccache manual:
"""
compiler_check (CCACHE_COMPILERCHECK)

    By default, ccache includes the modification time (“mtime”) and size of the compiler in the hash to ensure that results retrieved from the cache are accurate. This option can be used to select another strategy. Possible values are: 
[...]

 string:value

    Use value as the string to calculate hash from. This can be the compiler revision number you retrieved earlier and set here via environment variable.
a command string

    Hash the standard output and standard error output of the specified command. The string will be split on whitespace to find out the command and arguments to run. No other interpretation of the command string will be done, except that the special word %compiler% will be replaced with the path to the compiler. Several commands can be specified with semicolon as separator. Examples:

    %compiler% -v

    %compiler% -dumpmachine; %compiler% -dumpversion

    You should make sure that the specified command is as fast as possible since it will be run once for each ccache invocation.

    Identifying the compiler using a command is useful if you want to avoid cache misses when the compiler has been rebuilt but not changed.

    Another case is when the compiler (as seen by ccache) actually isn’t the real compiler but another compiler wrapper — in that case, the default mtime method will hash the mtime and size of the other compiler wrapper, which means that ccache won’t be able to detect a compiler upgrade. Using a suitable command to identify the compiler is thus safer, but it’s also slower, so you should consider continue using the mtime method in combination with the prefix_command setting if possible. See USING CCACHE WITH OTHER COMPILER WRAPPERS.
"""

Right now, the wiki suggests setting `compiler_check` (https://wiki.gentoo.org/wiki/Ccache#ccache.conf):
```
# Preserve cache across GCC rebuilds and
# introspect GCC changes through GCC wrapper.
compiler_check = %compiler% -v
```

I think we could do better and not invalidate caches on new GCC snapshots (various 12.2.1_p* for example where ABI should be stable and nothing notable should really be changing).

We could set CCACHE_COMPILERCHECK="string:$foo" where foo is the major version of GCC we've set to using gcc-config.
Comment 1 Larry the Git Cow gentoo-dev 2022-09-29 22:36:02 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/gcc-config.git/commit/?id=3e49f724d090d65b739be1a67f6c40574337a2f7

commit 3e49f724d090d65b739be1a67f6c40574337a2f7
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-09-29 22:12:12 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-09-29 22:35:57 +0000

    gcc-config: set CCACHE_COMPILERCHECK to major GCC version
    
    This avoids invalidating caches on every GCC snapshot bump for the same
    major version (e.g. 12.2.1_p*), but also for 12.1 -> 12.2. It should be fine
    to mix objects between such versions.
    
    If needed, we can swap this to just stripping .1_p* but that's a bit
    more awkward.
    
    Bug: https://bugs.gentoo.org/872971
    Signed-off-by: Sam James <sam@gentoo.org>

 gcc-config | 8 ++++++++
 1 file changed, 8 insertions(+)
Comment 2 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-10-08 22:10:44 UTC
Done in gcc-config-2.7.
Comment 3 Larry the Git Cow gentoo-dev 2022-10-10 14:09:48 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/gcc-config.git/commit/?id=681d39425a71977ba0850eb88448f831f84dd6eb

commit 681d39425a71977ba0850eb88448f831f84dd6eb
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-10-10 14:07:23 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-10-10 14:07:23 +0000

    Revert "gcc-config: set CCACHE_COMPILERCHECK to major GCC version"
    
    This isn't going to work where the compiler used is Clang but GCC
    is installed -- we'll end up always using a cache key of the
    major GCC version, even for say, different Clang versions.
    
    Fortunately, using %compiler% -dumpversion instead of %compiler% -v
    locally will handle it fine as a workaround (to be set in ccache.conf) because
    since  b7a62d5e3f5e6e05c929f225ead22a2981c9f64 in gentoo.git, GCC installs
    with just the major version in its paths, and -dumpversion reports said
    major version, not the whole version (can use gcc -dumpfullversion if needed).
    
    This reverts commit 3e49f724d090d65b739be1a67f6c40574337a2f7.
    
    Bug: https://bugs.gentoo.org/872971
    Signed-off-by: Sam James <sam@gentoo.org>

 gcc-config | 8 --------
 1 file changed, 8 deletions(-)
Comment 4 Larry the Git Cow gentoo-dev 2022-10-10 14:18:10 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=1f56ffa16dbca7cc38c49bbb977a384f894796e7

commit 1f56ffa16dbca7cc38c49bbb977a384f894796e7
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-10-10 14:16:50 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-10-10 14:17:59 +0000

    sys-devel/gcc-config: add 2.8
    
    Closes: https://bugs.gentoo.org/868639
    Bug: https://bugs.gentoo.org/872971
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-devel/gcc-config/Manifest              |  1 +
 sys-devel/gcc-config/gcc-config-2.8.ebuild | 60 ++++++++++++++++++++++++++++++
 2 files changed, 61 insertions(+)