Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 640958 - dev-util/ccache does not detect gcc upgrades due to /usr/bin/${CHOST}-gcc wrapper (was: sys-boot/lilo-24.0-r1 - ld: .../temp/ccvrWdC6.o: relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC)
Summary: dev-util/ccache does not detect gcc upgrades due to /usr/bin/${CHOST}-gcc wra...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: AMD64 Linux
: Normal normal (vote)
Assignee: Gentoo Toolchain Maintainers
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-12-13 20:47 UTC by Hartmut Figge
Modified: 2018-10-22 11:00 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build.log (build.log,6.10 KB, text/plain)
2017-12-13 20:50 UTC, Hartmut Figge
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Hartmut Figge 2017-12-13 20:47:17 UTC
My mostly stable Gentoo recently switched to gcc-6.4.0 and also required switching to a new profile, now default/linux/amd64/17.0

This required new compiling of the whole world. Three packages failed, one of them was lilo. Unstable lilo is also affected.

Major or critical? Hm. I will attach the log after submitting this bug.

Reproducible: Always
Comment 1 Hartmut Figge 2017-12-13 20:50:22 UTC
Created attachment 509966 [details]
build.log

The attachment shows the error log of trying to build lilo with gcc-6.4.0
Comment 2 Jeroen Roovers (RETIRED) gentoo-dev 2017-12-14 23:22:57 UTC
(In reply to Hartmut Figge from comment #0)
> My mostly stable Gentoo recently switched to gcc-6.4.0

But that wasn't it. LILO compiles and runs just fine with 6.4.0.

> switching to a new profile, now default/linux/amd64/17.0

That was it. Someone who thinks he's running toolchain@ started enforcing PIE in that profile set.
Comment 3 Hartmut Figge 2017-12-15 00:00:25 UTC
(In reply to Jeroen Roovers from comment #2)
> (In reply to Hartmut Figge from comment #0)

> > My mostly stable Gentoo recently switched to gcc-6.4.0
> 
> But that wasn't it. LILO compiles and runs just fine with 6.4.0.
> 
> > switching to a new profile, now default/linux/amd64/17.0
> 
> That was it. Someone who thinks he's running toolchain@ started enforcing
> PIE in that profile set.

Ah. But severity normal? Imagine a fresh install of Gentoo intended for lilo which now uses a profile 17.0 and gcc-6.4.0 and lilo fails to build. That borders on critical, don't you mean?

Regarding gcc-6.4.0: lilo builds fine fine under profile 17.0 after temporarily switching back to gcc-5.4.0. ;)
Comment 4 Sergei Trofimovich gentoo-dev 2017-12-17 11:04:15 UTC
Did you try dropping ccache cache after switching to 17.0 profile? I can't reproduce it locally.

Can you post your emerge --info as well?
Comment 5 Hartmut Figge 2017-12-17 11:20:54 UTC
(In reply to Sergei Trofimovich from comment #4)
> Did you try dropping ccache cache after switching to 17.0 profile? 

No. But now. Good guess. :) Now lilo compiles successfully under gcc-6.4.0. I will try to close the bug as WFM. As reporter I should have the permission to do that.
Comment 6 Sergei Trofimovich gentoo-dev 2017-12-17 11:58:46 UTC
(In reply to Hartmut Figge from comment #5)
> Now lilo compiles successfully under gcc-6.4.0.

Woohoo \o/. I think we have maanaged to fool ccache here because gcc version or gcc options did not seem to change from ccache's standpoint.

I wonder what exactly ccache considers as a unique identified of an object in the cache. My guess it's the tuple of (gcc-version, compiler-flags, preprocessed-output) which does not change with gcc[pie] vs. gcc[-pie]. Would be nice if ccache could detect this change.

Say, if ccache looks at 'gcc --version' output we could plug something there identifyng pie presence.
Comment 7 Sergei Trofimovich gentoo-dev 2017-12-17 12:22:59 UTC
Manual at https://ccache.samba.org/manual.html#_how_ccache_works says it checks for size of a compiler binary.

    /usr/bin/x86_64-pc-linux-gnu-gcc
is a 10KB wrapper in gentoo, but
    /usr/x86_64-pc-linux-gnu/gcc-bin/7.2.0/x86_64-pc-linux-gnu-gcc
is a real compiler.

It looks like at least for ebuilds ccache talks to the wrapper and not the real compiler (note second execve):

  $ strace -f -etrace=execve /usr/lib64/ccache/bin/x86_64-pc-linux-gnu-gcc -v
  execve("/usr/lib64/ccache/bin/x86_64-pc-linux-gnu-gcc", ["/usr/lib64/ccache/bin/x86_64-pc-"..., "-v"], 0x7ffec2acab38 /* 79 vars */) = 0

  execve("/usr/bin/x86_64-pc-linux-gnu-gcc", ["/usr/bin/x86_64-pc-linux-gnu-gcc", "-v"], 0x55936afe4370 /* 80 vars */) = 0

  execve("/usr/x86_64-pc-linux-gnu/gcc-bin/7.2.0/x86_64-pc-linux-gnu-gcc", ["/usr/x86_64-pc-linux-gnu/gcc-bin"..., "-v"], 0x7fff173d3fd0 /* 80 vars */) = 0

I think we hit the following gotcha from the manual above:

"""
Unless you set compiler_check to a suitable command (see the description of that configuration option), it is not recommended to use the form 'ccache anotherwrapper compiler args' as the compilation command. It’s also not recommended to use the masquerading technique for the other compiler wrapper. The reason is that by default, ccache will in both cases hash the mtime and size of the other wrapper instead of the real compiler, which means that:

    Compiler upgrades will not be detected properly.

    The cached results will not be shared between compilations with and without the other wrapper.
"""

In our case /usr/bin/x86_64-pc-linux-gnu-gcc is such a wrapper. Let's see if we can do something about it on toolchain@ side.
Comment 8 Larry the Git Cow gentoo-dev 2018-08-24 18:19:41 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/gcc-config.git/commit/?id=9b907ef80bc421df23515afc4c306e4d96c67649

commit 9b907ef80bc421df23515afc4c306e4d96c67649
Author:     Sergei Trofimovich <slyfox@gentoo.org>
AuthorDate: 2018-08-24 17:36:57 +0000
Commit:     Sergei Trofimovich <slyfox@gentoo.org>
CommitDate: 2018-08-24 18:09:17 +0000

    Convert binary wrapper to a symlink wrapper.
    
    Before the change:
       /usr/bin/gcc and friends were a copy of /usr/$(libexecdir)/misc/gcc-config
    After the change:
       /usr/bin/gcc is a symlink to a real compiler binary. Examples:
           /usr/${CTARGET}/gcc-bin/${GCC_VER}/gcc (native)
           /usr/${CHOST}/${CTARGET}/gcc-bin/${GCC_VER}/gcc (cross)
    
    gcc-config is a binary wrapper that does (or did) a few things:
    
    - [removed in 2011] injects additional CFLAGS_${ABI}
    
      Removed in commit 7ac40f3eb8434961f70485247d883f5b3009dcf2
      "Stop auto appending CFLAGS_<abi> from the env."
    
    - traverses PATH for real compiler binary and reexecutes it.
    
    - reads /etc/env.d/05gcc-${CTARGET} as a fallback if PATH is empty.
    
    Today binary wrapper does only PATH resolution and re-execution.
    
    This change has a few minor benefits:
    
    - PATH will not contain explicit /usr/${CHOST}/${CTARGET}/gcc-bin/${GCC_VER}
      entry. This will make PATH shorter for those who have mavy cross-compilers
      installed.
    
    - compiler switch will not require sourcing '. /etc/profile' as changes are
      applied as soon an symlink is switched.
    
    - ccache will see gcc binary changes directly and react accordingly.
      Previously in default configuration ccache cache depended on state of
      /usr/$(libexecdir)/misc/gcc-config
    
      See bug #640958 where ccache did not notice USE=-pie -> USE=pie switch.
    
    - Reasoning about PATH ordering is straightforward: all available binaries
      (as symlinks) are in /usr/bin.
    
      See bug #255695 where PATH ordering changed and bug #626606 where
      people get confused on what is in /usr/bin/gcc binary.
    
    Bug: https://bugs.gentoo.org/626606
    Bug: https://bugs.gentoo.org/255695
    Bug: https://bugs.gentoo.org/640958
    Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>

 .gitignore                                         |   3 -
 Makefile                                           |   8 +-
 README                                             |  69 +----
 gcc-config                                         |  33 +--
 tests/rw-multi-native-configs/test.select          |   4 +-
 .../usr/lib/gcc-config/wrapper                     |   0
 tests/source/test.whitespace                       |   2 +-
 wrapper.c                                          | 323 ---------------------
 8 files changed, 28 insertions(+), 414 deletions(-)
Comment 9 Larry the Git Cow gentoo-dev 2018-08-24 18:42:44 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=f9a5bc6b4fd38dca029e8591db140c85a82d1979

commit f9a5bc6b4fd38dca029e8591db140c85a82d1979
Author:     Sergei Trofimovich <slyfox@gentoo.org>
AuthorDate: 2018-08-24 18:42:23 +0000
Commit:     Sergei Trofimovich <slyfox@gentoo.org>
CommitDate: 2018-08-24 18:42:37 +0000

    sys-devel/gcc-config: bump up to 2.0
    
    Things of note:
    - gcc-config runs env-update on prefix as well
    - gcc wrapper gone away and gcc-config now just symlinks gcc binary
    - LTO plugin is symlinked into binutils
    
    Closes: https://bugs.gentoo.org/640958
    Package-Manager: Portage-2.3.48, Repoman-2.3.10

 sys-devel/gcc-config/Manifest              |  1 +
 sys-devel/gcc-config/gcc-config-2.0.ebuild | 54 ++++++++++++++++++++++++++++++
 2 files changed, 55 insertions(+)