Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 3028 - ccache not used by portage (several fixes included below)
Summary: ccache not used by portage (several fixes included below)
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All Linux
: High major (vote)
Assignee: Daniel Robbins (RETIRED)
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2002-05-25 11:19 UTC by Disconnect
Modified: 2002-06-17 11:11 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Disconnect 2002-05-25 11:19:47 UTC
If you install ccache (dev-util/ccache) to cache compilation results,
the portage system will not, by default, use it or be able to use it.

All the fixes below require that /var/cache/ccache be created by the
ccache package, set in the environment as CCACHE_DIR, and added to the
default sandbox writable directory list:

diff -u sandbox/sandbox.bashrc sandbox.new/sandbox.bashrc
--- sandbox/sandbox.bashrc	Wed Mar  6 04:51:02 2002
+++ sandbox.new/sandbox.bashrc	Fri May 24 13:21:48 2002
@@ -6,3 +6,4 @@
 export LD_PRELOAD="$SANDBOX_LIB"
 alias make="make LD_PRELOAD=$SANDBOX_LIB"
 alias su="su -c '/bin/bash -rcfile $SANDBOX_DIR/sandbox.bashrc'"
+export CCACHE_DIR="/var/cache/ccache/"
diff -u sandbox/sandbox.c sandbox.new/sandbox.c
--- sandbox/sandbox.c	Tue Apr 23 18:40:25 2002
+++ sandbox.new/sandbox.c	Fri May 24 13:21:48 2002
@@ -648,6 +648,8 @@
 				strcat(sandbox_write_var, ":");
 				strcat(sandbox_write_var, "/usr/lib/cf");
 				strcat(sandbox_write_var, ":");
+				strcat(sandbox_write_var, "/var/cache/ccache");
+				strcat(sandbox_write_var, ":");
 				if (NULL == portage_tmp_dir)
 				{
 					strcat(sandbox_write_var, "/tmp/");


The first (easiest) fix is to have ccache create 3 links in /bin:
lrwxr-xr-x    1 root     root           15 May 25 11:29 cpp -> /usr/bin/ccache*
lrwxr-xr-x    1 root     root           15 May 25 11:29 g++ -> /usr/bin/ccache*
lrwxr-xr-x    1 root     root           15 May 25 11:29 gcc -> /usr/bin/ccache*

That puts ccache ahead of the normal compilers in the path, and allows
it to work transparently.  This is the only fix that works for the
current qt ebuild.

Alternately, you can set the environment variables $CC/$CPP/$CXX to
"ccache <prog>" (eg "ccache gcc") which works for some ebuilds (such as
bitchx).

And still more alternately, you can modify the ebuild script itself to
detect the presence of ccache and prepend something like /usr/ccache/bin
(containing those links) to the path, with the same results as above,
but without some of the annoyances of putting inappropriate things in
/bin.

Using bitchx as an example, there is an initial compile time of 6
minutes (when there is nothing in the ccache cache directory) and after
that, only 1.5 minutes (mostly the configure/install time, as the object
files are already cached.)

A suggestion was made of using /tmp or similar for the cache directory,
but that defeats the purpose - it can get wiped due to age or reboots,
it might be stored on a ramdisk, etc.

If CCACHE_DIR is added to the main profile (and the permissions are
managed correctly) it could be used systemwide by users, root and the
portage scripts to speed up everything.  

CCache should have a decent cache hit rate even on upgraded packages, as
usually the compile flags and some/most of the source files won't change
between versions.
Comment 1 Disconnect 2002-05-25 11:31:04 UTC
Addendum: I just discovered that the environment variables must be added
to make.conf in order for some ebuilds to work - this works with the
symlinks in /bin/ without a problem:
CC="ccache gcc"
CPP="ccache cpp"
CXX="ccache g++"
CCACHE_DIR="/var/cache/ccache"

Comment 2 J Robert Ray 2002-05-26 18:09:20 UTC
CC="ccache gcc"
CPP="ccache cpp"
CXX="ccache g++"

Setting these environment variables caused the compilation of "imagemagick" to
fail.  I'm sorry I no longer have something to cut and paste, but it failed in
the middle of building some .lo's, I noticed in the middle of the output was a
ccache statistics summary and a ccache usage output, as if there was some bad
argument parsing and bad interaction with libtool.

The problem may be since $CC expands to two words 'ccache gcc' instead of one
word 'gcc' it screws up the argument list of a command.  Just a guess.
Comment 3 J Robert Ray 2002-05-26 18:10:55 UTC
Sorry, forgot to mention that I removed those environment vars and the
complication succeeded, and still appears to have used ccache because my
statistics summary now shows a lot of cache hits.
Comment 4 Disconnect 2002-05-27 21:14:30 UTC
The reason for the env variables in addition to the links is to   catch the cases where configure decides to use something like   i686-pc-linux-gnu-gcc (no cache) instead of the /bin/gcc symlink.      What may need to happen is a patch to those particular ebuilds to detect   ccache and act accordingly.    Another outstanding issue is how well (or if) it talks to colorgcc. 
Comment 5 Daniel Robbins (RETIRED) gentoo-dev 2002-06-17 11:11:07 UTC
Will appear in the next version of Portage (after 2.0.1).  I added the sandbox.c
fix to Portage.