| Summary: | portage should set CCACHE_BASEDIR | ||
|---|---|---|---|
| Product: | Portage Development | Reporter: | Martin Väth <martin> |
| Component: | Core - External Interaction | Assignee: | Portage team <dev-portage> |
| Status: | CONFIRMED --- | ||
| Severity: | enhancement | CC: | alex_y_xu, ansla80, bug, rhill |
| Priority: | High | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: |
A patch exporting the variable as requested
Dummy ebuild for testing |
||
(In reply to comment #0) > CCACHE_BASEDIR="${PORTAGE_TMPDIR}/portage" > With this value, build systems using absolute filenames will not > automatically cause cache misses for version changes (since under portage > the absolute compilation path always contains the version number...). Are you sure about that? Path relative to that would contain the version bump too. Shouldn't we use ${WORKDIR} instead then? Created attachment 254819 [details, diff]
A patch exporting the variable as requested
Well, in any case, does that export location work for you?
> Are you sure about that? Yes. ${PORTAGE_TMPDIR}/portage" suffices and is even better than $WORKDIR: The meaning of CCACHE_BASEDIR is only to define for which files ccache's "relative path storage treatment" should be enabled. It is *not* used to calculate the relative paths: The relative paths (for files in CCACHE_BASEDIR or its subdirectorues) are always calculated relative to the $CWD (of the ccache call). So if CCACHE_BASEDIR is /var/tmp/portage and you are calling ccache's wrapper from the directory /var/tmp/portage/foo/bar-version/workdir by the command g++ /var/tmp/portage/foo/bar-version/workdir/something/my.cc ccache will act as if you have used the command g++ something/my.cc The CCACHE_BASDIR is only used to check *whether* this transformation should be made (i.e. not *how* it is done). Summarizing: It is optimal to set CCACHE_BASEDIR to a directory as "high" as possible - only with the restriction that system files should not be inside. (E.g. "/" is "too" high since also include paths are transformed, and you will probably want that e.g. /usr/include/stdio.h is cached by its absolute path and not relative to $CWD). (In reply to comment #2) > Well, in any case, does that export location work for you? I haven't tried yet, but I would suggest to export it even if ccache is not in features: Also for people using PATH="/usr/lib/ccache/bin:/usr/bin/:..." globally (and thus not explicitly activating FEATURES=ccache) it makes sense that this variable is exported. (In reply to comment #3) > The meaning of CCACHE_BASEDIR is only to define for which files > ccache's "relative path storage treatment" should be enabled. > It is *not* used to calculate the relative paths: The relative paths > (for files in CCACHE_BASEDIR or its subdirectorues) are always calculated > relative to the $CWD (of the ccache call). Ok, that's what I needed to know. > > Well, in any case, does that export location work for you? > > I haven't tried yet, but I would suggest to export it even if ccache is > not in features: Also for people using > PATH="/usr/lib/ccache/bin:/usr/bin/:..." > globally (and thus not explicitly activating FEATURES=ccache) it makes sense > that this variable is exported. Let me quote Zac, ``if users are doing that, they can export CCACHE_BASEDIR themselves''. So, I'd appreciate if you could test my patch and let me know so we could push it into portage. (In reply to comment #4) > Let me quote Zac, ``if users are doing that, they can export CCACHE_BASEDIR > themselves''. Not really: It makes sense to include ccache in PATH globally (i.e. for all applications), but it does not make sense to set CCACHE_BASEDIR=/var/tmp/portage globally. The latter makes only sense when things inside this directory are compiled (i.e. when portage is running). > So, I'd appreciate if you could test my patch It does *not* seem to work, although I do not understand it: The attached ebuild does not print any CCACHE_BASEDIR, although PATH is set correctly in all phases. Moreover, if I set CCACHE_BASEDIR in /etc/portage/make.conf it becomes printed, although I guess it should have been overridden by the patch. I appears that the patch has no effect at all... Created attachment 255029 [details]
Dummy ebuild for testing
Looking at the cache man page, the CCACHE_BASEDIR variable seems non-essential and can even produce some negative effects as described here: • If you specify an absolute path to the source code file and compile with -g, the source code path stored in the object file may point to the wrong directory, which may prevent debuggers like GDB from finding the source code. Sometimes, a work-around is to change the directory explicitly with the “cd” command in GDB. Couldn't that interfere with things like splitdebug and installsources FEATURES? Perhaps it's better not to assume that this variable should be set, and leave it to the user to set it in bashrc if desired? Or, we could provide some way to control portage's behavior. It seems like it would be handy if ccache provided a way to exclude the file path from the hash. Isn't that how older cache behaved? (In reply to comment #8) > It seems like it would be handy if ccache provided a way to exclude the file > path from the hash. This is exactly what CCACHE_BASEDIR does (it does nothing else!). Only it does not strip the full path but only "up to the relative part" which is even more reliable, since in this way it can distinguish e.g. between a local (to the project) stdio.h file and the global /usr/include/stdio.h in the hash. (In reply to comment #7) > Couldn't that interfere with things like splitdebug and installsources > FEATURES? Not really: The paths after "installsources" are anyway not those used in the compilation call, i.e. they are not ${S}/.... (= /var/tmp/portage/...) but instead /usr/src/debug/..., so for tasks for which gdb relies on the absolute path name used during compilation you cannot use these sources anyway unless you manually copy them back to the original ${S} (or if you work with FEATURES=keepdir). > leave it to the user to set it in bashrc if desired? > Or, we could provide some way to control portage's behavior. I am not against some way to control portage's behavior, but IMHO setting it should be the default case, since the cases where you might not want it are very exceptional. You might also argue that somebody who does *not* want it can unset it (or set it to the empty string) in make.conf or bashrc. Another way might be to set (and document that normally it should be set) in make.conf (this is how I did it: Fortunately, the variables set there are automatically exported). In the latter case, it is a question of documenting it, since most users will not know that usually they want it :) Originally, I had documented in a postinst message in my ccache ebuild in my overlay that you should set it in make.conf, but this feels like a hack, and as mentioned in bug 313101, also the ccache maintainer thinks that this should better be handled by portage itself. So if you do not want to add some code, I suggest to at least add some documentation and code in make.conf.example, something like: # Usually you want to set this if you use >=ccache-3: CCACHE_BASEDIR="${PORTAGE_TMPDIR}/portage" (In reply to comment #9) > Not really: The paths after "installsources" are anyway not those used in > the compilation call, i.e. they are not ${S}/.... (= /var/tmp/portage/...) > but instead /usr/src/debug/..., so for tasks for which gdb relies on the > absolute path name used during compilation you cannot use these sources > anyway unless you manually copy them back to the original ${S} > (or if you work with FEATURES=keepdir). It's supposed to work with the sources directly in /usr/src/debug/, thanks to this line in /usr/lib/portage/bin/ebuild-helpers/prepstrip: debugedit -b "${WORKDIR}" -d "${prepstrip_sources_dir}" \ -l "${T}"/debug.sources "${x}" (In reply to comment #10) > debugedit -b "${WORKDIR}" -d "${prepstrip_sources_dir}" \ > -l "${T}"/debug.sources "${x}" Thanks. I did not know about debugedit before. So indeed, if you want to use this command you should avoid to strip the path for calculating the hash so that ccache does not reuse an old .o file which may contain an "old" ${WORKDIR}. This can be solved with modern gcc and clang using the -fdebug-prefix-map option. As an added benefit, CCACHE_HASHDIR no longer needs to be exported in that case. |
Since >=ccache-3 the variable CCACHE_BASEDIR is supported (causing ccache to treat subdirectories under this directory relative to their parent). For emerge actions, the only useful value of this variable is the directory CCACHE_BASEDIR="${PORTAGE_TMPDIR}/portage" With this value, build systems using absolute filenames will not automatically cause cache misses for version changes (since under portage the absolute compilation path always contains the version number...). Of course, it would not be reasonable to make this the global system default (e.g. if ccache is used outside of portage, this value makes no sense). Hence, I suggest that portage exports the above variable unconditionally: It makes no harm, if ccache is not used (or if earlier versions of ccache are used), but if ccache gets used (either by FEATURES=ccache or by including /usr/lib*/ccache/bin in $PATH), this is the only reasonable value during emerge. See also the discussion in bug 313101. I work with ccache-3 and the above variable in make.conf since several months (and since at least one emerge -e @world) without problems on x86 and amd64.