genone recently mentioned a suggestion that came up amongst the portage devs and on IRC, of moving the automatic management of ccache and distcc from portage into the gcc wrapper, so that portage can care nothing about the compiler. Additionally I've been considering also injecting -specs directives via the wrapper instead of using the GCC_SPECS modification we add to gcc. These two ideas dovetail nicely together. Getting the wrapper to modify the command to include -specs directives on its own would have been problematic, as ccache wouldn't know what the wrapper was going to get up to. However integrating use of ccache with the wrapper means that ccache understands the specs in a natural fashion, without needing any Gentoo-specific hack either to ccache (see bug #121336) or to gcc (i.e. the GCC_SPECS patch could be dropped from the compiler, although that's not necessary). To follow; experimental patches against the wrappers in gcc-config and eselect-compiler. distcc and/or ccache are enabled by having them in the environment variable COMPILER_FEATURES (much like portage' FEATURES). Note the wrappers also append the relevant paths to the sandbox variables, as portage currently does when enabling ccache/distcc. Obviously remove ccache/distcc from FEATURES before playing with this and portage otherwise you'll get multiple invocations of ccache/distcc which will probably make a mess. Both wrappers process GCC_SPECS from the environment if it's set (the gcc-config way); additionally eselect-compiler will get the specs from its config files if GCC_SPECS is not set (the eselect-compiler way). There are a couple of things on the TODO: 1) the gcc-config patch doesn't inhibit -specs injection for cross-compilers (the eselect-compiler one does). Not sure how to detect that in the gcc-config wrapper; there's some recursive voodoo that looks like it detects cross-compilers but I'm not clear how that works yet :) 2) paths to the distcc and ccache binaries are hard-coded and not checked for existence/executability
Created attachment 83908 [details, diff] Patch against CVS compiler-wrapper.c v1.8
Created attachment 83909 [details, diff] Patch against 1.4.7c of gcc-config's wrapper
Azarah is the primary author of the gcc-config. I know he has been busy as of late and this might take a bit of manual poking.
Created attachment 84032 [details, diff] Patch against CVS compiler-wrapper.c v1.8 Whoops. off-by-one error.
Created attachment 84033 [details, diff] Patch against 1.4.7c of gcc-config's wrapper
we can just encode the CHOST value when gcc-config was built and compare against that at runtime
I can understand the SPECS stuff - I however am not sure about the distcc/ccache stuff. My personal opinion would have been to not add that, as it adds the notion of the wrappers having to know about sandbox and portage variables that ferringb at least at the time was bitching not to do, as they would change or be removed. I would have said that once again the user should setup his path properly to have the wrappers before /usr/bin, and that distcc/ccache/colorgcc should then remove the path where they are located from PATH, and call the next gcc in the chain. You could then say 'what about SANDBOX setup???'. This I would say is really a Gentoo-{distcc,ccache} specific issue, and should either be handled by them, or if need be, I can try to get the /etc/sandbox.d/ stuff I wanted to add done (or maybe a $HOME/.sandbox.cfg if need be for user specific setup). Anyhow, just my POV, Mike/others guess should also comment, as I am not sure what the recent goals for gcc-config was, and really have NFI about compiler-config or whatever that other thing was called.
(In reply to comment #7) > I would have said that once again the user should > setup his path properly to have the wrappers before /usr/bin The problem with this, for me, is that ccache doesn't see the -specs directives. My reason for suggesting expanding GCC_SPECS into -specs directives in the wrapper, was so that ccache could see them. ccache includes -specs files in its checksum calculation, but obviously doesn't include anything listed in GCC_SPECS (see bug #121336). As it currently stands, switching specs causes ccache to provide objects built with the previous specs setting, which causes breakage. distcc also pays attention to -specs directives, in that it requires anything using -specs to be compiled locally (it'd be nice if it could send over the specs files as well, but that's another story ;) ). re. sandbox - an /etc/sandbox.d/ or /etc/sandbox.conf or ~/.sandboxrc or whatever would be great; I agree fiddling with the sandbox envvars in the wrapper isn't nice. Could then be more generic in the wrapper, e.g. it could prefix with anything in COMPILER_PREFIX - this would support whatever compiler prefixers the user chooses (and we could add COMPILER_PREFIX to emerge --info).
I think if its agreed to add these other stuff due to specs considerations, we should agree to only add wrapper-side support if they need to know the specs, and not for anything some user want (colorgcc for example do not belong in the wrapper for example). Also, you probably did think about this, but what if the user without knowing the wrapper do the distcc/ccache setups his paths to support those anyhow? You could end up with a call chain like: distcc -> ccache -> wrapper -> distcc -> ccache -> gcc or whatever. Point being, not nice.
(In reply to comment #9) > I think if its agreed to add these other stuff due to specs considerations, we > should agree to only add wrapper-side support if they need to know the specs, > and not for anything some user want (colorgcc for example do not belong in the > wrapper for example). It'll only happen if they're specified in the envvar (COMPILER_FEATURES, COMPILER_PREFIX, whataver - open to a better name for that, btw - perhaps COMPILER_WRAPPERS), which by default would be unset. Those setting up paths instead, would leave the envvar unset (and would need to flush the cache manually when switching specs themselves, which is how it currently has to be done). > Also, you probably did think about this, but what if the user without knowing > the wrapper do the distcc/ccache setups his paths to support those anyhow? > You could end up with a call chain like: > > distcc -> ccache -> wrapper -> distcc -> ccache -> gcc It's a good point; I didn't think about it as such, mostly figuring if someone sets the env-var, they're not going to set up paths as well. Patches above don't catch it at the moment, since by the time the wrapper is called the distcc/ccache from the original command is removed. I can't think of a neat way of picking that up without modifying ccache/distcc (simple but undesirable from the point of view of maintenance of those packages), or scanning the process hierarchy in /proc - open to suggestions of course :)
Created attachment 86738 [details, diff] Patch against 1.4.7c of gcc-config's wrapper Fix logic error typo ( '&' => '&&' ).
(In reply to comment #10) > (In reply to comment #9) > > > Also, you probably did think about this, but what if the user without knowing > > the wrapper do the distcc/ccache setups his paths to support those anyhow? > > You could end up with a call chain like: > > > > distcc -> ccache -> wrapper -> distcc -> ccache -> gcc > > It's a good point; I didn't think about it as such, mostly figuring if someone > sets the env-var, they're not going to set up paths as well. Patches above > don't catch it at the moment, since by the time the wrapper is called the > distcc/ccache from the original command is removed. I can't think of a neat > way of picking that up without modifying ccache/distcc (simple but undesirable > from the point of view of maintenance of those packages), or scanning the > process hierarchy in /proc - open to suggestions of course :) > Forgot to reply ... possibly an answer (/proc). I do not know distcc/ccache well enough to know if they set some vars that you can test if they already ran or not though. I think however that at some stage before this goes in, everybody have to sit down and decide exactly how the wrappers is supposed to respond - ie, should 'gcc' (no host added) call the next gcc in PATH, or the default for for the current host, or the next gcc in PATH for the current host. I thought at the time that it should check the first gcc in PATH, as you can still then change what gcc is used besides the default, but I really did it that way as I did not have anybody experienced in cross-compiling at the time to talk with. Not sure if Mike fixed it up like he wants for cross-compiling by now.
A call to 'gcc' and not ctarget-gcc should always return the host-gcc. This is the expected behavior of cross compile envioronments imho.
I suppose you meant chost-gcc in your last comment, as ctarget is valid only for compilers/linkers/assemblers/whatever-generates-code, if we're following autoconf conventions. Anyway, I hope you won't code gcc-config to check /proc, as that's unportable. And yes, gcc should call cbuild-gcc not chost-gcc when crosscompiling, but it wouldn't be wrong if all ebuilds called the right gcc command while building.
I'm really not going to just leave this open since there is absolutely no movement on this (since 2006). If you want to "fix" this, then please do so.
