Email from mbp@samba.org, ======= Hi, I was just looking at the current gentoo distcc bugs. It seems like some problems might be avoided if gentoo builds used an explicitly versioned compiler name, like gcc-3.3.1. That would trap any mismatched versions right away, and be both simpler and more reliable than trying to have distcc check them. It would also allow for machines to have several versions of gcc installed and their clients would use whichever one is appropriate. I certainly find that useful with machines which are on slightly different releases. The main catch here would be that masquerading alone might not be sufficient. You may need to explicitly set CC/CXX, and it will only work for packages that properly obey those variables. But I suspect that applies to most packages. ======= Have at it. ;)
Any noise on this? I won't set this by default without people testing and giving me directions, issues, and such... I'd prefer if distcc just compared the 'dumpversion' strings of the compilers.
this isnt so much a portage bug as a gcc-config bug ... we have gcc install entries that use full CHOST-gcc in the CC/CPP environment theres a few users floating around who have been doing this for sometime w/out issues and have been submitting patches when there are
I may think that there could be issues where it might cause cross-compile weirdness (or more to the point, where the configure thinks it should cross-compile - advancemame comes to mind). I can however bump a new gcc-config that do the 'right' thing regarding CC and CXX without changing all gcc ebuilds if nobody know about an issue that could be critical?
azarah: try doing that but package.mask it and at least i will give it a run through by doing `emerge world -e`
I have thought about this again - is there any way to figure out if distcc or ccache is running? I am not asking if there is a script to run, rather an env variable or such. If so, then I can modify the wrapper to rather exec the full binary (chost+version) if either are detected ...
Based on the PATH and envvars we should be able to tell what is running. If /usr/lib/ccache/bin is called first, and CCACHE_DISABLE is not set (='0') then ccache is called. Likewise, if /usr/lib/distcc/bin is before the path where gcc lives then distcc is being used.
Ok, could we change both to say export DISTCC=1 or CCACHE=1 if they are in use?
I think I have a plan to fix this in a future version of distcc, so you don't have to do it in Gentoo if you don't want to. Basically the distcc client will (optionally) convert the compiler name into a fully-qualified name before sending it to the server. So even if ebuild runs with CC=gcc, that will get turned into ia64-linux-gcc-3.3 on the server, avoiding the problem.
That will work. I must admit I have not given this much thought because of other things, until the other day. Then I relised that you have 3 options of fixing this: 1) Set CC to CHOST-gcc-VER. I do not like this though, as it will prob break on many apps, and they have to be fixed (sure, it is prob already done, but it just do not 'feel' right). 2) Do it like you said (ie distcc side). Problem then is that you still sit with ccache, etc. Less issues to fix though, but not 100% definate that upstream will accept. 3) Do it wrapper side. You still have the detection side though. Another issue is that the actual bin dir might be before the wrapper in the PATH, and thus break things again (as the wrapper will not be able to correct which gcc is run). So it seems 2) will be the most fool proof path to take. Except if I missed anything?
I agree that (1) is probably risky. I don't understand what you mean by "Problem then is that you still sit with ccache, etc." Compiler versions are not a big deal for ccache because it is purely local. It checks the mtime and size of the compiler frontend to see whether it has been upgraded. "Less issues to fix though, but not 100% definate that upstream will accept." Which upstream? I am the distcc upstream. If it works, I will put it in.
I think I got ccache into the picture by accident. I will see if I get time to come up with a patch (if you do not before me).
Martin is this bug still valid?
yes...i use the approach of setting my full compiler name in /etc/env.d/gcc/powerpc-unknown-linux-gnu-3.4.1 It *does* work flawlessly with the actual distcc toolchain via my athlon-xp crossdev cluster...however, when running ./configure scripts it is *abominably* slow. portageq uses up 100% cpu frequently and distcc keeps churning over and over. It lengthens the ./configure time by a factor of 5 or more. python 2.3.3 took TWENTY FIVE MINUTES to ./configure on my 1.2 GHz G4. Hopefully there is a work around :-/ What good is distcc if ./configure takes longer than one-computer make?
this is done now in unstable
Unfortunately there's a bug with this and Distcc (and possibly ccache): make: Entering directory `/home/gentoo/tmp/portage/baselayout-1.11.7-r2/work/rc-scripts-1.6.7/src' i686-pc-linux-gnu-gcc -march=pentium4 -Os -pipe -c -o consoletype.o consoletype.c distcc[2695] ERROR: compile /home/gentoo/tmp/ccache/consoletyp.tmp.lisa.2691.i on 192.168.0.2/4 failed /usr/bin/gcc-config: line 1: /proc/cmdline: Permission denied /usr/bin/gcc-config: line 1: /proc/cmdline: Permission denied /usr/bin/gcc-config: line 1: /proc/cmdline: Permission denied Could not run/locate "i686-pc-linux-gnu-gcc" make: *** [consoletype.o] Error 1 make: Leaving directory `/home/gentoo/tmp/portage/baselayout-1.11.7-r2/work/rc-scripts-1.6.7/src' 192.168.0.2 is set up as a i386 machine. From what I've overheard in #gentoo-portage it seems this only happens with applications that use toolchain. I will have to look into this further. From /var/log/daemon.log on 192.168.0.2:: Dec 8 02:11:49 www distccd[2916]: (dcc_check_client) connection from 192.168.0.254:49750 Dec 8 02:11:49 www distccd[2916]: (dcc_check_address) match client 0xfe00a8c0, value 0xa8c0, mask 0xffffff Dec 8 02:11:49 www distccd[2916]: (dcc_r_token_int) got DIST00000001 Dec 8 02:11:49 www distccd[2916]: (dcc_r_token_int) got ARGC00000008 Dec 8 02:11:49 www distccd[2916]: (dcc_r_argv) reading 8 arguments from job submission Dec 8 02:11:49 www distccd[2916]: (dcc_r_token_int) got ARGV00000015 Dec 8 02:11:49 www distccd[2916]: (dcc_r_argv) argv[0] = "i686-pc-linux-gnu-gcc" Dec 8 02:11:49 www distccd[2916]: (dcc_r_token_int) got ARGV0000000f Dec 8 02:11:49 www distccd[2916]: (dcc_r_argv) argv[1] = "-march=pentium4" Dec 8 02:11:49 www distccd[2916]: (dcc_r_token_int) got ARGV00000003 Dec 8 02:11:49 www distccd[2916]: (dcc_r_argv) argv[2] = "-Os" Dec 8 02:11:49 www distccd[2916]: (dcc_r_token_int) got ARGV00000005 Dec 8 02:11:49 www distccd[2916]: (dcc_r_argv) argv[3] = "-pipe" Dec 8 02:11:49 www distccd[2916]: (dcc_r_token_int) got ARGV00000002 Dec 8 02:11:49 www distccd[2916]: (dcc_r_argv) argv[4] = "-c" Dec 8 02:11:49 www distccd[2916]: (dcc_r_token_int) got ARGV00000002 Dec 8 02:11:49 www distccd[2916]: (dcc_r_argv) argv[5] = "-o" Dec 8 02:11:49 www distccd[2916]: (dcc_r_token_int) got ARGV0000002c Dec 8 02:11:49 www distccd[2916]: (dcc_r_argv) argv[6] = "/home/gentoo/tmp/ccache/tmp.hash.lisa.2691.o" Dec 8 02:11:49 www distccd[2916]: (dcc_r_token_int) got ARGV00000032 Dec 8 02:11:49 www distccd[2916]: (dcc_r_argv) argv[7] = "/home/gentoo/tmp/ccache/consoletyp.tmp.lisa.2691.i" Dec 8 02:11:49 www distccd[2916]: (dcc_r_argv) got arguments: i686-pc-linux-gnu-gcc -march=pentium4 -Os -pipe -c -o /home/gentoo/tmp/ccache/tmp.hash.lisa.2691.o /home/gentoo/tmp/ccache/consoletyp.tmp.lisa.2691.i Dec 8 02:11:49 www distccd[2916]: (dcc_scan_args) scanning arguments: i686-pc-linux-gnu-gcc -march=pentium4 -Os -pipe -c -o /home/gentoo/tmp/ccache/tmp.hash.lisa.2691.o /home/gentoo/tmp/ccache/consoletyp.tmp.lisa.2691.i Dec 8 02:11:49 www distccd[2916]: (dcc_scan_args) found object/output file "/home/gentoo/tmp/ccache/tmp.hash.lisa.2691.o" Dec 8 02:11:49 www distccd[2916]: (dcc_scan_args) found input file "/home/gentoo/tmp/ccache/consoletyp.tmp.lisa.2691.i" Dec 8 02:11:49 www distccd[2916]: compile from consoletyp.tmp.lisa.2691.i to tmp.hash.lisa.2691.o Dec 8 02:11:49 www distccd[2916]: (dcc_run_job) output file /home/gentoo/tmp/ccache/tmp.hash.lisa.2691.o Dec 8 02:11:49 www distccd[2916]: (dcc_input_tmpnam) input file /home/gentoo/tmp/ccache/consoletyp.tmp.lisa.2691.i Dec 8 02:11:49 www distccd[2916]: (dcc_r_token_int) got DOTI00007bab Dec 8 02:11:49 www distccd[2916]: (dcc_r_file) received 31659 bytes to file /tmp/distccd_c206a935.i Dec 8 02:11:49 www distccd[2916]: (dcc_r_file_timed) 31659 bytes received in 0.001309s, rate 23619kB/s Dec 8 02:11:49 www distccd[2916]: (dcc_set_input) changed input from "/home/gentoo/tmp/ccache/consoletyp.tmp.lisa.2691.i" to "/tmp/distccd_c206a935.i" Dec 8 02:11:49 www distccd[2916]: (dcc_set_input) command after: i686-pc-linux-gnu-gcc -march=pentium4 -Os -pipe -c -o /home/gentoo/tmp/ccache/tmp.hash.lisa.2691.o /tmp/distccd_c206a935.i Dec 8 02:11:49 www distccd[2916]: (dcc_set_output) changed output from "/home/gentoo/tmp/ccache/tmp.hash.lisa.2691.o" to "/tmp/distccd_c3f4a935.o" Dec 8 02:11:49 www distccd[2916]: (dcc_set_output) command after: i686-pc-linux-gnu-gcc -march=pentium4 -Os -pipe -c -o /tmp/distccd_c3f4a935.o /tmp/distccd_c206a935.i Dec 8 02:11:49 www distccd[2916]: (dcc_check_compiler_masq) /usr/bin/i686-pc-linux-gnu-gcc is not a symlink Dec 8 02:11:49 www distccd[2916]: (dcc_spawn_child) forking to execute: i686-pc-linux-gnu-gcc -march=pentium4 -Os -pipe -c -o /tmp/distccd_c3f4a935.o /tmp/distccd_c206a935.i Dec 8 02:11:49 www distccd[18410]: (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1 Dec 8 02:11:49 www distccd[2916]: (dcc_spawn_child) child started as pid18410 Dec 8 02:11:50 www distccd[2916]: (dcc_collect_child) cc child 18410 terminated with status 0x100 Dec 8 02:11:50 www distccd[2916]: (dcc_collect_child) cc times: user 0.070000s, system 0.030000s, 1813 minflt, 906 majflt Dec 8 02:11:50 www distccd[2916]: (dcc_x_token_int) send DONE00000001 Dec 8 02:11:50 www distccd[2916]: (dcc_x_token_int) send STAT00000100 Dec 8 02:11:50 www distccd[2916]: (dcc_x_file) send 231 byte file /tmp/distcc_8791a935.stderr with token SERR Dec 8 02:11:50 www distccd[2916]: (dcc_x_token_int) send SERR000000e7 Dec 8 02:11:50 www distccd[2916]: (dcc_x_file) send 0 byte file /tmp/distcc_876ba935.stdout with token SOUT Dec 8 02:11:50 www distccd[2916]: (dcc_x_token_int) send SOUT00000000 Dec 8 02:11:50 www distccd[2916]: (dcc_x_token_int) send DOTO00000000 Dec 8 02:11:50 www distccd[2916]: i686-pc-linux-gnu-gcc /home/gentoo/tmp/ccache/consoletyp.tmp.lisa.2691.i on localhost failed Dec 8 02:11:50 www distccd[2916]: job complete Dec 8 02:11:50 www distccd[2916]: (dcc_cleanup_tempfiles) deleted 4 temporary files
ok, i'm pretty sure the /usr/bin/gcc-config: line 1: /proc/cmdline: Permission denied is just a red herring here the way to reproduce is to have distccd start up at boot time ... if you ssh in and run `/etc/init.d/distccd restart`, the env will be different and the bug wont appear it seems like at boot time, $PATH doesnt contain the additional entries that it should ... the gcc wrappers then resort to running gcc-config, but that seems to fail too ...
Then it seems that to correct the distccd init script would be to specify the explicit path to gcc-config (instead of relying on the pre-set PATH). I'm not sure there's any way to test this without restarting a machine.
Summary renamed to be a bit more useful.
to be more specific, this problem seems to be an issue of running "i686-pc-linux-gnu-gcc" and "i586-pc-linux-gnu-gcc"
i doubt this will be fixed. But a workaround (nasty hack) is specified in the Gentoo Distcc Guide.
Hey... I've got a few extra machines bouncing around that I help administer for school if you want me to test anything out on them, I'm a willing victim. Just let me know what to go for and what you want tested.
