| Summary: | Simple distcc wrapper with toggle | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Eric Andresen <ndiin1> |
| Component: | [OLD] Development | Assignee: | Zach Welch (RETIRED) <zwelch> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | azarah, bcowan, converter42, gentoo, hannes, kevinc, mbp, stormlabs, ykoehler, zwelch |
| Priority: | High | ||
| Version: | 1.4_rc1 | ||
| Hardware: | x86 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | |||
| Bug Blocks: | 15048 | ||
| Attachments: |
wrapper tarball, see first post instructions
distcc-wrapper.tbz2 Fixed distcc-gcc-config-wrapper.tbz2 Full Replacement of above. See comments below bugfix for above. distcc-gcc-config-wrapper-r1.tbz2 distcc-gcc-config-wrapper-r2.tbz2 Fixed wrapper.c gcc-wrapper.tbz2 gcc-wrapper-r1.tbz2 gcc-wrapper-r2.tbz2 *feature enhancement* gcc-wrapper-r3.tbz2 distcc-1.1-r1.tar.bz2 distcc-1.1-r1.tar.bz2 distcc-1.1-r2.tar.bz2 distcc-1.1-r6.tar.bz2 distcc-1.1-r7.tar.bz2 |
||
|
Description
Eric Andresen
2003-01-14 01:41:56 UTC
Created attachment 7287 [details]
wrapper tarball, see first post instructions
Generic ccache support can easily be added, just change the wrapper scripts (/usr/bin/distcc-wrapper/gcc & g++) to use /usr/bin/ccache/ccache distcc gcc $* and g++ instead =) One thing I forgot to mention, this assumes that DISTCC_HOSTS is set elsewhere, but it can be set in /etc/profile.d/distcc as well I suppose. support for this in distcc and portage is a must.. it works! portage should check the path for distcc. also modify the script to check for ccache (and use it) we should get this in before 1.4 I've found that this wrapper works fine in a non-portage environment, but portage 2.0.46-r6 (And probably other versions) changes the path at line 137 of /usr/lib/portage/bin/ebuild.sh. There are two solutions: 1) Probably the easiest: Move the wrapper scripts to /usr/local/bin instead of /usr/bin/distcc-wrapper (The line in question uses $ROOTPATH, which /usr/local/bin is the first thing listed in by default) 2) Prefix the path on that line with /usr/bin/distcc-wrapper Either of these solutions will fix the problem. I'd recommend the first, since you do not have to worry about portage upgrades. Created attachment 7321 [details]
distcc-wrapper.tbz2 Fixed
This replacement tarball moves the wrapper scripts from /usr/bin/distcc-wrapper
to /usr/local/bin, and changed the scripts to check if you want distcc enabled
themselves, rather than depending on the PATH exclusively. This should take
care of all known problems.
*sigh*, fails on one part of XFree 4.2.1-r2 compile because of the way 1) the build uses \""Blah blah"\"... Which once passed through bash once, is no problem, but wrapping it again causes it to be seen as an empty string with extra garbage. I can't think of a solution, but to change the way the Makefile does it, or temporarily move the wrappers out of the path. Found another package that does the same thing.. ncompress I can't think of any solutions other than to temporarily move the wrappers out of the PATH... if anyone can think of something, let me know please! =) Another thing I should probably mention: If you use this same setup on multiple systems, you should use the modified init.d/distccd script as described in http://bugs.gentoo.org/show_bug.cgi?id=13625#c2 to prevent the wrapper from being called on the volunteer system as well. Created attachment 7352 [details]
distcc-gcc-config-wrapper.tbz2 Full Replacement of above. See comments below
This .tbz2 uses the new wrapper format from >=gcc-config-1.2.9
I've set it up to be more of a standard tarball, do not extract it to your root
drive anymore. Extract and run ./install for more info and installation.
Cheers!
.tbz2 contents:
distcc-wrapper/
distcc-wrapper/wrapper.c
distcc-wrapper/switch-distcc
distcc-wrapper/distcc
distcc-wrapper/install
distcc-wrapper/README
distcc-wrapper/init.d/
distcc-wrapper/init.d/distccd
Created attachment 7355 [details]
bugfix for above. distcc-gcc-config-wrapper-r1.tbz2
Oops, I got it working fine with distcc on, but forgot to fix something for
when 'switch-distcc' is 'off'. This fixes that. Enjoy!
Azarah, can u take a look at this....he has hacked your gcc wrapper to add distcc support. I can take care of the distcc end, but the gcc part is all yours :) Thanks Brad 1) Why does it use GCCBINPATH ? What is the whole point of getting wrapperbin if it does not use it ? 2) What is wrong with setting CC="distcc gcc" or however you did it before ? 1) I'll attach a new version tomorrow, I just wanted to get it working today for one of my machines.. Uses GCCBINPATH which it gets from /etc/profile.d/distcc to prevent the excess processing time for gcc-config to parse for the path every single time 'gcc' is called, which adds up quite quickly. Besides, if a user changes their gcc, they'll need to source /etc/profile again for it to work anyway. Doing it this way saves significant overhead IMO. 2) CC="distcc gcc" doesnt work for everything/many things without modifiying Makefiles by hand in some cases. By using a wrapper with the same name as the compiler that matches in the PATH sooner, it is guarunteed to be called. I'll clean it up and post a new version tomorrow. Just my 2 cents...about 25% of source breaks during compile with CC="distcc gcc" or CXX="distcc g++" partially due to libtool, and ALOT of source doesn't even recognize CC. This wrapper takes care of most all of those problems. Afaik X is one of the packages that doesn't recognize CC and last I checked mozilla too...these are some of the biggest compile time sources we as gentoo have and are examples of what distcc and ccache together should be used for. Created attachment 7359 [details]
distcc-gcc-config-wrapper-r2.tbz2
So I couldn't sleep, is that a crime? ;)
This should clean it up a bit, and completely removes the use of gcc-config
--get-bin-path in favor of the $GCCBINPATH variable. Also moved around
everything so none of the tokenizing takes place if it doesnt need to (w/
distcc on).
Hello? Do you understand how it works ? How it is now, it will only run gcc-config if PATH are not setup. No need for an extra non standard env variable ... why have gcc-config anyhow if you are going to start to use different env-variables all over the place ? Anyhow, I will not be adding this to gcc-config's wrapper, as this is not wat it was intended for. Do it like ccache or colorgcc .... add wrappers that calls 'distcc /usr/bin/gcc' and gcc-config's wrappers will do the rest: -----------/usr/bin/distcc/gcc----------------- #!/bin/sh exec /usr/bin/distcc/distcc /usr/bin/gcc "$*" ------------------------------------------------ Simple bash example that could be rewritten to C for speed. So no (like ccache) just add /usr/bin/distcc/ first in PATH. Created attachment 7386 [details]
Fixed wrapper.c
I agree that my change of looking for the gcc-config binpath to using
GCCBINPATH for when distcc is off was very idiotic, and thus changed. The use
of GCCBINPATH is 'hackish', but I see no harm in using it if you fall back on
gcc-config if it is not set.
My goal, however, is to attempt to unify known gcc wrappers into one wrapper
which would first of all make sure they are used in the correct order (eg.
distcc has to be last to gain the benefit of any other gcc wrappers) as well as
make switching the use of a wrapper on/off very easy, without changing the
environment to do that switch. By not relying on the environment, you then turn
the use of a particular wrapper off mid-build with no problems arising, which
can come in handy.
I realize this is not what the gcc-config wrappers were meant to do originally,
but this can eventually simplify the gcc wrapper problems significantly.
hey, i did all that with the latest wrapper.c and got this errors while trying to merge irssi.. would be nice if you can fix it.. -- checking whether the C compiler (gcc -march=i586 -O3 -pipe ) works... no configure: error: installation or configuration problem: C compiler cannot create executables. !!! ERROR: net-irc/irssi-0.8.6-r1 failed. !!! Function src_compile, Line 37, Exitcode 1 !!! ./configure failed --------------------------- ACCESS VIOLATION SUMMARY --------------------------- LOG FILE = "/tmp/sandbox-irssi-0.8.6-r1-9287.log" open_wr: /var/log/distcc.log open_wr: /var/log/distcc.log -------------------------------------------------------------------------------- The problem seems to be that DISTCC_LOG is set to an area offlimit to the sandbox. I cannot think of a way to get around this without sandbox giving permissions... And making the wrapper simply unset that environment variable wouldn't be a good solution either, because its only a problem if used in portage. So for now, I guess the best solution is to not use DISTCC_LOG. If I think of something better, I'll post it, or if you have any ideas, please let me know. it's working great here, but I don't have any logging on....with the lastest version is /etc/profile.d/distcc still needed and sourcing that file? another thing :) it is so nice to see all these sources taking advantage of distcc...seeing 2 athlon XP1800 boxes a 1.2 athlon box and a 1 gig duron box all at 100% on some sources is amazing :) Sourcing /etc/profile.d/distcc with the newest wrapper.c is not *necessary*, but it reduces the number of times that gcc-config --get-bin-path is called from "every time" to "only when source'ing /etc/profile". But it will work fine even if /etc/profile.d/distcc was not sourced. I should have replied some time back already. Currently I still need to get 1.3.0 into stable ... not even to talk about -r6 or later of gcc :/ Thus I am not even going to consider adding this. If you like, please consider the following: Start working on a wrapper that calls /usr/bin/gcc (g++, or whatever), and let my wrapper worry about the correct locations for the moment. Get this wrapper to correctly detect, and use colorgcc, ccache and distcc. Get it stress tested. We can then after 1.4 or there abouts think about integrating it. Created attachment 7476 [details]
gcc-wrapper.tbz2
Here is the latest copy of this wrapper. It no longer makes use of any
non-standard "GCCBINPATH" environment variable, and has support for ccache in
addition to distcc. It makes use of a new script for toggling usage of each
wrapper, named 'toggle-wrapper'. Feel free to remove /etc/profile.d/distcc and
any lines refering to it in your /etc/profile, as they are no longer needed
(Assuming that DISTCC_HOSTS is set elsewhere). Also, /sbin/switch-distcc can be
removed if you were using one of the older versions.
Currently, this is still a replacement for the gcc-config wrapper. Doing it
this way prevents any possible user error, and ensures that it is used.
Colorgcc support is not included because the colorgcc script is "dumb" in the
fact it cannot currently support having the program to execute as a parameter.
I will include a copy of colorgcc with this wrapper once I finish patching it
to allow for this behavior.
Please let me know about any bugs encountered. Thanks!
Erm, it appears there's a strange bug in this latest release..
Scenario:
toggle-wrapper --distcc=off --ccache=on
# gcc -march=athlon-tbird -O3 -pipe -fomit-frame-pointer -fforce-addr
-falign-functions=4 -D__LINUX__ -I/usr/X11R6/include -D__LINUX__ -DUSE_DIRENT
-DUSE_LPR_PRINT_CMD -c -o DialogF.o DialogF.c
Could not run/locate !
It appears that somehow the variable that holds ccache's bin path is being
destroyed somehow.. adding a printf("%s\n", ccachebin); right after it was set
makes it work properly with this scenario, but then distccbin is killed.. I'll
look into it later, but for now, avoid using this version.
There is another bug I've also found that is unrelated (that I know of), so I'll
post a revision to that along with the other major problem.
I appologize for being hasty, but this problem only rears its nose when using a
large number of arguments, but it does not appear to be a fixed number.
Created attachment 7483 [details] gcc-wrapper-r1.tbz2 This should fix the above problems. See comment #25 for information on usage. Created attachment 7491 [details]
gcc-wrapper-r2.tbz2 *feature enhancement*
This version adds the ability to set DISTCC_HOSTS in a file.
Set them with: toggle-wrappers --hosts="host1 host2 etc"
They are overridden by DISTCC_HOSTS env variable. Doing it this way allows for
two things: 1) changing hosts midbuild 2) fixes possible occurances of the
environment being changed/modified.
And of course, it allows you to not bother adding an "export DISTCC_HOSTS" line
to your profile/login scripts.
As usual, please report any problems to me.
Created attachment 7602 [details] gcc-wrapper-r3.tbz2 I've completely rewritten the DISTCC_HOSTS reading (from file), and it should now not have any problems with bad memory allocation in a few instances. See comment #28 for usage of the DISTCC_HOSTS reading. cc me re comment #7 BTW, the solution to problems with spaces in filenames may be to use "$@" rather than $*. This is generally better practice unless you specifically want the string to be split. RTFM. I was pointed at this bug last night as I investigated distcc. I used the gcc-config wrapper as a starting point. I rewote the current 1.3.1 version to be cleaner and modular. Before adding Eric's changes to this new version, I realized distcc needed its own wrappers. I am attaching distcc-1.1-r1.tar.bz2 which includes a new distcc ebuild and support files that allow distcc and ccache to work seamlessly with portage. distcc is enabled by simply by adding 'distcc' to FEATURES and setting DISTCC_HOSTS in make.conf. This version uses a new C wrapper that is installed in the /usr/bin/distcc directory (with copies under /usr/bin/distcc/ccache). The wrapper, when run, figures out whether it's in distcc-only or ccache-distcc mode (based on the path or program name), locates the real tool using gcc-config, then runs the lot of things. No changes to gcc-config were ever required to fix this bug; however, the new version is probably worth reviewing for inclusion, as it should be more forward maintainable when we start looking at per-user compiler configurations. While the version here does not include the same level of functionality provided by Eric's wrapper, the solution is much cleaner, integrates well with portage, and provides a solid foundation for future development. I think it wise to considering including this version for wider testing at once, and I will continute to add back the additional functionality in the next version. This version has been tested and compile all of kde over 3 or 4 machines; aspell and xmms specifically were previously problematic without this wrapper. Created attachment 7767 [details]
distcc-1.1-r1.tar.bz2
New distcc ebuild, wrapper, etc.
Created attachment 7768 [details]
distcc-1.1-r1.tar.bz2
New distcc ebuild, wrapper, etc.
IMPORTANT: ignore the above two attachments... this post came right after the bugzilla upgrade and they did not post correctly. If you want to get the files, they will be available at: http://gentoo.superlucidity.net/local-portage/sys-devel/distcc-1.1-r1.tar.bz2 as soon as I fix my RAID array and get the web server restarted. I will post here once Bugzilla is fixed. The distcc-1.1-r1.tar.bz2 package has a bug in it in that it passes an absolute pathname to distcc for the compiler. This only works if the remote distccd-running host has installed the compiler in exactly the same place as the client (which is not the case for me). While one might consider this a bug in distcc (in that it might be better if it ignored absolute paths and ran executables only from the startup PATH), since that's not how it currently works, the wrapper must set the PATH and pass a non-absolute compiler name to distcc (which will either run it locally based on the newly-set PATH, or remotely based on distccd's startup PATH). I'm working on an email that I'm sending to the distcc mailing list that will suggest how I think this should be solved, so feel free to join in the discussion there (if you're interested in solving this for all of distcc and not just for gentoo). The "not installed in the same path" puts an interesting twist on the problem, and I have some more ideas for re-writing the wrapper to, as you say, modify the PATH and pass a relative path to distcc. This will help performance too. I'll hack out a new version soon. In the meantime, I've updated the distcc ebuild to fix a couple of issues: 1) add symlink from /bin/discc to /usr/bin/distcc/distcc to provide backward compatability 2) fix portage problem if DISTCC_LOG is set I will attach the new version shortly. For everyone's reference, I am putting together a document that will hopefully help chart the new distcc support in Gentoo: http://gentoo.superlucidity.net/xcompile.html This document, while focusing on cross compiling issues, includes a healthy section on distcc that really gives reason for much of the work. Keep in mind the ideas presented will be refined and revised over time as I learn what everyone really wants and is implementing. I look forward to hearing further feedback on these matters. Created attachment 7900 [details]
distcc-1.1-r2.tar.bz2
This version fixes the two issues listed in the last comment.
I think the wrapper you've adapted is overly complex. I'm advocating turning distcc into its own wrapper, and I have created a patch for distcc to accomplish this. The email I sent to the distcc mailing list hasn't gone through (for some reason), you can see a copy of it here, if you like: http://www.clari.net/~wayne/distcc.html I also don't like the precedent set by ccache of putting a directory named ccache into /usr/bin, so I'd like to see us avoid that in a distcc package. I created a new ebuild that puts the regular binaries (distcc and distccd) into /usr/bin and the symlinks for the wrapper version into /usr/lib/distcc: http://www.clari.net/~wayne/distcc-1.1-r1.tar.bz2 I also created a new ccache ebuild that follows this example (by using /usr/lib/ccache for its wrappers and putting the ccache executable directly into /usr/bin): http://www.clari.net/~wayne/ccache-2.1.1-r1.tar.bz2 You don't need to install this version of ccache to try out the modified distcc, but you'll have to tweak the following patch if you leave the old version: Finally, to get these two things to work in portage (if "ccache" and "distcc" are specified in the FEATURES), apply this simple patch to ebuild.sh: http://www.clari.net/~wayne/ebuild.sh.patch Doing this allows a user to select what items to use just by setting the PATH (and DISTCC_HOSTS, as needed). The old gcc-config wrapper should be restored before running this version. I really like this move. I've started hacking on the distcc code itself, starting this time with the wrapper patch work already done by Wayne. My latest efforts are centered here, particularly the wrapper-1.1.patch I've started to generate. This version is not complete yet, but I'll finish it up in short order. Right now, it's awaiting runtime debugging. I managed to talk carpaski into integrating the required support for this into Portage, in addition to the changes he was making to support the traditional CC=distcc tricks Working together, I think we synthesized the best of both patches. In the end, we have portage-2.0.46-r11 which gives us the needed integration for this new wrapper to work. The new distcc functionality requires at least the patches from Wayne (which are in my tree as -r3), or my new patch that will become -r4. However, I'll be moving some of this discussion to the distcc mailing list, as we seem to have just about resolved the issue this bug was opened to to track, right folks? I'll be opening some more bugs for the cross compile issues soon, but this one has almost run its course. Also, please find me in irc to talk about these things - that's where most action seems to happen. Well I'm using distcc -r4 and portage -r11 and here's the list of bugs/gotchas. If your environment has CC and/or CXX redefined to auto use distcc for all make's a la the distcc user guide suggests then Portage doesn't work saying that the compiler is unable to build executibles. (for now I've removed those settings and added /usr/lib/distcc/bin to my path before /usr/bin and after /usr/bin/ccache) When using your own projects and just invoking "make" and not having DISTCC_HOSTS defined the compile will stop prior to the invoking itself recursivly error and complain that DISTCC_HOSTS needs to be defined. The wrapper I think should see DISTCC_HOSTS isn't defined and not invoke distcc. But for debugging purposes here's the info. afr bin # env (cut down) PATH=/bin:/sbin:/usr/bin/ccache:/usr/lib/distcc/bin:/usr/bin:/usr/sbin:/usr/local/bin:/opt/bin:/usr/i686-pc-linux-gnu/gcc-bin/3.2:/opt/Acrobat5:/usr/X11R6/bin:/opt/blackdown-jdk-1.4.1_beta/bin:/opt/blackdown-jdk-1.4.1_beta/jre/bin:/usr/qt/3/bin DISTCC_HOSTS=cessna localhost CC and CXX are not defined. afr bin # emerge --version whPortage 2.0.46-r11 (default-x86-1.4, gcc-3.2.2, glibc-2.3.1-r3) afr bin # which gcc /usr/bin/ccache/gcc afr bin # which distcc which: no distcc in (/bin:/sbin:/usr/bin/ccache:/usr/lib/distcc/bin:/usr/bin:/usr/sbin:/usr/local/bin:/opt/bin:/usr/i686-pc-linux-gnu/gcc-bin/3.2:/opt/Acrobat5:/usr/X11R6/bin:/opt/blackdown-jdk-1.4.1_beta/bin:/opt/blackdown-jdk-1.4.1_beta/jre/bin:/usr/qt/3/bin) afr bin # emerge info ...... FEATURES="sandbox ccache digest prelink distcc" Would be nice that someone obsolete some of the attachment that got overwritten by better version and that the -r4 patch that the last two comments talks about be provided ;-) There are only two marked as non-obsolete, and those two are separate entities. gcc-wrapper-r3.tbz2 is the original gcc-config wrapper replacement, and distcc-1.1-r2.tar.bz2 is the distcc ebuild replacement version. Comment on attachment 7900 [details]
distcc-1.1-r2.tar.bz2
This one was marked as obsolete but from the last comments it seems that it is
a mistake.
Created attachment 7982 [details]
distcc-1.1-r6.tar.bz2
This is the latest version of the wrapper, though I haven't seen Wayne's
latest changes yet today. I'll be committing either -r6 or -r7 today
sometime, along with Wayne's ccache 2.
Thanks for all the hard work guys :) this has become one of the largest bugs I've seen. Now that I'm gonna take over as release coodinator zwelch has shown interest in taking over this package, and I will gladly had it over seeing all of you guys now have hacked at it alot more than I ever intended to myself :) Thanks again Brad and again what the hell is the problem with my mozilla and wrapping with this new buzilla GRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR A couple of quick updates: 1) Wayne has pointed out a couple of shortcomings to this latest patch, so I'm busy updating it to release a -r7. 2) This next version should be able to work correctly even if the wrappers are called using a relative path. At least, it will be a start that way. I'm still hacking on it, in other words. But I'd like to commit something soon, as these changes or something similar should be integated with the next release of distcc, allowing us to stop worrying about it so much. In the meantime, it would be nice if we all had something in Portage. <p> The wrapping problem is a bugzilla issue: http://bugzilla.mozilla.org/show_bug.cgi?id=11901 </p> Created attachment 8056 [details]
distcc-1.1-r7.tar.bz2
This is the latest version of the distcc patch. This version has deviated
from Wayne's, but it provides slightly more functionality. Unless there
are reasonable objections, I'll be commiting this to CVS before too long.
For the curious, bug 15048 may track some interesting improvements to the distcc ebuild to improve Gentoo's distccd support for multiple users. Okay folks, this one has finally been put to rest. I have just committed distcc-1.1-r8 to portage which contains only minor modifications to the -r7 ebuild attached here. I am aware of further improvements that need be made, notably bug 15048; however, the reasons for this bug report to continue to exist are slim. I am resolving this as fixed and would appreciate further feedback in new bug reports. Thanks. *** Bug 15473 has been marked as a duplicate of this bug. *** |
