Go to:
Gentoo Home
Documentation
Forums
Lists
Bugs
Planet
Store
Wiki
Get Gentoo!
Gentoo's Bugzilla – Attachment 181213 Details for
Bug 230989
sys-devel/icecream: Please write some documentation on how to set it up
Home
|
New
–
[Ex]
|
Browse
|
Search
|
Privacy Policy
|
[?]
|
Reports
|
Requests
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
Cleaned up version
icecream.xml (text/plain), 11.90 KB, created by
Łukasz Damentko (RETIRED)
on 2009-02-07 03:41:44 UTC
(
hide
)
Description:
Cleaned up version
Filename:
MIME Type:
Creator:
Łukasz Damentko (RETIRED)
Created:
2009-02-07 03:41:44 UTC
Size:
11.90 KB
patch
obsolete
><?xml version="1.0" encoding="UTF-8"?> ><!DOCTYPE guide SYSTEM "/dtd/guide.dtd"> ><!-- $Header$ --> > ><guide link="/doc/en/guide.xml" lang="en"> ><title>Gentoo Icecream Documentation</title> > ><author title="Author"> > <mail link="bluebird@gentoo.org">Friedrich Oslage</mail> ></author> > ><abstract> >This document serves as a HOWTO for using Icecream with Gentoo. ></abstract> > ><!-- The content of this document is licensed under the CC-BY-SA license --> ><!-- See http://creativecommons.org/licenses/by-sa/2.5 --> ><license/> > ><version>1.0</version> ><date>2008-08-02</date> > ><chapter> ><title>Introduction</title> ><section> ><title>What is icecream?</title> ><body> > ><p> >Icecream is a program created by SUSE and is based on ideas and code by distcc. >Similar to distcc it takes compile jobs(both portage and manual compilations) >and distributes them to remote machines. The main difference to distcc is that >is uses a central server, the scheduler, to distribute the jobs. This pays off >mostly for shared computers, as it also takes the load of each machine into >account and will therefore only use (mostly) idle machines. ></p> > ></body> ></section> ><section> ><title>How icecream works</title> ><body> > ><p> >Let's have a look at how compiling a program works. Usually you execute >something like this: ></p> > ><pre caption="Sample compilation"> >$ <i>cc -c -o hello_world.o hello_world.c</i> >$ <i>ld -o hello_world hello_world.o</i> ></pre> > ><p> >After the installation of icecream your <c>$PATH</c> will be changed in a way >that whenever you execute cc or gcc icecream's binary <e>icecc</e> will be >execute instead of your regular compiler. Icecc will then send the job (and the >source file) to the scheduler which will distribute it to one of the available >helper nodes. ></p> > ><p> >To do the job the helper node needs a compiler (who would have thought of >that?). But it will <e>not</e> use the compiler of the helper box (if any is >installed at all), instead it will request a tarball (the icecc environment) >containing the compiler (the binary and <e>all</e> required libraries) from the >machine running the compilation(proxied through the scheduler). ></p> > ><note> >Because of this you may need to recompile your toolchain(gcc, glibc, binutils) >without processor-specific cflags, because with them there is a greater chance >that your compiler won't work on other machines. ></note> > ><p> >After the compilation is finished it will send the <path>.o</path> file back to >the scheduler which will forward it to the machine running the compilation. >Which will then run the linker (needs to be run locally and will not be replaced >by icecream) and create the final binary. ></p> > ></body> ></section> ></chapter> > ><chapter> ><title>Installation</title> ><section> ><title>Requirements</title> ><body> > ><p> >To use icecream you need at least two computers. They don't need to be of the >same architecture and they even don't need to run the same operating system, but >they do need to run the same version of icecream. However cross-compiling >requires some extra work, if you use identical systems (e.g. two Linux/i686 >systems) it works out of the box. ></p> > ><p> >There also need to have a fairly fast network connection to each other >(>10Mbit). The exact requirement depends on how much parallel jobs you are >planning to run... just don't try it using a dial-up connection. ;) ></p> > ></body> ></section> ><section> ><title>Install icecream</title> ><body> > ><p> >Install icecream on all boxes that you want to add to your compilation cluster. ></p> > ><pre caption="Emerge icecream"> >$ <i>emerge ">=sys-devel/icecream-0.9.1-r1"</i> ></pre> > ></body> ></section> ><section> ><title>Designate one box as scheduler</title> ><body> > ><p> >As mentioned in the introduction icecream uses one central server (the >scheduler) to distribute jobs across all nodes. This box needs to be online >every time you want to run a compilation on any of the other nodes. ></p> > ><p> >You don't need to dedicated a complete machine for this, the scheduler can also >be a helper node and do compilation jobs. ></p> > ><p> >After you made up your mind, edit <path>/etc/conf.d/icecream</path> on that >machine and set <c>ICECREAM_RUN_SCHEDULER="yes"</c>. ></p> > ></body> ></section> ><section> ><title>Set up portage to use icecream</title> ><body> > ><p> >As of 08/2008 portage has no explicit support for icecream (such as ><c>FEATURES="icecream"</c>). That means if you use icecream with portage, >portage won't know you are using icecream, it will just wonder what a super fast >machine you have. ;) ></p> > ><p> >On each machine that you want to benefit from your icecream setup do: ></p> > ><pre caption="Edit make.conf"> >$ <i>nano -w /etc/make.conf</i> ><comment>(Set N to a suitable number for your particular setup)</comment> ><comment>(A common strategy is setting N as twice the number of total CPUs available + 1)</comment> >MAKEOPTS="-jN" ><comment>(Use icecc instead of your normal cc)</comment> >PREROOTPATH="/usr/lib/icecc/bin" ></pre> > ></body> ></section> ><section> ><title>Set up manual compilations to use icecream</title> ><body> > ><p> >To make manually compilations (those you do by typing <c>./configure ; make -jN >; make install</c>) use icecream all you need to do is adding ><path>/usr/lib/icecc/bin</path> to the beginning of your PATH. ></p> > ><pre caption="Add icecream to PATH"> >$ <i>nano -w /etc/profile</i> ><comment>(Add this to the end of the file)</comment> >export PATH="/usr/lib/icecc/bin:$PATH" ></pre> > ><p> ><path>/etc/profile</path> is just one possible place to set it, another (on a >per-user basis) place to set it would be <path>~/.bashrc</path>. ></p> > ></body> ></section> ><section> ><title>Start icecream</title> ><body> > ><p> >Starting icecream is simple, just execute those two commands on all boxes >(scheduler and helper boxes): ></p> > ><pre caption="Start icecream and add it to the default runlevel"> >$ <i>/etc/init.d/icecream start</i> >$ <i>rc-update add icecream default</i> ></pre> > ></body> ></section> ></chapter> > ><chapter> ><title>Non-everyday setups</title> ><section> ><title>Using ccache which icecream</title> ><body> > ><p> >First, set up portage to use ccache them same way you'd set it up without >icecream. Information about setting up ccache can be found in the <uri >link="/doc/en/handbook/handbook-x86.xml?part=2&chap=3#doc_chap3">handbook</uri>. ></p> > ><p> >After you've set up ccache, all you need to do is add ><path>/usr/lib/ccache/bin</path> to the beginning of your <c>$PATH</c>: ></p> > ><p> >For portage: ></p> > ><pre caption="Edit make.conf"> >$ <i>nano -w /etc/make.conf</i> ><comment>(Use icecc with ccache instead of your normal cc)</comment> >PREROOTPATH="/usr/lib/ccache/bin:/usr/lib/icecc/bin" ></pre> > ><p> >For manual compilations: ></p> > ><pre caption="Edit /etc/profile"> >$ <i>nano -w /etc/profile</i> ><comment>(Add this to the end of the file)</comment> >export PATH="/usr/lib/ccache/bin:/usr/lib/icecc/bin:$PATH" ></pre> > ></body> ></section> ><section> ><title>Cross-compiling</title> ><body> > ><p> >Did you read the <uri link="/doc/en/cross-compiling-distcc.xml">DistCC >Cross-compiling Guide</uri>? If yes, please note that the way icecream works is >different, so most you read there doesn't apply here. ></p> > ><p> >Let's assume you have two machines, one <uri >link="/proj/en/base/x86/">Gentoo/i686</uri> and one <uri >link="/proj/en/base/sparc/">Gentoo/SPARC</uri> and you want the i686 box to help >the sparc box doing compilations. ></p> > ><p> >In that case you need a cross-compiler that runs on i686 and produces code for >sparc (<c>HOST=i686</c>, <c>BUILD=i686</c>, <c>TARGET=sparc</c>). ></p> > ><p> >To create one you can use <c>sys-devel/crossdev</c>. After that you need to >create a tarball (the icecc environment) containing the compiler. Basically you >need these binaries and their dependencies: ></p> > ><ul> > <li><c>as</c></li> > <li><c>gcc</c></li> > <li><c>g++</c></li> > <li><c>cc1</c></li> > <li><c>cc1plus</c></li> ></ul> > ><p> >Luckily there are lots of tools to help you to do that, you just need to execute >these three commands on the i686 box: ></p> > ><pre caption="Create an icecc environment for cross-compiling"> ><comment>(Install Gentoo's cross-toolchain generator)</comment> >$ <i>emerge sys-devel/crossdev</i> ><comment>(Create a cross-toolchain for sparc-unknown-linux-gnu)</comment> >$ <i>crossdev -t sparc-unknown-linux-gnu</i> ><comment>(Create the icecc environment)</comment> >$ <i>icecream-create-env sparc-unknown-linux-gnu</i> ></pre> > ><p> >Now you have a file with a name similar to ><e>df8f4a336bbfc23ed86a296966f50be4.tar.gz</e>, that's your icecc environment. ></p> > ><p> >Move it to the sparc machine. You can put it anywhere you want and give it a >more significant name, e.g. ><path>/var/icecc/host-i686-target-sparc.tar.gz</path>. ></p> > ><p> >The native environment icecream created automatically is located in ><path>/var/cache/icecream/native</path>. If it doesn't exist you can simply >create it by running <c>icecream-create-env</c>. ></p> > ><p> >It's a good practice to store all environments in the same directory. Let's >assume you copied it to <path>/var/icecc/host-sparc-target-sparc.tar.gz</path>. >But remember: Whenever you update binutils or gcc you should recreate them, >otherwise they'd still use the old version. ></p> > ><p> >After that you need to tell icecream about it by setting the ><c>ICECC_VERSION</c> environment variable. <c>ICECC_VERSION</c> contains a comma >separated list of icecc environments for different architectures, the format is: ></p> > ><pre caption="ICECC_VERSION format"> ><i>[ architecure(output of uname -m) ]</i><comment>:</comment><i>[ path to tar.gz icecc environment ]</i> ></pre> > ><p> >In our case you'd need to add this to <path>/etc/profile</path> on the sparc >box: ></p> > ><pre caption="Edit /etc/profile"> >$ <i>nano -w /etc/profile</i> ><comment>(Add this to the end of the file)</comment> >export ICECC_VERSION="i686:/var/icecc/host-i686-target-sparc.tar.gz,sparc64:/var/icecc/host-sparc-target-sparc.tar.gz" ></pre> > ><p> >Now restart icecream on the sparc box and your're done. ></p> > ><p> >If you also want the sparc box to assist the i686 box you need to create the >cross-icecc environment on the sparc box, copy it to the i686 and set ><c>ICECC_VERSION</c>. ></p> > ></body> ></section> ><section> ><title>More cross-compiling</title> ><body> > ><p> >Imagine you have an i686 box with an arm-linux-gnu cross-toolchain installed >which you are using to compile programs for your cell phone and you want to use >icecream to speed up compilations? ></p> > ><p> >First, create a few symlinks to invoke icecream instead of your normal compiler: ></p> > ><pre caption="Create cross-compiler symlinks"> >$ <i>icecream-config --install-links arm-linux-gnu</i> ></pre> > ><p> >After that you need to set <c>ICECC_VERSION</c> to point to the tarball(icecc >environment) for arm-linux-gnu. You can create is using ><c>icecream-create-env</c>. You probably don't want to set it permanently, just >on an as-needed basis. ></p> > ><pre caption="Create arm-linux-gnu icecc environment and set ICECC_VERSION"> ><comment>(Install Gentoo's cross-toolchain generator)</comment> >$ <i>emerge sys-devel/crossdev</i> ><comment>(Create a cross-toolchain for arm-linux-gnu)</comment> >$ <i>crossdev -t arm-linux-gnu</i> ><comment>(Create the icecc environment)</comment> >$ <i>icecream-create-env arm-linux-gnu</i> ></pre> > ></body> ></section> ></chapter> > ><chapter> ><title>Security concerns</title> ><section> ><body> > ><p> >Icecream has no authentication at all and the icecc daemon is run as root. ></p> > ><p> >Distcc has a --allow parameter to specify clients that are allowed to connect, >icecream has no such thing, it only provides the config option ><c>ICECREAM_ALLOW_REMOTE</c>, but no ACL. That means <e>anybody</e> can use >your system as a host for compilations and could potentially exploit bugs in gcc >to gain control over your box. ></p> > ><p> >Therefore it is best to block icecream's ports in your firewall and only allow >connections from trusted hosts. These are the ports icecream uses: ></p> > ><ul> > <li>TCP/10245 for the icecc daemon (required)</li> > <li>TCP/8765 for the scheduler (required)</li> > <li>TCP/8766 for the telnet interface to the scheduler (optional)</li> > <li>UDP/8765 for broadcast to find the scheduler (optional)</li> ></ul> > ></body> ></section> ></chapter> ></guide>
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 230989
: 181213