Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 87932 - Suggestion to simplify cross compiler setup for distcc in a hetrogenous network environment
Summary: Suggestion to simplify cross compiler setup for distcc in a hetrogenous netwo...
Status: RESOLVED LATER
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Development (show other bugs)
Hardware: All All
: High enhancement (vote)
Assignee: Gentoo Toolchain Maintainers
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-04-04 10:38 UTC by Andrew de Quincey
Modified: 2006-09-24 14:41 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
gcc 3.4.4 patch to disable libgcc (gcc-3.4.4-nolibgcc.patch,4.33 KB, patch)
2005-10-13 04:44 UTC, Andrew de Quincey
Details | Diff
Minimised patch to disabled libgcc (gcc-3.4.4-nolibgcc2.patch,1.10 KB, patch)
2005-10-14 04:05 UTC, Andrew de Quincey
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Andrew de Quincey 2005-04-04 10:38:07 UTC
Hi, I mentioned this idea on gentoo-dev, and I wanted to submit it properly. If you don't like it, fair enough, I won't ever mention it again.

The problem. Machine A is an i686-pc-linux-gnu. Machine B is a powerpc-pc-linux-gnu. I want to distribute distcc tasks from A onto machine B.

With crossdev, this is relatively easy - I install a cross compiler for i686-pc-linux-gnu on machine B. However, installing the whole of linux-headers and glibc just for distcc is overkill - they won't be used for a distcc compilation. I just need binutils + compilers.

crossdev supports a "--stage1" switch which does what I want for C - installs gcc without a libc. However, I also want to do this for g++.

Normally when you compile g++, you need a libc for the target machine because gcc insists on compiling libgcc (and possibly libstdc++) for it. However, prior to compiling all the target libraries, it has _already_ generated the full C and C++ cross compilers.

My modified ebuild adds a new USE flag to gcc - "distcc". If this is on, it applies a patch which disables the compilation of the target libraries - libgcc and libstdcc++ etc. Therefore you can get away with just having binutils and gcc/g++ on machine B.

I also modified crossdev so that when you use the --stage1 switch, it compiles gcc WITH c++ support, but turning on the new distcc USE flag, so you end up with a minimal C AND C++ compiler.

I know this works - I have it running perfectly here.. I've been test compiling random bits of kde for a day or so with this setup.

Anyway, the modified ebuild and crossdev are available at http://lidskialf.net/gcc
Comment 1 Lisa Seelye (RETIRED) gentoo-dev 2005-04-19 06:06:14 UTC
I'm not sure why this is assigned to me, perhaps the portage or crossdev team would be a better idea?
Comment 2 Radek Podgorny 2005-10-13 04:38:38 UTC
...are you sure you can compile gcc with c++ support without compiling glibc
first? Maybe I just didn't understand your terminology...
Comment 3 Andrew de Quincey 2005-10-13 04:43:43 UTC
yeah - I'm doing this on three difference architectures here (x86, amd64, and  
ppc). Actually, what it does is disable libgcc, which in turn means it doesn't 
need glibc. 
  
Attached is my latest patch to gcc 3.4.4 to disable libgcc.  
 
Comment 4 Andrew de Quincey 2005-10-13 04:44:15 UTC
Created attachment 70554 [details, diff]
gcc 3.4.4 patch to disable libgcc
Comment 5 SpanKY gentoo-dev 2005-10-13 05:49:51 UTC
one of the points to forcing a second stage gcc is to also properly build
threading support

gcc cannot build thread support properly if it does not have libc headers
Comment 6 Andrew de Quincey 2005-10-13 06:52:22 UTC
Does gcc need threading support compiled into the compilers themselves? 
 
This is _only_ for distcc cross compilation - where it takes a .c/.cc file and 
compiles it into a .o file... I wouldn't have expected threading to be a part 
of that process.  
 
Certainly at the link stage I can see the requirement, but linking is not done 
on the cross compiling slaves (which is where this special GCC runs). Its done 
on the machine generating the cross compile tasks, which will have a full gcc 
install including glibc/libgcc etc etc. 
 
 
Comment 7 Andrew de Quincey 2005-10-13 06:56:53 UTC
Actually, if you look at how the "nocxx" flag works, it doesn't build 
libgcc/require glibc either. Does threading work fine with a "nocxx"-built gcc? 
 
All this patch does is the same for g++. 
 
Comment 8 SpanKY gentoo-dev 2005-10-13 07:08:57 UTC
a stage1 cross-compiler builds with both --without-headers and --without-thread

a normal compiler with USE=nocxx builds both with headers and with threads
Comment 9 Andrew de Quincey 2005-10-13 07:19:47 UTC
Er yeah, but I don't remember saying I was disabling building stage2 anywhere? 
 
The patch disables libgcc yes, but thats the only change. There are no other   
changes to my ebuild (apart from applying the patch).   
 
I certainly don't supply --without-headers and --without-thread anywhere. The 
only change to the ebuild is to add a line to apply the patch. 
 
Comment 10 SpanKY gentoo-dev 2005-10-13 08:00:21 UTC
i was replying to your nocxx comment

a USE=nocxx compiler requires glibc/headers
Comment 11 Andrew de Quincey 2005-10-13 08:06:26 UTC
Ah - sorry - I was confused.  
 
Actually, reading back to my original comment, I see I do say I was using 
stage1. Sorry, its been months since I added this and I'd forgotten everything 
about it. 
 
 
Comment 12 Andrew de Quincey 2005-10-13 08:24:05 UTC
OK, sorry, I've changed the way I build things since I originally submitted  
this bug - I didn't realise it was so old. Here goes with what I do now (I no 
longer use crosstool to create the compilers). 
  
To cross compile things, I copy the directories /usr/portage/sys-devel/gcc  
and /usr/portage/sys-devel/binutils  
to /usr/local/portage/cross-powerpc-unknown-linux-gnu  
  
Then I  
patch /usr/local/portage/cross-powerpc-unknown-linux-gnu/gcc-3.4.4-r1.ebuild  
and add:  
  
        epatch "${FILESDIR}"/3.4.4/gcc-3.4.4-nolibgcc.patch  
 
where it says "fix cross compiling" 
 
Then I emerge cross-powerpc-unknown-linux-gnu/gcc. It builds gcc + binutils 
(and g++) 
 
The use flags for that gcc are: 
 
[ebuild   R   ] cross-powerpc-unknown-linux-gnu/gcc-3.4.4-r1  (-altivec) 
-bootstrap -boundschecking -build -fortran -gcj -gtk -hardened -ip28 
(-multilib) -multislot (-n32) (-n64) +nls -nocxx -nopie -nossp -objc -static 
-vanilla 0 kB [1] 
 
once its built, the cross compiler reports: 
 
powerpc-unknown-linux-gnu-gcc  -v 
Reading specs from /usr/lib/gcc/powerpc-unknown-linux-gnu/3.4.4/specs 
Configured with: /var/tmp/portage/gcc-3.4.4-r1/work/gcc-3.4.4/configure 
--prefix=/usr --bindir=/usr/powerpc-unknown-linux-gnu/gcc-bin/3.4.4 
--includedir=/usr/lib/gcc/powerpc-unknown-linux-gnu/3.4.4/include 
--datadir=/usr/share/gcc-data/powerpc-unknown-linux-gnu/3.4.4 
--mandir=/usr/share/gcc-data/powerpc-unknown-linux-gnu/3.4.4/man 
--infodir=/usr/share/gcc-data/powerpc-unknown-linux-gnu/3.4.4/info 
--with-gxx-include-dir=/usr/lib/gcc/powerpc-unknown-linux-gnu/3.4.4/include/g++-v3 
--host=x86_64-pc-linux-gnu --target=powerpc-unknown-linux-gnu 
--build=x86_64-pc-linux-gnu --disable-altivec --enable-nls 
--without-included-gettext --with-system-zlib --disable-checking 
--disable-werror --disable-libunwind-exceptions --disable-multilib 
--disable-libgcj --enable-languages=c,c++ --enable-shared 
--enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu 
Thread model: posix 
gcc version 3.4.4 (Gentoo 3.4.4-r1, ssp-3.4.4-1.0, pie-8.7.8) 
 
I don't need kernel header files (installed on the cross-compiler) since the 
preprocessing stage is done already by the machine distributing distcc tasks. 
 
Perhaps this is only suited to environments where all machines run the same 
glibc/kernel? Either way, this setup has been working for me for a long time 
with no problems. 
 
Comment 13 SpanKY gentoo-dev 2005-10-13 23:36:12 UTC
all in all, this is a pretty intrusive patch

anyway it can be minimized ?
Comment 14 Andrew de Quincey 2005-10-14 04:05:46 UTC
Created attachment 70650 [details, diff]
Minimised patch to disabled libgcc

How about this? It greatly decreases the number of lines changed... I hope this
is what you were meaning.
Comment 15 SpanKY gentoo-dev 2005-10-14 06:51:48 UTC
yes, that's what i meant, thanks
Comment 16 Mark Loeser (RETIRED) gentoo-dev 2005-12-14 23:46:56 UTC
Do we plan on adding this?  Also, calling it "distcc" might be misleading if I
understand this correctly.
Comment 17 SpanKY gentoo-dev 2006-09-24 14:41:14 UTC
at this point, no, i dont plan on adding it

there is no easy way to integrate this limited behavior so that it only happens when a user specifically wants it