Some packages do not compile when distcc is used. There are many such bugs reported in the Gentoo bugzilla and I've filed two of the most recent: Bug # 28275 and Bug # 28278 involving libgtop and gaim. Portage should maintain a list of packages, called for example, DISTCC-FRAGILE that do not compile when distcc is used. When such packages are to be compiled, Portage should behave as if FEATURES did not contain distcc and, additionally, compile with MAKEOPTS=jx where x does not reflect a value that would be used for a distributed compile. Additionally, when a distcc compile of a package fails, portage should ask the user whether it should try compiling without distcc. If the non-distcc compile fails, portage should ask the user whether it should update DISTCC-FRAGILE (mentioned above). Reproducible: Always Steps to Reproduce: 1. 2. 3. Portage 2.0.49-r3 (default-x86-1.4, gcc-3.3.1, glibc-2.3.2-r1, 2.4.20-gentoo-r5) ================================================================= System uname: 2.4.20-gentoo-r5 i686 Intel(R) Celeron(TM) CPU 1100MHz distcc 2.10 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled] ccache version 2.2 [enabled] ACCEPT_KEYWORDS="x86 ~x86" AUTOCLEAN="yes" CFLAGS="-march=pentium3 -O3 -pipe" CHOST="i686-pc-linux-gnu" COMPILER="gcc3" CONFIG_PROTECT="/etc /var/qmail/control /usr/kde/2/share/config /usr/kde/3/share/config /usr/X11R6/lib/X11/xkb /usr/kde/3.1/share/config /usr/share/config" CONFIG_PROTECT_MASK="/etc/gconf /etc/env.d" CXXFLAGS="-march=pentium3 -O3 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="sandbox autoaddcvs buildpkg ccache" GENTOO_MIRRORS="http://csociety-ftp.ecn.purdue.edu/pub/gentoo/ http://adelie.polymtl.ca/ http://gentoo.mirrors.pair.com/ http://gentoo.chem.wisc.edu/gentoo/ http://ds.thn.htu.se/linux/gentoo http://gentoo.seren.com/gentoo http://gentoo.inode.at/ http://www.fhh.opensource-mirror.de/gentoo.org/ http://darkstar.ist.utl.pt/gentoo/" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://mail.sarai.kit/gentoo-portage" USE="x86 oss apm avi crypt cups encode foomaticdb gif jpeg libg++ libwww mad mikmod mmx mpeg ncurses nls pdflib png quicktime spell truetype xml2 xmms xv zlib alsa gdbm berkdb slang readline arts aalib svga tcltk java mysql X sdl gpm tcpd pam ssl perl python esd imlib oggvorbis gnome gtk qt kde motif opengl mozilla ldap apache2 curl dga gtk2 gtkhtml i8x0 imap lcms mozaccess mozinterfaceinfo moznoirc moznomail mozp3p mozsvg mozxmlterm offensive samba sse tiff type1 wmf xml mbox"
TYPO: please amend last para of bug description to state Additionally, when a distcc compile of a package fails, portage should ask the user whether it should try compiling without distcc. If the non-distcc compile succeeds, portage should ask the user whether it should update DISTCC-FRAGILE with the name of the package.
This is a very good idea. This is a portage thing, not a distcc thing, though. I think the idea can be refined to packages that fail with parallel makes (like xfree and mozilla). Maybe something in an .ebuild: RESTRICT="distcc" That can be built upon to change gcc compilers on the fly: RESTRICT="gcc-2.95 -distcc"
your bugs are unrelated i believe ...
Lisa: Why do you say it's not a distcc bug but a portage bug ? When distcc was used to compile the programs, it didn't succeed. When it wasn't, it did. *slightly bewildered*
some packages cant compile in parallel, thats a simple fact distcc is a parallel compiler that needs to be restricted in the same way we restrict emake/make
Its a request for an improvement in Portage, not distcc.
Lisa: Correct. Just colour me slow today :)
The idea to query the user could cause problems, I think it's better for the emerge to 'cleanly' fail or recover, and if an ebuild cannot support -jN, or distcc, it ought to do the RESTRICT, and have bug filed if it requires human intervention. One reason (apart from it being inconsistent with UNIX CLI philosophy), it'd be a bad idea for emerge to hang around waiting for user input, is that portage doesn't support simultaneous usage (least did not earlier this year). Portage could be corrupted if an emerge is stopped or restarted when that window is found hours later, and another emerge was already started. Also it's bit pointless questioning the user, if re-trying is an acceptable solution. I'm sure yesterday when I set up distcc feature, but had not config-ed distcc portage printed a warning and continued build without distcc. It makes sense for a host with puny CPU, used say as a firewall, to pass off all compilation to another machine, even if it cannot parellise. So better for RESTRICT to make distinct : parallel-make remote-compile It seems possible to me, than an alternative to distcc may arise someday. If a bug report comes in "ebuild failed with distcc", then RESTRICT=distcc likely to be a 'no brainer', when in fact the culprit is really -jN or possibly really is the use of remote compilation (lisa is their info on possible causes of failure I can look at?).
Rob: An excellent point. What we have to consider is whether there would be any situations where a user would not want the compile to be run locally ... (embedded systems, gateways ? ). If this were the case, we could have a countdown-like display ..... Distributed compile failed, running compile locally in 5..4..3..2..1.. like the unmerge command displays. Aniruddha Shankar New Delhi, India
wrt comment 9, if you export DISTCC_FALLBACK='0', then a failed compilation will not be attempted locally. With this in mind you could do: DISTCC_FALLBACK='0' emake
OK, so local compilation can be prevented, then it must basically already be done, to re-try distcc's locally, in at least some circumstances. Those example failures in the bugs mentioned, look like linking steps failing, rather than compilation failures. Builds like Xfree, which don't work in parallel, ought they to work if distcc used with -j1 forced?
Ummmm... MAKEOPTS="${MAKEOPTS} -j1" And compilation via distcc shouldn't cause builds to fail. If it's distcc's fault, report it to them. No need to add extra things into portage that aren't needed.
Please add RESTRICT="distcc" support to portage. True most distcc problems are really parallel building problems. However, there is a perfectly valid scenario for wanting to RESTRICT="distcc" in the case of kernel modules.
*** Bug 235742 has been marked as a duplicate of this bug. ***
packages with parallel build problems get bugs and marked with -j1. distcc happens to exacerbate the issue, but it doesnt make it a distcc bug.
