This enhancement suggestion was inspired after reviewing the check-reqs.eclass I propose that an eclass is created, or an existing eclass is expanded, to include a check for FEATURES="distcc" during pkg_setup. This check would preemptively cause an ebuild to "die" with an informative error message during pkg_setup instead of having the ebuild "die" normally during src_compile. This check should be used in ebuilds that are known to fail when compiling with distcc. An ebuild inheriting this functionality should already have a bug opened about problems with distcc and those problems should be verified before the check is added. The main motivation for creating this functionality is that an ebuild does not need to fail during src_compile for an already known issue. With portage's --keep-going option, many of these errors can be ignored, but the ebuilds still proceed through src_unpack and src_compile. I have already tested a very simple, separate eclass, but I understand that this needs to be reviewed and refined before any possible inclusion. I welcome everyone's comments and additional suggestions. Reproducible: Always Steps to Reproduce:
Wouldn't it be easier to create a RESTRICT=distcc? See bug #28300 and bug #80894. Also, consider using /etc/portage/env/<cat>/<pkg> for this purpose, with each <cat>/<pkg> being a (shell script) file with an entry like FEATURES=-distcc. Anyway, there's no time like the present to discuss this feature again. :)
I do not believe you can use /etc/portage/env/<cat>/<pkg> to alter FEATURES. I am not looking for the ability to RESTRICT or automatically detect-and-correct. Those bugs were clearly closed. I guess my first question is should this be implemented in a new eclass or existing eclass. What additional functionality, if any, is needed?
(In reply to comment #2) > I do not believe you can use /etc/portage/env/<cat>/<pkg> to alter FEATURES. You haven't tried it, then?
Similar to ccache, there's no legitimate reason for a given ebuild not to work with distcc. Since it's only a compiler frontend, it's behavior should be 100% compatible with a normal compiler. Any incompatibility would either be a bug in the package's build system or else a bug in distcc itself. As such, disabling distcc only serves to hide the real problem. (In reply to comment #2) > I do not believe you can use /etc/portage/env/<cat>/<pkg> to alter FEATURES. In some cases you can if you define a pre_pkg_setup function and to it there. Unfortunately, it won't work in this case because of the way that distcc is setup in /usr/lib/portage/bin/ebuild.sh. However, something like this in /etc/portage/env/<cat>/<pkg> should work: CC=${CC#distcc } CXX=${CXX#distcc }
/etc/portage/env/<cat>/<pkg> did not work with portage-2.2_rc8 for FEATURES I am not disputing the legitimacy of ebuilds not working with distcc. The problems should not be hidden or ignored either. It is just that if an ebuild is known to fail (bug opened and verified on Gentoo Bugzilla) it should fail early and with a better error message/info block. At the very least, if the new workaround for /etc/portage/env/<cat>/<pkg> works, the ebuild should QA warn the user about the workaround.
I'd say the ebuild should either be fixed or else some workaround should be integrated directly into the ebuild.
OK, what should happen if the ebuild cannot be fixed, with or without an integrated workaround, in a reasonable amount of time, or ever? I do not expect maintainers to fix everything, especially if it is an upstream problem like a build system problem or a bug with distcc itself.
Well, something like this in the ebuild's src_compile should serve as a simple workaround: CC=${CC#distcc } CXX=${CXX#distcc } Just file a bug and the maintainer should be able to take care of it pretty quickly. You can do it yourself in /etc/portage/env/<cat>/<pkg> as a workaround until the ebuild is fixed.
The suggested workaround fails for the ebuild I test on: app-admin/keepassx-0.3.3, bug 214327 I tried: CC=${CC#distcc } CXX=${CXX#distcc } emake || die "emake failed" and: emake CC=${CC#distcc } CXX=${CXX#distcc } || die "emake failed" and: CC=${CC#distcc } CXX=${CXX#distcc } in /etc/portage/env/app-admin/keepassx.
And why exactly are you blaming distcc? Are you sure it's not just a general parallelization problem? Most problems that people blame on distcc are simply builds that break when done in parallel.
I am not blaming distcc. It is just that the flag in FEATURES is named distcc and it causes the build environment to be set up a certain way, from what I understand. I just ran this particular ebuild with FEATURES="distcc" MAKEOPTS="-j1" and it still failed. I do not know what else I can test for parallelism.
Can you attach a build log with the failure?
Sure. I am assuming you want temp/build.log for when I ran it with FEATURES="distcc" MAKEOPTS="-j1". I will post it in bug 214327 since it is specific to that package. If you want any additional logs for any combination of options, request it there and I will post it. Apparently, bug 222063 already requests something very similar to what I proposed above.
Build log is now posted in bug 214327.
(In reply to comment #9) > The suggested workaround fails for the ebuild I test on: Okay, now I've got a workaround which I've tested on app-admin/keepassx and it really works: PATH=${PATH#/usr/lib/distcc/bin:} I'll go ahead and put this workaround in the ebuild for bug #214327.
(In reply to comment #5) > /etc/portage/env/<cat>/<pkg> did not work with portage-2.2_rc8 for FEATURES Sorry. I forgot to check how exactly I use /etc/portage/env. The way I use it is like this: jeroen@elmer ~ $ cat /etc/portage/env/media-gfx/imagemagick FEATURES="${FEATURES/distcc/-distcc}" That seems to do it for me. :)
Great. Now that this workaround is known to work, and it is used in at least two ebuilds, shouldn't it be documented somewhere other than bugzilla? You could make an emake_nodistcc function, but a mention in the devmanual would be just as good I guess. Also, I believe there should be a QA message if the workaround is used in an ebuild. At the very least, it will bring more attention to the problem and capable users might get involved, fix the corresponding packages, and submit patches here or upstream.
Please add RESTRICT="distcc" support to portage. True most distcc problems are really parallel building problems. However, there is a perfectly valid scenario for wanting to RESTRICT="distcc" in the case of kernel modules.
(In reply to comment #18) > Please add RESTRICT="distcc" support to portage. True most distcc problems are > really parallel building problems. However, there is a perfectly valid > scenario for wanting to RESTRICT="distcc" in the case of kernel modules. > I'm not sure the issue has something to do with parallel building. Afterall , the -j1 flag is forced in every ebuild that doesn't support parallelism. It could be a distcc loop error during the compilation that is causing the issue. The gentoo distcc implementation isn't working for me for a while. I encountered divers problems like sandbox violations and the boring distcc error 111. I found a way that works pretty well with -j16 for instance. I forced the FEATURES="-distcc ccache" and configured ccache to handle distcc. I have set the variable CCACHE_PREFIX="distcc" in /etc/env.d/02ccache As Portage tmp directory can be set in memory (/dev/shm) it isn't permanent and distcc doesn't rebuild its directories. That's why I prefer /var/lib/cache for ccache and distcc trees. PORTAGE_TMPDIR/.distcc is no way the best environment. For example: CCACHE_DIR="/var/lib/cache/ccache" DISTCC_DIR="/var/lib/cache/distcc" Also I initialized the distcc client with, chown -R portage.portage /var/lib/cache/distcc chmod g+rws /var/lib/cache/distcc mkdir -p /var/lib/cache/distcc/state chmod g+w /var/lib/cache/distcc/state/ otherwise it complains about wrong right settings or a state directory missing. When using /var/lib/cache tree, it is still necessary to change your profile shell Add in your profile.bashrc, # cat /etc/portage/profile/profile.bashrc SANDBOX_WRITE="/var/log/ccache.log:/var/lib/cache/distcc:/var/log/distcc.log:${SANDBOX_WRITE}" Simply writing in /var/log/distcc.log could lead to a sandbox violation :) Also, distccd doesn't need to be configured if you use the --zeroconf option and declare some client subnets allowed to access the daemon. On the client side you have just to declare a +zeroconf host and activate avahi. Hope that helps
*** This bug has been marked as a duplicate of bug 28300 ***
