Me and mattst88 spent some time debugging miscompilation of glibc on #gentoo-mips. We found out distcc-pump to be the failure trigger. Looking at pump's assumptions they seem to be very strong: https://www.systutorials.com/docs/linux/man/1-pump/ """ Note that distcc's pump-mode assumes that sources files will not be modified during the lifetime of the include server, so modifying source files during a build may cause inconsistent results. """ I don't think it holds even for ./configure scripts. A few supporting cases for removal: - bug #636806: llvm fails at cmake configuration time - bug #672754#c3: statement by upstream maintainer that pump should not be ran on arbitrary code I suggest removing the FEATURES=distcc-pump entirely from portage and leave any plumbing to end-users if they need to. WDYT?
I think #507482 mostly broke distcc-pump as it added it to src_configure(). That difectly violates assumption of immutable sources. But other phases (src_compile()) can do it just as well. For example gcc builds stage2 and stage3 in the same directory and generates intermediate source and header files there.
FYI: I found one package, dev-util/cmake, which doesn't compile _without_ distcc-pump.
(In reply to Thomas Deutschmann from comment #2) > FYI: I found one package, dev-util/cmake, which doesn't compile _without_ > distcc-pump. Don't want to kill your enthusiasm but it probably just means your system is so broken you need to --emptytree.
(In reply to Thomas Deutschmann from comment #2) > FYI: I found one package, dev-util/cmake, which doesn't compile _without_ > distcc-pump. Compared to FEATURES=distcc? I think it's a bug and should not happen. Or there is an extra negation in the sentence?
It turned out to be a PEBKAC.
Created attachment 598656 [details, diff] 0001-emerge-drop-FEATURES-distcc-pump-support-bug-702146.patch
Sent https://archives.gentoo.org/gentoo-portage-dev/message/87fade273f739fb913942867cf27bbc9 for review.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=bc0c28bbcdb03506e6c0bafac9e63aff5ebb6894 commit bc0c28bbcdb03506e6c0bafac9e63aff5ebb6894 Author: Sergei Trofimovich <slyfox@gentoo.org> AuthorDate: 2019-12-06 21:00:46 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2019-12-08 21:29:45 +0000 emerge: drop FEATURES=distcc-pump support, bug #702146 'distcc' distributes code generation for preprocessed files. 'pump' distributes preprocessing and code generation of files and imposes very strict requirement: """ Note that distcc's pump-mode assumes that sources files will not be modified during the lifetime of the include server, so modifying source files during a build may cause inconsistent results. """ `src_configure()` (where we used to start include server before this change) almost always violates that requirement. It is not uncommon to generate more intermediate source files as a package builds (`bison`, `flex`, child `./configure` calls from `make`) and thus quite unsafe to use `pump`. This change drops `FEATURES=distcc-pump` and leaves only FEATURES=distcc. This way all the proprocessing happens as expected and only code generation is offloaded. Bug: https://bugs.gentoo.org/702146 Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org> bin/phase-functions.sh | 17 ----------------- lib/_emerge/EbuildPhase.py | 2 +- lib/portage/const.py | 1 - man/make.conf.5 | 3 --- 4 files changed, 1 insertion(+), 22 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=9557f104aa85b65c7d394c52c5c8d8727a111651 commit 9557f104aa85b65c7d394c52c5c8d8727a111651 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2019-12-15 22:53:50 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2019-12-15 22:56:34 +0000 sys-apps/portage: Bump to version 2.3.82 #310009 emerge: Show package USE in conflict messages #680456 display relevant FEATURES when unshare fails #693454 emerge-webrsync: support gentoo-YYYYMMDD snapshots #702146 emerge: drop FEATURES=distcc-pump support #702970 emerge-webrsync: enable xz snapshots for tarsync Bug: https://bugs.gentoo.org/701268 Bug: https://bugs.gentoo.org/310009 Bug: https://bugs.gentoo.org/680456 Bug: https://bugs.gentoo.org/693454 Bug: https://bugs.gentoo.org/702146 Bug: https://bugs.gentoo.org/702970 Package-Manager: Portage-2.3.82, Repoman-2.3.20 Signed-off-by: Zac Medico <zmedico@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-2.3.82.ebuild | 272 +++++++++++++++++++++++++++++++++ 2 files changed, 273 insertions(+)
Should remove distcc-pump usage from Handbook
(In reply to Reva Denis from comment #10) > Should remove distcc-pump usage from Handbook Can you link to a section where it's mentioned?
This is a not a good solution to the problem. Rather, it is better to modify /etc/portage/package.env to accommodate the unfriendly packages. For example, in package.env: dev-util/cmake nodistcc.conf And then in /etc/portage/env/nodistcc.conf to match: FEATURES="-distcc" This way, distcc and pump mode can remain enabled for all other packages. Please do change the fix here to this more appropriate solution. No need to remove entire features when such a simple accommodation is available.
(In reply to carbohydrates from comment #12) > This is a not a good solution to the problem. Rather, it is better to > modify /etc/portage/package.env to accommodate the unfriendly packages. For > example, in package.env: > > dev-util/cmake nodistcc.conf > > And then in /etc/portage/env/nodistcc.conf to match: > > FEATURES="-distcc" > > This way, distcc and pump mode can remain enabled for all other packages. > > Please do change the fix here to this more appropriate solution. No need to > remove entire features when such a simple accommodation is available. It's dead, Jim. FWIW, I've got a semi-defunct project like you mention here: https://gitweb.gentoo.org/proj/no-distcc-env.git/
