Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 622866 - llvm.eclass: llvm_pkg_setup() breaks PATH for ccache and distcc with <sys-devel/llvm-4
Summary: llvm.eclass: llvm_pkg_setup() breaks PATH for ccache and distcc with <sys-dev...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Eclasses (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: LLVM support project
URL: https://archives.gentoo.org/gentoo-de...
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-06-27 18:47 UTC by Matt Whitlock
Modified: 2017-08-12 21:29 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
llvm.eclass.patch (llvm.eclass.patch,705 bytes, patch)
2017-06-27 20:04 UTC, Matt Whitlock
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Matt Whitlock 2017-06-27 18:47:15 UTC
The following example concerns media-libs/mesa-17.0.6, but this bug would apply to any package that uses the llvm_pkg_setup() function from llvm.eclass when a slot of sys-devel/llvm lower than 4 is installed.

When Portage is configured to use ccache and distcc, the ebuild compile phase is initially launched with:

> PATH=/usr/lib/ccache/bin:/usr/lib/distcc/bin:/var/tmp/portage/._portage_reinstall_.4r0yxcgc/bin/ebuild-helpers:/usr/lib/portage/python3.4/ebuild-helpers:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/armv6j-hardfloat-linux-gnueabi/gcc-bin/5.4.0

However, then the llvm_pkg_setup() function from llvm.eclass prepends /usr/bin to PATH, which, along with the PATH modification made by python_wrapper_setup(), results in:

> PATH=/var/tmp/portage/media-libs/mesa-17.0.6/temp/python2.7/bin:/usr/bin:/usr/lib/ccache/bin:/usr/lib/distcc/bin:/var/tmp/portage/._portage_reinstall_.4r0yxcgc/bin/ebuild-helpers:/usr/lib/portage/python3.4/ebuild-helpers:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/armv6j-hardfloat-linux-gnueabi/gcc-bin/5.4.0

Thus, when the Makefile calls the compiler, the binary in /usr/bin is found before the wrapper in /usr/lib/ccache/bin or /usr/lib/distcc/bin, and thus ccache and distcc are bypassed, resulting in a much slower build.

Perhaps llvm_pkg_setup() ought to take a cue from python-utils-r1.eclass: create symlinks or wrappers in ${T}/llvm/bin and prepend ${T}/llvm/bin to PATH rather than /usr/bin. Alternatively, the older versions of sys-devel/llvm should use slotted installation directories to avoid llvm_pkg_setup()'s contaminating PATH with /usr/bin.
Comment 1 Matt Whitlock 2017-06-27 20:04:30 UTC
Created attachment 478252 [details, diff]
llvm.eclass.patch

Here's another alternative, which I've tested as working. This inserts the chosen LLVM's bin dir into PATH at a position where it will precede any/all other LLVMs but will follow any leading path elements that contain no llvm-config executable.
Comment 2 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-06-30 21:45:13 UTC
Could you test https://archives.gentoo.org/gentoo-dev/message/598df75c28e7049d9394689dab70854f, please?
Comment 3 Matt Whitlock 2017-06-30 21:54:44 UTC
(In reply to Michał Górny from comment #2)
> Could you test
> https://archives.gentoo.org/gentoo-dev/message/
> 598df75c28e7049d9394689dab70854f, please?

I believe your suggested fix is broken, as it will not move /usr/bin to an earlier position in the PATH than, e.g., /usr/lib/llvm/4/bin, in the case that both SLOT=0 and SLOT=4 of sys-devel/llvm are installed but LLVM_MAX_SLOT=0. My patch ensures that the version of LLVM required by LLVM_MAX_SLOT is the first version of LLVM found in the PATH.
Comment 4 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-06-30 21:58:13 UTC
Oh, right, you are correct. Then probably your idea is the best solution here, though I'd like to think if we can implement it easier. Maybe via some cheap variable substitution.
Comment 5 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-06-30 22:01:33 UTC
Wait a minute... Slot 0 blocks other slots, so you can't have it installed both in /usr/bin and /usr/lib/llvm/* (except for the short period between building new LLVM and removing the old one). In that case, even stale /usr/lib/llvm/* in PATH shouldn't be a problem since there are no files there anymore.
Comment 6 Matt Whitlock 2017-06-30 22:17:49 UTC
(In reply to Michał Górny from comment #5)
> Wait a minute... Slot 0 blocks other slots

Oh, good catch! If this is guaranteed always to be so, then your solution is fine.

Including the following since I had already written it:

(In reply to Michał Górny from comment #4)
> Maybe via some cheap variable substitution.

That would be great, though I don't know how variable substitution could test for the presence of an LLVM installation in various path elements. I suppose you could blindly replace all occurrences of :/usr/lib/llvm/?/bin: with : and then insert the chosen LLVM bin dir into PATH before :/usr/bin:. It'd be more fragile than my solution.

PATH=:${PATH}:
PATH=${PATH//':/usr/lib/llvm/'?'/bin:'/:}
PATH=${PATH%%:/usr/bin:*}:${llvm_prefix}/bin:/usr/bin:${PATH#*:/usr/bin:}
PATH=${PATH%:}
PATH=${PATH#:}

^ This works, though it won't remove any LLVM paths other than /usr/lib/llvm/?/bin.
Comment 7 Matt Whitlock 2017-07-01 02:09:31 UTC
(In reply to Michał Górny from comment #2)
> Could you test
> https://archives.gentoo.org/gentoo-dev/message/
> 598df75c28e7049d9394689dab70854f, please?

Confirming. This patch does allow media-libs/mesa to be built using ccache and distcc.
Comment 8 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-08-12 21:29:06 UTC
The fix is finally pushed. However, bug #627726 still applies; i.e. we still break ccache&co when using CC=clang/CXX=clang++.