Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 731264 - sys-devel/clang fails to cross compile (needs CBUILD clang-tblgen)
Summary: sys-devel/clang fails to cross compile (needs CBUILD clang-tblgen)
Status: UNCONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal with 2 votes (vote)
Assignee: LLVM support project
URL:
Whiteboard:
Keywords:
: 865009 (view as bug list)
Depends on:
Blocks:
 
Reported: 2020-07-07 10:36 UTC by Jakov Smolić
Modified: 2023-01-19 20:07 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jakov Smolić archtester gentoo-dev 2020-07-07 10:36:15 UTC
Hello, apologies in advance if this is an invalid bug.

I'm having some issues with clang cross compilation. Specifically, the missing `clang-tblgen` binary is causing problems. 

I have been investigating this for some time, and came across discussions in [1] and [2], but I wasn't able to figure this out as of yet. According to [2], the `tblgen` binary is not installed by the clang's build system.

Is there a workaround for this, would including the patch from [3] work here?



[1] https://bugs.gentoo.org/442958
[2] https://github.com/NixOS/nixpkgs/issues/40602
[3] https://reviews.llvm.org/D10916#change-9P8m0KkVhKUz
Comment 1 tt_1 2020-07-23 10:38:17 UTC
I tried to cross compile sys-devel/clang some time ago, with cross-emerge wrappers from crossdev, and while it worked for llvm, it didn't for clang. 

first problem: clang-tblgen cannot be found. 

first reason: the ebuild doesn't look at the correct path for the binary, that is old code from when clang wasn't slotted yet. It should be /usr/lib/llvm/X/bin/clang-tblgen

second reason: clang-tblgen binary is never emerged in the first place with the hosts clang. But it is compiled during src_compile. Which means that the patch should do the trick, I believe it can be dropped into /etc/portage/patches 

second problem: llvm/clang documentation is unclear

I still haven't found out what parameters to pass along for cross compilation to work once clang-tblgen is present, and the poor documentation doesn't help much either. Try your luck: https://llvm.org/docs/HowToCrossCompileLLVM.html

third problem: cross compiling the clang compiler doesn't allow to cross compile for the target you desire

if you want to use clang to cross compile a binary from amd64 to let's say arm, you need to enable LLVM_TARGETS for llvm/clang, that is enough to allow cross compile with clang as c compiler, strictly speaking. 

However, if you want to use the comfort of emerge wrappers and get a little binpkg in the end, you need to write a shell script as a wrapper. Have a look at https://llvm.org/docs/HowToCrossCompileLLVM.html#installing-and-using to get a basic idea of what it might look like.
Comment 2 Jakov Smolić archtester gentoo-dev 2020-07-24 09:12:29 UTC
Hi, thank you for the detailed reply.
As you pointed out, there are several problems, and this was giving me a lot of 
headaches over the last few weeks as the documentation does not provide clear instructions on how to cross compile and what steps to take when doing that. I got the feeling that a lot of it is left for the user to figure out which can be problematic.

Regarding first reason, yes, the location in which the tblgen binary is searched is wrong, and I believe that should be corrected in the ebuild.

Regarding second point, I tried to apply the above mentioned patch to the clang ebuild, but I wasn't able to install it that way.

The only workaround so far that I've found is to manually build the clang-tblgen,
and move it to /usr/bin directory so that the ebuild can find in on the host system.

With that covered, I managed to cross-compile clang using cross-emerge wrappers. Regarding passing additional parameters for cross compilation, I believe it is enough to add -DCMAKE_CROSSCOMPILING=ON and -DCLANG_TABLEGEN which are already covered in the clang ebuild, though the ebuild should look for the clang-tblgen binary at /usr/lib/llvm/${SLOT}/bin
Comment 3 tt_1 2020-07-24 12:23:50 UTC
Do you think you can provide more information on how you build your standalone clang-tblgen, and more information on how to understand why the patch doesn't work for you? Which slot are you on? 

I'm still on llvm/clang:9, and the patch for installing clang-tblgen is working for me, but I keep on running into errors about wrong binary format of /usr/armv7a-unknown-linux-gnueabihf/lib/llvm/9/bin/llvm-config when trying to cross-compile clang:9 via the cross-emerge wrappers
Comment 4 Jakov Smolić archtester gentoo-dev 2020-07-24 17:53:47 UTC
Sure,
I am currently on the slot 10. 

Regarding the patch, the cross-emerge of clang-10 package would always fail with "/usr/bin/clang-tblgen not found or usable". llvm was cross-compiled succesfully, this was the only bit that was causing problems.
So I modified the clang ebuild to include the patch to install clang-tblgen and used that on the host system to emerge clang,.
I created the patch and added to the ebuild:

```
PATCHES=(
	"${FILESDIR}/0001-clang-build-clang-tblgen-by-default.patch"
)

```

And then in the src_install() part of the ebuild:

```
dosym "clang-${clang_version}" "/usr/lib/llvm/${SLOT}/bin/clang-tblgen-${clang_version}"
dosym "clang-tblgen-${clang_version}" "/usr/lib/llvm/${SLOT}/bin/clang-tblgen"

```
But I would always run into the same error. Though when I think about it now, this probably worked okay, the issue likely was that the clang ebuild was looking the binary in /usr/bin, not in /usr/lib/llvm/${SLOT}/bin...


Then I just decided to try to build the clang-tblgen manually on the host system, but I struggled to find instruction on how to do that.
Basically I downloaded the llvm-10.0.0 source, and the did the following:

mkdir build && cd build
cmake ../llvm -DLLVM_ENABLE_PROJECTS='clang;compiler-rt;lld;clang-tools-extra' -GNinja
ninja clang-tblgen
mv bin/clang-tblgen /usr/bin


Maybe try switching to clang:10 and see if you will have the same issues with wrong binary format.
Comment 5 tt_1 2020-08-04 10:57:15 UTC
hey there, I did some more fiddeling with this. 

1. cross compiling clang needs a hosts clang-tblgen symlinked into /usr/bin/, no doubt about that. 

2. yes, that standalone clang-tblgen works wonderfully and does the job. why can't there be a standalone ebuild just for clang-tblgen binary? 

3. I still got failures and had troubles finding the error in the log, but they come down to two other binaries from llvm: llvm-tblgen and llvm-config. For unknown reason, the clang ebuild does look for and finds them in the foreign rootfs, and of course the compile stops at this point with some incomatible binary error. Fair enough, I symlinked the hosts llvm-config and llvm-tblgen into the rootfs , and only here I got a working cross compile of sys-devel/clang itself. 

I did extensive runtime testing with the binary, can't see any problems with it at runtime on the raspberry. 

so my solution for cross compiling sys-devel/clang is to add a clang-tblgen standalone ebuild, and also to look into the ebuilds and find out why they try to rely an the rootfs llvm binaries they so much need for cross compiling. 

and yes, I tested with clang:10, it's all fine and working as long as all three binaries are symlinked correctly.
Comment 6 Jakov Smolić archtester gentoo-dev 2020-08-13 13:12:34 UTC
Hi,
I was also doing more work related with this, trying to cross compile it to arm and arm64.

Basically I came to the same conclusion as you regarding llvm-config and llvm-tblgen binaries, I had to symlink host binaries to the target rootfs to get the compilation to work. Did you manage to find out why is Portage using binaries from target rootfs when cross-compiling the package?

Regarding point 2, perhaps it could be done so that clang-tblgen is installed as part of the  clang ebuild.
Comment 7 tt_1 2020-08-28 17:19:05 UTC
adding mgorny: can you please tell me if you still feel uncomfortable about installing clang-tblgen along with sys-devel/clang? We have proof that it is an absolute must for cross compiling a sys-devel/clang binary for weak systems such as arm, arm64, etc. 

thanks
Comment 8 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-08-28 21:46:35 UTC
I suppose it's fine to install it if the install rule works via distribution targets.
Comment 9 tt_1 2020-10-01 07:41:01 UTC
the problem regarding llvm-tblgen and llvm-config might be solved here: https://bugs.gentoo.org/745744
Comment 10 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-11-18 08:06:35 UTC
Same inherent issue as bug 865009 I think.
Comment 11 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-11-18 08:07:19 UTC
*** Bug 865009 has been marked as a duplicate of this bug. ***
Comment 12 tt_1 2022-11-18 11:30:08 UTC
this needs clang-tblgen binary installed on the host to work
Comment 13 Larry the Git Cow gentoo-dev 2023-01-07 13:34:45 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=23022edf6f87c765db32fac54586d6c2d2c668ae

commit 23022edf6f87c765db32fac54586d6c2d2c668ae
Author:     James Le Cuirot <chewi@gentoo.org>
AuthorDate: 2023-01-07 11:16:14 +0000
Commit:     James Le Cuirot <chewi@gentoo.org>
CommitDate: 2023-01-07 13:34:32 +0000

    sys-devel/clang: Fix cross-compiling of V16
    
    `LLVM_TOOLS_BINARY_DIR` is now used to find llvm-tblgen and
    clang-tblgen. clang-tblgen is still not normally installed, but it only
    needs an additional, albeit inappropriate, configure option now. An
    upstream fix is under review.
    
    Bug: https://bugs.gentoo.org/731264
    Signed-off-by: James Le Cuirot <chewi@gentoo.org>

 sys-devel/clang/clang-16.0.0.9999.ebuild        | 15 ++++++++++-----
 sys-devel/clang/clang-16.0.0_pre20221217.ebuild | 15 ++++++++++-----
 sys-devel/clang/clang-16.0.0_pre20221226.ebuild | 15 ++++++++++-----
 sys-devel/clang/clang-16.0.0_pre20230101.ebuild | 13 +++++++++----
 4 files changed, 39 insertions(+), 19 deletions(-)
Comment 14 James Le Cuirot gentoo-dev 2023-01-08 15:24:51 UTC
I was only half right before. LLVM_TOOLS_BINARY_DIR is only used to find llvm-tblgen, not clang-tblgen. The latter is found on the PATH, but we cannot rely on that because llvm.eclass adds ESYSROOT's LLVM bin to the PATH, and that breaks cross-compiling after the first time. Instead, we must override the location with CLANG_TABLEGEN, as we did previously. I've fixed this now.
Comment 15 tt_1 2023-01-19 20:03:55 UTC
how sensitiv is the crosscompiled sys-devel/clang to differences in gcc major version, i.e. does it break when its crosscompiled with a gcc-10 toolchain, but executed at runtime with a gcc-12 toolchain?
Comment 16 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-01-19 20:07:34 UTC
(In reply to tt_1 from comment #15)
> how sensitiv is the crosscompiled sys-devel/clang to differences in gcc
> major version, i.e. does it break when its crosscompiled with a gcc-10
> toolchain, but executed at runtime with a gcc-12 toolchain?

I don't see why that would ever matter. The only time it could be a problem is if you built with a newer GCC then tried to run it with an older libstdc++.