Summary: | sys-devel/clang fails to cross compile (needs CBUILD clang-tblgen) | ||
---|---|---|---|
Product: | Gentoo Linux | Reporter: | Jakov Smolić <jsmolic> |
Component: | Current packages | Assignee: | LLVM support project <llvm> |
Status: | UNCONFIRMED --- | ||
Severity: | normal | CC: | ev, herrtimson, luka.perkov, mgorny |
Priority: | Normal | ||
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
See Also: | https://bugs.gentoo.org/show_bug.cgi?id=442958 | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- |
Description
Jakov Smolić
2020-07-07 10:36:15 UTC
I tried to cross compile sys-devel/clang some time ago, with cross-emerge wrappers from crossdev, and while it worked for llvm, it didn't for clang. first problem: clang-tblgen cannot be found. first reason: the ebuild doesn't look at the correct path for the binary, that is old code from when clang wasn't slotted yet. It should be /usr/lib/llvm/X/bin/clang-tblgen second reason: clang-tblgen binary is never emerged in the first place with the hosts clang. But it is compiled during src_compile. Which means that the patch should do the trick, I believe it can be dropped into /etc/portage/patches second problem: llvm/clang documentation is unclear I still haven't found out what parameters to pass along for cross compilation to work once clang-tblgen is present, and the poor documentation doesn't help much either. Try your luck: https://llvm.org/docs/HowToCrossCompileLLVM.html third problem: cross compiling the clang compiler doesn't allow to cross compile for the target you desire if you want to use clang to cross compile a binary from amd64 to let's say arm, you need to enable LLVM_TARGETS for llvm/clang, that is enough to allow cross compile with clang as c compiler, strictly speaking. However, if you want to use the comfort of emerge wrappers and get a little binpkg in the end, you need to write a shell script as a wrapper. Have a look at https://llvm.org/docs/HowToCrossCompileLLVM.html#installing-and-using to get a basic idea of what it might look like. Hi, thank you for the detailed reply. As you pointed out, there are several problems, and this was giving me a lot of headaches over the last few weeks as the documentation does not provide clear instructions on how to cross compile and what steps to take when doing that. I got the feeling that a lot of it is left for the user to figure out which can be problematic. Regarding first reason, yes, the location in which the tblgen binary is searched is wrong, and I believe that should be corrected in the ebuild. Regarding second point, I tried to apply the above mentioned patch to the clang ebuild, but I wasn't able to install it that way. The only workaround so far that I've found is to manually build the clang-tblgen, and move it to /usr/bin directory so that the ebuild can find in on the host system. With that covered, I managed to cross-compile clang using cross-emerge wrappers. Regarding passing additional parameters for cross compilation, I believe it is enough to add -DCMAKE_CROSSCOMPILING=ON and -DCLANG_TABLEGEN which are already covered in the clang ebuild, though the ebuild should look for the clang-tblgen binary at /usr/lib/llvm/${SLOT}/bin Do you think you can provide more information on how you build your standalone clang-tblgen, and more information on how to understand why the patch doesn't work for you? Which slot are you on? I'm still on llvm/clang:9, and the patch for installing clang-tblgen is working for me, but I keep on running into errors about wrong binary format of /usr/armv7a-unknown-linux-gnueabihf/lib/llvm/9/bin/llvm-config when trying to cross-compile clang:9 via the cross-emerge wrappers Sure, I am currently on the slot 10. Regarding the patch, the cross-emerge of clang-10 package would always fail with "/usr/bin/clang-tblgen not found or usable". llvm was cross-compiled succesfully, this was the only bit that was causing problems. So I modified the clang ebuild to include the patch to install clang-tblgen and used that on the host system to emerge clang,. I created the patch and added to the ebuild: ``` PATCHES=( "${FILESDIR}/0001-clang-build-clang-tblgen-by-default.patch" ) ``` And then in the src_install() part of the ebuild: ``` dosym "clang-${clang_version}" "/usr/lib/llvm/${SLOT}/bin/clang-tblgen-${clang_version}" dosym "clang-tblgen-${clang_version}" "/usr/lib/llvm/${SLOT}/bin/clang-tblgen" ``` But I would always run into the same error. Though when I think about it now, this probably worked okay, the issue likely was that the clang ebuild was looking the binary in /usr/bin, not in /usr/lib/llvm/${SLOT}/bin... Then I just decided to try to build the clang-tblgen manually on the host system, but I struggled to find instruction on how to do that. Basically I downloaded the llvm-10.0.0 source, and the did the following: mkdir build && cd build cmake ../llvm -DLLVM_ENABLE_PROJECTS='clang;compiler-rt;lld;clang-tools-extra' -GNinja ninja clang-tblgen mv bin/clang-tblgen /usr/bin Maybe try switching to clang:10 and see if you will have the same issues with wrong binary format. hey there, I did some more fiddeling with this. 1. cross compiling clang needs a hosts clang-tblgen symlinked into /usr/bin/, no doubt about that. 2. yes, that standalone clang-tblgen works wonderfully and does the job. why can't there be a standalone ebuild just for clang-tblgen binary? 3. I still got failures and had troubles finding the error in the log, but they come down to two other binaries from llvm: llvm-tblgen and llvm-config. For unknown reason, the clang ebuild does look for and finds them in the foreign rootfs, and of course the compile stops at this point with some incomatible binary error. Fair enough, I symlinked the hosts llvm-config and llvm-tblgen into the rootfs , and only here I got a working cross compile of sys-devel/clang itself. I did extensive runtime testing with the binary, can't see any problems with it at runtime on the raspberry. so my solution for cross compiling sys-devel/clang is to add a clang-tblgen standalone ebuild, and also to look into the ebuilds and find out why they try to rely an the rootfs llvm binaries they so much need for cross compiling. and yes, I tested with clang:10, it's all fine and working as long as all three binaries are symlinked correctly. Hi, I was also doing more work related with this, trying to cross compile it to arm and arm64. Basically I came to the same conclusion as you regarding llvm-config and llvm-tblgen binaries, I had to symlink host binaries to the target rootfs to get the compilation to work. Did you manage to find out why is Portage using binaries from target rootfs when cross-compiling the package? Regarding point 2, perhaps it could be done so that clang-tblgen is installed as part of the clang ebuild. adding mgorny: can you please tell me if you still feel uncomfortable about installing clang-tblgen along with sys-devel/clang? We have proof that it is an absolute must for cross compiling a sys-devel/clang binary for weak systems such as arm, arm64, etc. thanks I suppose it's fine to install it if the install rule works via distribution targets. the problem regarding llvm-tblgen and llvm-config might be solved here: https://bugs.gentoo.org/745744 Same inherent issue as bug 865009 I think. *** Bug 865009 has been marked as a duplicate of this bug. *** this needs clang-tblgen binary installed on the host to work The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=23022edf6f87c765db32fac54586d6c2d2c668ae commit 23022edf6f87c765db32fac54586d6c2d2c668ae Author: James Le Cuirot <chewi@gentoo.org> AuthorDate: 2023-01-07 11:16:14 +0000 Commit: James Le Cuirot <chewi@gentoo.org> CommitDate: 2023-01-07 13:34:32 +0000 sys-devel/clang: Fix cross-compiling of V16 `LLVM_TOOLS_BINARY_DIR` is now used to find llvm-tblgen and clang-tblgen. clang-tblgen is still not normally installed, but it only needs an additional, albeit inappropriate, configure option now. An upstream fix is under review. Bug: https://bugs.gentoo.org/731264 Signed-off-by: James Le Cuirot <chewi@gentoo.org> sys-devel/clang/clang-16.0.0.9999.ebuild | 15 ++++++++++----- sys-devel/clang/clang-16.0.0_pre20221217.ebuild | 15 ++++++++++----- sys-devel/clang/clang-16.0.0_pre20221226.ebuild | 15 ++++++++++----- sys-devel/clang/clang-16.0.0_pre20230101.ebuild | 13 +++++++++---- 4 files changed, 39 insertions(+), 19 deletions(-) I was only half right before. LLVM_TOOLS_BINARY_DIR is only used to find llvm-tblgen, not clang-tblgen. The latter is found on the PATH, but we cannot rely on that because llvm.eclass adds ESYSROOT's LLVM bin to the PATH, and that breaks cross-compiling after the first time. Instead, we must override the location with CLANG_TABLEGEN, as we did previously. I've fixed this now. how sensitiv is the crosscompiled sys-devel/clang to differences in gcc major version, i.e. does it break when its crosscompiled with a gcc-10 toolchain, but executed at runtime with a gcc-12 toolchain? (In reply to tt_1 from comment #15) > how sensitiv is the crosscompiled sys-devel/clang to differences in gcc > major version, i.e. does it break when its crosscompiled with a gcc-10 > toolchain, but executed at runtime with a gcc-12 toolchain? I don't see why that would ever matter. The only time it could be a problem is if you built with a newer GCC then tried to run it with an older libstdc++. |