Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 667094 - sys-devel/clang-6.0.1 does not install clang-tblgen, but cross compile needs it
Summary: sys-devel/clang-6.0.1 does not install clang-tblgen, but cross compile needs it
Status: RESOLVED OBSOLETE
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: LLVM support project
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-09-25 22:44 UTC by tt_1
Modified: 2019-04-27 17:22 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
patch allowing clang to find the hosts llvm-config (llvm-config.patch,358 bytes, patch)
2018-09-26 22:24 UTC, tt_1
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description tt_1 2018-09-25 22:44:33 UTC
just like llvm has a file named llvm-tblgen, which is used by the llvm ebuild when it is going to be cross compiled, clang demands to have a file named clang-tblgen. But all efforts to cross compile fail, because the file is not installed on the host system and therefore it must fail during src_configure: 

	if tc-is-cross-compiler; then
		[[ -x "/usr/bin/clang-tblgen" ]] \
			|| die "/usr/bin/clang-tblgen not found or usable"eq


In fact, the file gets compiled and is present at: 

/var/tmp/portage/sys-devel/clang-6.0.1/work/x/y/cfe-6.0.1.src-abi_x86_64.amd64/bin/

but never gets copied through during src_install. 

Apart from not being copied, the clang ebuild is expecting it to be in the wrong folder if I'm not totally misstaken. It should be looking for it at /usr/lib/llvm/6/bin/ instead of at /usr/bin/ ; 

upon using google, I found this old message from 2015 about the same problem:  http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20150629/132268.html
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2018-09-26 07:22:30 UTC
This code was added as part of bug #442958 and hasn't been updated since.  I don't do cross, so I can't really test it.  If someone supplies a good patch, I can merge it.
Comment 2 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2018-09-26 08:02:30 UTC
Hmm, I see that LLVM has some CMake cross-compiling support which actually builds host tablegen.  Maybe removing our hacks would help?
Comment 3 tt_1 2018-09-26 08:12:33 UTC
there are two major problems here: 

- clang is probing the wrong folder 
- clang does compile clang-tblgen on the host side, but it doesn't get installed during src_install

I wonder what you mean by suggesting to remove some hacks? 

opened a pullrequest the minute your response came in, mainly for discussing: https://github.com/gentoo/gentoo/pull/9976

Please note: this is purely academic at the moment, because I can't cross compile llvm at the moment to beginn with, but as the wrong path and lack of clang-tblgen is so obvious, I don't see much of a problem to fix this before tackling llvm. You may disagree?
Comment 4 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2018-09-26 08:20:07 UTC
FWIU, you shouldn't need another LLVM/Clang installed.  LLVM should build a host tablegen for its own use, and use that.
Comment 5 tt_1 2018-09-26 08:47:04 UTC
My goal is to do a binpkg of llvm/clang for armv7 with cross-emerge, and if succeeded to copy it over to my armv7 device and install it with emerge -k ; so this is not about setting up a llvm/clang cross compiler with crossdev but to use the armv7a-unkown-linux-gnueabihf toolchain for this matter. It works pretty good for a base system, only all python related things are still a bit wacky. 

Why do I want to do this? arm devices are slow, but they are also memory restricted, the last time I tried to compile llvm nativly on an rpi2 it took around 12 hrs, and 16 hrs for clang, including heavy swapping. Using cross compile with cross-emerge, it would be < 10minutes for sure on a recent quadcore. 

This might be helpfull: https://llvm.org/docs/HowToCrossCompileLLVM.html

I'll try to understand it better and later on forge it into the ebuild.
Comment 6 James Le Cuirot gentoo-dev 2018-09-26 11:19:29 UTC
mgorny isn't talking about building a cross compiler. You're suggesting that LLVM should be installed on the build host first but that is a little excessive when you only need one small binary that isn't normally installed anyway.

Even if we were to do that, it is not currently possibly to express this kind of dependency. The experimental EAPI 5-hdepend could do it with HDEPEND="targetroot? ( sys-devel/clang )" but we decided not to adopt that feature in EAPI 7 because we prefer the other approach.

That is to build just clang-tblgen for the build host in a temporary location as part of the wider clang build process for the runtime host. I haven't looked into the specifics here but if mgorny says that is handling in CMake for this then I think I know what he means because I contributed changes to MariaDB some years ago to do exactly the same thing. My plate is full right now and I have no need for clang on ARM so I would appreciate some help. You'd probably need to use tc-env_build from toolchain-funcs.eclass. The following is the change I made to MariaDB, which explains how you use it.

https://github.com/MariaDB/server/commit/cdc38712e4d359348346758e47e1ebc7d1743f82
Comment 7 tt_1 2018-09-26 22:24:39 UTC
Created attachment 548008 [details, diff]
patch allowing clang to find the hosts llvm-config

Good news, I was able to cross-emerge the llvm:6 dep into the armv7 prefix. Bad news, clang now is unable to find llvm-config. 

I tried to patch the path of the hosts llvm-config into the ebuild, is it safe to do so?
Comment 8 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2018-09-27 07:02:44 UTC
I don't think using host's llvm-config is a good idea but it might work.  I think -9999 doesn't need llvm-config anymore, and will just use CMake files from the target system.

I've looked more into LLVM's cross support and it seems to use cheap hacks (i.e. spawning another cmake) to build native targets (tablegen and llvm-config).  I suppose we should just build them manually in the ebuild, and then point it to them.
Comment 9 tt_1 2018-09-27 21:14:30 UTC
Regarding the missing llvm-config: I can point to the hosts llvm-config, which produces ABI related errors, or point to the before cross-compiled llvm-config in the prefix, but that gives errors about tblgen. Not sure if about the cause, I'll provide the logs tomorrow. 

llvm-9999 can be cross compiled, clang-9999 not, tried it, same errors as before. I'm open to suggestions what else to try?
Comment 10 tt_1 2018-10-05 07:08:06 UTC
So I just emerged llvm:0 with USE="clang" on my host, and tried to cross-emerge llvm:0 with USE="clang" using cross-emerge. It gives a similar error to before when linking(?) : 

[99/100] Linking CXX static library lib/libLLVMSupport.a
[100/100] Linking CXX executable bin/llvm-config
FAILED: bin/llvm-config 
: && /usr/bin/armv7a-unknown-linux-gnueabihf-g++  -O2 -pipe -fomit-frame-pointer -fPIC -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -Werror=date-time -std=c++11 -ffunction-sections -fdata-sections -O3 -DNDEBUG  -Wl,-O1 -Wl,--as-needed -Wl,-allow-shlib-undefined    -Wl,-O3 -Wl,--gc-sections tools/llvm-config/CMakeFiles/llvm-config.dir/llvm-config.cpp.o  -o bin/llvm-config  -Wl,-rpath,"\$ORIGIN/../lib" lib/libLLVMSupport.a -lpthread -lrt -ldl -lcurses -lpthread -lz -lm && :
lib/libLLVMSupport.a: error adding symbols: Archive has no index; run ranlib to add one
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
Comment 11 tt_1 2019-04-27 17:22:01 UTC
this has come a long way. 

>=binutils-libs-2.31.1 can now be cross-emerged, which means that all llvm cross-emerges just fine without any problem. 

clang still doesn't, but I'll open a new bug since most of the things discussed here are outdated. hope that's ok.