Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 680652 - sys-devel/crossdev: handle LLVM, Clang, and Rust
Summary: sys-devel/crossdev: handle LLVM, Clang, and Rust
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal with 2 votes (vote)
Assignee: Gentoo Crossdev team
URL:
Whiteboard:
Keywords: PullRequest
Depends on:
Blocks: 917787 918451
  Show dependency tree
 
Reported: 2019-03-17 09:04 UTC by James Le Cuirot
Modified: 2024-02-10 07:49 UTC (History)
13 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Patch against dev-lang/rust ebuild to add armv7 std crates (rust-std-armv7.patch,1.72 KB, patch)
2019-03-17 10:24 UTC, James Le Cuirot
Details | Diff
patch against the cargo2.eclass (cargo2-cross.patch,2.84 KB, patch)
2019-03-18 12:09 UTC, tt_1
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description James Le Cuirot gentoo-dev 2019-03-17 09:04:28 UTC
I've been trying to cross-compile Firefox today and have it mostly working. The USE flags for LLVM, Clang, and Rust are not enforced though, which is annoying, and Rust is also missing the cross-compiled crates.

I've been thinking about how crossdev could help out here. It's different to the GNU toolchain as all targets are handled by a single installation of each.

I'm thinking that the LLVM_TARGETS flags could simply be handled by a mapping that's used to add to the package.use file. For example:

case ${CTARGET} in
arm*) LLVM_FLAG=ARM ;;
aarch64*) LLVM_FLAG=AArch64 ;;
...
esac

/etc/portage/package.use/cross-${CTARGET}:
dev-lang/rust ${LLVM_FLAG}
sys-devel/clang ${LLVM_FLAG}
sys-devel/llvm ${LLVM_FLAG}

That's the easy part. For the Rust crates, tt_1 sent me a patch against the ebuild that adds a rust-std-armv7 USE flag for a specific CTARGET but that's not flexible or scalable. I was thinking of building up a variable, possibly an array, of targets via /etc/portage/env but however I do it, I get syntax errors because it's not simply FOO=bar. You can put functions in these files but I don't know if that helps. Any ideas?

I have attached tt_1's patch so you can see roughly what is involved. This link may also be helpful. It implies that we could perhaps do the crates as separate packages?

https://github.com/japaric/rust-cross#cross-compiling-the-standard-crates

There's the question of crates outside std but I guess those would just live in SYSROOT like other libraries do. I'm not even sure if we have packaged any? There's no dev-rust category.
Comment 1 Sergei Trofimovich (RETIRED) gentoo-dev 2019-03-17 09:34:06 UTC
(In reply to James Le Cuirot from comment #0)

> I'm thinking that the LLVM_TARGETS flags could simply be handled by a
> mapping that's used to add to the package.use file. For example:
> 
> case ${CTARGET} in
> arm*) LLVM_FLAG=ARM ;;
> aarch64*) LLVM_FLAG=AArch64 ;;
> ...
> esac

Similar mapping is defined in multilib.eclass:multilib_env(). crossdev uses that to not embed most basic information about targets. You can do something similar.

> /etc/portage/package.use/cross-${CTARGET}:
> dev-lang/rust ${LLVM_FLAG}
> sys-devel/clang ${LLVM_FLAG}
> sys-devel/llvm ${LLVM_FLAG}

cross-${CTARGET} file normally contains bits only for cross-*/* packages. They try hard not to influence host's install: don't need host USE-flag or variable tweaks. Everything is self-contained in cross-*/* and ${SYSROOT}.

Affecting host's install sounds fragile. But whatever works for you.

I would suggest pkg_pretend() code in every rust package that would bail out if ${CHOST} requires some USE-flag from ${CBUILD}'s llvm and a clear message for user how they should change LLVM_TARGETS.

Then you would not need special crossdev support at all.

> That's the easy part. For the Rust crates, tt_1 sent me a patch against the
> ebuild that adds a rust-std-armv7 USE flag for a specific CTARGET but that's
> not flexible or scalable. I was thinking of building up a variable, possibly
> an array, of targets via /etc/portage/env but however I do it, I get syntax
> errors because it's not simply FOO=bar. You can put functions in these files
> but I don't know if that helps. Any ideas?
> 
> I have attached tt_1's patch so you can see roughly what is involved. This
> link may also be helpful.

I don't see a patch. Thus I don't really understand the mechanical problems of the above.

> It implies that we could perhaps do the crates as
> separate packages?
> 
> https://github.com/japaric/rust-cross#cross-compiling-the-standard-crates
> 
> There's the question of crates outside std but I guess those would just live
> in SYSROOT like other libraries do. I'm not even sure if we have packaged
> any? There's no dev-rust category.

For each end-user tool cargo.eclass packages every source crate dependency needed to build a tool standalone (effectively bypassing gentoo's package management). That could simplify cross-compilation if rust already knows how to build some of them for ${CBUILD} and some of them for ${CHOST}.
Comment 2 James Le Cuirot gentoo-dev 2019-03-17 10:24:55 UTC
Created attachment 569462 [details, diff]
Patch against dev-lang/rust ebuild to add armv7 std crates

Sorry, the patch upload messed up.
Comment 3 Sergei Trofimovich (RETIRED) gentoo-dev 2019-03-17 11:14:32 UTC
(In reply to James Le Cuirot from comment #2)
> Created attachment 569462 [details, diff] [details, diff]
> Patch against dev-lang/rust ebuild to add armv7 std crates

> --- a/dev-lang/rust/rust-1.32.0.ebuild
> +++ b/dev-lang/rust/rust-1.32.0.ebuild
> -IUSE="clippy cpu_flags_x86_sse2 debug doc libressl rls rustfmt system-llvm wasm ${ALL_LLVM_TARGETS[*]}"
> +IUSE="clippy cpu_flags_x86_sse2 debug doc libressl rls rustfmt rust-std-armv7 system-llvm wasm ${ALL_LLVM_TARGETS[*]}"
> ...
> @@ -126,6 +126,10 @@ src_configure() {
> ...
> +	if use rust-std-armv7; then
> +		rust_targets="${rust_targets},\"armv7-unknown-linux-gnueabihf\""
> +	fi
> ...
> +	if use rust-std-armv7; then
> +		cat <<- EOF >> "${S}"/config.toml
> +			[target.armv7-unknown-linux-gnueabihf]
> +			cc = "armv7a-unknown-linux-gnueabihf-gcc"
> +			cxx = "armv7a-unknown-linux-gnueabihf-g++"
> +			linker = "armv7a-unknown-linux-gnueabihf-gcc"
> +			ar = "armv7a-unknown-linux-gnueabihf-ar"
> +			llvm-config = "/usr/armv7a-unknown-linux-gnueabihf/usr/lib/llvm/7/bin/llvm-config"
> +		EOF
> +	fi
> }

You would need to decide on the flexibility of cross toolchains you want.
For example gentoo's gcc allows you to:
1. have multiple equivalent cross-toolchains:
  - x86_64-HEAD-linux-gnu
  - x86_64-pc-linux-gnu
2. Not couple cross-toolchain package versions with native package versions:
  - x86_64-pc-linux-gnu-gcc-8.3.0 with sanitizer enabled
  - x86_64-HEAD-linux-gnu-gcc-8.3.0-r1 with sanitizer disabled
  (also allows passing different CFLAGS/CXXFLAGS from default install)
3. cross-libc can have a different version.

If you want to follow similar flexibility you might end up needing a similar extra package:
    cross-${CTARGET}/rust (symlink to dev-lang/rust)
and install / DEPEND on what is missing.

If you don't need all that complexity/flexibility you could do static LLVM_TARGET->${CTARGET} mapping. I think you could even avoid most of gcc toolchain and use clang for C stuff:
    cc = "clang -<tuple arg?>"
    linker = "clang -<tuple arg?>"

Don't know if rust needs libc for basic setup to work. If rust does not libc you could avoid fiddling with crossdev completely and have LLVM_TARGET define everything for you.
Comment 4 James Le Cuirot gentoo-dev 2019-03-17 11:21:51 UTC
(In reply to Sergei Trofimovich from comment #1)
> Similar mapping is defined in multilib.eclass:multilib_env(). crossdev uses
> that to not embed most basic information about targets. You can do something
> similar.

Handy!

> cross-${CTARGET} file normally contains bits only for cross-*/* packages.
> They try hard not to influence host's install: don't need host USE-flag or
> variable tweaks. Everything is self-contained in cross-*/* and ${SYSROOT}.
> 
> Affecting host's install sounds fragile. But whatever works for you.
> 
> I would suggest pkg_pretend() code in every rust package that would bail out
> if ${CHOST} requires some USE-flag from ${CBUILD}'s llvm and a clear message
> for user how they should change LLVM_TARGETS.
> 
> Then you would not need special crossdev support at all.

I thought that the existing approach was more because the GNU toolchains have to be built separately. Whether we use package.use or pkg_pretend, the host's install is going to be affected regardless. The pkg_pretend approach requires more manual effort by the user so I felt it was a poor last resort. Not everyone will need this so it could be done through one of the "extra fun" crossdev options.

> For each end-user tool cargo.eclass packages every source crate dependency
> needed to build a tool standalone (effectively bypassing gentoo's package
> management). That could simplify cross-compilation if rust already knows how
> to build some of them for ${CBUILD} and some of them for ${CHOST}.

Knowing how Rust likes to statically link everything, that's what I guessed.
Comment 5 tt_1 2019-03-18 12:03:52 UTC
Not sure wether I can give some helfpull insight here, but I'll try nonetheless. 

When I first thought about this I wanted to cross-compile dev-lang/rust itself with the help of the cross-toolchain provided by crossdev+wrappers. Making this work is a mountain to climb, so I went for the approach to build a fitting rust-std instead. 

The patch I hacked together in #2 is a proof of concept, it lacks a few things such as stripping the foreign rust-std libraries with cross-binutils, but it does the job of providing dev-lang/rust with the ability to cross-compile. Given that there is a cross-gcc available, since rustc needs it for linking. 

The www-client/firefox ebuild can detect and use it for its compilation, which is what I had intended. 

So, given that users can think for themselves, I'd say to enable the ability to cross-emerge firefox/thunderbird it is sufficent to add a rust-std{-bin} ebuild to the tree, which is where going for different CHOSTS could be mapped to different LLVM_FLAGS. Which could be painfull on its own, so maybe even with a new use_extend of RUST_STD?
Comment 6 tt_1 2019-03-18 12:04:37 UTC
If the ebuild in question of cross-compilation does inherit cargo2.eclass, it needs some additional work. Cargo wants to be informed about the linker on a per target base, and it tries to find this in ~/.cargo/config.

Which works fine, unless you're using portage. The cargo2.eclass takes care of exporting all values to a local folder within the temp or work folder I believe, and probes here instead of at ~/.cargo; which works around user settings needed. 

So simply add this to cargo_gen_config() 

	[target.${TARGET_RUST_HOST]
	linker = "${TARGET_CHOST}-gcc"

Next cargo build has to be executed with the additional --target=${TARGET_RUST_HOST command, same goes for cargo install. 

This makes it possible to cross-compile with portage, given that there is a cross-compiler present. Stripping with cross-binutils not enabled yet.
Comment 7 tt_1 2019-03-18 12:09:31 UTC
Created attachment 569592 [details, diff]
patch against the cargo2.eclass

this is to demonstrate the problem, not meant for the tree.
Comment 8 tt_1 2019-03-23 15:35:59 UTC
for clang its best practice to use some kind of wrapper. 

at least these three values have to be added to each individual clang command: 

--target
--sysroot
-isysroot

I'll do some more research and post a wrapper once I got it to work.
Comment 9 tt_1 2019-12-13 23:14:44 UTC
here's a working wrapper for cross compiling with clang: 

#!/bin/bash

SYSROOT="/usr/armv7a-unknown-linux-gnueabihf"
TARGET="armv7a-linux-gnueabihf"
COMPILER_PATH="${SYSROOT}/usr/armv7a-unknown-linux-gnueabihf/gcc-bin/9.2.0/"

exec env COMPILER_PATH="${COMPILER_PATH}" \
	clang --target=${TARGET} \
		--sysroot="${SYSROOT}" \
		-isysroot "${SYSROOT}" \
		-L"${COMPILER_PATH}" \
		"$@"



I took libpng an test emerged it, it also passes firefox src_configure, given that it's put into /usr/lib/llvm/x/bin/ and made executeable. 

sys-devel/binutils must have multitarget use flag enabled.
Comment 10 Larry the Git Cow gentoo-dev 2020-06-12 00:31:09 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0326022d60e80c438d0ea90f489eebb722320417

commit 0326022d60e80c438d0ea90f489eebb722320417
Author:     Georgy Yakovlev <gyakovlev@gentoo.org>
AuthorDate: 2020-06-11 22:12:30 +0000
Commit:     Georgy Yakovlev <gyakovlev@gentoo.org>
CommitDate: 2020-06-12 00:30:49 +0000

    dev-lang/rust: add experimental cross support to 1.44.0
    
    Brief usage how-to in the ebuild
    Bug: https://bugs.gentoo.org/680652
    Bug: https://bugs.gentoo.org/680652
    Package-Manager: Portage-2.3.100, Repoman-2.3.22
    Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org>

 dev-lang/rust/rust-1.44.0.ebuild | 57 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 57 insertions(+)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0326022d60e80c438d0ea90f489eebb722320417

commit 0326022d60e80c438d0ea90f489eebb722320417
Author:     Georgy Yakovlev <gyakovlev@gentoo.org>
AuthorDate: 2020-06-11 22:12:30 +0000
Commit:     Georgy Yakovlev <gyakovlev@gentoo.org>
CommitDate: 2020-06-12 00:30:49 +0000

    dev-lang/rust: add experimental cross support to 1.44.0
    
    Brief usage how-to in the ebuild
    Bug: https://bugs.gentoo.org/680652
    Bug: https://bugs.gentoo.org/680652
    Package-Manager: Portage-2.3.100, Repoman-2.3.22
    Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org>

 dev-lang/rust/rust-1.44.0.ebuild | 57 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 57 insertions(+)
Comment 11 Larry the Git Cow gentoo-dev 2020-09-25 07:38:26 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=e2701fa0f4b3b3a3d971c2274c36b4ea4bce7181

commit e2701fa0f4b3b3a3d971c2274c36b4ea4bce7181
Author:     Georgy Yakovlev <gyakovlev@gentoo.org>
AuthorDate: 2020-09-25 07:26:47 +0000
Commit:     Georgy Yakovlev <gyakovlev@gentoo.org>
CommitDate: 2020-09-25 07:37:58 +0000

    sys-devel/rust-std: new package, for crossdev #679878
    
    EXPERIMENTAL!
    
    Bug: https://bugs.gentoo.org/680652
    Bug: https://bugs.gentoo.org/679878
    Bug: https://bugs.gentoo.org/689336
    Package-Manager: Portage-3.0.8, Repoman-3.0.1
    Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org>

 sys-devel/rust-std/Manifest               |   1 +
 sys-devel/rust-std/metadata.xml           |  21 +++++
 sys-devel/rust-std/rust-std-1.46.0.ebuild | 146 ++++++++++++++++++++++++++++++
 3 files changed, 168 insertions(+)
Comment 12 Georgy Yakovlev archtester gentoo-dev 2020-09-25 07:39:28 UTC
# rust-std + crossdev = <3

EXPERIMENTAL! not sure if stage0 stdlib is good enough, but it produces working binaries just fine.

TLDR: (your file paths may differ)

assuming you already have `aarch64-unknown-linux-gnu` toolchain bootstrapped via crossdev.

```bash
cd /var/db/repos/crossdev/cross-aarch64-unknown-linux-gnu
ln -s /var/db/repos/gentoo/sys-devel/rust-std
echo 'cross-aarch64-unknown-linux-gnu/rust-std **' >> /etc/portage/package.accept_keywords/aarch64
emerge cross-aarch64-unknown-linux-gnu/rust-std -av
cd /some/rust/source
export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-unknown-linux-gnu-gcc
cargo build --target aarch64-unknown-linux-gnu
```

for crates that link to libstdc++ you may need to add something like this:
`CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_RUSTFLAGS="-C link-arg=-Wl,-rpath-link,/usr/lib/gcc/aarch64-unknown-linux-gnu/9.3.0"`
More info on that: https://bugs.gentoo.org/549994


## Package if used as cross- will install new stdlib to `/usr/lib64/rust-<ver>/rustlib/aarch64-unknown-linux-gnu`, for example.

currently this method is not as flexible as fully custom target support via rust ebuild itself,
because rust-toolchain.eclass' rust_abi() function needs better triple transformations.
for example it will always transform CHOST x86-64-gentoo-linux-musl to RHOST x86-64-unknown-linux-musl
but it can be fixed, of course. https://bugs.gentoo.org/671736

There's a workaround, ERUST_STD_RTARGET=whatever-weird-rust-triple allows setting any custom one.
that will set `build.target` in `config.toml`

it's very fast, one can get a new target in couple of minutes, instead of waiting for full bootstrap
of dev-lang/rust with RUST_CROSS_TARGETS.

Please provide feedback in https://bugs.gentoo.org/679878
Comment 13 SpanKY gentoo-dev 2021-10-27 08:49:33 UTC
i don't see what value crossdev can add for llvm.  it's a monolithic build and includes dedicated knobs for enabling specific arches.  having crossdev drop flags onto it independent of what already exists feels like overstepping.

Rust also is a monolithic build, but requires the list of tuples at its build time.  so also seems like an issue a bit out of scope for crossdev as the Rust ebuild would figure out how to expose the config to users.

so i'm not sure what the expectation is for this bug.
Comment 14 Larry the Git Cow gentoo-dev 2023-08-22 17:05:36 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/crossdev.git/commit/?id=bc2c2acbc92f2119db9633fd186978265eed8f03

commit bc2c2acbc92f2119db9633fd186978265eed8f03
Author:     Alfred Persson Forsberg <cat@catcream.org>
AuthorDate: 2023-07-05 18:59:29 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2023-08-22 17:05:32 +0000

    Support standalone LLVM/Clang as crosscompiler
    
    This change makes it possible to use Clang instead of GCC in
    Crossdev. As LLVM is already able to target other architectures,
    provided that LLVM_TARGETS is set accordingly, the only thing needed
    to do is compile builtins (compiler-rt) for the target triple. Note
    that compiler-rt needs libc headers to target when building, and in
    turn linux-headers needs to be installed for Linux targets, so most
    stages except binutils and GCC are still there.
    
    Currently having both a GCC and LLVM Crossdev environment installed
    for the same triple is not supported since many ebuilds just use
    /usr/${CTARGET} as a hardcoded sysroot, but I plan adding support.
    
    Note: by standalone I mean a pure LLVM toolchain not dependent on an
    existing GCC toolchain.
    
    Bug: https://bugs.gentoo.org/680652
    Signed-off-by: Alfred Persson Forsberg <cat@catcream.org>
    Closes: https://github.com/gentoo/crossdev/pull/10
    Signed-off-by: Sam James <sam@gentoo.org>

 crossdev                  | 243 ++++++++++++++++++++++++++++++++++++----------
 wrappers/cross-pkg-config |   1 +
 wrappers/emerge-wrapper   |  20 ++++
 3 files changed, 215 insertions(+), 49 deletions(-)
Comment 15 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-08-22 18:08:28 UTC
commit 29e85f853557ceb18cb712675df14ae3f3ad99c5 (HEAD -> master, origin/master, origin/HEAD)
Author: Alfred Persson Forsberg <cat@catcream.org>
Date:   Fri Aug 18 22:52:17 2023 +0200

    sys-devel/clang-crossdev-wrappers: new package, add 16, 17, and 18

    Creates wrappers and symlinks for clang, clang++, and clang-cpp so that
    llvm.org's fix version function is happy with ${CTARGET}-clang

    Signed-off-by: Alfred Persson Forsberg <cat@catcream.org>
    Closes: https://github.com/gentoo/gentoo/pull/32136
    Signed-off-by: Sam James <sam@gentoo.org>

commit 3bc0eebf391b1811343f454314c82402ebc6b38c
Author: Alfred Persson Forsberg <cat@catcream.org>
Date:   Wed Aug 2 00:33:30 2023 +0200

    sys-libs/compiler-rt: Support LLVM/Clang crossdev

    This adds functionality to cross compile compiler-rt for a target
    architecture. Needed for LLVM/Clang crossdev.

    Thanks wikky for help!

    Signed-off-by: Alfred Persson Forsberg <cat@catcream.org>
    Signed-off-by: Sam James <sam@gentoo.org>

commit 6472469a0333ea9088e815c0741c2dfb200987b5
Author: Alfred Persson Forsberg <cat@catcream.org>
Date:   Wed Aug 2 00:32:35 2023 +0200

    kernel-2.eclass: Use cross.eclass for figuring out CTARGET

    Use crossdev.eclass instead of manually parsing category. This makes it
    automatically support LLVM/Clang crossdev.

    Signed-off-by: Alfred Persson Forsberg <cat@catcream.org>
    Signed-off-by: Sam James <sam@gentoo.org>

commit 7a2c1f90a07b2adc0b556d6db320a64b74a30ad6
Author: Alfred Persson Forsberg <cat@catcream.org>
Date:   Wed Aug 2 00:29:25 2023 +0200

    sys-libs/musl: Support LLVM/Clang crossdev

    This replaces inline code with functions from crossdev.eclass. Now also
    supports LLVM/Clang crossdev because the eclass handles that
    automatically.

    Thanks wikky for help!

    Signed-off-by: Alfred Persson Forsberg <cat@catcream.org>
    Signed-off-by: Sam James <sam@gentoo.org>

commit 00ee14eab35e0dad294da8efe1c9f6606c3e4794
Author: Alfred Persson Forsberg <cat@catcream.org>
Date:   Wed Aug 2 00:14:15 2023 +0200

    crossdev.eclass: new eclass

    This eclass includes convenience wrappers for use in ebuilds used by the
    Crossdev tool. Mostly to avoid duplicating code for checking category
    and setting CTARGET in ebuilds.

    Signed-off-by: Alfred Persson Forsberg <cat@catcream.org>
    Signed-off-by: Sam James <sam@gentoo.org>
Comment 16 Larry the Git Cow gentoo-dev 2023-08-31 01:34:46 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ac3e1265008b02455c9714b356d36fbbff2c019f

commit ac3e1265008b02455c9714b356d36fbbff2c019f
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2023-08-31 01:34:13 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2023-08-31 01:34:39 +0000

    sys-devel/crossdev: add 20230831
    
    Bug: https://bugs.gentoo.org/680652
    Closes: https://bugs.gentoo.org/912849
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-devel/crossdev/Manifest                 |  1 +
 sys-devel/crossdev/crossdev-20230831.ebuild | 40 +++++++++++++++++++++++++++++
 sys-devel/crossdev/crossdev-99999999.ebuild |  3 +--
 3 files changed, 42 insertions(+), 2 deletions(-)
Comment 17 Larry the Git Cow gentoo-dev 2023-12-07 13:01:00 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=aa19bed00fa0eb61339b16cf23ce060120fcafb2

commit aa19bed00fa0eb61339b16cf23ce060120fcafb2
Author:     Matoro Mahri <matoro_gentoo@matoro.tk>
AuthorDate: 2023-11-26 04:32:02 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2023-12-07 13:00:23 +0000

    llvm.eclass: add tuple -> LLVM_TARGETS translate function
    
    To be used in support of dev-lang/rust under crossdev.  Cross-compiled
    rust needs LLVM with support for both host and destination targets.
    
    Bug: https://bugs.gentoo.org/680652
    Signed-off-by: Matoro Mahri <matoro_gentoo@matoro.tk>
    Closes: https://github.com/gentoo/gentoo/pull/33996
    Signed-off-by: Sam James <sam@gentoo.org>

 eclass/llvm.eclass | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
Comment 18 12101111 2024-01-15 16:45:21 UTC
Some issue I found:
1. sys-devel/clang-crossdev-wrappers:17 don't have keywords but sys-libs/compiler-rt:17 have.

2. mingw64-runtime-11.0.0.ebuild don't handle CATEGORY == cross_llvm-*

 *   Invalid configuration, please see: https://wiki.gentoo.org/wiki/Mingw
 *
 * Call stack:
 *                       ebuild.sh, line 136:  Called pkg_setup
 *   mingw64-runtime-11.0.0.ebuild, line  31:  Called die
 * The specific snippet of code:
 *              die "Invalid configuration, please see: https://wiki.gentoo.org/wiki/Mingw"
 *

3. In /etc/portage/env/cross_llvm*/llvm.conf created by crossdev(line 1594),
AS=llvm-as is incorrect. llvm-as convert LLVM IR to LLVM Bytecode. 
It should be a wrapper script or AS="${CTARGET}-clang -c".

the configure script of mingw64-runtime will check ${CTARGET}-gcc and ${CTARGET}-as and other binutils with CTARGET prefix. so symlinks for them like sys-devel/llvm-toolchain-symlinks and sys-devel/clang-toolchain-symlinks[gcc-symlinks] are needed.

4. In cmake.eclass(line 485), it check KERNEL variable. But when compiling cross_llvm-*/compiler-rt, KERNEL=linux is set and the output file is broken for mingw32.

A simple helloword c program don't build: undefined symbol: ___chkstk_ms

Also a patch is needed for compiler-rt: https://github.com/llvm/llvm-project/commit/885d7b759b5c166c07c07f4c58c6e0ba110fb0c2

5. The path of installed compiler-rt for mingw32 is not recognized.
Run `strace -e trace=stat x86_64-w64-mingw32-clang --print-runtime-dir` and find out that clang will use those directories as path to compiler-rt:

/usr/lib/clang/17/lib/x86_64-w64-windows-gnu
/usr/lib/llvm/17/lib/x86_64-w64-windows-gnu
/usr/lib/clang/17/lib/windows/x86_64

But compiler-rt is installed into /usr/lib/clang/17/lib/x86_64-w64-mingw32

6. llvm-libunwind need CMAKE_CXX_COMPILER_WORKS=1 to build
Comment 19 Larry the Git Cow gentoo-dev 2024-01-15 17:33:07 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=5c800f1e7874f263845bc22ba72e7e6484feae75

commit 5c800f1e7874f263845bc22ba72e7e6484feae75
Author:     12101111 <w12101111@gmail.com>
AuthorDate: 2024-01-15 16:16:30 +0000
Commit:     Ionen Wolkens <ionen@gentoo.org>
CommitDate: 2024-01-15 17:32:10 +0000

    dev-util/mingw64-runtime: Use crossdev.eclass
    
    crossdev.eclass contains code of parsing CTARGET from category and
    include the support of crossdev with LLVM/Clang compiler
    
    Bug: https://bugs.gentoo.org/680652
    Signed-off-by: Han Puyu <w12101111@gmail.com>
    Closes: https://github.com/gentoo/gentoo/pull/34820
    Signed-off-by: Ionen Wolkens <ionen@gentoo.org>

 dev-util/mingw64-runtime/mingw64-runtime-11.0.0.ebuild | 12 +++---------
 1 file changed, 3 insertions(+), 9 deletions(-)
Comment 20 Alfred Persson 2024-01-16 15:26:26 UTC
(In reply to 12101111 from comment #18)

> 1-5

Thanks, I'll look at those.

> 6. llvm-libunwind need CMAKE_CXX_COMPILER_WORKS=1 to build

I got libc++, libc++abi, and libunwind to work by setting them up manually but never got around to making ebuilds due to some issue. I'll fix all of them together once I get some free time. I think I'll install them after compiler-rt in a separate crossdev stage, probably with some option to disable it.
Comment 21 Larry the Git Cow gentoo-dev 2024-01-26 09:31:08 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/crossdev.git/commit/?id=a5daa83d8dd73d73035403aa57b24f7e057e4e19

commit a5daa83d8dd73d73035403aa57b24f7e057e4e19
Author:     Alfred Persson Forsberg <cat@catcream.org>
AuthorDate: 2024-01-26 00:05:30 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-01-26 09:30:46 +0000

    llvm.conf: set AS="clang -c" instead of llvm-as
    
    Bug: https://bugs.gentoo.org/680652
    Signed-off-by: Alfred Persson Forsberg <cat@catcream.org>
    Closes: https://github.com/gentoo/crossdev/pull/18
    Signed-off-by: Sam James <sam@gentoo.org>

 crossdev | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Comment 22 Larry the Git Cow gentoo-dev 2024-01-26 09:31:55 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=bcf011b03a776ad52a809f4ec35fa5b83e137dcb

commit bcf011b03a776ad52a809f4ec35fa5b83e137dcb
Author:     Alfred Persson Forsberg <cat@catcream.org>
AuthorDate: 2024-01-26 01:14:41 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-01-26 09:31:19 +0000

    sys-devel/clang-crossdev-wrappers: add 19
    
    Bug: https://bugs.gentoo.org/680652
    Signed-off-by: Alfred Persson Forsberg <cat@catcream.org>
    Closes: https://github.com/gentoo/gentoo/pull/35023
    Signed-off-by: Sam James <sam@gentoo.org>

 .../clang-crossdev-wrappers-19.ebuild              | 43 ++++++++++++++++++++++
 1 file changed, 43 insertions(+)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=c4e0bd372eb746030368abfc17c0dcb1fdfb2cff

commit c4e0bd372eb746030368abfc17c0dcb1fdfb2cff
Author:     Alfred Persson Forsberg <cat@catcream.org>
AuthorDate: 2024-01-26 01:08:05 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-01-26 09:31:19 +0000

    sys-devel/clang-crossdev-wrappers: keyword 17
    
    Bug: https://bugs.gentoo.org/680652
    Signed-off-by: Alfred Persson Forsberg <cat@catcream.org>
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-devel/clang-crossdev-wrappers/clang-crossdev-wrappers-16.ebuild | 2 +-
 sys-devel/clang-crossdev-wrappers/clang-crossdev-wrappers-17.ebuild | 5 ++---
 2 files changed, 3 insertions(+), 4 deletions(-)
Comment 23 Larry the Git Cow gentoo-dev 2024-02-10 07:49:48 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=869a67efcaedbf7d08fe16057466f58ac1f21a84

commit 869a67efcaedbf7d08fe16057466f58ac1f21a84
Author:     Matoro Mahri <matoro_gentoo@matoro.tk>
AuthorDate: 2024-02-09 15:48:17 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-02-10 07:49:17 +0000

    dev-lang/rust: crossdev support
    
    This allows Rust to be cross-compiled with crossdev.  This is the last
    item for bug 680652, closing it.  Note that this is for a foreign CHOST,
    i.e., CBUILD != CHOST == CTARGET.  It does not cover the scenario for
    CBUILD == CHOST != CTARGET(s), which is tracked separately in bug
    679878.
    
    A number of changes were required in order to make this work:
    
    * Force USE=system-bootstrap, since we cannot specify the appropriate
      boostrap URL based on CBUILD in SRC_URI
    * LLVm is compiled twice as part of the build, once for host and once
      for target.  However, the rust build uses the same settings from
      config.toml for both builds.  Therefore we cannot override flags nor
      default-linker and must let rust choose them for us.
    * Set the appropriate build/host variables which correspond to
      CBUILD/CHOST.  This works as expected.
    * Set PKG_CONFIG and OPENSSL_*_DIR variables; cargo needs these for some
      reason.
    * Enforce that LLVM_TARGETS is set correctly for both host and target
      arches.  This uses the new llvm_tuple_to_target function, introduced
      in https://github.com/gentoo/gentoo/pull/33996
    * Lastly a small patch to rust source is needed, to tell it to link with
      system libz.  It's unclear why this scenario was excluded under
      cross-compile conditions in upstream rust.
    
    See: https://paste.sr.ht/~kchibisov/682321e0fd4a3ece4a4b7b71591896f5cd3cdb22
    Bug: https://bugs.gentoo.org/679878
    Closes: https://bugs.gentoo.org/680652
    Signed-off-by: Matoro Mahri <matoro_gentoo@matoro.tk>
    Closes: https://github.com/gentoo/gentoo/pull/35246
    Signed-off-by: Sam James <sam@gentoo.org>

 .../rust/files/1.74.1-cross-compile-libz.patch     | 19 +++++++++
 dev-lang/rust/rust-1.74.1.ebuild                   | 46 ++++++++++++++++------
 dev-lang/rust/rust-1.75.0.ebuild                   | 46 ++++++++++++++++------
 3 files changed, 89 insertions(+), 22 deletions(-)