Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 679878 - dev-lang/rust - add cross-compile support
Summary: dev-lang/rust - add cross-compile support
Status: IN_PROGRESS
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal with 2 votes (vote)
Assignee: Gentoo Rust Project
URL:
Whiteboard:
Keywords:
: 919202 (view as bug list)
Depends on: 671736 747760
Blocks:
  Show dependency tree
 
Reported: 2019-03-09 21:51 UTC by tt_1
Modified: 2024-03-28 18:49 UTC (History)
15 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
config.toml for additional rust-std (config.toml,1.30 KB, text/plain)
2019-03-09 21:51 UTC, tt_1
Details
hello world test file with instructions (hello.tar.gz,247 bytes, application/octet-stream)
2019-03-09 22:36 UTC, tt_1
Details

Note You need to log in before you can comment on or make changes to this bug.
Description tt_1 2019-03-09 21:51:21 UTC
Created attachment 568354 [details]
config.toml for additional rust-std

According to my research it isn't too difficult to cross-compile with a rustc toolchain, but for that to work rust needs to have the fitting rustlib installed alongside the hosts native rustlib. 

for the case of wanting to cross-compile for armv7a-unknown-linux-gnueabihf, the following needs to be appended to the config.toml: 

1. target has to be added in [build]:
target = ["x86_64-unknown-linux-gnu","armv7-unknown-linux-gnueabihf"]

2. target has to be configured:
[target.armv7-unknown-linux-gnueabihf]
cc = "armv7a-unknown-linux-gnueabihf-gcc"
cxx = "armv7a-unknown-linux-gnueabihf-g++"
linker = "armv7a-unknown-linux-gnueabihf-gcc"
ar = "armv7a-unknown-linux-gnueabihf-ar"
llvm-config = "/usr/armv7a-unknown-linux-gnueabihf/usr/lib/llvm/7/bin/llvm-config"

full config.toml is attached.
Comment 1 tt_1 2019-03-09 22:36:41 UTC
Created attachment 568364 [details]
hello world test file with instructions

It requires a whole cross-compile toolchain, set up with crossdev beforehand, until rust may be built without gcc as a linker at some point in the future.

Also I used system-llvm useflag when testing this approach. System llvm has ARM added to llvm_targets.
Comment 2 David Michael 2020-02-27 17:26:53 UTC
Could this maybe do something like GRUB and use CTARGET to include a platform?  Ideally, Rust would support packaging any number of platforms (same with GRUB), but just one target would at least solve my current use case.

--- dev-lang/rust/rust-1.41.0.ebuild
+++ dev-lang/rust/rust-1.41.0.ebuild
@@ -171,7 +171,7 @@
 	if use wasm; then
 		rust_targets="${rust_targets},\"wasm32-unknown-unknown\""
 	fi
-	rust_targets="${rust_targets#,}"
+	rust_targets="${rust_targets#,}${CTARGET:+,\"$(rust_abi "${CTARGET}")\"}"
 
 	local extended="true" tools="\"cargo\","
 	if use clippy; then
@@ -234,6 +234,15 @@
 		src-tarball = false
 	EOF
 
+	test -n "${CTARGET}" &&
+	cat <<- EOF >> "${S}"/config.toml
+		[target.$(rust_abi "${CTARGET}")]
+		cc = "$(tc-getCC "${CTARGET}")"
+		cxx = "$(tc-getCXX "${CTARGET}")"
+		linker = "$(tc-getCC "${CTARGET}")"
+		ar = "$(tc-getAR "${CTARGET}")"
+	EOF
+
 	for v in $(multilib_get_enabled_abi_pairs); do
 		rust_target=$(rust_abi $(get_abi_CHOST ${v##*.}))
 		arch_cflags="$(get_abi_CFLAGS ${v##*.})"
Comment 3 Georgy Yakovlev archtester gentoo-dev 2020-06-11 04:36:06 UTC
I'm thinking on tackling it in nearest future.

question, is cross-llvm really required? or native llvm with appropriate target enabled is enough?
Comment 4 tt_1 2020-06-11 06:11:44 UTC
I think that a full cross gcc toolchain is needed for linking, both during bootstraping the cross crates and when cross compiling rust code with those cross crates.

I'm not sure about llvm, but I have dropped the foreign llvm config and now use this config: 

if use rust-std-armv7; then
	cat <<- EOF >> "${S}"/config.toml
		[target.armv7-unknown-linux-gnueabihf]
		cc = "armv7a-unknown-linux-gnueabihf-gcc"
		cxx = "armv7a-unknown-linux-gnueabihf-g++"
		linker = "armv7a-unknown-linux-gnueabihf-gcc"
		ar = "armv7a-unknown-linux-gnueabihf-ar"
	EOF
Comment 5 David Michael 2020-06-11 13:37:27 UTC
The steps I use are:
  1. Run crossdev for the target to produce a linker, compilers, etc.
  2. Add the target architecture to LLVM_TARGETS and rebuild LLVM if needed.
  3. Build Rust with a [target.${RUST_TARGET}] section added to config.toml.
Comment 6 tt_1 2020-06-11 15:51:24 UTC
(In reply to David Michael from comment #5)
> The steps I use are:
>   1. Run crossdev for the target to produce a linker, compilers, etc.
>   2. Add the target architecture to LLVM_TARGETS and rebuild LLVM if needed.
>   3. Build Rust with a [target.${RUST_TARGET}] section added to config.toml.

I follow the same steps, but I have to remove -march=znver1 from cflags since rust-1.36.0 I believe.
Comment 7 Georgy Yakovlev archtester gentoo-dev 2020-06-11 19:53:39 UTC
thanks, that answers my question.

currently idea is that user defines something like that:

> RUST_CROSS_TARGETS=(
>         "AArch64:aarch64-unknown-linux-gnu:aarch64-unknown-linux-gnu"
>         "X86:x86_64-unknown-linux-musl:x86_64-gentoo-linux-musl"
> )

where 1st element is required llvm target,
2nd element is rust target (one from rustc --print target-list)
3rd element is C target (the one you have installed with crossdev)

and runs emerge, it should magically work.

It may be even easier with clang, because it may work without cross toolchain, just proper llvm_target set (for CLANG). I'll look at it later, no


currently no advanced error checking or dep tracking planned, user is supposed to pre-install cross toolchain and enable LLVM_TARGETS, so not much will change for you, but maybe you could stop forking the ebuild.

maybe I'll try to at least trigger LLVM_TARGETS dependency. But I can't require cross toolchain within a ebuild.
At least without re-writing it to use ${CATEGORY} dependant logic.

as for C(XX)FLAGS, nice catch. I'll try to filter out cpu specific flags for foreign arch targets.
Comment 8 Georgy Yakovlev archtester gentoo-dev 2020-06-12 00:32:36 UTC
https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0326022d60e80c438d0ea90f489eebb722320417

comitted
forgot to properly reference this bug.

please try  it out and let me know =)
Comment 9 Georgy Yakovlev archtester gentoo-dev 2020-06-12 00:43:56 UTC
cat /etc/portage/env/dev-lang/rust 
> I_KNOW_WHAT_I_AM_DOING_CROSS=yes
> RUST_CROSS_TARGETS=( "AArch64:aarch64-unknown-linux-gnu:aarch64-unknown-linux-gnu" )


this is what I used for test (without system-llvm this time)
Comment 10 Georgy Yakovlev archtester gentoo-dev 2020-06-12 02:40:15 UTC
cargo.eclass needs some changes as noted, but I was able to compile working ripgrep with minimal changes to eclass.


-inherit multiprocessing toolchain-funcs                                                                                                                                                                           
+inherit multiprocessing rust-toolchain toolchain-funcs

...
        [build]                                                                                                                                                                                                    
        jobs = $(makeopts_jobs)
+       target = "${build_target}"                                                                                                                                                                                 
+       target-dir = "${S}/target/release"                                                                                                                                                                         
+                                                                                                                                                                                                                  
+       [target.${build_target}]                                                                                                                                                                                   
+       linker = "$(tc-getCC)"



this will need more work for crates that link to C libraries.
I'll test above changes if they cause breakage to native builds and will commit soon, and will tackle more complex builds (with C libs involved) a bit later.
Comment 11 Georgy Yakovlev archtester gentoo-dev 2020-06-12 02:42:56 UTC
missed this line from simplified eclass diff

> local build_target="$(rust_abi $(get_abi_CHOST ${v##*.}))"
Comment 12 David Michael 2020-06-12 03:34:25 UTC
I'll try to build it tomorrow, but at a glance, could it maybe skip adding a target if it's already defined?  It fails if a section is defined twice, and I have automated scripts that could end up cross-compiling for the same architecture as the build system.  (If not, I can just add a special case to exclude the arch from the array when CHOST==CBUILD.)  Maybe add this in the new loop:

grep -Fqx "[target.${cross_rust_target}]" "${S}"/config.toml && continue

Regarding the comment about just bootstrapping the libraries: I implemented it that way a while ago, but I haven't looked at it since.  Here was the latest iteration of that package, if it helps as a starting point:

https://github.com/coreos/coreos-overlay/tree/8bf82c04b77a7be90cf86bfe13838535980ab847^/dev-libs/rustlib/rustlib-1.29.1.ebuild
Comment 13 Georgy Yakovlev archtester gentoo-dev 2020-06-12 06:30:49 UTC
in that scenario you've described, how does same target gets in there? you inject it to config file somehow?


the idea with stdlib was to use it as follows:

have rust-std ebuild.

it defines dependencies like this

if cross-compiling; then
  "${CATEGORY}/gcc"
  "${CATEGORY}/binutils"
fi

for example.
so with crossdev support it becomes

cross-aarch64-unknown-linux-gnu/rust-std


which properly pulls in toolchain with portage and will be installed to /usr/lib/rustlib/aarch64-unknown-linux-gnu


obvious problems here is that we still need valid mappings between rust triples and possible C triples, because without custom target spec rust will not like x86_64-pc-linux-gnu (standard gentoo one), it wants x86_64-unknown-linux-gnu.
that's what https://bugs.gentoo.org/show_bug.cgi?id=671736 for, it has pretty bad matcher and needs to be re-written to support more stuff  and libs and be more generic. it's probably better to revbump it and start fresh, because it's very hard to modify it currently, as it affects all the rust ebuilds in the tree.

and it needs crossdev support, at least basic one.



But current approach also kinda works. I wish we could define RUST_TARGETS like llvm does, and put well known targets in there, and make it extensible, but cross-toolchain requirement is a show-stopper, it may be possible with clang though, not sure if it works and how it'll mix with gcc, but not all hope is lost =)
Comment 14 tt_1 2020-06-12 06:54:15 UTC
Regarding cross compiling: for some crates, you will have to provide a symlink of cross gcc/g++ to match it with debian name patterns. The reason for that is, some crates have c/c++ code within them, and the crate responsible for finding the c compiler only scans for debian name patterns. 

That is /usr/bin/arm-linux-gnueabihf-gcc instead of /usr/bin/armv7a-unknown-linux-gnueabihf-gcc ; but it is again stupid enough to follow the symlink.
Comment 15 David Michael 2020-06-12 15:12:01 UTC
(In reply to Georgy Yakovlev from comment #13)
> in that scenario you've described, how does same target gets in there? you
> inject it to config file somehow?

Yes, I'd plan to have it append an array element for each $CHOST when I'm cross-compiling for a new platform.  Since my scripts are based around the $CHOST triplet, it would also break the build if different vendor strings are used for the same target hardware (e.g. powerpc-gentoo-linux-gnu and powerpc-pc-linux-gnu).

> obvious problems here is that we still need valid mappings between rust
> triples and possible C triples, because without custom target spec rust will
> not like x86_64-pc-linux-gnu (standard gentoo one), it wants
> x86_64-unknown-linux-gnu.

Maybe some eselect-like setup would help, where crossdev builds /usr/lib/rustlib/$CHOST which is symlinked to a builtin Rust triplet?  (That probably wouldn't be needed if they ever fixed https://github.com/rust-lang/rust/issues/16351 , which prevented automatically using new platforms in /usr/lib/rustlib when I last looked into this a couple years ago.)
Comment 16 tt_1 2020-06-12 17:09:53 UTC
I've tested this a bit, it does work for aarch64-unknown-linux-gnu

It not working when the rust and gcc target names don't match, for instance:

>I_KNOW_WHAT_I_AM_DOING_CROSS=yes
>RUST_CROSS_TARGETS=( "ARM:armv7-unknown-linux-gnueabihf:armv7a-unknown-linux-gnueabihf" )


>>> Source prepared.
>>> Configuring source in /var/tmp/portage/dev-lang/rust-1.44.0/work/rustc-1.44.0-src ...
 * ERROR: dev-lang/rust-1.44.0::gentoo failed (configure phase):
 *   need armv7-unknown-linux-gnueabihf cross toolchain
 * 

and fails as well with the extra target needed for neon optimizations:

> "ARM:thumbv7neon-unknown-linux-gnueabihf:armv7a-unknown-linux-gnueabihf"
Comment 17 Georgy Yakovlev archtester gentoo-dev 2020-06-12 17:18:46 UTC
(In reply to tt_1 from comment #16)
> I've tested this a bit, it does work for aarch64-unknown-linux-gnu
> 
> It not working when the rust and gcc target names don't match, for instance:
> 
> >I_KNOW_WHAT_I_AM_DOING_CROSS=yes
> >RUST_CROSS_TARGETS=( "ARM:armv7-unknown-linux-gnueabihf:armv7a-unknown-linux-gnueabihf" )
> 
> 
> >>> Source prepared.
> >>> Configuring source in /var/tmp/portage/dev-lang/rust-1.44.0/work/rustc-1.44.0-src ...
>  * ERROR: dev-lang/rust-1.44.0::gentoo failed (configure phase):
>  *   need armv7-unknown-linux-gnueabihf cross toolchain
>  * 
> 
> and fails as well with the extra target needed for neon optimizations:
> 
> > "ARM:thumbv7neon-unknown-linux-gnueabihf:armv7a-unknown-linux-gnueabihf"

all it does is
> command -v ${cross_toolchain}-gcc > /dev/null 2>&1 || die "need ${cross_toolchain} cross toolchain"

basically checks if armv7-unknown-linux-gnueabihf-gcc is in PATH.

is it? or check misfires?
crossdev should have created symlinks, mine is below:
/usr/bin/aarch64-unknown-linux-gnu-gcc -> /usr/powerpc64le-unknown-linux-gnu/aarch64-unknown-linux-gnu/gcc-bin/9.3.0/aarch64-unknown-linux-gnu-gcc
Comment 18 tt_1 2020-06-12 17:21:13 UTC
mind the a ;-) 

it's armv7a (gcc) vs armv7 (rust)
Comment 19 Larry the Git Cow gentoo-dev 2020-06-12 17:24:52 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=f0cecc7b52bd20fbc674e81496008feff72efc8d

commit f0cecc7b52bd20fbc674e81496008feff72efc8d
Author:     Georgy Yakovlev <gyakovlev@gentoo.org>
AuthorDate: 2020-06-12 17:23:31 +0000
Commit:     Georgy Yakovlev <gyakovlev@gentoo.org>
CommitDate: 2020-06-12 17:24:41 +0000

    dev-lang/rust: fix cross triple detection
    
    Bug: https://bugs.gentoo.org/679878
    Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org>

 dev-lang/rust/rust-1.44.0.ebuild | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Comment 20 Georgy Yakovlev archtester gentoo-dev 2020-06-12 17:25:08 UTC
nvm, found it, fix comitted.
Comment 21 Georgy Yakovlev archtester gentoo-dev 2020-06-12 17:26:36 UTC
or maybe not, damn I need coffee. hold on, will commit actually fixed version soon.
Comment 22 Larry the Git Cow gentoo-dev 2020-06-12 17:42:26 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=8e0668379b81eaaa034547c554b0e5ab59672e89

commit 8e0668379b81eaaa034547c554b0e5ab59672e89
Author:     Georgy Yakovlev <gyakovlev@gentoo.org>
AuthorDate: 2020-06-12 17:40:03 +0000
Commit:     Georgy Yakovlev <gyakovlev@gentoo.org>
CommitDate: 2020-06-12 17:42:02 +0000

    dev-lang/rust: more cross triple fixes
    
    Bug: https://bugs.gentoo.org/679878
    Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org>

 dev-lang/rust/rust-1.44.0.ebuild | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
Comment 23 Georgy Yakovlev archtester gentoo-dev 2020-06-12 17:43:28 UTC
not it should check gcc triple correctly.
it was the intention, just got string splitting wrong.
Comment 24 tt_1 2020-06-12 17:50:57 UTC
seems legit, it doesn't error anymore and the config.toml looks to me: 


>I_KNOW_WHAT_I_AM_DOING_CROSS=yes
>RUST_CROSS_TARGETS=(
>	"AArch64:aarch64-unknown-linux-gnu:aarch64-unknown-linux-gnu"
>	"ARM:armv7-unknown-linux-gnueabihf:armv7a-unknown-linux-gnueabihf"
>	"ARM:thumbv7neon-unknown-linux-gnueabihf:armv7a-unknown-linux-gnueabihf"
>)



I'll give it a test now.
Comment 25 tt_1 2020-06-12 20:02:14 UTC
it seems to work pretty good, I just finished cross compiling firefox-78 beta for armv7a with said neon optimizations. 

I haven't yet tested any true rust project, such as cbindgen, but there's likely a few hickups left to deal with in cargo.eclass to make them work


if you're bored, the *so files of the foreign libs are with full debug info .. ;-)
Comment 26 Georgy Yakovlev archtester gentoo-dev 2020-06-12 20:16:39 UTC
thanks for testing!

yeah, cargo.eclass needs a lot of massage. I'll take step-by-step approach with it, so changes will be rolling as time goes and I find-fix issues.

it feels so great to just cross-emerge a rust package when it does not have external deps, with EAPI=7 rust stays on the host side =)
I removed EAPI=6 support from cargo.eclass btw, so it can start using cross features from 7.

as for debuginfo, that's probably because host strip which portage uses does not know how to strip foreign arch stuff. not sure it can be fixed in ebuild/eclass. we don't let packages strip themselves, it's portage job so FEATURES=nostrip or splitdebug is respected.

EAPI7 has some strip improvements, I'll check it.

thanks again for testing and doing the initial research. it was kinda easy to add TBH, because most of the work already happened.
Comment 27 Georgy Yakovlev archtester gentoo-dev 2020-06-12 21:04:33 UTC
as for strip. I was told that binutils build with [multitarget] will handle stripping of foreign arches just fine.

I may add a message to the ebuild recommending this.
Comment 29 Cănărău Constantin 2020-07-04 17:51:04 UTC
Just a curiosity: after this will be done can we hope that rust to be integrated in distcc and cross-compile firefox and thunderbird with raspberry pi as host and a powerful x86 machine on other distcc side ?
As far I understand from comments and attached archive, right now it is possible to compile firefox for arm on host x86 machine. It is correct ? 
Resulted binary work correctly ? I am talking about the fact than on qemu some binaries compile correctly, but are not usable. Can I expect same bad behavior using this setup ?
Comment 30 tt_1 2020-07-04 17:59:20 UTC
yes, cross compiling firefox is possible and has been possible for some time now, using crossdev's emerge-wrappers. I'm doing it all the time, testing betas, finding and fixing bugs.
Comment 31 Georgy Yakovlev archtester gentoo-dev 2020-07-04 22:48:25 UTC
I heard reports from firefox Maintainer that re-building firefox with ccache + sccache(for rust parts) takes about 3 minutes on under-powered machine.

so there's hope. 

trying using distributed sccache (it can work as distcc) for rust + distcc cross for C++ might be interesting project, but not something I'm going to do anytime soon.
Comment 32 Cănărău Constantin 2020-07-05 06:58:39 UTC
Yes, I know that it is possible to compile firefox with distcc, but distcc is used for C/C++ sources. Rust parts are still compiled on host (raspberry pi), at least that top show me.
Am I doing something wrong ?
Comment 33 Georgy Yakovlev archtester gentoo-dev 2020-07-05 07:57:33 UTC
(In reply to Cănărău Constantin from comment #32)
> Yes, I know that it is possible to compile firefox with distcc, but distcc
> is used for C/C++ sources. Rust parts are still compiled on host (raspberry
> pi), at least that top show me.
> Am I doing something wrong ?

https://github.com/mozilla/sccache

packaged as dev-util/sccache

it's supposed to work as ccache and/or distcc, and supports rust, at least locally.
I haven't used is as distcc for rust (only for C/C++), you can try =)
Comment 34 Georgy Yakovlev archtester gentoo-dev 2020-07-05 07:59:05 UTC
from their readme: 

sccache also provides icecream-style distributed compilation (automatic packaging of local toolchains) for all supported compilers (including Rust).

not sure how it works for cross... probably not.
Comment 35 Larry the Git Cow gentoo-dev 2020-09-25 07:38:28 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=e2701fa0f4b3b3a3d971c2274c36b4ea4bce7181

commit e2701fa0f4b3b3a3d971c2274c36b4ea4bce7181
Author:     Georgy Yakovlev <gyakovlev@gentoo.org>
AuthorDate: 2020-09-25 07:26:47 +0000
Commit:     Georgy Yakovlev <gyakovlev@gentoo.org>
CommitDate: 2020-09-25 07:37:58 +0000

    sys-devel/rust-std: new package, for crossdev #679878
    
    EXPERIMENTAL!
    
    Bug: https://bugs.gentoo.org/680652
    Bug: https://bugs.gentoo.org/679878
    Bug: https://bugs.gentoo.org/689336
    Package-Manager: Portage-3.0.8, Repoman-3.0.1
    Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org>

 sys-devel/rust-std/Manifest               |   1 +
 sys-devel/rust-std/metadata.xml           |  21 +++++
 sys-devel/rust-std/rust-std-1.46.0.ebuild | 146 ++++++++++++++++++++++++++++++
 3 files changed, 168 insertions(+)
Comment 36 Georgy Yakovlev archtester gentoo-dev 2020-09-25 07:38:58 UTC
# rust-std + crossdev = <3

EXPERIMENTAL! not sure if stage0 stdlib is good enough, but it produces working binaries just fine.

TLDR: (your file paths may differ)

assuming you already have `aarch64-unknown-linux-gnu` toolchain bootstrapped via crossdev.

```bash
cd /var/db/repos/crossdev/cross-aarch64-unknown-linux-gnu
ln -s /var/db/repos/gentoo/sys-devel/rust-std
echo 'cross-aarch64-unknown-linux-gnu/rust-std **' >> /etc/portage/package.accept_keywords/aarch64
emerge cross-aarch64-unknown-linux-gnu/rust-std -av
cd /some/rust/source
export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-unknown-linux-gnu-gcc
cargo build --target aarch64-unknown-linux-gnu
```

for crates that link to libstdc++ you may need to add something like this:
`CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_RUSTFLAGS="-C link-arg=-Wl,-rpath-link,/usr/lib/gcc/aarch64-unknown-linux-gnu/9.3.0"`
More info on that: https://bugs.gentoo.org/549994


## Package if used as cross- will install new stdlib to `/usr/lib64/rust-<ver>/rustlib/aarch64-unknown-linux-gnu`, for example.

currently this method is not as flexible as fully custom target support via rust ebuild itself,
because rust-toolchain.eclass' rust_abi() function needs better triple transformations.
for example it will always transform CHOST x86-64-gentoo-linux-musl to RHOST x86-64-unknown-linux-musl
but it can be fixed, of course. https://bugs.gentoo.org/671736

There's a workaround, ERUST_STD_RTARGET=whatever-weird-rust-triple allows setting any custom one.
that will set `build.target` in `config.toml`

it's very fast, one can get a new target in couple of minutes, instead of waiting for full bootstrap
of dev-lang/rust with RUST_CROSS_TARGETS.
Comment 37 tt_1 2020-09-25 07:42:36 UTC
sounds like a good reason to upgrade to llvm-10 and >=rust-1.46 , will test soon
Comment 38 Georgy Yakovlev archtester gentoo-dev 2020-09-25 08:17:09 UTC
you can easily copy ebuild into crossdev repo as any version.
but it will conflict with rust ebuild internal cross of course, if you install same targets.

otherwise, waiting for feedback.

I've compiled simple programs (ripgrep cbindgen fd etc) just fine. even ones that link to libstdc++, with workaround mentioned, and generally to cross libs.

also, cargo.eclass cross is almost ready, stay tuned. will land soon, maybe this weekend.
Comment 39 tt_1 2020-09-25 12:19:59 UTC
I need two targets in cross-armv7a-unknown-linux-gnueabihf/rust-std, once armv7-unknown-linux-gnueabihf and, additional I need thumbv7neon-unknown-linux-gnueabihf 

how can I config that, please?
Comment 40 Georgy Yakovlev archtester gentoo-dev 2020-09-25 12:48:05 UTC
don't think it's possible with current rust-std ebuild, so looks like you are still stuck with rust-ebuild's cross target thingie.

I'll think on how to get it done, but a bit later.
it could be a useflag if same C cross- toolchain can link both rust targets, or maybe something else.

need to get some sleep, later.
Comment 41 tt_1 2020-09-25 17:27:06 UTC
that's okay, I can stick around with the more cpu intensive approach that already exists from dev-lang/rust-1.44.1 onwards. 

thumbv7neon-unknown-linux-gnueabihf is needed for -mthumb and neon optimization in firefox, and demands a regular armv7a-unknown-linux-gnueabihf cross compiler chain to bootstrap 

one thing already mentioned, the *.so files are merged unstripped, propably unless there's binutils with USE=multitarget available. maybe this issue is worth an upstream bug, even though it is kind of minor in my opinion.
Comment 42 Georgy Yakovlev archtester gentoo-dev 2020-09-27 22:11:24 UTC
we can't really mitigate binutils[multitarget] requirement.

because portage does strip and it has no idea that you are installing binaries for 2+ architectures in a single ebuild.

I can't ask it use different strips for different sub-directories, so  multitarget is the best solution.

It's good flag anyway, not that it changes a lot in binutils, just tells it to handle different arches, not just native one, kinda like llvm already does.


if rust-std approach proves itself (after some work) it will be a non-issue anyway, because it will be an ebuld per target/arch and portage will use proper strip.
Comment 43 Bartosz Stebel 2020-10-02 19:04:43 UTC
I was attempting to use dev-lang/rust for embedded dev and also wasn't able to build rust-std correctly (eventually settled on rustup). The target in Rust's convention is thumbv7em-none-eabihf. The corresponding crossdev target seems to be arm-hardfloat-eabi, with hardfloat flags as per https://wiki.gentoo.org/wiki/ARM#Enable_hardfloat_support
Comment 44 Georgy Yakovlev archtester gentoo-dev 2020-10-02 19:14:15 UTC
Yeah, I know, it’s in progress and I don’t have a working solution yet for rust-std, but it should work via rust ebuild cross support where you can define arbitrary targets.

Current target logic is in https://github.com/gentoo/gentoo/blob/f4de0f1f06807bcb6a0506c58c355bda9340582f/eclass/rust-toolchain.eclass#L35 and is rather primitive.
Comment 45 tt_1 2020-10-10 08:31:55 UTC
can you please bump the rust-std ebuild as well?
Comment 46 Georgy Yakovlev archtester gentoo-dev 2020-10-10 09:38:20 UTC
after 1.47.0 is unmasked, sure =) can't build firefox with it anyway due to https://bugzilla.mozilla.org/show_bug.cgi?id=1670245

std ebuild will need some changes as well as I changed installation path again...
Comment 47 Johannes Geiss 2020-11-23 13:31:15 UTC
(In reply to tt_1 from comment #24)
> 
> >RUST_CROSS_TARGETS=(
> >	"AArch64:aarch64-unknown-linux-gnu:aarch64-unknown-linux-gnu"
> >	"ARM:armv7-unknown-linux-gnueabihf:armv7a-unknown-linux-gnueabihf"
> >	"ARM:thumbv7neon-unknown-linux-gnueabihf:armv7a-unknown-linux-gnueabihf"
> >)

I always get an error when I use this RUST_CROSS_TARGETS assignment. I executed

ebuild /var/db/repos/gentoo/dev-lang/rust/rust-1.46.0.ebuild clean
"/etc/portage/env/rust.conf", line 4: Invalid token 'ARM:armv7-unknown-linux-gnueabihf:armv7a-unknown-linux-gnueabihf' (not '=')

If I use only one line without the (), it works:

RUST_CROSS_TARGETS="AArch64:aarch64-unknown-linux-gnu:aarch64-unknown-linux-gnu"

ebuild /var/db/repos/gentoo/dev-lang/rust/rust-1.46.0.ebuild configure
[...]

Did anyone have this error?
Comment 48 Georgy Yakovlev archtester gentoo-dev 2020-11-23 21:59:54 UTC
you need spaces after ( and before )

> ("AArch64:aarch64-unknown-linux-gnu:aarch64-unknown-linux-gnu") - incorrect
> ( "AArch64:aarch64-unknown-linux-gnu:aarch64-unknown-linux-gnu" ) - correct
Comment 49 Johannes Geiss 2020-11-24 07:59:53 UTC
(In reply to Georgy Yakovlev from comment #48)
> you need spaces after ( and before )
> 
> > ("AArch64:aarch64-unknown-linux-gnu:aarch64-unknown-linux-gnu") - incorrect
> > ( "AArch64:aarch64-unknown-linux-gnu:aarch64-unknown-linux-gnu" ) - correct

I tried this, but it is not working either.

If I use only one target line, like

RUST_CROSS_TARGETS="ARM:thumbv7neon-unknown-linux-gnueabihf:armv7a-pandora_gentoo-linux-gnueabihf"

This is working (Note: without ())

But when I use more targets, I cannot do this:

RUST_CROSS_TARGETS=(
        "ARM:thumbv7neon-unknown-linux-gnueabihf:armv7a-pandora_gentoo-linux-gnueabihf"
        "ARM:arm-unknown-linux-gnueabihf:armv6j-raspberry-linux-gnueabihf"
        "AArch64:aarch64-unknown-linux-gnu:aarch64-raspberry-linux-gnu"
)

I get the error

"/etc/portage/env/rust.conf", line 4: Invalid token 'ARM:arm-unknown-linux-gnueabihf:armv6j-raspberry-linux-gnueabihf' (not '=')

Regardless if I write it in one line or with 5 as shown.


As a workaround, I copied rust-1.46.0.ebuild into my own private overlay and inserted the 5 lines there. This is working, so I assume the parsing of /etc/portage/env/rust.conf does not work as expected.
Comment 50 Georgy Yakovlev archtester gentoo-dev 2020-11-24 17:59:43 UTC
(In reply to Johannes Geiss from comment #49)
> (In reply to Georgy Yakovlev from comment #48)
> > you need spaces after ( and before )
> > 
> > > ("AArch64:aarch64-unknown-linux-gnu:aarch64-unknown-linux-gnu") - incorrect
> > > ( "AArch64:aarch64-unknown-linux-gnu:aarch64-unknown-linux-gnu" ) - correct
> 
> I tried this, but it is not working either.
> 
> If I use only one target line, like
> 
> RUST_CROSS_TARGETS="ARM:thumbv7neon-unknown-linux-gnueabihf:armv7a-
> pandora_gentoo-linux-gnueabihf"
> 
> This is working (Note: without ())
> 
> But when I use more targets, I cannot do this:
> 
> RUST_CROSS_TARGETS=(
>        
> "ARM:thumbv7neon-unknown-linux-gnueabihf:armv7a-pandora_gentoo-linux-
> gnueabihf"
>         "ARM:arm-unknown-linux-gnueabihf:armv6j-raspberry-linux-gnueabihf"
>         "AArch64:aarch64-unknown-linux-gnu:aarch64-raspberry-linux-gnu"
> )
> 
> I get the error
> 
> "/etc/portage/env/rust.conf", line 4: Invalid token
> 'ARM:arm-unknown-linux-gnueabihf:armv6j-raspberry-linux-gnueabihf' (not '=')
> 
> Regardless if I write it in one line or with 5 as shown.
> 
> 
> As a workaround, I copied rust-1.46.0.ebuild into my own private overlay and
> inserted the 5 lines there. This is working, so I assume the parsing of
> /etc/portage/env/rust.conf does not work as expected.

ok got it, you are using wrong env file =)

there are 2 type of env files

/etc/portage/env/* - those are parsed by python, and should not contain any complex bash structures. that's why string variable works for you, but ebuild expects array for more than 1 target.
those work as per-package /etc/portage/make.conf

/etc/portage/env/<category>/<pkg> - those are parsed using bash, a bit later than above file.

IMPORTANT: there's no need to reference /etc/portage/env/<category>/<pkg> in /etc/portage/package.env
just adding bash code to /etc/portage/env/<category>/<pkg> is enough to activate.
it works like per-package /etc/portage/bashrc

so you should place target definition into (do not add .conf, it should be exactly this path)

> /etc/portage/env/dev-lang/rust
Comment 51 Georgy Yakovlev archtester gentoo-dev 2020-11-24 18:01:46 UTC
more info on per-package bashrc
https://dev.gentoo.org/~zmedico/portage/doc/portage.html#config-bashrc-locations
Comment 52 Johannes Geiss 2020-11-25 12:46:40 UTC
(In reply to Georgy Yakovlev from comment #50)

> /etc/portage/env/* - those are parsed by python, and should not contain any
> complex bash structures. that's why string variable works for you, but
> ebuild expects array for more than 1 target.
> [...]
> /etc/portage/env/<category>/<pkg> - those are parsed using bash, a bit later
> than above file.

You made my day. Thank you so much. It's now working.
Comment 53 Christopher Head 2022-01-31 17:12:13 UTC
I’m not sure whether this is in scope for this ticket or not, but something I would be interested in is using Rust to cross-compile for foreign targets on the *same* architecture—for example, one of the x86_64-pc-windows triples. I’m not pushing hard for this to happen—it’s just something I’d be interested in playing around with some day—but just an idea to bear in mind when thinking about cross compilation, if it’s reasonable to implement.
Comment 54 Alec Moskvin 2022-09-04 02:14:18 UTC
Could the rust-std package be bumped along with each rust release? Current version is 1.59.0, and it blocks newer versions of rust from being installed.
Comment 55 Georgy Yakovlev archtester gentoo-dev 2023-12-06 19:45:44 UTC
*** Bug 919202 has been marked as a duplicate of this bug. ***
Comment 56 Larry the Git Cow gentoo-dev 2024-02-10 07:49:49 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=869a67efcaedbf7d08fe16057466f58ac1f21a84

commit 869a67efcaedbf7d08fe16057466f58ac1f21a84
Author:     Matoro Mahri <matoro_gentoo@matoro.tk>
AuthorDate: 2024-02-09 15:48:17 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-02-10 07:49:17 +0000

    dev-lang/rust: crossdev support
    
    This allows Rust to be cross-compiled with crossdev.  This is the last
    item for bug 680652, closing it.  Note that this is for a foreign CHOST,
    i.e., CBUILD != CHOST == CTARGET.  It does not cover the scenario for
    CBUILD == CHOST != CTARGET(s), which is tracked separately in bug
    679878.
    
    A number of changes were required in order to make this work:
    
    * Force USE=system-bootstrap, since we cannot specify the appropriate
      boostrap URL based on CBUILD in SRC_URI
    * LLVm is compiled twice as part of the build, once for host and once
      for target.  However, the rust build uses the same settings from
      config.toml for both builds.  Therefore we cannot override flags nor
      default-linker and must let rust choose them for us.
    * Set the appropriate build/host variables which correspond to
      CBUILD/CHOST.  This works as expected.
    * Set PKG_CONFIG and OPENSSL_*_DIR variables; cargo needs these for some
      reason.
    * Enforce that LLVM_TARGETS is set correctly for both host and target
      arches.  This uses the new llvm_tuple_to_target function, introduced
      in https://github.com/gentoo/gentoo/pull/33996
    * Lastly a small patch to rust source is needed, to tell it to link with
      system libz.  It's unclear why this scenario was excluded under
      cross-compile conditions in upstream rust.
    
    See: https://paste.sr.ht/~kchibisov/682321e0fd4a3ece4a4b7b71591896f5cd3cdb22
    Bug: https://bugs.gentoo.org/679878
    Closes: https://bugs.gentoo.org/680652
    Signed-off-by: Matoro Mahri <matoro_gentoo@matoro.tk>
    Closes: https://github.com/gentoo/gentoo/pull/35246
    Signed-off-by: Sam James <sam@gentoo.org>

 .../rust/files/1.74.1-cross-compile-libz.patch     | 19 +++++++++
 dev-lang/rust/rust-1.74.1.ebuild                   | 46 ++++++++++++++++------
 dev-lang/rust/rust-1.75.0.ebuild                   | 46 ++++++++++++++++------
 3 files changed, 89 insertions(+), 22 deletions(-)