Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 747760 - dev-lang/rust{,-bin}: introduce RUST_TARGETS
Summary: dev-lang/rust{,-bin}: introduce RUST_TARGETS
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal with 2 votes (vote)
Assignee: Gentoo Rust Project
URL: https://archives.gentoo.org/gentoo-de...
Whiteboard:
Keywords:
Depends on:
Blocks: 679878 680652 684896 689336
  Show dependency tree
 
Reported: 2020-10-11 03:10 UTC by Georgy Yakovlev
Modified: 2021-06-14 22:16 UTC (History)
7 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Georgy Yakovlev gentoo-dev 2020-10-11 03:10:34 UTC
Tracking/discussion bug for RUST_TARGETS use expand introduction.
Comment 1 Georgy Yakovlev gentoo-dev 2020-10-11 03:32:34 UTC
I would like to introduce new USE_EXPAND in profiles for rusts
3 consumers in the tree:
dev-lang/rust
dev-lang/rust-bin
sys-devel/rust-std
and maybe virtual, but not sure yet, may be required for firefox.

The idea:
Rust supports building as many targets as one wants, as long as C toolchan can do it and matching llvm target is enabled on rust or llvm.

There are some special rust targets that require same host toolchain but are treated like separate targets in rust.
For example x86_64-unknown-linux-gnu target is primary on amd64,
but on multilib profiles we can easily enable i686-unknown-linux-gnu target and it will just work (that's how multilib in rust works now)

Another example is arm:

armv7-unknown-linux-gnueabihf is primary toolchain.
but rust supports additional target thumbv7neon-unknown-linux-gnueabihf,
which requires exactly same host toolchain, but is capable of emitting thumb-neon binaries. This is required by firefox if --enable-neon is passed in firefox build.

Another example: ppc64

We can easily build both ppc64 and ppc64le targets in a single bootstrap with host toolchain, and rust will support emitting code for both endians.

Another example: musl

On musl we need 1 extra target with gentoo vendor field.
So installing both
  x86_64-unknown-linux-musl
  x86_64-gentoo-linux-musl
will be very easy, as both targets use same C toolchain, but slightly different settings.

And last, but not the least, it allows cross support if matching toolchain is installed (tested with crossdev)
A user can unmask the following targets on amd64
  rust_targets_thumbv7neon-unknown-linux-gnueabihf
  rust_targets_armv7-unknown-linux-gnueabihf

And as long as they have /usr/armv7a-unknown-linux-gnueabihf toolchain available at build time, and llvm_targets_ARM enabled, rust will bootstrap itself in a way that it will be able to emit code for any of 3 targets enable. This is optional opt-in functionality.

And just to clarify, native host toolchain is not going to be a use-expand, at least that's how I imagine it now.
Native target matching CHOST is always build without any flags, and is the default target for rust.
But I may re-evaluate it and introduce requirements like

 abi_x86_64?? ( rust_targets_x86_64-unknown-linux-gnu )
 abi_x86_32? ( rust_targets_i686-unknown-linux-gnu )

for multilib profiles.

and something like this for non-multilib arches.

 ppc64? ( ||
    rust_targets_powerpc64le-unknown-linux-gnu
    rust_targets_powerpc64-unknown-linux-gnu
)

for example, similar to as llvm does with LLVM_TARGETS.
Of course default targets will be use-forced via profiles.
But if possible I'd avoid it.


Proof of concept exists (without using expand flags for now) in both rust and rust-bin ebuilds. Multilib is implemented that way and has been working great for more than a year, also experimental cross support in rust implemented that way, and users are already able to define as many targets as they want, but it feels hacky and I would like to move forward to targets approach and make it trackable via flags, and not magic variables nobody knows about.


Initial rollout will require a bit of arch profile masking (like llvm does it) and maybe new eclass to reduce boilerplate code in ebuilds, but nothing to complex.

WDYT? 


sent RFC to gentoo-dev with the same text.
Comment 2 Dirkjan Ochtman gentoo-dev 2020-10-11 12:34:02 UTC
It sounds okay to me. My primary doubt is how much overlap there would be between RUST_TARGETS and LLVM_TARGETS, can you go into that a bit more!

(BTW, your work on the rust ebuilds is awesome, thanks so much for getting it done!)
Comment 3 David Michael 2020-10-11 15:03:47 UTC
Does this imply that there must be a predefined list of supported targets to declare IUSE?  If so, I'd suggest declaring it with the output of "rustc --print target-list" after each new Rust release so at least all builtin targets are available.

There are more odd cases than your examples, such as from the fact that Rust nonsensically assumes i686 implies SSE2.  For i686 chips without SSE, I need to use the i586-unknown-linux-gnu target to build usable code which is not currently in the eclass's list of supported targets.  I am only able to use it because RUST_CROSS_TARGETS accepts arbitrary values.

A predefined list of USE flags would still exclude non-builtin targets and custom builtin targets from patching, but I haven't seriously used Gentoo for this yet, so I won't consider handling that a priority.
Comment 4 Georgy Yakovlev gentoo-dev 2020-10-11 15:55:15 UTC
thanks for encouragement =) I like working on it.

rust<->llvm  target mapping is not going to be too complex to implement, because llvm targets are very broad, but pretty verbose REQUIRED_USE will be needed, better emitted by eclass (rust-toolchain or a new one) via arch to llvm mapping.

result will look like:

REQUIRED_USE=
    rust_target_thumbv7neon-unknown-linux-gnueabihf? ( llvm_targets_ARM )
    rust_targets_armv7-unknown-linux-gnueabihf? ( llvm_targets_ARM )
    rust_targets_powerpc64le-unknown-linux-gnu? ( llvm_targets_PowerPC )
    rust_targets_x86_64-unknown-linux-gnu? ( llvm_targets_X86 )
    rust_targets_i686-unknown-linux-gnu? ( llvm_targets_X86 )
...
"

this can be achieved via many to some style bash array mapping which can be updated on releases.
hardcoded list of rust targets and dynamic intersection with llvm_targets based on patterns.

like arm* -> ARM
thumb*neon -> ARM
ppc* -> PPC

etc

simple iterator will do.

like:
> for target in ${ALL_RUSTC_TARGETS[@]}; do
>     if [[ ${target} == arm* ]]; then  
>         requse+=( "${target}? ( llvm_targets_ARM )" )
>     fi
...
but it can be done smarter with 2 arrays.

some relevant pointers are in https://bugs.gentoo.org/651908


(In reply to David Michael from comment #3)
> Does this imply that there must be a predefined list of supported targets to
> declare IUSE?  If so, I'd suggest declaring it with the output of "rustc
> --print target-list" after each new Rust release so at least all builtin
> targets are available.
> 
> There are more odd cases than your examples, such as from the fact that Rust
> nonsensically assumes i686 implies SSE2.  For i686 chips without SSE, I need
> to use the i586-unknown-linux-gnu target to build usable code which is not
> currently in the eclass's list of supported targets.  I am only able to use
> it because RUST_CROSS_TARGETS accepts arbitrary values.
> 
> A predefined list of USE flags would still exclude non-builtin targets and
> custom builtin targets from patching, but I haven't seriously used Gentoo
> for this yet, so I won't consider handling that a priority.

yes, subset of '--print target-list'.
defining _ALL_ targets from that list is excessive I think,
but we'll need a list of at least 1 target per each keyword + musl + extras like thumb* and wasm*

we can dump all targets to eclass on releases, but only convert what we want to IUSE/REQUIRED_USE.

no plans to introduce foreign-os, android, and none, for example, at least initially.
custom cross thing will stay hidden for power users like it is now.

and yes, there are plans to expand/rewrite/improve rust_abi() mapper based on that target list, instead of making very rigid assumptions like it is now.