Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 731764 - dev-lang/rust: support system-bootstrap in a sysroot
Summary: dev-lang/rust: support system-bootstrap in a sysroot
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Georgy Yakovlev
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-07-08 15:35 UTC by David Michael
Modified: 2020-11-23 12:49 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
cross.patch (cross.patch,2.27 KB, patch)
2020-07-11 19:47 UTC, David Michael
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description David Michael 2020-07-08 15:35:25 UTC
I'd like to resolve the dependency loop with system-bootstrap when building in a sysroot.  It can use the Rust installation on the host without needing a bootstrap version in the sysroot as is done with Go.  This may be related to bug #689336.

Below is a patch that fixes it by putting the bootstrap dependencies in BDEPEND.  It also moves cmake and ninja to BDEPEND for !system-llvm since there is no need to install them in the sysroot.  (The eselect package was copied to BDEPEND as in bug #716334 to handle both sysroot and native root installations, but it should really be in IDEPEND.)

This patch does not support cross-compiling.  More changes are required around cargo.toml etc. for that, and I didn't get it fully working in general yet.

Reproducible: Always

Steps to Reproduce:
1. USE=system-bootstrap emerge -v dev-lang/rust  # with a sysroot profile

Actual Results:  
Circular dependency: dev-lang/rust -> dev-lang/rust

Expected Results:  
It should use the native root's dev-lang/rust installation.

--- a/dev-lang/rust/rust-1.44.1.ebuild
+++ b/dev-lang/rust/rust-1.44.1.ebuild
@@ -58,7 +58,19 @@
 "
 LLVM_MAX_SLOT=10
 
-BOOTSTRAP_DEPEND="|| ( >=dev-lang/rust-1.$(($(ver_cut 2) - 1)) >=dev-lang/rust-bin-1.$(($(ver_cut 2) - 1)) )"
+BDEPEND="
+	>=app-eselect/eselect-rust-20190311
+	system-bootstrap? (
+		|| (
+			>=dev-lang/rust-1.$(($(ver_cut 2) - 1))
+			>=dev-lang/rust-bin-1.$(($(ver_cut 2) - 1))
+		)
+	)
+	!system-llvm? (
+		dev-util/cmake
+		dev-util/ninja
+	)
+"
 
 # libgit2 should be at least same as bungled into libgit-sys #707746
 COMMON_DEPEND="
@@ -81,11 +93,6 @@
 		>=sys-devel/gcc-4.7
 		>=sys-devel/clang-3.5
 	)
-	system-bootstrap? ( ${BOOTSTRAP_DEPEND}	)
-	!system-llvm? (
-		dev-util/cmake
-		dev-util/ninja
-	)
 "
 
 RDEPEND="${COMMON_DEPEND}
Comment 1 Georgy Yakovlev gentoo-dev 2020-07-09 04:49:37 UTC
that's good thing to support and I wanted to move some build deps to BDEPEND anyway.
Comment 2 David Michael 2020-07-11 19:47:59 UTC
Created attachment 648974 [details, diff]
cross.patch

Here is a patch I used for cross-compiling.  Some notes:

  - It includes the fix for bug #732222 to fix a bad curl cross-compiling environment.
  - It sets "build" to the native triplet to fix using cross-tools on native objects.
  - It writes dummy CFLAGS for the native triplet to override defaulting to passing cross-compiler CFLAGS to the native compiler.
  - The ranlib addition isn't necessary; I just left that there.

It needs this environment when running emerge:

  - I_KNOW_WHAT_I_AM_DOING_CROSS=yes
  - LLVM_TARGETS=X86
  - PKG_CONFIG_SYSROOT_DIR=/usr/armv5tel-gentoo-linux-gnueabi
  - RUST_CROSS_TARGETS=X86:x86_64-unknown-linux-gnu:x86_64-pc-linux-gnu
  - USE='system-bootstrap -system-llvm'

Don't use system-llvm since the build system will try to execute cross-compiled binaries for it.  The LLVM target is needed to compile LLVM compatible with the build system.  The cross-target definition for the native system isn't really needed; it might just need to define the linker for that Rust target, but the cross-target environment variable defines that easily.  (It should probably hard-code that configuration instead when $CBUILD != $CHOST so that's not needed.)
Comment 3 David Michael 2020-07-11 19:51:03 UTC
(I forgot to note that the pkg-config sysroot setting is required for it to use the cross-compiled libgit2 version instead of trying to use the native one.)
Comment 4 David Michael 2020-07-11 19:54:09 UTC
Might as well note this for completeness, too, since I pasted the sysroot definition for armv5te...  This change is required for cross-compiling to armv5te with at least GCC 9 because of https://github.com/rust-lang/rust/issues/48625 .

--- a/vendor/compiler_builtins/build.rs
+++ b/vendor/compiler_builtins/build.rs
@@ -69,7 +69,6 @@
 
     // Only emit the ARM Linux atomic emulation on pre-ARMv6 architectures.
     if llvm_target[0] == "armv4t" || llvm_target[0] == "armv5te" {
-        println!("cargo:rustc-cfg=kernel_user_helpers")
     }
 }
Comment 5 Larry the Git Cow gentoo-dev 2020-07-17 06:36:13 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=75a4d1ee6422c5e2d3984e1e7c51e65dcce66c96

commit 75a4d1ee6422c5e2d3984e1e7c51e65dcce66c96
Author:     Georgy Yakovlev <gyakovlev@gentoo.org>
AuthorDate: 2020-07-17 00:39:23 +0000
Commit:     Georgy Yakovlev <gyakovlev@gentoo.org>
CommitDate: 2020-07-17 06:35:02 +0000

    dev-lang/rust: bump to 1.45.0
    
    move a lot of deps to BDEPEND
    
    Bug: https://bugs.gentoo.org/703744
    Bug: https://bugs.gentoo.org/683420
    Bug: https://bugs.gentoo.org/731764
    Package-Manager: Portage-2.3.103, Repoman-2.3.23
    Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org>

 dev-lang/rust/Manifest           |  15 ++
 dev-lang/rust/rust-1.45.0.ebuild | 503 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 518 insertions(+)
Comment 6 David Michael 2020-07-17 14:07:08 UTC
It looks like this can be closed, unless you want to use it to track support for cross-compiling the compiler.  (The bug should probably be renamed in that case.)  I have no need for cross-compiling the compiler anymore since Gentoo will not allow non-upstream bootstrap binaries, so I'm not going to pursue it further.
Comment 7 Georgy Yakovlev gentoo-dev 2020-10-13 03:30:17 UTC
I have equipment for building upstream-like dist tarballs, but just need automation (docker or similar) and I NEED this to solve rust musl problem.

so basically if a docker setup which can take couple of local patches and emit upstream-like dist tarballs, I can try bootstrapping a bit more arches, but will strictly keep using upstream tarballs which they already ship.

I will research this topic as time permits, but any pointers will be helpful. Hadn't had time to properly dig this. I bet upstream has some container-like setup we can fork and re-use for lesser arches + local patches.
Comment 8 David Michael 2020-10-13 14:19:18 UTC
I have not looked into how to produce upstream binaries, only Gentoo binpkgs.  For that, automation should probably take four options: Rust version, LLVM_TARGETS architecture, CHOST triplet, and Rust target.  The process would be:

  - Enter a stage3 container
  - Run emerge-webrsync
  - Apply the Rust ebuild patch and update manifest
  - Write the native environment vars LLVM_TARGETS, RUST_CROSS_TARGETS, etc.
  - Run emerge to update the stage3 and install crossdev, virtual/rust, and rust-bin (or take the extra time to compile dev-lang/rust ... or start campaigning to get Rust built into the stage3 images)
  - Run crossdev to build cross-compilers
  - Update $SYSROOT/etc/portage with the target environment and any patches
  - Run cross-emerge to install dev-lang/rust and create a binpkg

That's what my own automation does anyway.  It's for a much more generalized build environment, though, so it will probably be easier to write a simple procedure from scratch rather than try to make sense of my scripts.
Comment 9 Georgy Yakovlev gentoo-dev 2020-10-13 17:01:56 UTC
I've started using docker only recently myself and have this setup

https://github.com/gyakovlev/gendo/

it's a bit dumb and pre-alpha but it works.

beauty of it is you can do

DOCKER_HOST="ssh://arm-host-with-docker" make rust

and it will emit arm binpkgs to _local_ directory after completion.

it does not always succeed and needs a lot of work, so just proof of concept for now.


another great thing about rust bootstrap is that `./x.py dist` creates dist tarballs for all components, so ebuild can emit/expose those with some MAINTAINER_ONLY variable.


I'll be researching this subject more at time permits, but that's all I have for now.