We recently switched to using the Rust ecloss to enforce the rule-of-thumb[1] that Rust can be built with the same version or one minor release behind: ``` RUST_MAX_VER=${PV} RUST_MIN_VER="$(ver_cut 1).$(($(ver_cut 2) - 1)).0" ``` At about the same time we have enforced the (previously optional) USE=system-bootstrap path. Unfortunately we currently lack a full 'bootstrap path' for Rust from our oldest in-tree slot to the newest following this rule, meaning that at least 3 slots of dev-lang/rust-bin are required to bootstrap 1.71.1 -> 1.82.0.[2] Missing slots: * 1.72.1 * 1.73.0 * 1.76.0 * 1.78.0 The following slots are outdated and should be updated: * 1.77.1 -> 1.77.2 As a stretch goal, the following slots are required to bootstrap a modern Rust without downloading a prebuilt Rust compiler[3][4]: * 1.70.0 * 1.69.0 * 1.68.2 * 1.67.1 * 1.66.1 * 1.65.0 * 1.64.0 * 1.63.0 * 1.62.1 * 1.61.0 * 1.60.0 * 1.59.0 * 1.58.1 * 1.57.0 * 1.56.1 * 1.55.0 * 1.54.0 1: this is not actually an official rule, it mostly works and provides us with a good baseline to tighten up from. 2: https://releases.rs/ 3: https://github.com/thepowersgang/mrustc 4 There has been some desire from users to avoid downloading even a single binary to bootstrap Rust (ignoring that dev-lang/rust would previously download the same version of dev-lang/rust-bin internally without USE=system-bootstrap)
The quick-start to this one is: ``` git log --pretty=format: --name-only --diff-filter=A | grep -i dev-lang/rust | sort -u > /tmp/oxidized-files rust_slots=( 1.54.0 ... ) for slot in "${rust_slots[@]}" do for f in $(grep ${slot} /tmp/oxidized-files) do # assume the last commit this file appears is is its final state and check out the parent commit git checkout $(git log --oneline -- ${f} | head -n 1 | awk '{print $1}')~ -- ${f} done done ```
N.B.: One of the primary goals of the gentoo-bootstrap[1] overlay is to provide a complete path to a modern Rust installation that is free of precompiled binaries. It does this using the mrustc mentioned in footnote 3 of comment #0. [1] https://github.com/gentoo-mirror/gentoo-bootstrap
Created attachment 911389 [details] brief old rust -> slotted rust porting guide Added some notes from porting 1.54 to slotted rust. Note that this uses an eclass feature not currently merged to override some crates that do not work - OVERRIDE_CRATES. This impl may change, This recipe should hold for every version that we need; there _should_ be no requirement to replace crates for later versions: 1.54.0 is the only one we need which cannot use openssl 3.x.
I'm sending off patches to -dev for review shortly that will enable us to get Rust 1.54.0 in-tree. Re non-rust bootstrap (mrustc): - We're already past hurdle 1 (openssl)[1] - the eclass changes will allow us to sub in a version that "just works" with modern openssl. - We can crib some notes from the bootstrap overlay[2] (thanks Matt) My current thoughts on approaching this are: 1. Package dev-lang/mrustc as a standalone package. It's not super useful aside from bootstrapping rust but there's no reason to prevent users from using it, and it makes the bootstrap path more clear. 2. Use RUST_OPTIONAL and manually set the dependencies for our bootstrap version. If mrustc proves reliable we can just always bootstrap with it, otherwise we can come up wtih a `mrustc-bootstrap` USE to select that path. 3. ??? 4. PROFIT! Overall it seems pretty straightforward, and [1]: https://orowith2os.gitlab.io/posts/I-lost-my-mind-trying-to-bootstrap-rustc-with-mrustc-and-you-will-too/ (good read, recommended) [2]: https://github.com/gentoo-mirror/gentoo-bootstrap/blob/master/dev-lang/rust/rust-1.54.0.ebuild
(In reply to Matt Jolly from comment #4) > - We can crib some notes from the bootstrap overlay[2] (thanks Matt) I can point out that there is no need to 'emake test' or 'emake local_tests' in src_compile(). I deleted those lines from rust-1.54.0.ebuild and had no issues building the Rust toolchain or subsequently using it. I also dropped all the '-j1' arguments from the emake commands and encountered no problems with parallel building, but of course that's not conclusive. For what it's worth, I currently have installed on my system the very latest, slotted dev-lang/rust-1.82.0-r100::gentoo, with "built strictly from source" provenance going all the way back to dev-lang/rust-1.54.0::gentoo-bootstrap, which was built using mrustc compiled from sources by my system C compiler. The only trick I had to do was, toward the end of the long series of manual stepwise upgrades, to manually change the SLOT (in /var/db/pkg) of dev-lang/rust-1.79.0::gentoo-bootstrap from "stable/1.79" to "1.79.0" so that I could use it to build dev-lang/rust-1.79.0-r100::gentoo. After that, I could simply 'emerge -u1 dev-lang/rust', and the new slotting took care of getting me 1.80.1-r100, 1.81.0-r100, and 1.82.0-r100.
(In reply to Matt Whitlock from comment #5) > (In reply to Matt Jolly from comment #4) > For what it's worth, I currently have installed on my system the very > latest, slotted dev-lang/rust-1.82.0-r100::gentoo, with "built strictly from > source" provenance going all the way back to > dev-lang/rust-1.54.0::gentoo-bootstrap, which was built using mrustc > compiled from sources by my system C compiler. Excellent. The intent here is to enable doing this with nothing but ::gentoo, some disk space, and a _lot_ of patience. Now that the required eclass changes are off for review we can see about filling in the bootstrap gaps and getting the mrustc bootstrap into 1.54.0 soon!
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=04a43a50b1d930e55f8803d420b3371e6c921ff9 commit 04a43a50b1d930e55f8803d420b3371e6c921ff9 Author: Matt Jolly <kangie@gentoo.org> AuthorDate: 2024-11-21 22:19:21 +0000 Commit: Matt Jolly <kangie@gentoo.org> CommitDate: 2024-12-02 07:54:17 +0000 dev-lang/rust{,-bin}: Add 1.54.0 These are the the old ebuilds, updated to be slotted. Other changes: - `USE=rls` was dropped at some point, this is now hard-enabled. - As the LLVM is far too old for Gentoo we are using the bundled LLVM exclusively. In particular, dev-lang/rust:1.54.0 forms an important part of the upcoming 'bootstrap Rust without Rust binaries' path. Bug: https://bugs.gentoo.org/943706 Closes: https://github.com/gentoo/gentoo/pull/39464 Signed-off-by: Matt Jolly <kangie@gentoo.org> dev-lang/rust-bin/Manifest | 34 ++ dev-lang/rust-bin/rust-bin-1.54.0.ebuild | 188 +++++++ dev-lang/rust/Manifest | 4 + ....0-ignore-broken-and-non-applicable-tests.patch | 75 +++ .../files/1.49.0-gentoo-musl-target-specs.patch | 164 ++++++ dev-lang/rust/files/1.53.0-rustversion-1.0.5.patch | 234 +++++++++ dev-lang/rust/files/1.54.0-parallel-miri.patch | 43 ++ .../rust/files/llvm/12/cstdint-signals-h.patch | 24 + dev-lang/rust/rust-1.54.0.ebuild | 559 +++++++++++++++++++++ eclass/rust.eclass | 2 + 10 files changed, 1327 insertions(+)
So far: commit 0051c03c3a9586cbc17c1d81772ebf403e1bce00 Author: Arthur Zamarin <arthurzam@gentoo.org> Date: Sat Dec 7 20:43:50 2024 +0000 dev-lang/rust: add 1.76.0-r100 Signed-off-by: Arthur Zamarin <arthurzam@gentoo.org> commit d77ef3b187c1bc207bc1df5f244b00878ca1bad0 Author: Arthur Zamarin <arthurzam@gentoo.org> Date: Sat Dec 7 20:09:15 2024 +0000 dev-lang/rust: add 1.78.0-r100 Signed-off-by: Arthur Zamarin <arthurzam@gentoo.org>
After lurking in #mrustc for a while it looks like a release targeting 1.74.0 is likely to come out sometime around the new year, and the prerelease version of mrustc is already working on Linux. Switching to target that, 1.54.0 mrustc is already building; hopefully it's a straightforward update.
Mrustc with support to build 1.74 just got released: https://www.reddit.com/r/rust/comments/1he0xbf/announcing_mrustc_0110_with_rust_174_support/
Created attachment 915617 [details] Steps make takes to build rustc/cargo I've attached a file containing the steps make takes in order to build rustc/cargo 1.74.0 with mrustc, following [upstream instructions][0] Things to note: 1. It starts with downloading rustc-${VERSION}-src.tar.gz, but that can easily be done done "manually". 2. mrustc and minicargo are built quite early in the process. 3. It is using the LLVM that is bundled with the rust project. Creating an mrustc ebuild shouldn't be too hard. We'd have to follow the attachment until and including step 9. Installing would be three things: 1. dobin bin/mrustc 2. dobin bin/minicargo 3. install of output-${RUSTC_VERSION} contents (the mrustc library) Maybe we should install mrustc into mrustc and treat it as an eselect-able rust alternative? Then rustc 1.74.0 can BDEPEND on eith rustc-bin, rust or mrustc (of appropriate versions). But rust ebuild is fairly complicated already and I don't feel like messing with it on my own, to allow mrustc. [0]: https://github.com/thepowersgang/mrustc?tab=readme-ov-file#linux-gnu-and-macos
Kangie's already working on integration of mrustc at https://github.com/Kangie/gentoo/tree/rustc-bootstrap (and been discussing it heavily on IRC at #gentoo-rust).
PR up for review (and testing!). I opted to add a bootstrap stage to 1.74.1 using system-installed mrustc built against 1.74.1 complete with system libs. This means that mrustc and minicargo are installed and, if desired, can be used standalone to build "things", as some users in #gentoo-chat thought an alternative compiler would be fun. We don't use the upstream Makefile, opting instead for a bootstrap function in the ebuild in pure bash that does pretty much the same thing. Works with system-llvm and is wired up to build a llvm for use in the bootstrap if required. Potential future enhancements: - maybe we can use our output Rust directly, worth looking into. - is it possible to reuse the bootstrap llvm rather than rebuilding it during the actual Rust build? - needs testing (and likely upstream updates) for other arch support - at the very least I expect the patch will need to be updated. I hope nobody will hold it against me if I leave that to those interested :)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=b488b5c1480da343723d383547e174f728cc0545 commit b488b5c1480da343723d383547e174f728cc0545 Author: Matt Jolly <kangie@gentoo.org> AuthorDate: 2025-01-11 02:44:31 +0000 Commit: Matt Jolly <kangie@gentoo.org> CommitDate: 2025-01-19 08:33:53 +0000 dev-lang/rust: add USE=mrustc_bootstrap This USE flag enables the use of dev-lang/mrustc to bootstrap a rust sysroot which is then used in place of system dev-lang/rust{,-bin}. If required a bootstrap LLVM will be built from the Rust source tree, however USE=system-llvm also works with bootstrap. With this change we finally have a method of bootstrapping dev-lang/rust from source, and we can use this to build later versions of the package. Signed-off-by: Matt Jolly <kangie@gentoo.org> Closes: https://bugs.gentoo.org/943706 Closes: https://github.com/gentoo/gentoo/pull/40095 dev-lang/rust/metadata.xml | 1 + dev-lang/rust/rust-1.74.1-r100.ebuild | 324 +++++++++++++++++++++++++++++++--- 2 files changed, 304 insertions(+), 21 deletions(-)
I've managed to bootstrap rust 1.74 with mrustc, using @Kangie's branch, before it was merged. I've noticed two things that may or may not have been intentional: - Compiling mrustc ignores CFLAGS from make.conf. - mrustc is not visible in the eselect rust list. If that those are intentional, I can guess that the reason is that mrustc is made for bootstraping purposes.
