| Summary: | dev-lang/rust: Enable extended build | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Oleg <O01eg> |
| Component: | Current packages | Assignee: | Gentoo Rust Project <rust> |
| Status: | RESOLVED FIXED | ||
| Severity: | enhancement | CC: | gnome, herrtimson, hiwatari.seiji, martin, mlen, mozilla, O01eg, pacman616, qa, quentin, xaviermiller, zerochaos |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | |||
| Bug Blocks: | 647912 | ||
| Attachments: |
Patch to customize extended tools
Rust without cargo installed Extending rust with cargo installation |
||
|
Description
Oleg
2018-01-23 18:30:57 UTC
gnome-extra/gnome-builder is a consumer of RLS, CCing us to keep track for builder elog purposes I've started by adding an "extended" flag for rust-1.23.0-r1, which enables this. Given the way upstream is going, I'm inclined to think we should enable the extended build by default and get rid of the separate rustfmt and cargo builds (this would need an analogous solution in rust-bin). What do others think about that? Will `eselect rust` also manage version of cargo and other tools? While I think the idea with "extended" is great, it currently breaks installations: 1. dev-lang/rust[extended] should block dev-util/cargo, because of a file conflict with /usr/bin/cargo, and worse: 2. Consumers of cargo (e.g. www-client/firefox-58.0) should not directly depend on dev-util/cargo (thus forcing the conflict) but also be happy with dev-lang/rust[extended]. Perhaps the cleanest solution to 2. is to introduce a virtual/cargo with dependence on || ( dev-lang/rust[extended] dev-util/cargo ). Maybe this restructuring could also be used as an opportunity to introduce a USE=bootstrap or something similar so that - once rust and/or cargo have been installed and work - it is finally no longer necessary to pull in a full obsolete binary distribution of rust and cargo when rebuilding/upgrading only for bootstrapping. Could it allow to have two or more rust[extended] from different slots? If separate dev-util/cargo is not needed, then I vote for dropping it.
Currently dev-util/cargo ebuild also install some documentation, man pages, BASH and ZSH completion.
This code should be copied (in properly updated form) to dev-lang/rust ebuild:
...
dodoc src/tools/cargo/README.md
newbashcomp src/tools/cargo/src/etc/cargo.bashcomp.sh cargo
insinto /usr/share/zsh/site-functions
doins src/tools/cargo/src/etc/_cargo
doman src/tools/cargo/src/etc/man/*
(In reply to Arfrever Frehtes Taifersar Arahesis from comment #6) > Currently dev-util/cargo ebuild also install some documentation, man pages, > BASH and ZSH completion. > This code should be copied (in properly updated form) to dev-lang/rust > ebuild: Currently when having done emerge dev-lang/rust (USE -doc) --- [ebuild R ] dev-lang/rust-1.23.0-r1:stable/1.23::pmaci USE="-debug -doc extended jemalloc" 0 KiB --- There are following cargo related files: --- # qlist dev-lang/rust|grep cargo /usr/share/rust-1.23.0/man/man1/cargo-rustc.1 /usr/share/rust-1.23.0/man/man1/cargo-publish.1 /usr/share/rust-1.23.0/man/man1/cargo-uninstall.1 /usr/share/rust-1.23.0/man/man1/cargo-doc.1 /usr/share/rust-1.23.0/man/man1/cargo-search.1 /usr/share/rust-1.23.0/man/man1/cargo-update.1 /usr/share/rust-1.23.0/man/man1/cargo-package.1 /usr/share/rust-1.23.0/man/man1/cargo-yank.1 /usr/share/rust-1.23.0/man/man1/cargo-rustdoc.1 /usr/share/rust-1.23.0/man/man1/cargo-version.1 /usr/share/rust-1.23.0/man/man1/cargo-owner.1 /usr/share/rust-1.23.0/man/man1/cargo-new.1 /usr/share/rust-1.23.0/man/man1/cargo-login.1 /usr/share/rust-1.23.0/man/man1/cargo.1 /usr/share/rust-1.23.0/man/man1/cargo-run.1 /usr/share/rust-1.23.0/man/man1/cargo-init.1 /usr/share/rust-1.23.0/man/man1/cargo-fetch.1 /usr/share/rust-1.23.0/man/man1/cargo-generate-lockfile.1 /usr/share/rust-1.23.0/man/man1/cargo-clean.1 /usr/share/rust-1.23.0/man/man1/cargo-test.1 /usr/share/rust-1.23.0/man/man1/cargo-check.1 /usr/share/rust-1.23.0/man/man1/cargo-pkgid.1 /usr/share/rust-1.23.0/man/man1/cargo-metadata.1 /usr/share/rust-1.23.0/man/man1/cargo-install.1 /usr/share/rust-1.23.0/man/man1/cargo-build.1 /usr/share/rust-1.23.0/man/man1/cargo-bench.1 /usr/share/zsh/site-functions/_cargo /usr/lib64/rustlib/manifest-cargo /usr/bin/cargo /etc/bash_completion.d/cargo --- # cargo --version cargo 0.24.0-dev --- Look at the '-dev' version appendix, I don't know what that means, having cargo with 'extended' rust? What about disable cargo compilation with some patches? Sent patch to upstream: https://github.com/rust-lang/rust/pull/48015 At least it works for me and rust installs without cargo and with other tools enabled. would be super cool if rust didn't conflict with cargo right now. please fix your deps :-( The patch to select extended tools is accepted and merged to rust. Created attachment 519090 [details, diff]
Patch to customize extended tools
Created attachment 519106 [details]
Rust without cargo installed
Ebuild for rust with RLS only extended tools.
Maybe instead of using extended flag it would be a good idea to introduce RUST_COMPONENTS variable, where individual components could be enabled. This would provide granular control over what gets installed Rust 1.24.0 has been released, but without https://github.com/rust-lang/rust/commit/7be8e2fbb3745602ac864fc079a040dd3a80d91d and https://github.com/rust-lang/rust/commit/78a0b7fd466093003841cec6fd20d65270cf2175 yet. A new USE_EXPAND for a single package does not make sense. IUSE="... cargo rls rust-analysis rust-src rustfmt" would suffice. I suggest to abandon separate dev-util/cargo and dev-util/rustfmt packages if possible. What about slots for rust? If we have rustc from different slots which cargo should be used? I don't understand why people want one massive ebuild that will take 2+ hours to build with a bundle of different features in it? These components are in the same upstream source tarball only because of git submodules. The result will be if you need to turn on USE=XYZ because it wasn't defaulted to on the user will incur a 2+ hour rebuild. What's the advantage in that? I hate the fact that e.g. dev-util/cargo ebuild would want to download 125 mostly small files. This is very ineffective and makes manual management of DISTDIR tiresome.
Majority of these files also violate policy that ${P} (or something similar) should be included in names of downloaded files.
(An example of better handling is dev-java/icedtea. For files whose upstream names do not start with "icedtea", "icedtea-${ICEDTEA_BRANCH}-" (e.g. "icedtea-3.7-") prefix is added at Gentoo level.)
dev-lang/rust ebuild would download only 2 files with good names.
If USE flags were a problem, I would be fine with dev-lang/rust unconditionally installing cargo et al.
But I think that a good compromise would be to have USE flags enabled by default:
IUSE="... +cargo +rls +rust-analysis +rust-src +rustfmt"
Those small files are sources of dev-util/cargo dependencies not cargo itself. To avoid it it's possible to use vendoring and install them before as separate ebuild. Rust compilation also downloads them within src_compile step. > I don't understand why people want one massive ebuild that will take 2+ hours to build with a bundle of different features in it? These components are in the same upstream source tarball only because of git submodules. The result will be if you need to turn on USE=XYZ because it wasn't defaulted to on the user will incur a 2+ hour rebuild. What's the advantage in that?
Upstream has been moving to integrate a number of tools to create a complete rust distribution. This means that for example, the pinned dependencies for any particular cargo version now live in the rust repo, not in the cargo repo. As such, maintaining separate ebuilds for stuff like this goes against the grain of what upstream is doing, making ebuild maintenance harder. Also, as a user experience, Rust developers will more and more expect the installation of Rust to encompass other things, including RLS and rustfmt.
I think there is one division that makes sense, and which we should probably expose as a USE flag (which can be "extended", or replace "extended"): Rust developers vs users of Rust-compiled packages. Rust developers will likely also want rustfmt, clippy and RLS, whereas other users will just want to have rustc and cargo.
Finally, as far as I'm aware I've been the main person maintaining the rust and cargo ebuilds. I'm personally not interested in spending extra effort to split up packages that upstream wants to distribute as a whole. (I'm also aware there is a bunch more stuff to be ironed out; however, I will soon have more time to spend on these sorts of things, and I hope to have a much better situation for Rust in Gentoo around the beginning of April.)
(In reply to Dirkjan Ochtman from comment #20) > however, I will soon > have more time to spend on these sorts of things, and I hope to have a much > better situation for Rust in Gentoo around the beginning of April.) Please don't have a broken tree meanwhile (everything else depends on dev-util/cargo meanwhile, which collides with rust[extended]). At least heavily consider package.use.mask rust[extended] meanwhile, if nothing else. That said, I personally would prefer to not have 20+ copies of many common crates installed on my system, statically linked and all separate versions with no individual updates performed. So I would like if we tried to move towards properly packaged rust-based libraries from crates, with appropriate parallel installable slotting based on semantic versioning and so on. That said, it's not like I'm volunteering to do this work at this point of time (maybe in the rather distant future, if I do get involved with rust programming language as I kind of hope). However, if we move to a monolith rust package now, I imagine this ideal goal to become harder. I'm imagining a world where cargo crate deps themselves are also from packages, with only initial build with USE=build doing some static linking to bootstrap. I don't consider distfiles proliferation a valid concern, we have tools for that. The tree is only broken if you use extended, which is disabled by default. I will work to fix it, but again, I have limited time. (In reply to Dirkjan Ochtman from comment #22) > The tree is only broken if you use extended, which is disabled by default. I > will work to fix it, but again, I have limited time. Exactly as I said. Please package.use.mask rust[extended] meanwhile or ACK doing it. Unhandled file collisions == broken tree. We have users (and developers) spending a day compiling rust (as it builds llvm in the process; probably with most LLVM_TARGETS) after adding this new cool looking USE=extended (documented to give you cargo, rls and rustfmt; who wouldn't want that, right?), only to end up with file collisions after rust is emerged and the deptree proceeds to dev-util/cargo. And then they get to spend another day recompiling rust. This is not an acceptable state of the tree for months now, with unknown timeline for fixing it properly (at least 3 weeks it seems). Fine, you have my ACK. But I'd prefer if someone set up a virtual/cargo that depends on ( cargo || cargo-bin || rust[extended] ) and fixed up all the dependents to moved to that, so we can make actual progress. (In reply to Dirkjan Ochtman from comment #24) > I'd prefer if someone set up a virtual/cargo that > depends on ( cargo || cargo-bin || rust[extended] ) If the long-term plan is to have cargo included in rust package, it would be better to add "cargo" USE flag to virtual/rust and update reverse dependencies to depend on virtual/rust[cargo]. How is it going to have cargo from different rust slots? In practice, there is only 1 Rust slot: "stable"
(If ${PV} = *beta*, then slot would be "beta".)
There also "git" slot from overlay which is nightly equivalent. Actually, in my mind cargo should be installed unconditionally if rust is installed, as rustc is practically useless without cargo anyway. So maybe we should change other packages to depend on virtual/rust only, and make sure all variants of it keep cargo installed somehow. The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=03005eca7f8563a4964c7f9990e6654152aa0319 commit 03005eca7f8563a4964c7f9990e6654152aa0319 Author: Mart Raudsepp <leio@gentoo.org> AuthorDate: 2018-04-01 11:30:33 +0000 Commit: Mart Raudsepp <leio@gentoo.org> CommitDate: 2018-04-01 11:31:51 +0000 profiles: p.use.mask rust[extended] due to unsolved file collisions for months Acked-by: Dirkjan Ochtman <djc@gentoo.org> Bug: https://bugs.gentoo.org/645498 Bug: https://bugs.gentoo.org/645562 profiles/base/package.use.mask | 5 +++++ 1 file changed, 5 insertions(+)} Created attachment 534400 [details]
Extending rust with cargo installation
There is a ebuild for stable rust without conflicts with cargo.
Also it allows to install sources.
(In reply to Oleg from comment #31) > > There is a ebuild for stable rust without conflicts with cargo. Why would anybody using a source-based distribution want to install the binary cargo tarball if he can have cargo compiled from source? Anyway, as long as the binary tarball remains optional (and can be selected/unselected with "eselect cargo") I am not against it. (In reply to Martin Väth from comment #32) > (In reply to Oleg from comment #31) > > > > There is a ebuild for stable rust without conflicts with cargo. > > Why would anybody using a source-based distribution want to install the > binary cargo tarball if he can have cargo compiled from source? > > Anyway, as long as the binary tarball remains optional (and can be > selected/unselected with "eselect cargo") I am not against it. It could be used with cargo built from sources. Does the extended useflag install the cargo binary from the prebuilt stage, or does it bootstrap cargo from source? In any way, it would be ideal to provide one virtual/cargo ebuild to satisfy all implementations of cargo as in dev-util/cargo, dev-lang/rust[extended], _and_ dev-lang/rust-bin[extended] in one single package, where other packages such as firefox can depend on! (In reply to tt_1 from comment #34) > Does the extended useflag install the cargo binary from the prebuilt stage, > or does it bootstrap cargo from source? In any way, it would be ideal to > provide one virtual/cargo ebuild to satisfy all implementations of cargo as > in dev-util/cargo, dev-lang/rust[extended], _and_ > dev-lang/rust-bin[extended] in one single package, where other packages such > as firefox can depend on! I think it will be better to have a separate USE flag for cargo so it will be possible to have a dev-lang/rust[extended,-cargo] which won't conflict with dev-util/cargo and dev-lang/rust[extended,cargo] which will conflict with dev-util/cargo. Starting with the dev-lang/rust-bin-1.26 ebuild, there are a few changes in how dev-util/cargo is pulled into the dep graph, or to be more precise, >=dev-lang/rust-bin-1.26 now forces the cargo version to be rust version + 1 Personally I don't see much of a chance to fix this via a useflag; but I'm not a dev so it may be possible. Can you provide a patch for testing? (In reply to tt_1 from comment #36) > Starting with the dev-lang/rust-bin-1.26 ebuild, there are a few changes in > how dev-util/cargo is pulled into the dep graph, or to be more precise, > >=dev-lang/rust-bin-1.26 now forces the cargo version to be rust version + 1 > > Personally I don't see much of a chance to fix this via a useflag; but I'm > not a dev so it may be possible. Can you provide a patch for testing? Attached ebuild have a same restriction: PDEPEND=">=dev-util/cargo-${CARGO_DEPEND_VERSION}" This has been fixed by now. |
