|
Description
Dennis Schridde
2017-05-07 16:19:10 UTC
Created attachment 475276 [details]
rust-1.17.0.ebuild
Created attachment 475278 [details]
cargo-0.18.0.ebuild
Created attachment 475280 [details, diff]
patch: cargo-0.18.0.ebuild (from 0.17.0)
The ebuild for Rust is more or less a complete rewrite, since I switched to the now recommended `x.py` build-system and `config.toml`. The ebuild for Cargo is not that different from the previous version, hence I provided a patch. (In reply to Dennis Schridde from comment #2) > Created attachment 475278 [details] > cargo-0.18.0.ebuild This is great work, would be even better tho if it actually supported building rustc for musl as the work has landed upstream in this release. I can't see patches used in 1.17 ebuild found patches in overlay, there is output with clang and llvm flags on: https://gist.github.com/mpkh/1119ec439bfc6efd8a133c0834fe1f2d Created attachment 475588 [details, diff]
rust-1.17.0-bootstrap-verbose.patch
Created attachment 475590 [details, diff]
rust-1.17.0-bootstrap-output-name-of-failed-config.patch
(In reply to Cynede from comment #8) > cargo fault: https://gist.github.com/mpkh/ca1d307012d9a917e6c78fe53eec0318 This looks very similar to the problems I was facing with Rust initially, until I used the "rust" (instead of the "rustc") tarball for building, which also includes the contents of the "rust-std" tarball -- i.e. the standard library. Why this also hits Cargo now I am not sure -- Rust's debug output is very scarce [1] and even RUST_LOG=info does give anything useful. My suspicion is, that the libraries being installed to /usr/lib64/rust-1.17.0 might confuse the loader, but I still have to investigate. [1]: The report_error function in librustc_metadata/locator outputs only the name of the missing crate, but not any directories or library names it tried: https://github.com/rust-lang/rust/blob/1.17.0/src/librustc_metadata/locator.rs#L324 It's already 1.18 out: https://blog.rust-lang.org/2017/06/08/Rust-1.18.html https://www.rust-lang.org/en-US/other-installers.html#standalone ie rust-bin if someone bumps rust-bin.... a number of prebuilt... https://static.rust-lang.org/rustup/dist/aarch64-unknown-linux-gnu/rustup-init Beta (1.19) Nightly (1.20) Created attachment 476730 [details] rust-1.17.0.ebuild Fix comment #8 (std not being found). Cargo 0.18.0 as attached compiles now. (In reply to Jory A. Pratt from comment #5) > (In reply to Dennis Schridde from comment #2) > > Created attachment 475278 [details] > > cargo-0.18.0.ebuild > > This is great work, would be even better tho if it actually supported > building rustc for musl as the work has landed upstream in this release. That would be bug #615030. I'd handle it separately. Created attachment 476742 [details, diff] rust-1.18.0-bootstrap-use-configured-rustc-cargo-paths.patch Rust 1.18.0 needs https://github.com/rust-lang/rust/pull/42695 to bootstrap using local cargo / rustc. Created attachment 476744 [details]
rust-1.18.0.ebuild
Rust 1.18.0 version bump.
rust-1.17.0-bootstrap-output-name-of-failed-config.patch and rust-1.17.0-bootstrap-verbose.patch help with debugging the build, but are not strictly necessary, hence they are not anymore used in this version.
Created attachment 476746 [details]
cargo-0.19.0.ebuild
Cargo 0.19.0 version bump
Created attachment 476748 [details, diff]
patch: cargo-0.19.0.ebuild (from 0.18.0)
The Rust 1.18.0 and Cargo 0.19.0 version bumps correspond to: * https://github.com/devurandom/gentoo-overlay/commit/1f646d22f39c05028b95da0cc3b4e38e2b660464 * https://github.com/devurandom/gentoo-overlay/commit/98e1e4ac85b204f68870900d951024d533985471 My installed Rust is able to build Firefox 54, hence I consider it to be actually working and tested. rust cannot be built with USE="-llvm" (In reply to Perfect Gentleman from comment #22) > rust cannot be built with USE="-llvm" Could you please give me a build.log? (I assume that the issue involves llvm-config -- is that correct?) (In reply to Dennis Schridde from comment #23) > (In reply to Perfect Gentleman from comment #22) > > rust cannot be built with USE="-llvm" > > Could you please give me a build.log? (I assume that the issue involves > llvm-config -- is that correct?) I can give ebuild with which I had rust built. Created attachment 477190 [details]
ebuild with which I had rust built
I've got working version in rust overlay currently, please test it out, fixes are welcome via pull requests https://github.com/gentoo/gentoo-rust (In reply to Cynede from comment #26) > I've got working version in rust overlay currently, please test it out, > fixes are welcome via pull requests https://github.com/gentoo/gentoo-rust I've tested that ebuilds too. They don't work. (In reply to Perfect Gentleman from comment #27) > (In reply to Cynede from comment #26) > > I've got working version in rust overlay currently, please test it out, > > fixes are welcome via pull requests https://github.com/gentoo/gentoo-rust > > I've tested that ebuilds too. They don't work. they do for me... maybe some use flags don't, alike clang can cause problems potentially home λ rustc --version rustc 1.18.0-dev home λ cargo --version cargo 0.19.0 home λ alacritty --version alacritty 0.1.0 > they do for me... maybe some use flags don't, alike clang can cause problems
> potentially
Don't for me
------------
* rustc-1.18.0-src.tar.gz SHA256 SHA512 WHIRLPOOL size ;-) ... [ ok ]
* rust-1.17.0-x86_64-unknown-linux-gnu.tar.gz SHA256 SHA512 WHIRLPOOL size ;-) ... [ ok ]
* checking ebuild checksums ;-) ... [ ok ]
* checking auxfile checksums ;-) ... [ ok ]
* checking miscfile checksums ;-) ... [ ok ]
* ERROR: dev-lang/rust-1.18.0::pg_overlay failed (setup phase):
* No LLVM slot <= 4 found in PATH!
*
* Call stack:
* ebuild.sh, line 115: Called pkg_setup
* rust-1.18.0.ebuild, line 74: Called llvm_pkg_setup
* llvm.eclass, line 133: Called get_llvm_prefix '4'
* llvm.eclass, line 114: Called die
* The specific snippet of code:
* die "No LLVM slot${1:+ <= ${1}} found in PATH!"
*
* If you need support, post the output of `emerge --info '=dev-lang/rust-1.18.0::pg_overlay'`,
* the complete build log and the output of `emerge -pqv '=dev-lang/rust-1.18.0::pg_overlay'`.
* The complete build log is located at '/tmp/portage/dev-lang/rust-1.18.0/temp/build.log'.
* The ebuild environment file is located at '/tmp/portage/dev-lang/rust-1.18.0/temp/environment'.
* Working directory: '/tmp/portage/dev-lang/rust-1.18.0/homedir'
* S: '/tmp/portage/dev-lang/rust-1.18.0/work/rustc-1.18.0-src'
---------------------------------------------------------------
and cargo ------------ * cargo-0.19.0.tar.gz SHA256 SHA512 WHIRLPOOL size ;-) ... [ ok ] * cargo-0.19.0-x86_64-unknown-linux-gnu.tar.gz SHA256 SHA512 WHIRLPOOL size ;-) ... [ ok ] * checking ebuild checksums ;-) ... [ ok ] * checking miscfile checksums ;-) ... [ ok ] >>> Unpacking source... >>> Unpacking cargo-0.19.0.tar.gz to /tmp/portage/dev-util/cargo-0.19.0-r1/work >>> Unpacking cargo-0.19.0-x86_64-unknown-linux-gnu.tar.gz to /tmp/portage/dev-util/cargo-0.19.0-r1/work >>> Source unpacked in /tmp/portage/dev-util/cargo-0.19.0-r1/work >>> Preparing source in /tmp/portage/dev-util/cargo-0.19.0-r1/work/cargo-0.19.0 ... >>> Source prepared. >>> Configuring source in /tmp/portage/dev-util/cargo-0.19.0-r1/work/cargo-0.19.0 ... ./configure --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --target=x86_64-unknown-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --docdir=/usr/share/doc/cargo-0.19.0-r1 --libdir=/usr/lib64 --prefix=/usr --host=x86_64-unknown-linux-gnu --build=x86_64-unknown-linux-gnu --target=x86_64-unknown-linux-gnu --cargo=/tmp/portage/dev-util/cargo-0.19.0-r1/work/cargo-0.19.0-x86_64-unknown-linux-gnu/cargo/bin/cargo --enable-optimize --release-channel=stable --disable-verify-install --disable-debug --disable-cross-tests configure: looking for configure programs configure: found cmp configure: found mkdir configure: found printf configure: found cut configure: found head configure: found grep configure: found xargs configure: found cp configure: found find configure: found uname configure: found date configure: found tr configure: found sed configure: found cmake configure: found make configure: recreating config.tmp configure: configure: processing ./configure args configure: configure: CFG_DISABLE_DEBUG := 1 configure: CFG_DISABLE_VERIFY_INSTALL := 1 configure: CFG_DISABLE_CROSS_TESTS := 1 configure: CFG_PREFIX := /usr configure: CFG_LOCAL_RUST_ROOT := configure: CFG_CARGO := /tmp/portage/dev-util/cargo-0.19.0- ... configure: CFG_RUSTC := rustc configure: CFG_RUSTDOC := rustdoc configure: CFG_CARGO := /tmp/portage/dev-util/cargo-0.19.0- ... configure: CFG_RUSTC := /usr/bin/rustc (1.18.0-dev) configure: CFG_RUSTDOC := /usr/bin/rustdoc (1.18.0-dev) configure: CFG_BUILD := x86_64-unknown-linux-gnu configure: CFG_HOST := x86_64-unknown-linux-gnu configure: CFG_TARGET := x86_64-unknown-linux-gnu configure: CFG_LOCALSTATEDIR := /var/lib configure: CFG_SYSCONFDIR := /etc configure: CFG_DATADIR := /usr/share configure: CFG_INFODIR := /usr/share/info configure: CFG_DOCDIR := /usr/share/doc/cargo-0.19.0-r1 configure: CFG_MANDIR := /usr/share/man configure: CFG_LIBDIR := /usr/lib64 configure: git: no git directory. Changing default release channel to stable configure: CFG_RELEASE_CHANNEL := stable configure: configure: validating ./configure args configure: configure: configure: looking for build programs configure: configure: CFG_CC := /usr/bin/cc (6.3.0) configure: GIT := /usr/bin/git (2.13.1) configure: configure: writing configuration configure: configure: CFG_SRC_DIR := /tmp/portage/dev-util/cargo-0.19.0- ... configure: CFG_BUILD_DIR := /tmp/portage/dev-util/cargo-0.19.0- ... configure: CFG_CONFIGURE_ARGS := --prefix=/usr --build=x86_64-pc-lin ... configure: CFG_PREFIX := /usr configure: CFG_BUILD := x86_64-unknown-linux-gnu configure: CFG_HOST := x86_64-unknown-linux-gnu configure: CFG_TARGET := x86_64-unknown-linux-gnu configure: CFG_DATADIR := /share configure: CFG_DOCDIR := /share/doc/cargo-0.19.0-r1 configure: CFG_INFODIR := /share/info configure: CFG_MANDIR := /share/man configure: CFG_LIBDIR := /lib64 configure: CFG_RUSTC := /usr/bin/rustc configure: CFG_RUSTDOC := /usr/bin/rustdoc configure: CFG_CARGO := /tmp/portage/dev-util/cargo-0.19.0- ... configure: CFG_GIT := configure: configure: cp /tmp/portage/dev-util/cargo-0.19.0-r1/work/cargo-0.19.0/Makefile.in ./Makefile configure: mv config.tmp config.mk configure: configure: complete configure: configure: >>> Source configured. >>> Compiling source in /tmp/portage/dev-util/cargo-0.19.0-r1/work/cargo-0.19.0 ... make -j9 VERBOSE=1 PKG_CONFIG_PATH= /usr/bin/rustc -V rustc 1.18.0-dev /tmp/portage/dev-util/cargo-0.19.0-r1/work/cargo-0.19.0-x86_64-unknown-linux-gnu/cargo/bin/cargo --version cargo 0.19.0 (28d1d60d4 2017-05-16) /tmp/portage/dev-util/cargo-0.19.0-r1/work/cargo-0.19.0-x86_64-unknown-linux-gnu/cargo/bin/cargo build --target x86_64-unknown-linux-gnu --manifest-path /tmp/portage/dev-util/cargo-0.19.0-r1/work/cargo-0.19.0//Cargo.toml --release --verbose error: no matching package named `shell-escape` found (required by `cargo`) location searched: registry https://github.com/rust-lang/crates.io-index version required: = 0.1.3 make: *** [Makefile:130: cargo-x86_64-unknown-linux-gnu] Error 101 * ERROR: dev-util/cargo-0.19.0-r1::pg_overlay failed (compile phase): * emake failed * * If you need support, post the output of `emerge --info '=dev-util/cargo-0.19.0-r1::pg_overlay'`, * the complete build log and the output of `emerge -pqv '=dev-util/cargo-0.19.0-r1::pg_overlay'`. * The complete build log is located at '/tmp/portage/dev-util/cargo-0.19.0-r1/temp/build.log'. * The ebuild environment file is located at '/tmp/portage/dev-util/cargo-0.19.0-r1/temp/environment'. * Working directory: '/tmp/portage/dev-util/cargo-0.19.0-r1/work/cargo-0.19.0' * S: '/tmp/portage/dev-util/cargo-0.19.0-r1/work/cargo-0.19.0' --------------------------------------------------------------- $ equery uses rust [ Legend : U - final flag setting for installation] [ : I - package is installed with flag ] [ Colors : set, unset ] * Found these USE flags for dev-lang/rust-1.18.0: U I - - clang : Use sys-devel/clang for building - - debug : Enable extra debug codepaths, like asserts and extra output. If you want to get meaningful backtraces see https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Backtraces - - doc : Add extra documentation (API, Javadoc, etc). It is recommended to enable per package instead of globally + + jemalloc : Use dev-libs/jemalloc for allocations - - llvm : Build with sys-devel/llvm I was talking about rust overlay, not pg_overlay (In reply to Cynede from comment #32) > I was talking about rust overlay, not pg_overlay the same, I meant ebuilds from rust overlay (In reply to Cynede from comment #32) > I was talking about rust overlay, not pg_overlay but I wasn't using eclass from that overlay Created attachment 477200 [details]
build.log
For me the ebuild compiles fine as long as the llvm useflag is deactivated. If I activate it, however, it fails with a lot of unknown command line argument failures. I have attached the build.log of the failed emerge for further information.
Created attachment 477202 [details]
emerge --info
Created attachment 477204 [details]
environment
Created attachment 477206 [details]
emerge -pqv
I've dropped clang and llvm flags from rust overlay for now until someone will find a way to fix that Created attachment 477632 [details] rust-1.18.0.ebuild Removed IUSE=llvm, corresponds to https://github.com/devurandom/gentoo-overlay/commit/550bc08f4fefcc63be891739eac5fa6c45741259 (In reply to Cynede from comment #39) > I've dropped clang and llvm flags from rust overlay for now until someone > will find a way to fix that USE=clang works with my ebuild. Just bumped rust{,-bin}-1.19.0 and cargo-0.20.0. I had to substantially overhaul the ebuilds to deal with build system changes, please try it out and file bugs.
(In reply to Dirkjan Ochtman from comment #42) > Just bumped rust{,-bin}-1.19.0 and cargo-0.20.0. I had to substantially > overhaul the ebuilds to deal with build system changes, please try it out > and file bugs. I have a few questions regarding your ebuild, after I compared it to my proposal, discarding merely stylistic differences: * Why is the creation of config.toml a compilation step (src_compile) and not a configuration step (src_configure)? * Downloading rustc, rust-std and cargo binary tarballs separately is not saving space compared to downloading the combined rust tarball. In fact I think the three together were larger than the single tarball distribution. What is the benefit that made you select this route? * Upstream seemed to require CLang to be from a certain version range. How did you get it to build with any version of sys-devel/clang? * Same goes for GCC: How did you get it to build with GCC <4.7, which upstream appeared to require? * I also listed CMake as a build-time dependency, iirc since there were some Cargo.tomls using it. Are you sure it is not necessary? * My toml_usex bash function seemed generally useful, e.g. reducing the amount of code required to check whether to enable flags or not. Did you disregard it for a specific reason? * The $cache_dir copying might be unnecessary, if you use the install.sh included in the binary distribution tarballs to install those binaries into a local directory and specify the cargo and rustc paths in config.toml. * Did you not set the llvm.release-debuginfo flag for a specific reason? * Why do you set build.docs to false? * Why do you not set the build.{build,host,target} triplets? I included them, since they would allow to easily add additional targets and possibly cross-compilation later. (Related: There exists a target.$triplet section, which might be helpful to select the correct (cross-)compiler: CC, CXX and LLVM. * IIRC build.vendor was only effective if build.locked-deps was also set, though I might have made a mistake or the code might have changed further from 1.18 to 1.19. * Why did you not enable build.verbose, `x.py build --verbose` and RUST_BACKTRACE? These were necessary for me to debug the build process, in those cases where there were problems and they might help you as a maintainer in case bugreports are being filed for the ebuild. * AFAIK the values of rust.default-{linker,ar} are used at runtime, not at compile-time. config.toml.example says: "The default linker that will be used by the generated compiler. Note that this is not the linker used to link said compiler." * Did you not set rust.debuginfo and rust.debug-assertions for a specific reason? * There is also the possibility to set rust.use-jemalloc. Did you find problems with that? * You opted to link against LLVM statically. Is there a specific reason for that? * I found bootstrap.py to not complain when it cannot find config.toml. Hence I specified the path explicitly in the call to `x.py build` and iirc upstream added code to fail hard if the explicitly specified file cannot be found, which helped a lot in debugging. Did you not encounter such problems? * Why did you decide to not use `x.py dist --install`, but instead copy the files manually? Was there a problem with the install target, that should be fixed upstream? P.S. One more question: You disregard the SRC_URI handling for betas, but keep the beta handling logic, including the SRC variable, in the beginning of the ebuild. Why is that? Hey, thanks for the detailed comments.
First off, I didn't actually see your ebuild proposal until now. Second, upstream is moving to this rustbuild build system, and I wanted to get Gentoo on board with that rather than staying on the old way, because upstream's intention seems to be to deprecate the non-rustbuild build system anyway. Third, I did quite a bit of digging to get it to compile in a sane way, but then wanted to get it out there in the tree (since version bumps have stalled quite a bit already) without waiting to long for polishing. However, it's clear that there's a lot to improve still!
* Why is the creation of config.toml a compilation step (src_compile) and not a configuration step (src_configure)?
I had it in src_configure, but since x.py just sort of gets going and compiles, it didn't seem to make sense to me to split it up. If you disagree, I'm definitely open to having this change.
* Downloading rustc, rust-std and cargo binary tarballs separately is not saving space compared to downloading the combined rust tarball. In fact I think the three together were larger than the single tarball distribution. What is the benefit that made you select this route?
The new build system seems to always manage the 3 downloads separately. I started off working from a single tarball, but that actually seemed to require more work (and thus more integration -> more fragility) in trying to get the build system to find and use all the pieces.
* Upstream seemed to require CLang to be from a certain version range. How did you get it to build with any version of sys-devel/clang?
I have to admit I did not test building with a system clang/LLVM. Upstream also does not support building with clang, and is quite fussy about the version of clang/LLVM that they build with. I think it would be better to just deprecate building with the system clang/LLVM/libcxx.
* Same goes for GCC: How did you get it to build with GCC <4.7, which upstream appeared to require?
See above.
* I also listed CMake as a build-time dependency, iirc since there were some Cargo.tomls using it. Are you sure it is not necessary?
I'm not fully sure, no. It's probably required for building LLVM.
* My toml_usex bash function seemed generally useful, e.g. reducing the amount of code required to check whether to enable flags or not. Did you disregard it for a specific reason?
As referred to before, I just missed it. However, I think it might no longer make sense with the new build system?
* The $cache_dir copying might be unnecessary, if you use the install.sh included in the binary distribution tarballs to install those binaries into a local directory and specify the cargo and rustc paths in config.toml.
That seems workable for cargo and rustc, but I had the most trouble with rust-std. However, if there's a way to make this more simple, I'm definitely open to it!
* Did you not set the llvm.release-debuginfo flag for a specific reason?
I went over all the flags and tried to copy what we had in the older ebuilds, it's definitely possible I missed something, though.
* Why do you set build.docs to false?
Looks like an oversight, that should probably be conditional on the USE flag.
* Why do you not set the build.{build,host,target} triplets? I included them, since they would allow to easily add additional targets and possibly cross-compilation later. (Related: There exists a target.$triplet section, which might be helpful to select the correct (cross-)compiler: CC, CXX and LLVM.
I have no real way/motivation to test cross-compilation, but I'm happy if someone can test it and contribute changes to enable it!
* IIRC build.vendor was only effective if build.locked-deps was also set, though I might have made a mistake or the code might have changed further from 1.18 to 1.19.
build.vendor seemed to be effective.
* Why did you not enable build.verbose, `x.py build --verbose` and RUST_BACKTRACE? These were necessary for me to debug the build process, in those cases where there were problems and they might help you as a maintainer in case bugreports are being filed for the ebuild.
Might be better. I did enable them locally while trying to get it to work.
* AFAIK the values of rust.default-{linker,ar} are used at runtime, not at compile-time. config.toml.example says: "The default linker that will be used by the generated compiler. Note that this is not the linker used to link said compiler."
* Did you not set rust.debuginfo and rust.debug-assertions for a specific reason?
Nope.
* There is also the possibility to set rust.use-jemalloc. Did you find problems with that?
Nope.
* You opted to link against LLVM statically. Is there a specific reason for that?
Just the defaults.
* I found bootstrap.py to not complain when it cannot find config.toml. Hence I specified the path explicitly in the call to `x.py build` and iirc upstream added code to fail hard if the explicitly specified file cannot be found, which helped a lot in debugging. Did you not encounter such problems?
No, I didn't find problems like that, probably because I just wrote it to the default location anyway.
* Why did you decide to not use `x.py dist --install`, but instead copy the files manually? Was there a problem with the install target, that should be fixed upstream?
I was not aware that there is an --install target. In fact, one problem I found is that the x.py script --help function did not work for me (and looking at the source it didn't seem to be wired up to much, if anything). I would be very happy to replace my crummy hand install with something more polished.
Sorry if you spent a lot of time investigating my changes. I really just wanted to use the new upstream build system and get something working that seems robust against upstream changes. That's also a reason for me to make sure we opt for upstream's defaults in many places. On the other hand, there are clearly places where Gentoo users are used to being able to make changes which should be honored here.
I'm happy to work with you to further improve the ebuild, but for me personally the most important thing is being able to do regular version bumps when upstream releases new versions without lots of work.
(In reply to Dirkjan Ochtman from comment #45) > Second, > upstream is moving to this rustbuild build system, and I wanted to get > Gentoo on board with that rather than staying on the old way, because > upstream's intention seems to be to deprecate the non-rustbuild build system > anyway. My proposal uses the new build-system. In fact most effort in writing my ebuild was to bring it from the old build-system to the new one and in in a few cases I had to work with upstream to get the build-system to run more smoothly and fix bugs. (See e.g. the See-Also-s of this bugreport.) > Third, I did quite a bit of digging to get it to compile in a sane > way, I had the same issue. :) Hence the PRs upstream. > * Why is the creation of config.toml a compilation step (src_compile) and > not a configuration step (src_configure)? > > I had it in src_configure, but since x.py just sort of gets going and > compiles, it didn't seem to make sense to me to split it up. If you > disagree, I'm definitely open to having this change. For me the separation is a logical one, independent of how much time each step takes. But I don't know what the Gentoo consensus is on that. > * Downloading rustc, rust-std and cargo binary tarballs separately is not > saving space compared to downloading the combined rust tarball. In fact I > think the three together were larger than the single tarball distribution. > What is the benefit that made you select this route? > > The new build system seems to always manage the 3 downloads separately. I > started off working from a single tarball, but that actually seemed to > require more work (and thus more integration -> more fragility) in trying to > get the build system to find and use all the pieces. For me it was the other way around. I started with three and in the end decided: "Why bother, if there is no benefit?" After using the combined tarball and setting build.{cargo,rustc} in config.toml, it went more smoothly. Maybe you can compare my ebuild with yours to see what I did differently from your initial approaches? > * Upstream seemed to require CLang to be from a certain version range. How > did you get it to build with any version of sys-devel/clang? > > I have to admit I did not test building with a system clang/LLVM. Upstream > also does not support building with clang, and is quite fussy about the > version of clang/LLVM that they build with. I think it would be better to > just deprecate building with the system clang/LLVM/libcxx. I removed the use-flag to build with a system LLVM for that exact reason (see also comments #22, #35 and #40 here). But I kept the ability to select whether to bootstrap using GCC or CLang. The dependencies seemed to be documented better for this case, too. > * My toml_usex bash function seemed generally useful, e.g. reducing the > amount of code required to check whether to enable flags or not. Did you > disregard it for a specific reason? > > As referred to before, I just missed it. However, I think it might no longer > make sense with the new build system? Yes, this function was written for the new build-system and was not useful with the old build-system. > * The $cache_dir copying might be unnecessary, if you use the install.sh > included in the binary distribution tarballs to install those binaries into > a local directory and specify the cargo and rustc paths in config.toml. > > That seems workable for cargo and rustc, but I had the most trouble with > rust-std. However, if there's a way to make this more simple, I'm definitely > open to it! I just finished building rust-1.19.0 and the ebuild proposed for 0.18 seems to work with only minor modifications in src_install(). > * Why do you not set the build.{build,host,target} triplets? I included > them, since they would allow to easily add additional targets and possibly > cross-compilation later. (Related: There exists a target.$triplet section, > which might be helpful to select the correct (cross-)compiler: CC, CXX and > LLVM. > > I have no real way/motivation to test cross-compilation, but I'm happy if > someone can test it and contribute changes to enable it! Might be helpful to tackle bug #615030. > * AFAIK the values of rust.default-{linker,ar} are used at runtime, not at > compile-time. config.toml.example says: "The default linker that will be > used by the generated compiler. Note that this is not the linker used to > link said compiler." I wrote this, since you use tc-getBUILD_CC/AR, which would be wrong in a cross-compilation case. > * I found bootstrap.py to not complain when it cannot find config.toml. > Hence I specified the path explicitly in the call to `x.py build` and iirc > upstream added code to fail hard if the explicitly specified file cannot be > found, which helped a lot in debugging. Did you not encounter such problems? > > No, I didn't find problems like that, probably because I just wrote it to > the default location anyway. Might be helpful to catch bugs in the ebuild more quickly. > * Why did you decide to not use `x.py dist --install`, but instead copy the > files manually? Was there a problem with the install target, that should be > fixed upstream? > > I was not aware that there is an --install target. In fact, one problem I found > is that the x.py script --help function did not work for me (and looking at the > source it didn't seem to be wired up to much, if anything). I would be very > happy to replace my crummy hand install with something more polished. I just tested with 1.19: The command is now `x.py install`, not anymore `x.py dist --install`. But after changing that, it still works. > I'm happy to work with you to further improve the ebuild, but for me > personally the most important thing is being able to do regular version > bumps when upstream releases new versions without lots of work. Would be great if you could compare my ebuilds at [1] and [2] with yours and see what makes sense for you to merge into the official ebuild. I also dug quite a bit into bootstrap.py and its helpers to be able to fix upstream's build-system for our use-cases, so if there are any questions or I can be of additional help, please don't hesitate to contact me. Regarding the Cargo ebuild I have just one question: * Is there a reason for not using `cargo install --root="${ED}/usr"`? * You removed src_test(). Is there a reason not to call `cargo test`? * The Cargo docs say that Python is required. This is missing in the current ebuild. On the other hand there are several sys-apps listed that are not mentioned in the docs. Is that intentional? Also the list of differences compared to my current ebuild is tiny: * In src_configure() I pass using `:`. * I pass `--verbose --verbose` to Cargo, to ease debugging in case of trouble. [1]: https://github.com/devurandom/gentoo-overlay/tree/master/dev-lang/rust [2]: https://github.com/devurandom/gentoo-overlay/tree/master/dev-util/cargo Could you try to build Rust Language Server with build.extended = true? (In reply to Dirkjan Ochtman from comment #45) > I'm happy to work with you to further improve the ebuild, but for me > personally the most important thing is being able to do regular version > bumps when upstream releases new versions without lots of work. Dirkjan: I would like to take you up on that offer. What shall I do to get my proposed changes merged? Hey, yeah, sorry for stalling on this, I've been very busy. I looked at your ebuild and figured it's probably better on most counts than what I've come up with. So I started looking at your ebuild and hacking it a little bit, here are the changes I did so far: - Remove the clang USE flag: what purpose does this serve exactly? See also bug 591930. - Inlined all local variables in src_configure() that are single-use. This is more of a style point, prefer to not have the abstraction if not needed. - Decreased verbose = 2 to 0. For releasing to the entire user base, I don't think the verbose builds are a good default. - Moved eselect-rust dependency from PDEPEND to RDEPEND, see bug 622654. - Made the jemalloc USE flag +, to reflect upstream defaults. - I've replaced ${D} with ${ED} in a bunch of places -- I thought this was due to a bug, can't find it now though. See here: https://dirkjan.ochtman.nl/files/rust-1.19.0-r1.ebuild. However, on testing this didn't seem to work, so I got stalled. In general, we need to take all the bugs into account for dev-lang/rust, especially the ones filed against 1.19.0. Your help is much appreciated! (In reply to Dirkjan Ochtman from comment #49) > So I started looking at your ebuild and hacking it a little bit, here are > the changes I did so far: > > - Remove the clang USE flag: what purpose does this serve exactly? See also > bug 591930. I agree with the rationale of bug 591930. > - Inlined all local variables in src_configure() that are single-use. This > is more of a style point, prefer to not have the abstraction if not needed. Agreed, the fields in config.toml are verbose enough. > - Decreased verbose = 2 to 0. For releasing to the entire user base, I don't > think the verbose builds are a good default. In many other ebuilds the default will be to pass e.g. VERBOSE=1 to make, hence I did the same here. For debugging and getting good build.logs, I would prefer to have at least some verbosity enabled. The difference between level 1 and 2 was iirc, that 1 prints invocations of rust, while 2 also prints the programs that rust calls. > - Moved eselect-rust dependency from PDEPEND to RDEPEND, see bug 622654. Seems reasonable. > - Made the jemalloc USE flag +, to reflect upstream defaults. OK. > - I've replaced ${D} with ${ED} in a bunch of places -- I thought this was > due to a bug, can't find it now though. I vaguely remembered something similar, but couldn't find the official policy on which one to use. I think ED was when building for a ROOT? > See here: https://dirkjan.ochtman.nl/files/rust-1.19.0-r1.ebuild. However, > on testing this didn't seem to work, so I got stalled. Did you have a look at https://github.com/devurandom/gentoo-overlay/tree/master/dev-lang/rust, yet? It contains some more changes, e.g. does it handle rust-lldb correctly and it supports USE=system-llvm as suggested in comment #39. I also merged some changes you made in the -r1 ebuild proposed in comment #49. Some notes on the r1 ebuild: * It sets LDPATH and MANPATH to include ED, which seems wrong. * Where can I find rust-1.19.0-bootstrap-use-configured-rustc-cargo-paths.patch, or which upstream commit is this based on? I would like to rebuild with that, to try to reproduce the problems you're seeing. There also another rust overlay: https://github.com/gentoo/gentoo-rust/tree/master/dev-lang/rust (In reply to Oleg from comment #51) > There also another rust overlay: > https://github.com/gentoo/gentoo-rust/tree/master/dev-lang/rust 1.19.0-r2 looks quite similar to what I have. Will check the diff in more detail later. (In reply to Dennis Schridde from comment #50) > * Where can I find > rust-1.19.0-bootstrap-use-configured-rustc-cargo-paths.patch, or which > upstream commit is this based on? I would like to rebuild with that, to try > to reproduce the problems you're seeing. I assume this refers to https://github.com/rust-lang/rust/pull/42695 ? It was merged upstream before 1.19.0 and hence should no longer be necessary. I don't really like system-llvm supports, as it seems the LLVM versions allowed differ significantly from the version upstream pins. I don't want our builds to differ from upstream that much, as I think it would create additional differences, and thus additional maintenance burden. For non-system-LLVM builds, it would probably be very helpful for build times (and disk usage) if we can reduce the targets the rust-internal LLVM build does, as in bug 627288. With Rust 1.20 coming up this week, I will be doing a fresh round of ebuilds probably towards the end of this week. I think I'll start from what Dennis has in his overlay, if people want to suggest improvements on top of that, now would be a good time. Created attachment 493740 [details, diff] Differences between my cargo-0.20.0 and cargo-0.21.0 in Gentoo Main (In reply to Dirkjan Ochtman from comment #55) > With Rust 1.20 coming up this week, I will be doing a fresh round of ebuilds > probably towards the end of this week. I think I'll start from what Dennis > has in his overlay, if people want to suggest improvements on top of that, > now would be a good time. It seems that dev-util/cargo-0.21.0 was bumped without incorporating the changes from my overlay. I see the attached differences. Assuming the changes to CRATES being fine, I wonder why the other changes, like src_configure, verbosity and backtraces on faults, using cargo in src_install and having a proper src_test, were dropped. That's because when I mentioned your "ebuilds", I was primarily thinking of the ones for dev-lang/rust, so I hadn't taken your changes to cargo into account. I did try to copy your ebuild for rust-1.20.0, but it failed for me while building rustc_llvm. Can you try that and see if you get an error? After trying through ebuild, I ran it manually as root in the work dir and it seemed to get further into the build, so there might be something indeterministic going on. (In reply to Dirkjan Ochtman from comment #58) > I did try to copy your ebuild for rust-1.20.0, but it failed for me while > building rustc_llvm. Can you try that and see if you get an error? After > trying through ebuild, I ran it manually as root in the work dir and it > seemed to get further into the build, so there might be something > indeterministic going on. I called get_llvm_prefix from llvm.eclass in the wrong way. f79e4de9b92db0179c4a9b82ff6a6c0cbc836613 in my overlay fixes that. I also bumped dev-lang/rust-1.20.0 and dev-util/cargo-0.21.0 in my overlay. I believe bugs #591930, #622654 and #627288 to be fixed in my version of the ebuilds. Rust-1.21 is out. https://blog.rust-lang.org/2017/10/12/Rust-1.21.html i modified the ebuild. Now there is no need in dev-util/cargo as it is said at Cargo's page "Cargo is distributed by default with Rust, so if you've got rustc installed locally you probably also have cargo installed locally." So mozconfig-v6.57.eclass also should modified to remove dev-util/cargo from deps. Created attachment 506794 [details]
rust-1.22.1.ebuild
Since we have much more recent versions in the tree now, I'm going to close this. If you have still improvements for the ebuild, please report those as separate bugs with specific patches. |
