Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 617744 - dev-lang/rust-1.18.0, dev-util/cargo-0.19.0 - version bump
Summary: dev-lang/rust-1.18.0, dev-util/cargo-0.19.0 - version bump
Status: RESOLVED OBSOLETE
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal with 1 vote (vote)
Assignee: Gentoo Rust Project
URL: https://blog.rust-lang.org/2017/06/08...
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-05-07 16:19 UTC by Dennis Schridde
Modified: 2018-06-06 08:52 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
rust-1.17.0.ebuild (rust-1.17.0.ebuild,5.17 KB, text/plain)
2017-06-05 17:31 UTC, Dennis Schridde
Details
cargo-0.18.0.ebuild (cargo-0.18.0.ebuild,4.36 KB, text/plain)
2017-06-05 17:33 UTC, Dennis Schridde
Details
patch: cargo-0.18.0.ebuild (from 0.17.0) (cargo-0.18.0.ebuild.patch,4.05 KB, patch)
2017-06-05 17:35 UTC, Dennis Schridde
Details | Diff
rust-1.17.0-bootstrap-verbose.patch (rust-1.17.0-bootstrap-verbose.patch,3.71 KB, patch)
2017-06-08 07:05 UTC, Dennis Schridde
Details | Diff
rust-1.17.0-bootstrap-output-name-of-failed-config.patch (rust-1.17.0-bootstrap-output-name-of-failed-config.patch,991 bytes, patch)
2017-06-08 07:05 UTC, Dennis Schridde
Details | Diff
rust-1.17.0.ebuild (rust-1.17.0.ebuild,5.16 KB, text/plain)
2017-06-17 09:08 UTC, Dennis Schridde
Details
rust-1.18.0-bootstrap-use-configured-rustc-cargo-paths.patch (rust-1.18.0-bootstrap-use-configured-rustc-cargo-paths.patch,1.77 KB, patch)
2017-06-17 12:53 UTC, Dennis Schridde
Details | Diff
rust-1.18.0.ebuild (rust-1.18.0.ebuild,5.12 KB, text/plain)
2017-06-17 15:32 UTC, Dennis Schridde
Details
cargo-0.19.0.ebuild (cargo-0.19.0.ebuild,4.36 KB, text/plain)
2017-06-17 15:32 UTC, Dennis Schridde
Details
patch: cargo-0.19.0.ebuild (from 0.18.0) (file_617744.txt,9.02 KB, patch)
2017-06-17 15:34 UTC, Dennis Schridde
Details | Diff
ebuild with which I had rust built (rust-1.18.0-r1.ebuild,5.09 KB, text/plain)
2017-06-18 21:11 UTC, Perfect Gentleman
Details
build.log (build.log,438.82 KB, text/x-log)
2017-06-19 10:05 UTC, Christian Widmer
Details
emerge --info (emerge-info.txt,6.59 KB, text/plain)
2017-06-19 10:07 UTC, Christian Widmer
Details
environment (environment,125.41 KB, text/plain)
2017-06-19 10:07 UTC, Christian Widmer
Details
emerge -pqv (emerge-pqv.txt,220 bytes, text/plain)
2017-06-19 10:11 UTC, Christian Widmer
Details
rust-1.18.0.ebuild (rust-1.18.0.ebuild,4.85 KB, text/plain)
2017-06-22 17:31 UTC, Dennis Schridde
Details
Differences between my cargo-0.20.0 and cargo-0.21.0 in Gentoo Main (cargo-0.20.0-vs-0.21.0.patch,5.43 KB, patch)
2017-09-10 15:07 UTC, Dennis Schridde
Details | Diff
rust-1.22.1.ebuild (rust-1.22.1.ebuild,4.48 KB, text/plain)
2017-11-27 05:38 UTC, Perfect Gentleman
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Dennis Schridde 2017-05-07 16:19:10 UTC
Rust 1.17 is out: https://blog.rust-lang.org/2017/04/27/Rust-1.17.html
Comment 1 Dennis Schridde 2017-06-05 17:31:17 UTC
Created attachment 475276 [details]
rust-1.17.0.ebuild
Comment 2 Dennis Schridde 2017-06-05 17:33:51 UTC
Created attachment 475278 [details]
cargo-0.18.0.ebuild
Comment 3 Dennis Schridde 2017-06-05 17:35:24 UTC
Created attachment 475280 [details, diff]
patch: cargo-0.18.0.ebuild (from 0.17.0)
Comment 4 Dennis Schridde 2017-06-05 17:36:53 UTC
The ebuild for Rust is more or less a complete rewrite, since I switched to the now recommended `x.py` build-system and `config.toml`. The ebuild for Cargo is not that different from the previous version, hence I provided a patch.
Comment 5 Jory A. Pratt gentoo-dev 2017-06-06 14:29:09 UTC
(In reply to Dennis Schridde from comment #2)
> Created attachment 475278 [details]
> cargo-0.18.0.ebuild

This is great work, would be even better tho if it actually supported building rustc for musl as the work has landed upstream in this release.
Comment 6 Cynede gentoo-dev 2017-06-07 07:51:25 UTC
I can't see patches used in 1.17 ebuild
Comment 7 Cynede gentoo-dev 2017-06-07 08:11:55 UTC
found patches in overlay, there is output with clang and llvm flags on: https://gist.github.com/mpkh/1119ec439bfc6efd8a133c0834fe1f2d
Comment 8 Cynede gentoo-dev 2017-06-07 08:36:56 UTC
cargo fault: https://gist.github.com/mpkh/ca1d307012d9a917e6c78fe53eec0318
Comment 9 Dennis Schridde 2017-06-08 07:05:09 UTC
Created attachment 475588 [details, diff]
rust-1.17.0-bootstrap-verbose.patch
Comment 10 Dennis Schridde 2017-06-08 07:05:39 UTC
Created attachment 475590 [details, diff]
rust-1.17.0-bootstrap-output-name-of-failed-config.patch
Comment 11 Dennis Schridde 2017-06-08 07:12:09 UTC
(In reply to Cynede from comment #8)
> cargo fault: https://gist.github.com/mpkh/ca1d307012d9a917e6c78fe53eec0318

This looks very similar to the problems I was facing with Rust initially, until I used the "rust" (instead of the "rustc") tarball for building, which also includes the contents of the "rust-std" tarball -- i.e. the standard library.  Why this also hits Cargo now I am not sure -- Rust's debug output is very scarce [1] and even RUST_LOG=info does give anything useful.  My suspicion is, that the libraries being installed to /usr/lib64/rust-1.17.0 might confuse the loader, but I still have to investigate.

[1]: The report_error function in librustc_metadata/locator outputs only the name of the missing crate, but not any directories or library names it tried: https://github.com/rust-lang/rust/blob/1.17.0/src/librustc_metadata/locator.rs#L324
Comment 12 Oleg 2017-06-08 17:32:18 UTC
It's already 1.18 out: https://blog.rust-lang.org/2017/06/08/Rust-1.18.html
Comment 13 Michael Lawrence 2017-06-09 01:36:59 UTC
https://www.rust-lang.org/en-US/other-installers.html#standalone
ie rust-bin  if someone bumps rust-bin.... a number of prebuilt... 

https://static.rust-lang.org/rustup/dist/aarch64-unknown-linux-gnu/rustup-init

Beta (1.19) Nightly (1.20)
Comment 14 Dennis Schridde 2017-06-17 09:08:07 UTC
Created attachment 476730 [details]
rust-1.17.0.ebuild

Fix comment #8 (std not being found). Cargo 0.18.0 as attached compiles now.
Comment 15 Dennis Schridde 2017-06-17 09:16:21 UTC
(In reply to Jory A. Pratt from comment #5)
> (In reply to Dennis Schridde from comment #2)
> > Created attachment 475278 [details]
> > cargo-0.18.0.ebuild
> 
> This is great work, would be even better tho if it actually supported
> building rustc for musl as the work has landed upstream in this release.

That would be bug #615030. I'd handle it separately.
Comment 16 Dennis Schridde 2017-06-17 12:53:02 UTC
Created attachment 476742 [details, diff]
rust-1.18.0-bootstrap-use-configured-rustc-cargo-paths.patch

Rust 1.18.0 needs https://github.com/rust-lang/rust/pull/42695 to bootstrap using local cargo / rustc.
Comment 17 Dennis Schridde 2017-06-17 15:32:08 UTC
Created attachment 476744 [details]
rust-1.18.0.ebuild

Rust 1.18.0 version bump.

rust-1.17.0-bootstrap-output-name-of-failed-config.patch and rust-1.17.0-bootstrap-verbose.patch help with debugging the build, but are not strictly necessary, hence they are not anymore used in this version.
Comment 18 Dennis Schridde 2017-06-17 15:32:52 UTC
Created attachment 476746 [details]
cargo-0.19.0.ebuild

Cargo 0.19.0 version bump
Comment 19 Dennis Schridde 2017-06-17 15:34:10 UTC
Created attachment 476748 [details, diff]
patch: cargo-0.19.0.ebuild (from 0.18.0)
Comment 21 Dennis Schridde 2017-06-17 17:16:58 UTC
My installed Rust is able to build Firefox 54, hence I consider it to be actually working and tested.
Comment 22 Perfect Gentleman 2017-06-18 14:39:08 UTC
rust cannot be built with USE="-llvm"
Comment 23 Dennis Schridde 2017-06-18 20:42:45 UTC
(In reply to Perfect Gentleman from comment #22)
> rust cannot be built with USE="-llvm"

Could you please give me a build.log? (I assume that the issue involves llvm-config -- is that correct?)
Comment 24 Perfect Gentleman 2017-06-18 21:11:05 UTC
(In reply to Dennis Schridde from comment #23)
> (In reply to Perfect Gentleman from comment #22)
> > rust cannot be built with USE="-llvm"
> 
> Could you please give me a build.log? (I assume that the issue involves
> llvm-config -- is that correct?)

I can give ebuild with which I had rust built.
Comment 25 Perfect Gentleman 2017-06-18 21:11:38 UTC
Created attachment 477190 [details]
ebuild with which I had rust built
Comment 26 Cynede gentoo-dev 2017-06-19 07:20:27 UTC
I've got working version in rust overlay currently, please test it out, fixes are welcome via pull requests https://github.com/gentoo/gentoo-rust
Comment 27 Perfect Gentleman 2017-06-19 07:33:28 UTC
(In reply to Cynede from comment #26)
> I've got working version in rust overlay currently, please test it out,
> fixes are welcome via pull requests https://github.com/gentoo/gentoo-rust

I've tested that ebuilds too. They don't work.
Comment 28 Cynede gentoo-dev 2017-06-19 08:07:33 UTC
(In reply to Perfect Gentleman from comment #27)
> (In reply to Cynede from comment #26)
> > I've got working version in rust overlay currently, please test it out,
> > fixes are welcome via pull requests https://github.com/gentoo/gentoo-rust
> 
> I've tested that ebuilds too. They don't work.

they do for me... maybe some use flags don't, alike clang can cause problems potentially

home λ rustc --version
rustc 1.18.0-dev
home λ cargo --version
cargo 0.19.0
home λ alacritty --version
alacritty 0.1.0
Comment 29 Perfect Gentleman 2017-06-19 08:41:40 UTC
> they do for me... maybe some use flags don't, alike clang can cause problems
> potentially

Don't for me
------------
 * rustc-1.18.0-src.tar.gz SHA256 SHA512 WHIRLPOOL size ;-) ...                                                                                                                                                                       [ ok ]
 * rust-1.17.0-x86_64-unknown-linux-gnu.tar.gz SHA256 SHA512 WHIRLPOOL size ;-) ...                                                                                                                                                   [ ok ]
 * checking ebuild checksums ;-) ...                                                                                                                                                                                                  [ ok ]
 * checking auxfile checksums ;-) ...                                                                                                                                                                                                 [ ok ]
 * checking miscfile checksums ;-) ...                                                                                                                                                                                                [ ok ]
 * ERROR: dev-lang/rust-1.18.0::pg_overlay failed (setup phase):
 *   No LLVM slot <= 4 found in PATH!
 * 
 * Call stack:
 *            ebuild.sh, line 115:  Called pkg_setup
 *   rust-1.18.0.ebuild, line  74:  Called llvm_pkg_setup
 *          llvm.eclass, line 133:  Called get_llvm_prefix '4'
 *          llvm.eclass, line 114:  Called die
 * The specific snippet of code:
 *      die "No LLVM slot${1:+ <= ${1}} found in PATH!"
 * 
 * If you need support, post the output of `emerge --info '=dev-lang/rust-1.18.0::pg_overlay'`,
 * the complete build log and the output of `emerge -pqv '=dev-lang/rust-1.18.0::pg_overlay'`.
 * The complete build log is located at '/tmp/portage/dev-lang/rust-1.18.0/temp/build.log'.
 * The ebuild environment file is located at '/tmp/portage/dev-lang/rust-1.18.0/temp/environment'.
 * Working directory: '/tmp/portage/dev-lang/rust-1.18.0/homedir'
 * S: '/tmp/portage/dev-lang/rust-1.18.0/work/rustc-1.18.0-src'

---------------------------------------------------------------
Comment 30 Perfect Gentleman 2017-06-19 08:42:57 UTC
and cargo
------------
 * cargo-0.19.0.tar.gz SHA256 SHA512 WHIRLPOOL size ;-) ...                                                                                                                                                                           [ ok ]
 * cargo-0.19.0-x86_64-unknown-linux-gnu.tar.gz SHA256 SHA512 WHIRLPOOL size ;-) ...                                                                                                                                                  [ ok ]
 * checking ebuild checksums ;-) ...                                                                                                                                                                                                  [ ok ]
 * checking miscfile checksums ;-) ...                                                                                                                                                                                                [ ok ]
>>> Unpacking source...
>>> Unpacking cargo-0.19.0.tar.gz to /tmp/portage/dev-util/cargo-0.19.0-r1/work
>>> Unpacking cargo-0.19.0-x86_64-unknown-linux-gnu.tar.gz to /tmp/portage/dev-util/cargo-0.19.0-r1/work
>>> Source unpacked in /tmp/portage/dev-util/cargo-0.19.0-r1/work
>>> Preparing source in /tmp/portage/dev-util/cargo-0.19.0-r1/work/cargo-0.19.0 ...
>>> Source prepared.
>>> Configuring source in /tmp/portage/dev-util/cargo-0.19.0-r1/work/cargo-0.19.0 ...
./configure --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --target=x86_64-unknown-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --docdir=/usr/share/doc/cargo-0.19.0-r1 --libdir=/usr/lib64 --prefix=/usr --host=x86_64-unknown-linux-gnu --build=x86_64-unknown-linux-gnu --target=x86_64-unknown-linux-gnu --cargo=/tmp/portage/dev-util/cargo-0.19.0-r1/work/cargo-0.19.0-x86_64-unknown-linux-gnu/cargo/bin/cargo --enable-optimize --release-channel=stable --disable-verify-install --disable-debug --disable-cross-tests
configure: looking for configure programs
configure: found cmp
configure: found mkdir
configure: found printf
configure: found cut
configure: found head
configure: found grep
configure: found xargs
configure: found cp
configure: found find
configure: found uname
configure: found date
configure: found tr
configure: found sed
configure: found cmake
configure: found make
configure: recreating config.tmp
configure: 
configure: processing ./configure args
configure: 
configure: CFG_DISABLE_DEBUG    := 1 
configure: CFG_DISABLE_VERIFY_INSTALL := 1 
configure: CFG_DISABLE_CROSS_TESTS := 1 
configure: CFG_PREFIX           := /usr 
configure: CFG_LOCAL_RUST_ROOT  :=  
configure: CFG_CARGO            := /tmp/portage/dev-util/cargo-0.19.0- ...
configure: CFG_RUSTC            := rustc 
configure: CFG_RUSTDOC          := rustdoc 
configure: CFG_CARGO            := /tmp/portage/dev-util/cargo-0.19.0- ...
configure: CFG_RUSTC            := /usr/bin/rustc (1.18.0-dev)
configure: CFG_RUSTDOC          := /usr/bin/rustdoc (1.18.0-dev)
configure: CFG_BUILD            := x86_64-unknown-linux-gnu 
configure: CFG_HOST             := x86_64-unknown-linux-gnu 
configure: CFG_TARGET           := x86_64-unknown-linux-gnu 
configure: CFG_LOCALSTATEDIR    := /var/lib 
configure: CFG_SYSCONFDIR       := /etc 
configure: CFG_DATADIR          := /usr/share 
configure: CFG_INFODIR          := /usr/share/info 
configure: CFG_DOCDIR           := /usr/share/doc/cargo-0.19.0-r1 
configure: CFG_MANDIR           := /usr/share/man 
configure: CFG_LIBDIR           := /usr/lib64 
configure: git: no git directory. Changing default release channel to stable
configure: CFG_RELEASE_CHANNEL  := stable 
configure: 
configure: validating ./configure args
configure: 
configure: 
configure: looking for build programs
configure: 
configure: CFG_CC               := /usr/bin/cc (6.3.0)
configure: GIT                  := /usr/bin/git (2.13.1)
configure: 
configure: writing configuration
configure: 
configure: CFG_SRC_DIR          := /tmp/portage/dev-util/cargo-0.19.0- ...
configure: CFG_BUILD_DIR        := /tmp/portage/dev-util/cargo-0.19.0- ...
configure: CFG_CONFIGURE_ARGS   := --prefix=/usr --build=x86_64-pc-lin ...
configure: CFG_PREFIX           := /usr 
configure: CFG_BUILD            := x86_64-unknown-linux-gnu 
configure: CFG_HOST             := x86_64-unknown-linux-gnu 
configure: CFG_TARGET           := x86_64-unknown-linux-gnu 
configure: CFG_DATADIR          := /share 
configure: CFG_DOCDIR           := /share/doc/cargo-0.19.0-r1 
configure: CFG_INFODIR          := /share/info 
configure: CFG_MANDIR           := /share/man 
configure: CFG_LIBDIR           := /lib64 
configure: CFG_RUSTC            := /usr/bin/rustc 
configure: CFG_RUSTDOC          := /usr/bin/rustdoc 
configure: CFG_CARGO            := /tmp/portage/dev-util/cargo-0.19.0- ...
configure: CFG_GIT              :=  
configure: 
configure: cp /tmp/portage/dev-util/cargo-0.19.0-r1/work/cargo-0.19.0/Makefile.in ./Makefile
configure: mv config.tmp config.mk
configure: 
configure: complete
configure: 
configure: 
>>> Source configured.
>>> Compiling source in /tmp/portage/dev-util/cargo-0.19.0-r1/work/cargo-0.19.0 ...
make -j9 VERBOSE=1 PKG_CONFIG_PATH= 
/usr/bin/rustc -V
rustc 1.18.0-dev
/tmp/portage/dev-util/cargo-0.19.0-r1/work/cargo-0.19.0-x86_64-unknown-linux-gnu/cargo/bin/cargo --version
cargo 0.19.0 (28d1d60d4 2017-05-16)
/tmp/portage/dev-util/cargo-0.19.0-r1/work/cargo-0.19.0-x86_64-unknown-linux-gnu/cargo/bin/cargo build --target x86_64-unknown-linux-gnu --manifest-path /tmp/portage/dev-util/cargo-0.19.0-r1/work/cargo-0.19.0//Cargo.toml --release  --verbose 
error: no matching package named `shell-escape` found (required by `cargo`)
location searched: registry https://github.com/rust-lang/crates.io-index
version required: = 0.1.3
make: *** [Makefile:130: cargo-x86_64-unknown-linux-gnu] Error 101
 * ERROR: dev-util/cargo-0.19.0-r1::pg_overlay failed (compile phase):
 *   emake failed
 * 
 * If you need support, post the output of `emerge --info '=dev-util/cargo-0.19.0-r1::pg_overlay'`,
 * the complete build log and the output of `emerge -pqv '=dev-util/cargo-0.19.0-r1::pg_overlay'`.
 * The complete build log is located at '/tmp/portage/dev-util/cargo-0.19.0-r1/temp/build.log'.
 * The ebuild environment file is located at '/tmp/portage/dev-util/cargo-0.19.0-r1/temp/environment'.
 * Working directory: '/tmp/portage/dev-util/cargo-0.19.0-r1/work/cargo-0.19.0'
 * S: '/tmp/portage/dev-util/cargo-0.19.0-r1/work/cargo-0.19.0'
---------------------------------------------------------------
Comment 31 Perfect Gentleman 2017-06-19 08:45:50 UTC
$ equery uses rust
[ Legend : U - final flag setting for installation]
[        : I - package is installed with flag     ]
[ Colors : set, unset                             ]
 * Found these USE flags for dev-lang/rust-1.18.0:
 U I
 - - clang    : Use sys-devel/clang for building
 - - debug    : Enable extra debug codepaths, like asserts and extra output. If you want to get meaningful backtraces see https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Backtraces
 - - doc      : Add extra documentation (API, Javadoc, etc). It is recommended to enable per package instead of globally
 + + jemalloc : Use dev-libs/jemalloc for allocations
 - - llvm     : Build with sys-devel/llvm
Comment 32 Cynede gentoo-dev 2017-06-19 09:13:04 UTC
I was talking about rust overlay, not pg_overlay
Comment 33 Perfect Gentleman 2017-06-19 09:43:25 UTC
(In reply to Cynede from comment #32)
> I was talking about rust overlay, not pg_overlay

the same, I meant ebuilds from rust overlay
Comment 34 Perfect Gentleman 2017-06-19 09:45:31 UTC
(In reply to Cynede from comment #32)
> I was talking about rust overlay, not pg_overlay

but I wasn't using eclass from that overlay
Comment 35 Christian Widmer 2017-06-19 10:05:34 UTC
Created attachment 477200 [details]
build.log

For me the ebuild compiles fine as long as the llvm useflag is deactivated. If I activate it, however, it fails with a lot of unknown command line argument failures. I have attached the build.log of the failed emerge for further information.
Comment 36 Christian Widmer 2017-06-19 10:07:06 UTC
Created attachment 477202 [details]
emerge --info
Comment 37 Christian Widmer 2017-06-19 10:07:57 UTC
Created attachment 477204 [details]
environment
Comment 38 Christian Widmer 2017-06-19 10:11:15 UTC
Created attachment 477206 [details]
emerge -pqv
Comment 39 Cynede gentoo-dev 2017-06-19 10:27:25 UTC
I've dropped clang and llvm flags from rust overlay for now until someone will find a way to fix that
Comment 40 Dennis Schridde 2017-06-22 17:31:27 UTC
Created attachment 477632 [details]
rust-1.18.0.ebuild

Removed IUSE=llvm, corresponds to https://github.com/devurandom/gentoo-overlay/commit/550bc08f4fefcc63be891739eac5fa6c45741259
Comment 41 Dennis Schridde 2017-06-22 19:06:24 UTC
(In reply to Cynede from comment #39)
> I've dropped clang and llvm flags from rust overlay for now until someone
> will find a way to fix that

USE=clang works with my ebuild.
Comment 42 Dirkjan Ochtman gentoo-dev 2017-07-24 13:38:08 UTC
Just bumped rust{,-bin}-1.19.0 and cargo-0.20.0. I had to substantially overhaul the ebuilds to deal with build system changes, please try it out and file bugs.
Comment 43 Dennis Schridde 2017-07-29 10:53:18 UTC
(In reply to Dirkjan Ochtman from comment #42)
> Just bumped rust{,-bin}-1.19.0 and cargo-0.20.0. I had to substantially
> overhaul the ebuilds to deal with build system changes, please try it out
> and file bugs.

I have a few questions regarding your ebuild, after I compared it to my proposal, discarding merely stylistic differences:
* Why is the creation of config.toml a compilation step (src_compile) and not a configuration step (src_configure)?
* Downloading rustc, rust-std and cargo binary tarballs separately is not saving space compared to downloading the combined rust tarball. In fact I think the three together were larger than the single tarball distribution. What is the benefit that made you select this route?
* Upstream seemed to require CLang to be from a certain version range. How did you get it to build with any version of sys-devel/clang?
* Same goes for GCC: How did you get it to build with GCC <4.7, which upstream appeared to require?
* I also listed CMake as a build-time dependency, iirc since there were some Cargo.tomls using it. Are you sure it is not necessary?
* My toml_usex bash function seemed generally useful, e.g. reducing the amount of code required to check whether to enable flags or not. Did you disregard it for a specific reason?
* The $cache_dir copying might be unnecessary, if you use the install.sh included in the binary distribution tarballs to install those binaries into a local directory and specify the cargo and rustc paths in config.toml.
* Did you not set the llvm.release-debuginfo flag for a specific reason?
* Why do you set build.docs to false?
* Why do you not set the build.{build,host,target} triplets? I included them, since they would allow to easily add additional targets and possibly cross-compilation later. (Related: There exists a target.$triplet section, which might be helpful to select the correct (cross-)compiler: CC, CXX and LLVM.
* IIRC build.vendor was only effective if build.locked-deps was also set, though I might have made a mistake or the code might have changed further from 1.18 to 1.19.
* Why did you not enable build.verbose, `x.py build --verbose` and RUST_BACKTRACE? These were necessary for me to debug the build process, in those cases where there were problems and they might help you as a maintainer in case bugreports are being filed for the ebuild.
* AFAIK the values of rust.default-{linker,ar} are used at runtime, not at compile-time. config.toml.example says: "The default linker that will be used by the generated compiler. Note that this is not the linker used to link said compiler."
* Did you not set rust.debuginfo and rust.debug-assertions for a specific reason?
* There is also the possibility to set rust.use-jemalloc. Did you find problems with that?
* You opted to link against LLVM statically. Is there a specific reason for that?
* I found bootstrap.py to not complain when it cannot find config.toml. Hence I specified the path explicitly in the call to `x.py build` and iirc upstream added code to fail hard if the explicitly specified file cannot be found, which helped a lot in debugging. Did you not encounter such problems?
* Why did you decide to not use `x.py dist --install`, but instead copy the files manually? Was there a problem with the install target, that should be fixed upstream?
Comment 44 Dennis Schridde 2017-07-29 11:01:02 UTC
P.S. One more question: You disregard the SRC_URI handling for betas, but keep the beta handling logic, including the SRC variable, in the beginning of the ebuild. Why is that?
Comment 45 Dirkjan Ochtman gentoo-dev 2017-07-29 11:13:20 UTC
Hey, thanks for the detailed comments.

First off, I didn't actually see your ebuild proposal until now. Second, upstream is moving to this rustbuild build system, and I wanted to get Gentoo on board with that rather than staying on the old way, because upstream's intention seems to be to deprecate the non-rustbuild build system anyway. Third, I did quite a bit of digging to get it to compile in a sane way, but then wanted to get it out there in the tree (since version bumps have stalled quite a bit already) without waiting to long for polishing. However, it's clear that there's a lot to improve still!

* Why is the creation of config.toml a compilation step (src_compile) and not a configuration step (src_configure)?

I had it in src_configure, but since x.py just sort of gets going and compiles, it didn't seem to make sense to me to split it up. If you disagree, I'm definitely open to having this change.

* Downloading rustc, rust-std and cargo binary tarballs separately is not saving space compared to downloading the combined rust tarball. In fact I think the three together were larger than the single tarball distribution. What is the benefit that made you select this route?

The new build system seems to always manage the 3 downloads separately. I started off working from a single tarball, but that actually seemed to require more work (and thus more integration -> more fragility) in trying to get the build system to find and use all the pieces.

* Upstream seemed to require CLang to be from a certain version range. How did you get it to build with any version of sys-devel/clang?

I have to admit I did not test building with a system clang/LLVM. Upstream also does not support building with clang, and is quite fussy about the version of clang/LLVM that they build with. I think it would be better to just deprecate building with the system clang/LLVM/libcxx.

* Same goes for GCC: How did you get it to build with GCC <4.7, which upstream appeared to require?

See above.

* I also listed CMake as a build-time dependency, iirc since there were some Cargo.tomls using it. Are you sure it is not necessary?

I'm not fully sure, no. It's probably required for building LLVM.

* My toml_usex bash function seemed generally useful, e.g. reducing the amount of code required to check whether to enable flags or not. Did you disregard it for a specific reason?

As referred to before, I just missed it. However, I think it might no longer make sense with the new build system?

* The $cache_dir copying might be unnecessary, if you use the install.sh included in the binary distribution tarballs to install those binaries into a local directory and specify the cargo and rustc paths in config.toml.

That seems workable for cargo and rustc, but I had the most trouble with rust-std. However, if there's a way to make this more simple, I'm definitely open to it!

* Did you not set the llvm.release-debuginfo flag for a specific reason?

I went over all the flags and tried to copy what we had in the older ebuilds, it's definitely possible I missed something, though.

* Why do you set build.docs to false?

Looks like an oversight, that should probably be conditional on the USE flag.

* Why do you not set the build.{build,host,target} triplets? I included them, since they would allow to easily add additional targets and possibly cross-compilation later. (Related: There exists a target.$triplet section, which might be helpful to select the correct (cross-)compiler: CC, CXX and LLVM.

I have no real way/motivation to test cross-compilation, but I'm happy if someone can test it and contribute changes to enable it!

* IIRC build.vendor was only effective if build.locked-deps was also set, though I might have made a mistake or the code might have changed further from 1.18 to 1.19.

build.vendor seemed to be effective.

* Why did you not enable build.verbose, `x.py build --verbose` and RUST_BACKTRACE? These were necessary for me to debug the build process, in those cases where there were problems and they might help you as a maintainer in case bugreports are being filed for the ebuild.

Might be better. I did enable them locally while trying to get it to work.

* AFAIK the values of rust.default-{linker,ar} are used at runtime, not at compile-time. config.toml.example says: "The default linker that will be used by the generated compiler. Note that this is not the linker used to link said compiler."
* Did you not set rust.debuginfo and rust.debug-assertions for a specific reason?

Nope.

* There is also the possibility to set rust.use-jemalloc. Did you find problems with that?

Nope.

* You opted to link against LLVM statically. Is there a specific reason for that?

Just the defaults.

* I found bootstrap.py to not complain when it cannot find config.toml. Hence I specified the path explicitly in the call to `x.py build` and iirc upstream added code to fail hard if the explicitly specified file cannot be found, which helped a lot in debugging. Did you not encounter such problems?

No, I didn't find problems like that, probably because I just wrote it to the default location anyway.

* Why did you decide to not use `x.py dist --install`, but instead copy the files manually? Was there a problem with the install target, that should be fixed upstream?

I was not aware that there is an --install target. In fact, one problem I found is that the x.py script --help function did not work for me (and looking at the source it didn't seem to be wired up to much, if anything). I would be very happy to replace my crummy hand install with something more polished.

Sorry if you spent a lot of time investigating my changes. I really just wanted to use the new upstream build system and get something working that seems robust against upstream changes. That's also a reason for me to make sure we opt for upstream's defaults in many places. On the other hand, there are clearly places where Gentoo users are used to being able to make changes which should be honored here.

I'm happy to work with you to further improve the ebuild, but for me personally the most important thing is being able to do regular version bumps when upstream releases new versions without lots of work.
Comment 46 Dennis Schridde 2017-07-29 15:41:49 UTC
(In reply to Dirkjan Ochtman from comment #45)
> Second,
> upstream is moving to this rustbuild build system, and I wanted to get
> Gentoo on board with that rather than staying on the old way, because
> upstream's intention seems to be to deprecate the non-rustbuild build system
> anyway.

My proposal uses the new build-system. In fact most effort in writing my ebuild was to bring it from the old build-system to the new one and in in a few cases I had to work with upstream to get the build-system to run more smoothly and fix bugs. (See e.g. the See-Also-s of this bugreport.)

> Third, I did quite a bit of digging to get it to compile in a sane
> way,

I had the same issue. :) Hence the PRs upstream.

> * Why is the creation of config.toml a compilation step (src_compile) and
> not a configuration step (src_configure)?
> 
> I had it in src_configure, but since x.py just sort of gets going and
> compiles, it didn't seem to make sense to me to split it up. If you
> disagree, I'm definitely open to having this change.

For me the separation is a logical one, independent of how much time each step takes. But I don't know what the Gentoo consensus is on that.

> * Downloading rustc, rust-std and cargo binary tarballs separately is not
> saving space compared to downloading the combined rust tarball. In fact I
> think the three together were larger than the single tarball distribution.
> What is the benefit that made you select this route?
> 
> The new build system seems to always manage the 3 downloads separately. I
> started off working from a single tarball, but that actually seemed to
> require more work (and thus more integration -> more fragility) in trying to
> get the build system to find and use all the pieces.

For me it was the other way around. I started with three and in the end decided: "Why bother, if there is no benefit?" After using the combined tarball and setting build.{cargo,rustc} in config.toml, it went more smoothly. Maybe you can compare my ebuild with yours to see what I did differently from your initial approaches?

> * Upstream seemed to require CLang to be from a certain version range. How
> did you get it to build with any version of sys-devel/clang?
> 
> I have to admit I did not test building with a system clang/LLVM. Upstream
> also does not support building with clang, and is quite fussy about the
> version of clang/LLVM that they build with. I think it would be better to
> just deprecate building with the system clang/LLVM/libcxx.

I removed the use-flag to build with a system LLVM for that exact reason (see also comments #22, #35 and #40 here). But I kept the ability to select whether to bootstrap using GCC or CLang. The dependencies seemed to be documented better for this case, too.

> * My toml_usex bash function seemed generally useful, e.g. reducing the
> amount of code required to check whether to enable flags or not. Did you
> disregard it for a specific reason?
> 
> As referred to before, I just missed it. However, I think it might no longer
> make sense with the new build system?

Yes, this function was written for the new build-system and was not useful with the old build-system.

> * The $cache_dir copying might be unnecessary, if you use the install.sh
> included in the binary distribution tarballs to install those binaries into
> a local directory and specify the cargo and rustc paths in config.toml.
> 
> That seems workable for cargo and rustc, but I had the most trouble with
> rust-std. However, if there's a way to make this more simple, I'm definitely
> open to it!

I just finished building rust-1.19.0 and the ebuild proposed for 0.18 seems to work with only minor modifications in src_install().

> * Why do you not set the build.{build,host,target} triplets? I included
> them, since they would allow to easily add additional targets and possibly
> cross-compilation later. (Related: There exists a target.$triplet section,
> which might be helpful to select the correct (cross-)compiler: CC, CXX and
> LLVM.
> 
> I have no real way/motivation to test cross-compilation, but I'm happy if
> someone can test it and contribute changes to enable it!

Might be helpful to tackle bug #615030.

> * AFAIK the values of rust.default-{linker,ar} are used at runtime, not at
> compile-time. config.toml.example says: "The default linker that will be
> used by the generated compiler. Note that this is not the linker used to
> link said compiler."

I wrote this, since you use tc-getBUILD_CC/AR, which would be wrong in a cross-compilation case.

> * I found bootstrap.py to not complain when it cannot find config.toml.
> Hence I specified the path explicitly in the call to `x.py build` and iirc
> upstream added code to fail hard if the explicitly specified file cannot be
> found, which helped a lot in debugging. Did you not encounter such problems?
> 
> No, I didn't find problems like that, probably because I just wrote it to
> the default location anyway.

Might be helpful to catch bugs in the ebuild more quickly.

> * Why did you decide to not use `x.py dist --install`, but instead copy the
> files manually? Was there a problem with the install target, that should be
> fixed upstream?
>
> I was not aware that there is an --install target. In fact, one problem I found
> is that the x.py script --help function did not work for me (and looking at the
> source it didn't seem to be wired up to much, if anything). I would be very
> happy to replace my crummy hand install with something more polished.

I just tested with 1.19: The command is now `x.py install`, not anymore `x.py dist --install`. But after changing that, it still works.

> I'm happy to work with you to further improve the ebuild, but for me
> personally the most important thing is being able to do regular version
> bumps when upstream releases new versions without lots of work.

Would be great if you could compare my ebuilds at [1] and [2] with yours and see what makes sense for you to merge into the official ebuild. I also dug quite a bit into bootstrap.py and its helpers to be able to fix upstream's build-system for our use-cases, so if there are any questions or I can be of additional help, please don't hesitate to contact me.

Regarding the Cargo ebuild I have just one question:
* Is there a reason for not using `cargo install --root="${ED}/usr"`?
* You removed src_test(). Is there a reason not to call `cargo test`?
* The Cargo docs say that Python is required. This is missing in the current ebuild. On the other hand there are several sys-apps listed that are not mentioned in the docs. Is that intentional?

Also the list of differences compared to my current ebuild is tiny:
* In src_configure() I pass using `:`.
* I pass `--verbose --verbose` to Cargo, to ease debugging in case of trouble.

[1]: https://github.com/devurandom/gentoo-overlay/tree/master/dev-lang/rust
[2]: https://github.com/devurandom/gentoo-overlay/tree/master/dev-util/cargo
Comment 47 Oleg 2017-07-30 06:25:25 UTC
Could you try to build Rust Language Server with build.extended = true?
Comment 48 Dennis Schridde 2017-08-19 15:33:42 UTC
(In reply to Dirkjan Ochtman from comment #45)
> I'm happy to work with you to further improve the ebuild, but for me
> personally the most important thing is being able to do regular version
> bumps when upstream releases new versions without lots of work.

Dirkjan: I would like to take you up on that offer.  What shall I do to get my proposed changes merged?
Comment 49 Dirkjan Ochtman gentoo-dev 2017-08-21 08:57:15 UTC
Hey, yeah, sorry for stalling on this, I've been very busy. I looked at your ebuild and figured it's probably better on most counts than what I've come up with.

So I started looking at your ebuild and hacking it a little bit, here are the changes I did so far:

- Remove the clang USE flag: what purpose does this serve exactly? See also bug 591930.

- Inlined all local variables in src_configure() that are single-use. This is more of a style point, prefer to not have the abstraction if not needed.

- Decreased verbose = 2 to 0. For releasing to the entire user base, I don't think the verbose builds are a good default.

- Moved eselect-rust dependency from PDEPEND to RDEPEND, see bug 622654.

- Made the jemalloc USE flag +, to reflect upstream defaults.

- I've replaced ${D} with ${ED} in a bunch of places -- I thought this was due to a bug, can't find it now though.

See here: https://dirkjan.ochtman.nl/files/rust-1.19.0-r1.ebuild. However, on testing this didn't seem to work, so I got stalled.

In general, we need to take all the bugs into account for dev-lang/rust, especially the ones filed against 1.19.0.

Your help is much appreciated!
Comment 50 Dennis Schridde 2017-08-21 19:16:14 UTC
(In reply to Dirkjan Ochtman from comment #49)
> So I started looking at your ebuild and hacking it a little bit, here are
> the changes I did so far:
> 
> - Remove the clang USE flag: what purpose does this serve exactly? See also
> bug 591930.

I agree with the rationale of bug 591930.

> - Inlined all local variables in src_configure() that are single-use. This
> is more of a style point, prefer to not have the abstraction if not needed.

Agreed, the fields in config.toml are verbose enough.

> - Decreased verbose = 2 to 0. For releasing to the entire user base, I don't
> think the verbose builds are a good default.

In many other ebuilds the default will be to pass e.g. VERBOSE=1 to make, hence I did the same here.  For debugging and getting good build.logs, I would prefer to have at least some verbosity enabled.  The difference between level 1 and 2 was iirc, that 1 prints invocations of rust, while 2 also prints the programs that rust calls.

> - Moved eselect-rust dependency from PDEPEND to RDEPEND, see bug 622654.

Seems reasonable.

> - Made the jemalloc USE flag +, to reflect upstream defaults.

OK.

> - I've replaced ${D} with ${ED} in a bunch of places -- I thought this was
> due to a bug, can't find it now though.

I vaguely remembered something similar, but couldn't find the official policy on which one to use.  I think ED was when building for a ROOT?

> See here: https://dirkjan.ochtman.nl/files/rust-1.19.0-r1.ebuild. However,
> on testing this didn't seem to work, so I got stalled.

Did you have a look at https://github.com/devurandom/gentoo-overlay/tree/master/dev-lang/rust, yet?  It contains some more changes, e.g. does it handle rust-lldb correctly and it supports USE=system-llvm as suggested in comment #39.  I also merged some changes you made in the -r1 ebuild proposed in comment #49.

Some notes on the r1 ebuild:
* It sets LDPATH and MANPATH to include ED, which seems wrong.
* Where can I find rust-1.19.0-bootstrap-use-configured-rustc-cargo-paths.patch, or which upstream commit is this based on?  I would like to rebuild with that, to try to reproduce the problems you're seeing.
Comment 51 Oleg 2017-08-21 19:54:41 UTC
There also another rust overlay: https://github.com/gentoo/gentoo-rust/tree/master/dev-lang/rust
Comment 52 Dennis Schridde 2017-08-22 08:12:16 UTC
(In reply to Oleg from comment #51)
> There also another rust overlay:
> https://github.com/gentoo/gentoo-rust/tree/master/dev-lang/rust

1.19.0-r2 looks quite similar to what I have.  Will check the diff in more detail later.
Comment 53 Dennis Schridde 2017-08-22 08:14:49 UTC
(In reply to Dennis Schridde from comment #50)
> * Where can I find
> rust-1.19.0-bootstrap-use-configured-rustc-cargo-paths.patch, or which
> upstream commit is this based on?  I would like to rebuild with that, to try
> to reproduce the problems you're seeing.

I assume this refers to https://github.com/rust-lang/rust/pull/42695 ?  It was merged upstream before 1.19.0 and hence should no longer be necessary.
Comment 54 Dirkjan Ochtman gentoo-dev 2017-08-22 09:14:13 UTC
I don't really like system-llvm supports, as it seems the LLVM versions allowed differ significantly from the version upstream pins. I don't want our builds to differ from upstream that much, as I think it would create additional differences, and thus additional maintenance burden.

For non-system-LLVM builds, it would probably be very helpful for build times (and disk usage) if we can reduce the targets the rust-internal LLVM build does, as in bug 627288.
Comment 55 Dirkjan Ochtman gentoo-dev 2017-08-29 19:14:58 UTC
With Rust 1.20 coming up this week, I will be doing a fresh round of ebuilds probably towards the end of this week. I think I'll start from what Dennis has in his overlay, if people want to suggest improvements on top of that, now would  be a good time.
Comment 57 Dennis Schridde 2017-09-10 15:07:11 UTC
Created attachment 493740 [details, diff]
Differences between my cargo-0.20.0 and cargo-0.21.0 in Gentoo Main

(In reply to Dirkjan Ochtman from comment #55)
> With Rust 1.20 coming up this week, I will be doing a fresh round of ebuilds
> probably towards the end of this week. I think I'll start from what Dennis
> has in his overlay, if people want to suggest improvements on top of that,
> now would  be a good time.

It seems that dev-util/cargo-0.21.0 was bumped without incorporating the changes from my overlay.  I see the attached differences.  Assuming the changes to CRATES being fine, I wonder why the other changes, like src_configure, verbosity and backtraces on faults, using cargo in src_install and having a proper src_test, were dropped.
Comment 58 Dirkjan Ochtman gentoo-dev 2017-09-10 18:04:59 UTC
That's because when I mentioned your "ebuilds", I was primarily thinking of the ones for dev-lang/rust, so I hadn't taken your changes to cargo into account.

I did try to copy your ebuild for rust-1.20.0, but it failed for me while building rustc_llvm. Can you try that and see if you get an error? After trying through ebuild, I ran it manually as root in the work dir and it seemed to get further into the build, so there might be something indeterministic going on.
Comment 59 Dennis Schridde 2017-09-10 23:09:47 UTC
(In reply to Dirkjan Ochtman from comment #58)
> I did try to copy your ebuild for rust-1.20.0, but it failed for me while
> building rustc_llvm. Can you try that and see if you get an error? After
> trying through ebuild, I ran it manually as root in the work dir and it
> seemed to get further into the build, so there might be something
> indeterministic going on.

I called get_llvm_prefix from llvm.eclass in the wrong way.  f79e4de9b92db0179c4a9b82ff6a6c0cbc836613 in my overlay fixes that.

I also bumped dev-lang/rust-1.20.0 and dev-util/cargo-0.21.0 in my overlay.  I believe bugs #591930, #622654 and #627288 to be fixed in my version of the ebuilds.
Comment 60 Perfect Gentleman 2017-10-14 08:33:24 UTC
Rust-1.21 is out.
https://blog.rust-lang.org/2017/10/12/Rust-1.21.html
Comment 61 Perfect Gentleman 2017-11-27 05:37:25 UTC
i modified the ebuild. Now there is no need in dev-util/cargo as it is said at Cargo's page "Cargo is distributed by default with Rust, so if you've got rustc installed locally you probably also have cargo installed locally."
So mozconfig-v6.57.eclass also should modified to remove dev-util/cargo from deps.
Comment 62 Perfect Gentleman 2017-11-27 05:38:28 UTC
Created attachment 506794 [details]
rust-1.22.1.ebuild
Comment 63 Dirkjan Ochtman gentoo-dev 2018-06-06 08:52:55 UTC
Since we have much more recent versions in the tree now, I'm going to close this. If you have still improvements for the ebuild, please report those as separate bugs with specific patches.