As a result of the change linked below, my Rust now force-links every Rust build with x86_64-pc-linux-gnu-cc as the linker: https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=f65ef3fddd9ef9cc3d2d172f6da9a39231b1e2d1 This causes all Rust builds to fail with a variation on this error: [38;5;31m38;5;9merror: linking with `x86_64-pc-linux-gnu-cc` failed: exit status: 1 [38;5;31m38;5;12m= note: /usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/bin/ld: /usr/lib/rust/1.85.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-8a71ed6c63583c9f.rlib: error adding symbols: file format not recognized collect2: error: ld returned 1 exit status Appending "-C linker=ld.lld" / "-C link-args=-fuse-ld=lld" to RUSTFLAGS does not help; the GNU linker is still forced. I build most of my system with LLVM/clang/lld, including dev-lang/rust; maybe that could be the reason why ld.bfd doesn't understand these rlibs. Either way, forcing the GNU linker is suboptimal. Reverting the commit & rebuilding Rust immediately fixes the problem. Reproducible: Always Steps to Reproduce: 1. Build dev-lang/rust with default-linker = "${CHOST}-cc" (building with CC=clang/LD=ld.lld might be necessary) 2. Build any Rust package, e.g. sys-apps/fd Actual Results: Rust packages fail to link with the force GNU ld. Expected Results: Rust packages build and link fine just like before this change.
> Either way, forcing the GNU linker is suboptimal. It's not forcing the GNU linker, though. llvm-core/clang-toolchain-symlinks installs /usr/lib/llvm/18/bin/x86_64-pc-linux-gnu-cc and so on. It's just that I suspect the GNU version in /usr/bin comes first in PATH for you.
It's not clear what a good solution here is. Maybe to not set `default-linker` at all. Eli had done some archaeology and found it was added in like 2015 with no rationale given. https://doc.rust-lang.org/rustc/codegen-options/index.html#linker says that if you don't specify it, it'll be `cc`. That'll surely have the same effect for you as the current value we're passing?
As a further note, Nate emailed me earlier and said that he *hadn't* explicitly set CC=clang-19, which makes the original fix (f65ef3fddd9ef9cc3d2d172f6da9a39231b1e2d1) more important, because it's being done by the eclass when it "fixes" the tool versions to be consistent.
If it actually inspects ${CC} at the time if you don't specify anything, we may be able to that (drop specifying it). If not, we may just have to tell people to add -C linker to RUSTFLAGS (or adjust PATH manually). Certainly the previous state of affairs isn't acceptable.
Indeed, those /usr/lib/llvm symlinks come after /usr/bin; although that weird call there didn't feel like it was a result of a $PATH resolution (/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/bin/ld). I'm happy to rebuild Rust with no default-linker set and see what happens. If it still fails by default but lets me override it with "-C linker" at least (unlike now), that'd be an improvement. I'll give it a go and let you know.
(In reply to Petr Šabata from comment #5) > Indeed, those /usr/lib/llvm symlinks come after /usr/bin; although that > weird call there didn't feel like it was a result of a $PATH resolution > (/usr/lib/gcc/x86_64-pc-linux-gnu/14/../../../../x86_64-pc-linux-gnu/bin/ld). > It's using the thing it finds via PATH resolution as the compiler driver which then invokes ld. The driver happens to be GCC here because it comes first via PATH, and then it invokes bfd using the usual lookup it does. > I'm happy to rebuild Rust with no default-linker set and see what happens. > If it still fails by default but lets me override it with "-C linker" at > least (unlike now), that'd be an improvement. It won't let you override it now? That's surprising. > > I'll give it a go and let you know. Thanks!
(In reply to Sam James from comment #6) > > I'm happy to rebuild Rust with no default-linker set and see what happens. > > If it still fails by default but lets me override it with "-C linker" at > > least (unlike now), that'd be an improvement. > > It won't let you override it now? That's surprising. Indeed. I noted that in the original comment. I've tried with both relative and absolute paths, just to be sure; there was no difference. > > I'll give it a go and let you know. > > Thanks! So yeah, now it indeed defaults to "cc" as you noted, and fails just the same. And passing "-C linker" still doesn't help. Attaching an example build.log if you're curious. I don't know what the right solution is, but for the time being I'll have to use a local ebuild with the change reverted, otherwise I can't build any Rust code.
Created attachment 921648 [details] An example build.log (sys-apps/fd)
(In reply to Petr Šabata from comment #7) > (In reply to Sam James from comment #6) > > > I'm happy to rebuild Rust with no default-linker set and see what happens. > > > If it still fails by default but lets me override it with "-C linker" at > > > least (unlike now), that'd be an improvement. > > > > It won't let you override it now? That's surprising. > > Indeed. I noted that in the original comment. I've tried with both > relative and absolute paths, just to be sure; there was no difference. > I asked again because it's against the documentation, expectation, and the code. But indeed it doesn't seem to be in your log. > > > I'll give it a go and let you know. > > > > Thanks! > > So yeah, now it indeed defaults to "cc" as you noted, and fails just the > same. And passing "-C linker" still doesn't help. Attaching an example > build.log if you're curious. > Interestingly, it appears twice with clang and then a full path to lld. But ah, in cargo.eclass, we do: local TRIPLE=${TRIPLE^^} LD_A=( $(tc-getCC) ${LDFLAGS} ) local -Ix CARGO_TARGET_"${TRIPLE}"_RUSTFLAGS+=" -C strip=none -C linker=${LD_A[0]} -C target-feature=-crt-static" which makes things even more mysterious (note that the original change was made to help people using rustc outside of ebuilds), given it doesn't seem to have come up before. One hypothesis would be that your clang doesn't default to lld, and that when two `-C linker` arguments appear, cargo or rustc doesn't behave correctly. But then that doesn't explain how it worked before. I'd suggest running one of the failing commands manually with --verbose to see if we can look at what it's invoking and why. > I don't know what the right solution is, but for the time being I'll have to > use a local ebuild with the change reverted, otherwise I can't build any > Rust code. Right. The change we made seems right, especially so given the default has the same behaviour (and the old behaviour was definitely wrong, just it happened to work well for you as a side-effect).
Ah, wait, on that line: ``` [0m [0m[0m[1m[38;5;12m= [0m[0m[1mnote[0m[0m: LC_ALL="C" PATH="/usr/lib/rust/1.85.1/lib/rustlib/x86_64-unknown-linux-gnu/bin:/usr/lib/rust/1.85.1//bin:/usr/lib/portage/python3.13/ebuild-helpers/xattr:/usr/lib/portage/python3.13/ebuild-helpers:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin:/usr/lib/llvm/20/bin:/usr/lib/llvm/19/bin" VSLANG="1033" "cc" "-m64" "/var/tmp/portage/sys-apps/fd-10.2.0-r1/temp/rustcrlGJ3H/symbols.o" "<3 object files omitted>" "-Wl,--as-needed" "-Wl,-Bstatic" "/usr/lib/rust/1.85.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/ [...] ``` ... it's not using clang at all? It's using `cc`? Again looking at https://docs.rs/cc/latest/cc/#external-configuration-via-environment-variables, it should respect CC and CXX just fine.
Another thing to play with would be -C linker-flavor.
My clang defaults to lld; llvm-core/clang-common and llvm-core/clang-runtime are built with USE="default-lld", and the relevant gentoo-runtimes.cfg includes -fuse-ld=lld as a result. I've tried "-C linker-flavor=ld.lld", which makes no difference. It still calls "cc". Calling one of those failing commands with --verbose produces JSON output which is not particularly helpful; it doesn't tell you why it decides to call "cc" despite being instructed to use clang, lld or anything else. It just includes all the filenames; but I'm going to attach that also. As for the key env variables, they're CC="clang", CXX="clang++", LD="ld.lld".
Created attachment 921651 [details] Verbose call failure
Oh and to comment on "-C linker" being passed twice: The first one (clang) is from the build itself, the second one is from my RUSTFLAGS; it being defined twice isn't the root cause here because my starting RUSTFLAGS didn't include this definition at all and it was failing just the same. It's just ignoring the value, it seems.
Baffling. I've looked through cc-rs/src/lib.rs and I don't see anything obvious. May be worth triyng to `cargo init` some example project with a trivial C file and see if you can reproduce there (with it using the `cc` crate).
(In reply to Sam James from comment #15) > Baffling. I've looked through cc-rs/src/lib.rs and I don't see anything > obvious. > > May be worth triyng to `cargo init` some example project with a trivial C > file and see if you can reproduce there (with it using the `cc` crate). Now, that's different! Still fails by default, using "cc" and ignoring the CC/CXX/LD env vars. But it doesn't ignore the linker argument and if I set it to "-C linker=clang" via env RUSTFLAGS, it actually builds and runs fine. So... progress. Portage must be fiddling with something else causing it to behave differently in that environment then.
A very silly idea and I kind of hope it's wrong, but the eclass does RUSTFLAGS_XXX where XXX is a Rust target. I wonder if cc sees RUSTFLAGS_XXX and then decides it'll only look at *_XXX rather than the generic versions. So, if you say set in make.conf/export before calling emerge: CC_x86_64_unknown_linux_gnu="clang", does that help? (You may need to adjust the variable name to adjust whatever the eclass uses, check the environment file at /var/tmp/portage/sys-apps/fd-10.2.0-r1/temp/environment for whatever the RUSTFLAGS_XXX suffix is.)
(In reply to Sam James from comment #17) > A very silly idea and I kind of hope it's wrong, but the eclass does > RUSTFLAGS_XXX where XXX is a Rust target. I wonder if cc sees RUSTFLAGS_XXX > and then decides it'll only look at *_XXX rather than the generic versions. > > So, if you say set in make.conf/export before calling emerge: > CC_x86_64_unknown_linux_gnu="clang", does that help? > > (You may need to adjust the variable name to adjust whatever the eclass > uses, check the environment file at > /var/tmp/portage/sys-apps/fd-10.2.0-r1/temp/environment for whatever the > RUSTFLAGS_XXX suffix is.) There's nothing like that defined in the environment. I tried setting it anyway and it had no impact on the outcome. I used x86_64_unknown_linux_gnu as that's the only x86_64 Rust target on my system. But we can ignore this because... I've also tried rebuilding Rust without "-C linker-plugin-lto" in my RUSTFLAGS and now I can build Rust ebuilds again. However, the core issue is likely still present: it still calls the default linker, effectively ld.bfd, it's just now the linker doesn't trip over those rlibs. So to confirm (since this not logged unless it fails), I've looked at strace and it actually... calls both! That made me grep all the "Running" lines and not all of them append RUSTFLAGS and those very same lines also don't pass "-C linker=clang" from the build itself. So now I think it isn't rustc ignoring the linker argument; it's that it's not passed to it consistently. You might be able to notice some patterns and tie it to specific phases of the build, perhaps, so I'm also attaching a successful sys-apps/fd build.log.
Created attachment 921764 [details] Successful sys-apps/fd build.log
These look relevant: https://github.com/rust-lang/cargo/issues/10111 https://github.com/rust-lang/cargo/issues/10321 https://github.com/rust-lang/cargo/pull/9601 Looking at the docs (https://doc.rust-lang.org/cargo/reference/environment-variables.html), maybe using CARGO_ENCODED_RUSTFLAGS instead of CARGO_TARGET_${TRIPLE}_RUSTFLAGS would be the way to address this behavior.
I've reproduced this by building with CC=clang and pointing /usr/bin/${CHOST}-cc to /bin/false. I think I know what it is. I can't explain right now, but the bad news is that I think fixing it requires the host-config nightly feature.
We joked on IRC about how I'd broken something again. Turns out I really did break this, or at least that's how it looks to me right now. When I wrote my last comment, I thought that Cargo always behaved like it was cross-compiling, applying flags to the target but totally ignoring any configuration for the "host" (the build system). The host-config feature allows you to configure the host and target separately, even if they have the same tuple. However, if you don't set CARGO_BUILD_TARGET (or pass --target), then it doesn't actually split the configuration like this, and everything works as expected. I made cargo.eclass set this a few months back to aid cross-compiling, but I applied it for native builds too. One reason was to have a consistent CARGO_TARGET_DIR. We can still have that by setting CARGO_TARGET_DIR ourselves, but I'm trying to remember if there was also another reason. The good news is that you can still set flags via CARGO_TARGET_<TRIPLE>_RUSTFLAGS rather than just RUSTFLAGS, so the cargo.eclass logic doesn't need to change too much. I'll need to do some testing around this. Trying to get it right last time took quite some effort.
*** Bug 951804 has been marked as a duplicate of this bug. ***
Also discussed at e.g. https://public-inbox.gentoo.org/gentoo-user/58bab509-ec7f-4450-901e-562610305eef@gentoo.org/T/#u (In reply to Petr Šabata from comment #18) > So to confirm (since this not logged unless it fails), I've looked at strace > and it actually... calls both! That made me grep all the "Running" lines > and not all of them append RUSTFLAGS and those very same lines also don't > pass "-C linker=clang" from the build itself. > > So now I think it isn't rustc ignoring the linker argument; it's that it's > not passed to it consistently. You might be able to notice some patterns > and tie it to specific phases of the build, perhaps, so I'm also attaching a > successful sys-apps/fd build.log. Yup. Well, it does show rustc invocations whether it fails or not, and you can indeed see that some targets currently don't have any linker flags passed, seemingly at this point the distinction is --crate-name build_script_build (In reply to Petr Šabata from comment #20) > Looking at the docs > (https://doc.rust-lang.org/cargo/reference/environment-variables.html), > maybe using CARGO_ENCODED_RUSTFLAGS instead of > CARGO_TARGET_${TRIPLE}_RUSTFLAGS would be the way to address this behavior. I don't think so -- I think that crates are supposed to be coded to look up the value of *_ENCODED_RUSTFLAGS, not that you are supposed to export it in the environment.
(In reply to Petr Šabata from comment #18) > There's nothing like that defined in the environment. I tried setting it > anyway and it had no impact on the outcome. I used x86_64_unknown_linux_gnu > as that's the only x86_64 Rust target on my system. But we can ignore this > because... Just to clarify this bit: I was wrong, as we don't export it in general, we do cargo_env, so it won't show up there.
*** Bug 951868 has been marked as a duplicate of this bug. ***
Created attachment 922208 [details, diff] Proposed fix unsetting CARGO_BUILD_TARGET Try this on for size. I was wrong about CARGO_TARGET_DIR, that only sets the initial "target" part of the output path. Luckily, we have the cargo_target_dir helper, so I just have to put this back to how it was before.
(In reply to James Le Cuirot from comment #27) > Created attachment 922208 [details, diff] [details, diff] > Proposed fix unsetting CARGO_BUILD_TARGET > > Try this on for size. I was wrong about CARGO_TARGET_DIR, that only sets the > initial "target" part of the output path. Luckily, we have the > cargo_target_dir helper, so I just have to put this back to how it was > before. That appears to have fixed it. Comparing the build logs, the linker argument is now passed every time, and running strace on the build, I no longer see any invocations of the GNU linker.
As a sanity check, I've also just rebuilt Rust with -Clinker-plugin-lto and tried rebuilding all the builds that had previously failed. All but one, =gnome-base/librsvg-2.58.5, have passed. As for librsvg, it's the same problem (GNU linker used, not understanding the rlibs).
Did it fail to build the 32-bit version by any chance?
(In reply to James Le Cuirot from comment #30) > Did it fail to build the 32-bit version by any chance? No, that doesn't seem to be the case. Also I'm on a no-multilib profile.
Ah, I know. It's not using my code. Try changing multilib_src_compile to this: > multilib_src_compile() { > cargo_env gnome2_src_compile > } We probably need to do something similar for the tests.
(In reply to James Le Cuirot from comment #32) > Ah, I know. It's not using my code. Try changing multilib_src_compile to > this: > > > multilib_src_compile() { > > cargo_env gnome2_src_compile > > } > > We probably need to do something similar for the tests. Nope, that does not help, unfortunately. Isn't this lovely.
Ignore the previous comment. It does work, librsvg builds. I forgot to also patch the eclass in that branch.
Naively applying.. ``` --- a/gnome-base/librsvg/librsvg-2.58.5.ebuild +++ b/gnome-base/librsvg/librsvg-2.58.5.ebuild @@ -380,8 +380,7 @@ multilib_src_configure() { ) fi - ECONF_SOURCE=${S} \ - gnome2_src_configure "${myconf[@]}" + ECONF_SOURCE=${S} cargo_env gnome2_src_configure "${myconf[@]}" if multilib_is_native_abi; then ln -s "${S}"/doc/html doc/html || die @@ -393,11 +392,11 @@ src_compile() { } multilib_src_compile() { - gnome2_src_compile + cargo_env gnome2_src_compile } src_test() { - multilib-minimal_src_test + cargo_env multilib-minimal_src_test } src_install() { @@ -405,7 +404,7 @@ src_install() { } multilib_src_install() { - gnome2_src_install + cargo_env gnome2_src_install } multilib_src_install_all() { ``` and I got: ``` Finished `release` profile [optimized] target(s) in 59.73s libtool: link: x86_64-pc-linux-gnu-ar cr .libs/librsvg_c_api.a .libs/_rsvg_dummy.o libtool: link: x86_64-pc-linux-gnu-ranlib .libs/librsvg_c_api.a libtool: link: ( cd ".libs" && rm -f "librsvg_c_api.la" && ln -s "../librsvg_c_api.la" "librsvg_c_api.la" ) cp: cannot stat '/var/tmp/portage/gnome-base/librsvg-2.58.5/work/librsvg-2.58.5-abi_x86_64.amd64/target/release/liblibrsvg_c.a': No such file or directory make[2]: *** [Makefile:1639: librsvg_c_api.la] Error 1 make[1]: *** [Makefile:1159: all-recursive] Error 1 make: *** [Makefile:798: all] Error 2 * ERROR: gnome-base/librsvg-2.58.5::gentoo failed (compile phase): * emake failed ```
Ah, it works once applied Chewi's patch to the eclass.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=5fe28b3e44b8b264413decae969e24a5aa811c1a commit 5fe28b3e44b8b264413decae969e24a5aa811c1a Author: James Le Cuirot <chewi@gentoo.org> AuthorDate: 2025-03-23 22:19:40 +0000 Commit: James Le Cuirot <chewi@gentoo.org> CommitDate: 2025-03-29 15:57:43 +0000 gnome-base/librsvg: Use cargo_env to ensure proper RUSTFLAGS are applied This almost removes the need to set cross_compiling=yes but unfortunately, this is still needed for the build to determine the right target directory. Closes: https://bugs.gentoo.org/951740 Signed-off-by: James Le Cuirot <chewi@gentoo.org> gnome-base/librsvg/librsvg-2.58.5.ebuild | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=6937fcb873120179a5bb32055b26612ac4a72c52 commit 6937fcb873120179a5bb32055b26612ac4a72c52 Author: James Le Cuirot <chewi@gentoo.org> AuthorDate: 2025-03-23 11:12:00 +0000 Commit: James Le Cuirot <chewi@gentoo.org> CommitDate: 2025-03-29 15:57:39 +0000 cargo.eclass: Only tell Cargo to cross-compile when actually needed This avoids the build host vs target flag separation issue. Sometimes it is important for the build host to use the right flags. We cannot fix this for cross-compiling now, but it should at least work for native builds. Closes: https://bugs.gentoo.org/951740 Signed-off-by: James Le Cuirot <chewi@gentoo.org> eclass/cargo.eclass | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
Thanks! Everything appears to build correctly now.
