Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 371961 - sys-devel/llvm: Add a USE-flag for changing the built-in linker from ld to ld.gold
Summary: sys-devel/llvm: Add a USE-flag for changing the built-in linker from ld to ld...
Status: RESOLVED WONTFIX
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: Normal enhancement with 1 vote (vote)
Assignee: LLVM support project
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-06-16 21:07 UTC by Matthias Maier
Modified: 2017-02-11 11:58 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Proposed ebuild with USE-flag "gold" (clang-2.9-r1.ebuild.patch,1.21 KB, patch)
2011-06-16 21:09 UTC, Matthias Maier
Details | Diff
-fuse-ld support to select gold linker (llvm-3.4.2-use-ld.patch,11.79 KB, patch)
2014-09-05 14:58 UTC, Holger Hoffstätte
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Matthias Maier gentoo-dev 2011-06-16 21:07:26 UTC
Enabling link-time optimization (LTO) support with -O4 in clang makes the use of the gold linker necessary. (The plugin LLVMgold.so has to be loaded).

So it is desirable that clang uses the gold linker instead of the system ld (that happens to be GNU ld at the moment), so that the use of clang with -O4 is a bit more transparent to current build scripts: Invoking clang with "-O4 -Wl,--plugin=/usr/lib64/llvm/LLVMgold.so" is much less painfull than changing the build scripts of (whatever) project to compile only to object files with clang and afterwards link the object files with ld.gold by hand. (Not to mention to get all necessary libraries correctly on the command line...)

So I propose to add a USE-flag "gold", that simply replaces the invocation of the system ld with ld.gold . Please note that with the current clang binaries it is neither possible to define an alternative linker on the command line, nor to specify the to be used linker at configure time prior to compilation...

Shure, the USE-flag will be obsolete as soon as the gold linker replaces the old GNU ld. But this could take some more time and in the meantime it would be a huge improvement.

Reproducible: Always
Comment 1 Matthias Maier gentoo-dev 2011-06-16 21:09:11 UTC
Created attachment 277299 [details, diff]
Proposed ebuild with USE-flag "gold"

The proposed ebuild with support for a "gold" USE-flag.

This would go along something like

sys-devel/clang:gold - Replace the system linker invocation with the gold linker

in the use.local.desc file
Comment 2 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-07-16 08:32:25 UTC
To be honest, I'd rather see a patch which would enable 'ld.gold' in runtime whenever LTO is enabled.
Comment 3 Richard Yao gentoo-dev 2012-07-16 13:01:45 UTC
(In reply to comment #2)
> To be honest, I'd rather see a patch which would enable 'ld.gold' in runtime
> whenever LTO is enabled.

I was under the impression that there was a flag that could be used to change the linker dynamically, but I have not had a chance to play with -O4 to examine that.
Comment 4 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-07-31 20:56:17 UTC
(In reply to Richard Yao from comment #3)
> (In reply to comment #2)
> > To be honest, I'd rather see a patch which would enable 'ld.gold' in runtime
> > whenever LTO is enabled.
> 
> I was under the impression that there was a flag that could be used to
> change the linker dynamically, but I have not had a chance to play with -O4
> to examine that.

I know that gcc has -fuse-ld=gold for that but I don't know if clang supports it as well.
Comment 5 Holger Hoffstätte 2014-08-01 14:37:55 UTC
After remerging llvm with +gold just to be chased in circles for hours, here is how it "works". Or not, depending on your expectations. I found the solution here: https://bugs.launchpad.net/ubuntu/+source/llvm-toolchain-snapshot/+bug/1254970 - though it's a bit confusing since the first part about linking LLVMgold.so does not apply in our case.

There does not seem to be a flag for dynamic linker selection, since Apple as main clang developers need only one linker; it apparently happens based on search path, and as regular non-gold ld is always in /usr/bin, it gets found first and used. However clang understands gcc's -B switch (undocumented?!), which allows setting a prefix directory for finding executables - and in our case ld.gold:

holger>which ld
/usr/bin/ld
holger>echo 'int main() {}' | clang -xc -flto - 
/tmp/--57b31a.o: file not recognized: File format not recognized
clang: error: linker command failed with exit code 1 (use -v to see invocation)

holger>sudo mkdir /usr/bin/gold && sudo ln -s /usr/bin/ld.gold /usr/bin/gold/ld
holger>ll /usr/bin/gold/ld 
lrwxrwxrwx 1 root root 16 Aug  1 16:11 /usr/bin/gold/ld -> /usr/bin/ld.gold*

holger>echo 'int main() {}' | clang -B/usr/bin/gold -xc -flto - 
holger>ll a.out                                                
-rwxr-xr-x 1 holger users 5.6K Aug  1 16:12 a.out*

Consequently adding "-B/usr/bin/gold" to LDFLAGS (in addition to -flto) now allows building a 'real' project. The resulting executable was quite a bit smaller and still worked, despite being in C++. :)

Obviously this is not a "real" solution, but at least it does not break anything and allows activation of LTO-aware linking when requested by simply adding the prefix path to LDFLAGS.

The real fix indeed seems to be a patch to the clang driver to look first for ld.gold instead of ld when -flto is requested.
Comment 6 Holger Hoffstätte 2014-08-01 15:50:43 UTC
Upstream status of runtime-selectable ld:
http://llvm.org/bugs/show_bug.cgi?id=18257
Comment 7 Holger Hoffstätte 2014-08-01 18:14:27 UTC
(In reply to Holger Hoffstätte from comment #6)
> Upstream status of runtime-selectable ld:
> http://llvm.org/bugs/show_bug.cgi?id=18257

Sorry for talking to myself, but some good news: -fuse-ld was re-added after all, and - according to the 3.5 release branch - will be part of 3.5:
https://github.com/llvm-mirror/clang/commit/bab68c96f15867af6938d9c8f922d59d31351cad

Maybe we should just backport this if 3.4.2 is an eventual stable candidate? The changes do not look too complicated or intrusive, and we had handpicked patches before..
Comment 8 Holger Hoffstätte 2014-08-05 16:56:09 UTC
(In reply to Holger Hoffstätte from comment #7)
> Maybe we should just backport this if 3.4.2 is an eventual stable candidate?

So I just made this work, and behold! Working LTO without toolchain/search path fiddling:

holger>echo "#include <iostream>\n int main() {std::cout << \":)\\\n\";}" | clang++ -xc++ -flto -fuse-ld=gold -
holger>./a.out                                                                    :)

It's a single patch generated from llvm's git and slightly massaged, applied here via epatch_user to 3.4.2. If anybody is interested just let me know and I'll open a proper new bug with attachment.
Comment 9 Holger Hoffstätte 2014-09-05 14:58:27 UTC
Created attachment 384238 [details, diff]
-fuse-ld support to select gold linker

I've been asked to attach the patch here, so here it is. Drop it into /etc/portage/patches/sys-devel/llvm and off you go.
Comment 10 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-02-11 11:58:19 UTC
Upstream clang supports -fuse-ld for some time already. They also support setting a default at build time but I don't think we have a strong reason to expose that at the moment. Plus we're having sys-devel/lld as an alternative now.
Comment 11 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-02-11 11:58:47 UTC
Plus ld.bfd supports plugins too; however, I haven't checked if LLVMgold works with it.