Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 946486 - llvm-runtimes/*: Allow to build LLVM runtime (compiler-rt, libunwind) with GNU extensions (llvm-libgcc)
Summary: llvm-runtimes/*: Allow to build LLVM runtime (compiler-rt, libunwind) with GN...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal enhancement
Assignee: LLVM support project
URL:
Whiteboard:
Keywords: PullRequest
Depends on:
Blocks:
 
Reported: 2024-12-14 21:00 UTC by vadorovsky
Modified: 2025-02-27 06:05 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description vadorovsky 2024-12-14 21:00:11 UTC
Rust binaries provided by the upstream community are linked to libgcc_s. To remove the hard dependency on GCC for musl-llvm profile, we initially tried to use llvm-libunwind as a drop-in replacement:

https://github.com/gentoo/gentoo/pull/39495

But that caused the following issue:

https://bugs.gentoo.org/946321

Which makes sense, because libunwind is not a drop-in replacement for libgcc_s. libgcc_s is a monolithic library, while LLVM ships the same functionality as two separate libraries (compiler-rt, libunwind). The bug mentioned above was triggered by that - rustc was looking for a symbol belonging to compiler-rt and failed to find it in libunwind.

However, there is a solution. LLVM provides a project called llvm-libgcc (https://github.com/llvm/llvm-project/tree/main/llvm-libgcc), which is a shim layer on top of llvm-libunwind and compiler-rt which provides a linker script mimicking the GNU ABI, links compiler-rt to libunwind and overall makes sure that all the symbols which would normally be present in libgcc_s, are provided by libunwind.so.

If we package llvm-libgcc, we could easily provide this predicate, which wouldn't require GCC on musl-llvm systems:

	|| (
		sys-devel/gcc:*
		llvm-runtime/llvm-libgcc
	)

We could ship llvm-libgcc in two ways.

1. The standard way envisioned by the LLVM upstream

It nails down to the following steps:

- Building compiler-rt and libunwind, which in contrast to a default build:
  - Have GNU extensions.
  - Have symbols from compiler-rt as a part of libunwind.so.
- Installiation of these libraries:
  - libclang_rt.builtins*.a
  - libunwind.{a,so}
- Symlinks, which serve as a drop-in replacement for libgcc_s:
  - libclang_rt.builtins*.a -> libgcc.a
  - ibunwind.a -> libgcc_eh.a
  - libunwind.so.1.0 -> libgcc_s.so.1.0

However, that requires building compiler-rt and libunwind together. Currently Gentoo provides them as separate ebuilds. We would probably need to merge them into one ebuild. After doing so we could add a USE flag (`gnu`?) which enables the llvm-libgcc shim.

Using LLVM runtime with llvm-libgcc would obviously conflict with GCC (they would both provide the same files). In practice, using it would make sense only on profiles which never-ever require GCC - which is only musl-llvm. I think it would make sense to unmask that USE flag only on musl-llvm.

2. My hacky solution

I have an ebuild of llvm-libgcc in my personal overlay:

https://github.com/vadorovsky/overlay/tree/main/llvm-runtimes/llvm-libgcc

Which does the following:

- Builds llvm-libgcc (compiler-rt and libunwind).
- Manually installs (moves files, instead of symlinking like upstream):
  - libclang_rt.builtins*.a -> libgcc.a
  - ibunwind.a -> libgcc_eh.a
  - libunwind.so.1.0 -> libgcc_s.so.1.0

This way, the system ends up with two copies of LLVM runtime:

- The default one, without the libgcc shim.
- The GNU-compatible one.

This hack works, but it's quite wasteful (bloats your system with two runtime copies). It also goes against upstream and manually installs stuff, instead of just using CMake for installation.

However, what's good about this solution is that it's less intrusive and we wouldn't have to change the core LLVM runtime packages.

LMK what solution would you prefer.
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2024-12-16 14:34:33 UTC
Please close your eyes because I'm having scary ideas.

I don't mind enabling GNU stuff on compiler-rt and libunwind, as I think that's unlikely to break anything.  However, I'd rather not compile stuff twice.

Do I understand correctly that:

1) as far as static libraries are concerned, libgcc.a and libgcc_eh.a are roughly equivalent to the static libs built right now + enabled GNU extensions?

2) then, libgcc_s.so is pretty much libunwind.so + libgcc.a?

Which gives me an idea: can't we just make libgcc_s.so that would include libgcc.a and link to libunwind.so?
Comment 2 vadorovsky 2024-12-16 17:14:09 UTC
(In reply to Michał Górny from comment #1)
> Please close your eyes because I'm having scary ideas.
> 
> I don't mind enabling GNU stuff on compiler-rt and libunwind, as I think
> that's unlikely to break anything.  However, I'd rather not compile stuff
> twice.
> 
> Do I understand correctly that:
> 
> 1) as far as static libraries are concerned, libgcc.a and libgcc_eh.a are
> roughly equivalent to the static libs built right now + enabled GNU
> extensions?

I think so. To be precise, I think that, when building llvm-libgcc, libunwind.a receives the GNU extensions (through the gcc_s.ver linker script) and then it's symlinked to libgcc_eh.a.

> 
> 2) then, libgcc_s.so is pretty much libunwind.so + libgcc.a?

libunwind.so also receives GNU extensions (gcc_s.ver, https://github.com/llvm/llvm-project/blob/290f38cd1a9fa7b1a91ddb25632807ecb5308dc7/llvm-libgcc/CMakeLists.txt#L113-L115), is linked to clang-rt.builtins (https://github.com/llvm/llvm-project/blob/290f38cd1a9fa7b1a91ddb25632807ecb5308dc7/llvm-libgcc/CMakeLists.txt#L117-L120) and then symlinked to libgcc.so.

> 
> Which gives me an idea: can't we just make libgcc_s.so that would include
> libgcc.a and link to libunwind.so?

You mean producing the libgcc_s.so manually, without using CMake, but following what llvm-libgcc/CMakeLists.txt does? And not having any symlinks, but instead just a separate libgcc_s.so with all the symbols, while the system-wide crt/libunwind remain untouched. That's also and option. But also, would require checking each release whether nothing changed in the way of building these libraries. And we would still have duplicate libs (the same downside as my current ebuild). Not saying that I'm against the solution, just something to keep in mind.
Comment 3 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2024-12-16 19:07:19 UTC
Well, I don't see much of a difference in duplication.  I'm saying we should modify compiler-rt and libunwind to have these .a compatible with llvm-libgcc, so there's no duplication there (no problem in adding the extra component, not sure how much of build system you can reuse).

And then, since compiler-rt doesn't install shared builtins, libgcc_s.so will always be duplicate.  Though the way I see it, we can make it "less duplicate" by making it link to libunwind.so rather than duplicating symbols.  That is, unless I'm missing something.
Comment 4 vadorovsky 2024-12-16 20:09:17 UTC
(In reply to Michał Górny from comment #3)
> Well, I don't see much of a difference in duplication.  I'm saying we should
> modify compiler-rt and libunwind to have these .a compatible with
> llvm-libgcc, so there's no duplication there (no problem in adding the extra
> component, not sure how much of build system you can reuse).

Oh, that's what you mean. Yes, I agree with this approach.

I guess that merging compiler-rt and libunwind (to be able to just do DLLVM_ENABLE_RUNTIMES="llvm-libgcc") is out of question? If so, I will need to manually do what llvm-libgcc/CMakeLists.txt does.

What's your take on that? Separate packages or merging crt/libunwind into one?

> And then, since compiler-rt doesn't install shared builtins, libgcc_s.so
> will always be duplicate.  Though the way I see it, we can make it "less
> duplicate" by making it link to libunwind.so rather than duplicating
> symbols.  That is, unless I'm missing something.

Yes, exactly, if we ship libunwind.so with GNU and crt symbols, libgcc_s.so can be just a link.
Comment 5 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2024-12-17 03:24:39 UTC
(In reply to vadorovsky from comment #4)
> What's your take on that? Separate packages or merging crt/libunwind into
> one?

I'd prefer keeping them separate, at least unless it turns out too hard.

> > And then, since compiler-rt doesn't install shared builtins, libgcc_s.so
> > will always be duplicate.  Though the way I see it, we can make it "less
> > duplicate" by making it link to libunwind.so rather than duplicating
> > symbols.  That is, unless I'm missing something.
> 
> Yes, exactly, if we ship libunwind.so with GNU and crt symbols, libgcc_s.so
> can be just a link.

Not symlink.  "Link" as in ld.
Comment 6 Larry the Git Cow gentoo-dev 2025-01-26 20:01:09 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ccf5fbc90ecfe4104a510a2ec5dc9070f0ded9e1

commit ccf5fbc90ecfe4104a510a2ec5dc9070f0ded9e1
Author:     Michal Rostecki <vadorovsky@protonmail.com>
AuthorDate: 2025-01-09 17:50:22 +0000
Commit:     Michał Górny <mgorny@gentoo.org>
CommitDate: 2025-01-26 20:00:16 +0000

    llvm-runtimes/libgcc: New package, 19.1.7 and 20.x
    
    llvm-libgcc[0] is the LLVM subproject which provides a link script,
    which can be used to build a library linked to compiler-rt and
    llvm-libunwind, which serves as a full replacement of libgcc_s.
    
    The main purpose of having it in Gentoo is ability to run binaries,
    linked to libgcc_s, on musl-llvm systems.
    
    [0] https://github.com/llvm/llvm-project/tree/main/llvm-libgcc
    
    Bug: https://bugs.gentoo.org/946486
    Signed-off-by: Michal Rostecki <vadorovsky@protonmail.com>
    Pull-Request: https://github.com/gentoo/gentoo/pull/39868
    Signed-off-by: Michał Górny <mgorny@gentoo.org>

 llvm-runtimes/libgcc/Manifest                      |   3 +
 llvm-runtimes/libgcc/libgcc-19.1.7.ebuild          | 156 +++++++++++++++++++++
 llvm-runtimes/libgcc/libgcc-20.0.0.9999.ebuild     | 155 ++++++++++++++++++++
 .../libgcc/libgcc-20.0.0_pre20250125.ebuild        | 155 ++++++++++++++++++++
 llvm-runtimes/libgcc/metadata.xml                  |  14 ++
 5 files changed, 483 insertions(+)
Comment 7 Larry the Git Cow gentoo-dev 2025-02-27 06:05:48 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=f59546d6e2ad8c0a8f3148efc06a641d860e7df6

commit f59546d6e2ad8c0a8f3148efc06a641d860e7df6
Author:     Michal Rostecki <vadorovsky@protonmail.com>
AuthorDate: 2025-01-29 17:46:56 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2025-02-27 06:03:44 +0000

    dev-lang/rust-bin: Allow to use llvm-libgcc instead of GCC
    
    We recently added llvm-runtimes/libgcc ebuild, which provides a full
    replacement of libgcc_s for systems without GCC.
    
    To leverage that ebuild in Rust binary packages, make them depend either
    on llvm-runtimes/libgcc or sys-devel/gcc.
    
    Bug: https://bugs.gentoo.org/946486
    Signed-off-by: Michal Rostecki <vadorovsky@protonmail.com>
    Closes: https://github.com/gentoo/gentoo/pull/40737
    Signed-off-by: Sam James <sam@gentoo.org>

 dev-lang/rust-bin/rust-bin-1.85.0-r1.ebuild        | 322 +++++++++++++++++++++
 .../rust-bin-1.86.0_beta20250218-r1.ebuild         | 320 ++++++++++++++++++++
 dev-lang/rust-bin/rust-bin-9999.ebuild             |   5 +-
 3 files changed, 646 insertions(+), 1 deletion(-)