Bug 853280 – media-libs/alsa-lib making the case in favor of -flto

• Product: Gentoo Linux
• Status: RESOLVED WONTFIX

[Description unhappy-ending 2022-06-21 07:11:48 UTC]

Created attachment 786542 [details]
GCC building alsa-lib and passing tests with using single threaded LTO

Please Note: This is a system built using the Clang and LLVM toolchain system wide. This was also built using _FLAG_O_MATIC_ECLASS=1 to override ebuild restrictions.

It looks like filter-flags -flto=* -flto has been added to the most recent ebuild. This causes a configure phase error when checking if the compiler works, since Clang specific -flto options such as -fwhole-program-vtables requires the -flto flag.

media-sound/alsa-lib successfully builds and passes tests when using Clang. Including a build log to show this is the case. There are no build errors complaining about symbols so bug https://bugs.gentoo.org/669086 and https://bugs.gentoo.org/616108 do not apply.

I've been using -flto and alsa-lib for over a year now and haven't had issues with other packages building against it nor audio issues when using my system.

I'd also like to note, that back on June 2, 2021 I reported GCC is capable of using -flto but you need to pass the -flto-partition=one. I believe the reason it works is because GCC defaults to splitting up chunks of code into "partitions" and then merges them back together during link time. This is how it achieves multithreading during link time optimizations. I'm not an expert on GCC but I do know that when Clang does this using thinLTO which has similar behavior that splits code into chunks for multithreading, you lose symbol and call visibility. When using a single thread, behavior is such that you get the full visibility of calls and symbols at the cost of speed. The Clang feature -fvirtual-function-elimination requires -flto=full for this reason. I'm guessing it's similar for GCC, because GCC has no problem running -flto when it has a single block of code that hasn't been split up into pieces.

And, just to be sure, I ran a test trying Clang with -flto=thin passed. The same exact error in bug 616108 using GCC with the default multithreaded partitions happens. This is a problem with visibility to symbols and calls caused by splitting the code into chunks, and not actually link-time optimization itself.

I know it would complicate things a little, but some compiler checks could be done and append flags would make building with -flto a reality rather than simply filtering the flags. Rather than filtering the flags, would it be possible to append -flto-partition=one to the flags if -flto is found when using GCC? And, for Clang we could append-flags -flto=full when -flto or -flto=thin is found when using Clang. IMO, this isn't a hack since this is the expected behavior from multithreaded LTO that breaks compiled code into chunks to later be merged together. If code is sensitive to threaded LTO, then single thread it and there are flags that do exactly that to change the compiler's default behavior.

Attaching build logs.

[Comment 1 unhappy-ending 2022-06-21 07:13:35 UTC]

Created attachment 786545 [details]
Clang building alsa-lib and passing tests using single threaded LTO

No problems with single threaded (-flto=full) LTO using Clang

[Comment 2 unhappy-ending 2022-06-21 07:18:21 UTC]

Created attachment 786548 [details]
Clang fails to build alsa-lib when using mutlithreaded LTO

Yet we can see once you start multithreading by breaking the code into chunks during LTO is when we have complaints about symbols, since Clang doesn't see everything the same as it does when it's a single code chunk running in a single thread.

[Comment 3 Sam James 2022-06-21 22:05:53 UTC]

I really don't want to do this.

The whole reason I filtered it is because it's *dangerous*. If it fails with partitioning, it means it's bad code, and it needs to be fixed. Tests passing doesn't mean that it's safe.

I understand it builds and appears to work, but:
1. there's no guarantee it'll *continue* to work;
2. we've been given a huge warning sign. Ignoring it is foolish.

Working around it isn't great. If someone cares deeply about this, they should fix the upstream issue.