Bug 754654 – media-video/ffmpeg: LTO breaks endianness test

• Product: Gentoo Linux
• Status: RESOLVED FIXED

[Description David Michael 2020-11-15 02:07:13 UTC]

CFLAGS apparently isn't supported upstream.  The configure script tests for a big-endian system by compiling a global integer and dumping the object file as hex.  When CFLAGS enables LTO, the value gets optimized out, so PowerPC builds fail due to big-endian macros not getting set.

https://github.com/FFmpeg/FFmpeg/blob/master/configure#L5758-L5762

Reproducible: Always

Steps to Reproduce:
1. powerpc-gentoo-linux-gnu-emerge -v media-video/ffmpeg  # with -flto in CFLAGS

Actual Results:  
...
big-endian                no
...
src/libavcodec/ppc/audiodsp.c: In function ‘scalarproduct_int16_altivec’:
src/libavutil/ppc/util_altivec.h:123:5: error: implicit declaration of function ‘vec_vsx_ld’; did you mean ‘vec_vsel’? [-Werror=implicit-function-declaration]
  123 |     vec_vsx_ld(offset, b)
      |     ^~~~~~~~~~
...

Expected Results:  
It should build and install.

Fixing this particular problem can be done by filtering out the -flto* flags like below.  (The --enable-lto option adds back a plain -flto later, without the jobserver setting.)  If that's not doable, then "use ppc && local -x bigendian=yes" also fixes it.

--- media-video/ffmpeg/ffmpeg-4.3.1.ebuild
+++ media-video/ffmpeg/ffmpeg-4.3.1.ebuild
@@ -426,6 +426,7 @@
 
 	# LTO support, bug #566282
 	is-flagq "-flto*" && myconf+=( "--enable-lto" )
+	filter-flags "-flto*"
 
 	# Mandatory configuration
 	myconf=(

[Comment 1 Larry the Git Cow 2021-03-22 10:07:00 UTC]

The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=4e7e25bb617a8202fb1a0f0381949dcd380de614

commit 4e7e25bb617a8202fb1a0f0381949dcd380de614
Author:     David Michael <[e-mail]>
AuthorDate: 2021-03-22 10:06:25 +0000
Commit:     David Seifert <[e-mail]>
CommitDate: 2021-03-22 10:06:25 +0000

    media-video/ffmpeg: work around cross-endian tests failing
    
    Cross-compiling with LTO prevents the build system from detecting
    endianness.  Upstream says that enabling LTO with CFLAGS is not
    supported, so filter -flto* out of CFLAGS after adding --enable-lto
    to the configure command (which just adds back -flto after endian
    tests pass).
    
    Closes: https://bugs.gentoo.org/754654
    Package-Manager: Portage-3.0.13, Repoman-3.0.2
    Signed-off-by: David Michael <[e-mail]>
    Signed-off-by: David Seifert <[e-mail]>

 media-video/ffmpeg/ffmpeg-4.3.2.ebuild | 3 ++-
 media-video/ffmpeg/ffmpeg-9999.ebuild  | 5 +++--
 2 files changed, 5 insertions(+), 3 deletions(-)