Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 547194 - media-libs/libvpx build fails on NEON capable ARM if -mfpu is in CFLAGS
Summary: media-libs/libvpx build fails on NEON capable ARM if -mfpu is in CFLAGS
Status: UNCONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Library (show other bugs)
Hardware: ARM Linux
: Normal normal (vote)
Assignee: media-video herd
URL:
Whiteboard:
Keywords:
: 665782 (view as bug list)
Depends on:
Blocks:
 
Reported: 2015-04-20 16:16 UTC by John Bowler
Modified: 2020-10-02 20:40 UTC (History)
10 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge --info (emerge.info,5.06 KB, text/plain)
2015-04-20 16:20 UTC, John Bowler
Details
make.conf (make.conf,1.33 KB, text/plain)
2015-04-20 16:22 UTC, John Bowler
Details
emerge --info '=media-libs/libvpx-1.4.0::gentoo' (emerge.info.full,5.62 KB, text/plain)
2015-04-20 16:25 UTC, John Bowler
Details
emerge -pqv '=media-libs/libvpx-1.4.0::gentoo' (emerge.pqv,103 bytes, text/plain)
2015-04-20 16:26 UTC, John Bowler
Details
build.log (build.log,292.68 KB, text/plain)
2015-04-20 16:27 UTC, John Bowler
Details
environment (environment,90.90 KB, text/plain)
2015-04-20 16:28 UTC, John Bowler
Details
libvpx-1.4.0.ebuild.patch (libvpx-1.4.0.ebuild.patch,1.56 KB, patch)
2015-05-19 18:03 UTC, Alexander Tsoy
Details | Diff
libvpx-1.4.0.ebuild-V2.patch (libvpx-1.4.0.ebuild-V2.patch,1.45 KB, patch)
2015-05-27 07:03 UTC, Alexander Tsoy
Details | Diff
libvpx-1.4.0.ebuild.patch (libvpx-1.4.0.ebuild-V3.patch,1.57 KB, patch)
2015-05-27 07:23 UTC, Alexander Tsoy
Details | Diff
libvpx-1.4.0.ebuild.patch (libvpx-1.4.0.ebuild.patch,1.19 KB, patch)
2015-05-27 13:13 UTC, Alexander Tsoy
Details | Diff
emerge --info media-libs/libvpx (libvpx-einfo,5.12 KB, text/plain)
2015-08-21 15:26 UTC, David J Cozatt
Details

Note You need to log in before you can comment on or make changes to this bug.
Description John Bowler 2015-04-20 16:16:52 UTC
libvpx build fails with the particular system configuration I have.
Apparently the neon intrinsics code is being enabled even though my build (make.conf) puts -mfpu=vfpv4 into CFLAGS; this results in a gcc call of the general form:

gcc ... -mfpu=neon ... -mfpu=vfpv4

followed by a (valid) #error from arm_neon.h and, in fact, GCC does fail to compile the files with NEON instructions in them.

Reproducible: Always

Steps to Reproduce:
1. On an appropriate armv7-a build put -mfpu=vfpv4 into CFLAGS
2. emerge libvpx
3.
Actual Results:  
Many .c files fail to compile (the build log I have attached was doing a parallel make).

Expected Results:  
libvpx configure to *not* add -mfpu=neon if the emerge environment specifies a different FPU.


Alternatively there could be a "neon" use flag to disable the NEON code.

The problem would not arise if the tests for NEON inside libvpx configure included CFLAGS and "-mfpu=neon" in exactly the same order as the actual build.  If -mfpu=neon was added after CFLAGS then libvpx would force NEON even if the system build said something else, if the -mfpu=neon was included before CFLAGS in the configure tests then the system build could override the default.
Comment 1 John Bowler 2015-04-20 16:20:27 UTC
Created attachment 401682 [details]
emerge --info
Comment 2 John Bowler 2015-04-20 16:22:12 UTC
Created attachment 401684 [details]
make.conf

This is /etc/portage/make.conf; it shows the somewhat wacky -mfpu= settings I use (necessary to get distcc builds to work; my remote compilers have to know what fpu to use.)
Comment 3 John Bowler 2015-04-20 16:25:47 UTC
Created attachment 401686 [details]
emerge --info '=media-libs/libvpx-1.4.0::gentoo'

This includes the package settings, the other emerge.info attachment didn't have them.
Comment 4 John Bowler 2015-04-20 16:26:25 UTC
Created attachment 401688 [details]
emerge -pqv '=media-libs/libvpx-1.4.0::gentoo'
Comment 5 John Bowler 2015-04-20 16:27:41 UTC
Created attachment 401690 [details]
build.log
Comment 6 John Bowler 2015-04-20 16:28:23 UTC
Created attachment 401692 [details]
environment
Comment 7 John Bowler 2015-04-20 16:54:26 UTC
I can confirm that simply removing the system architecture settings from CFLAGS (and CXXFLAGS) hides the problem; the build completes.
Comment 8 Alexander Tsoy 2015-05-19 18:03:37 UTC
Created attachment 403628 [details, diff]
libvpx-1.4.0.ebuild.patch

Could you test this patch with USE=neon and USE=-neon?

And I'm not sure if filter-flags() in multilib_src_configure() is a good idea. :) Probably something like pkg_setup() is a better place.
Comment 9 Alexander Tsoy 2015-05-19 18:57:38 UTC
Comment on attachment 403628 [details, diff]
libvpx-1.4.0.ebuild.patch

>+	use neon && filter-flags "-mfpu=*" #547194

The foolowing should also work:

use neon & append-cflags -mfpu=neon
Comment 10 John Bowler 2015-05-19 21:18:38 UTC
I think that is a minimal ebuild level work-round, but the NEON mess is actually quite complicated:

https://gcc.gnu.org/onlinedocs/gcc/ARM-Options.html

Adding -mfpu=*neon* is a requirement of armv7 and armv8 to cause gcc to recognize the intrinsics, however with aarch64 it produces an 'unrecognized command line option', even though aarch64 gcc always recognizes the NEON intrinsics.

So any of these -mfpu options with 32-bit systems (should) turn on neon intrinsics:

‘neon’, ‘neon-fp16’, ‘fp-armv8’, ‘neon-fp-armv8’, and ‘crypto-neon-fp-armv8’

But none are necessary with aarch64.

I still earnestly believe that ebuilds should not mess with the compiler arch-specific flags; take a look at the way I enabled the optimizations in libpng (latest versions of 1.6).  There are all sorts of reasons for choosing different FPU options on ARM systems; there's no one right answer.

One thing about all this is that there seems to be an upstream bug in the libvpx source code.  It is, apparently, checking the wrong #defines; it should compile fine with any string of -mfpu and, effectively, just use the last one.  It seems that the authors didn't read the official documentation:

http://infocenter.arm.com/help/topic/com.arm.doc.ihi0053c/IHI0053C_acle_2_0.pdf

I believe GCC does conform to that specification, in particular it supports the requirements for source code in section 4.4 by defining __ARM_NEON:

 #ifdef __ARM_NEON
 #include <arm_neon.h>
 #endif /* __ARM_NEON */

(Note that the __ARM_NEON__ version is deprecated and doesn't work with AArch64.)  So the source code should look for __ARM_NEON, e.g. with a configure compile test on:

#ifdef __ARM_NEON
#error ARM NEON is on
#endif

(I.e. if the compile fails the build script should include anything needed for NEON.)
Comment 11 Alexander Tsoy 2015-05-27 00:49:39 UTC
(In reply to John Bowler from comment #10)
> I think that is a minimal ebuild level work-round

Yes, of course. I think the situation is similar to pulseaudio:
http://lists.freedesktop.org/archives/pulseaudio-discuss/2013-March/016753.html

They even added a configure check to fail the build as early as possible. :) Quote:

"this does not fix
http://lists.freedesktop.org/archives/pulseaudio-discuss/2012-December/015570.html
but at least makes the build fail in configure (and not while compiling stuff)
and gives better diagnostics"
Comment 12 Mike Gilbert gentoo-dev 2015-05-27 01:46:49 UTC
I think adding $(use_enable neon) to the econf call should suffice here. I would prefer that to messing with CFLAGS.

Can someone test that?
Comment 13 Alexander Tsoy 2015-05-27 06:33:02 UTC
There is no difference between neon enabled automagically and neon enabled via --enable-neon. :) BTW, I found the culprit:

https://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/media-libs/libvpx/libvpx-1.1.0.ebuild?r1=1.3&r2=1.4

Due to this change system CFLAGS are appended to CFLAGS variable used by the build system (note "... -mfpu=vfpv3 ... -mfpu=neon ... -mfpu=vfpv3 ..."):

# grep ^CFLAGS /var/tmp/portage/media-libs/libvpx-1.4.0/work/libvpx-1.4.0-.arm/libs-armv7-linux-gcc.mk 
CFLAGS  = -O2 -march=armv7-a -mtune=cortex-a8 -mfpu=vfpv3 -mfloat-abi=hard -pipe -march=armv7-a -mfloat-abi=hard -mfpu=neon -DNDEBUG -O3 -fPIC -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -march=armv7-a -mtune=cortex-a8 -mfpu=vfpv3 -mfloat-abi=hard -pipe -Wall -Wdeclaration-after-statement -Wdisabled-optimization -Wpointer-arith -Wtype-limits -Wcast-qual -Wvla -Wimplicit-function-declaration -Wuninitialized -Wunused-variable -Wunused-but-set-variable -Wno-unused-function


(In reply to Alexander Tsoy from comment #11)
...
> Yes, of course. I think the situation is similar to pulseaudio:
> http://lists.freedesktop.org/archives/pulseaudio-discuss/2013-March/016753.
> html

Ignore my comment about pulseaudio. pulseaudio has a broken build system, but this bug seems to be a packaging issue.
Comment 14 Alexander Tsoy 2015-05-27 07:03:25 UTC
Created attachment 404050 [details, diff]
libvpx-1.4.0.ebuild-V2.patch

- add neon USE flag (otherwise it is enabled automagically - very bad for binary packages)
- drop --extra-cflags as it breaks build with USE=neon

Tested with USE=neon and USE=-neon. The only problem is "-O3" optimization which is not overrided by CFLAGS.
Comment 15 Alexander Tsoy 2015-05-27 07:14:43 UTC
(In reply to Alexander Tsoy from comment #14)
...
> The only problem is "-O3" optimization which is not overrided by CFLAGS.

How about --disable-optimizations ?
Comment 16 Alexander Tsoy 2015-05-27 07:23:55 UTC
Created attachment 404054 [details, diff]
libvpx-1.4.0.ebuild.patch

- do not override global optimization options
Comment 17 Alexander Tsoy 2015-05-27 13:13:57 UTC
Created attachment 404092 [details, diff]
libvpx-1.4.0.ebuild.patch

flag-o-matic is not needed
Comment 18 David J Cozatt 2015-08-21 15:23:46 UTC
quoting from comment #10

"I still earnestly believe that ebuilds should not mess with the compiler arch-specific flags"

I whole heartedly agree...for instance if a user wishes to enable -fno-strict-aliasing or -D_FORTIFY_SOURCE=1 which may be the issue for my case. Not filing a new bug because I expect that this is the result of imposing/appending CFLAGS on a package and that seems to be the issue here.

Configuring for target 'x86_64-linux-gcc'
  enabling x86_64
  enabling runtime_cpu_detect
  using yasm
  enabling use_x86inc
Requested extra CFLAGS '-march=native -O2 -pipe -Wall -D_FORTIFY_SOURCE=1' not supported by compiler

Configuration failed. This could reflect a misconfiguration of your
toolchains, improper options selected, or another problem. If you
don't see any useful error messages above, the next step is to look
at the configure error log file (config.log) to determine what
configure was trying to do when it died.
 * ERROR: media-libs/libvpx-1.4.0::gentoo failed (configure phase):
 *   (no error message)
 * 
 * Call stack:
 *     ebuild.sh, line   93:  Called src_configure
 *   environment, line 2428:  Called multilib-minimal_src_configure
 *   environment, line 1701:  Called multilib_foreach_abi 'multilib-minimal_abi_src_configure'
 *   environment, line 1908:  Called multibuild_foreach_variant '_multilib_multibuild_wrapper' 'multilib-minimal_abi_src_configure'
 *   environment, line 1632:  Called _multibuild_run '_multilib_multibuild_wrapper' 'multilib-minimal_abi_src_configure'
 *   environment, line 1630:  Called _multilib_multibuild_wrapper 'multilib-minimal_abi_src_configure'
 *   environment, line  339:  Called multilib-minimal_abi_src_configure
 *   environment, line 1695:  Called multilib_src_configure
 *   environment, line 2146:  Called die
 * The specific snippet of code:
 *       "${S}/configure" --prefix="${EPREFIX}"/usr --libdir="${EPREFIX}"/usr/$(get_libdir) --enable-pic --enable-vp8 --enable-shared --extra-cflags="${CFLAGS}" $(use_enable altivec) $(use_enable cpu_flags_x86_avx avx) $(use_enable cpu_flags_x86_avx2 avx2) $(use_enable cpu_flags_x86_mmx mmx) $(use_enable postproc) $(use cpu_flags_x86_sse2 && use_enable cpu_flags_x86_sse sse || echo --disable-sse) $(use_enable cpu_flags_x86_sse2 sse2) $(use_enable cpu_flags_x86_sse3 sse3) $(use_enable cpu_flags_x86_sse4_1 sse4_1) $(use_enable cpu_flags_x86_ssse3 ssse3) $(use_enable static-libs static) $(use_enable test unit-tests) $(use_enable threads multithread) ${myconf} || die


wile I have 

igloo david # qlist -I | wc
    662     662   55445

662 pkgs installed that say the compiler works just fine
Comment 19 David J Cozatt 2015-08-21 15:26:07 UTC
Created attachment 409766 [details]
emerge --info media-libs/libvpx
Comment 20 John Bowler 2016-07-22 20:09:40 UTC
The bug is in libvpx 1.6.0 as well.

The easy work round is:

CFLAGS= emerge --oneshot libvpx

This causes gst-plugins-vpx to be recompiled with the same CFLAGS (empty) but that does work, as does an explicit emerge --oneshot gst-plugins-vpx immediately afterward.

I don't know what happens when the FPUs are mixed in this way, for all I know the result crashes, but I don't care because I don't use libvpx.

I tried adding '-vpx' to the USE flags in make.conf; I would really like to never see libvpx again, but that flag apparently doesn't work (media-video/cheese doesn't honor it).
Comment 21 Michael Mair-Keimberger (iamnr3) 2017-10-29 19:26:34 UTC
I can confirm this with libvpx-1.6.1 too.
BTW, there is another bug which looks like a dup of this: https://bugs.gentoo.org/562024
Comment 22 hanetzer 2018-09-15 05:22:12 UTC
Experiencing the same bug on media-libs/libvpx-1.7.0.
Switching CFLAGS="-O2 -pipe -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=hard"
to "-O2 -march=armv7-a -mfpu=neon -mfloat-abi=hard" allowed it to build.
Machine is an asus c201 4GiB chromebook, using one of the 17.0 armv7a profiles.
Comment 23 Sam James archtester gentoo-dev Security 2020-10-02 20:40:00 UTC
*** Bug 665782 has been marked as a duplicate of this bug. ***