Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 752507 - media-video/ffmpeg-4.3.1 compilation fails without mmxext cpuflag
Summary: media-video/ffmpeg-4.3.1 compilation fails without mmxext cpuflag
Status: UNCONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: x86 Linux
: Normal normal (vote)
Assignee: Gentoo Media-video project
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-11-03 13:08 UTC by Maiken Akka
Modified: 2021-11-26 11:20 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build.log (build.log.xz,29.04 KB, application/x-xz)
2020-11-03 13:08 UTC, Maiken Akka
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Maiken Akka 2020-11-03 13:08:29 UTC
Created attachment 669884 [details]
build.log

Emerging fails on an ancient computer without mmxext.

Portage 3.0.8 (python 3.7.9-final-0, default/linux/x86/17.0, gcc-9.3.0, glibc-2.32-r2, 5.4.66-gentoo i686)
=================================================================
                         System Settings
=================================================================
System uname: Linux-5.4.66-gentoo-i686-Pentium_II_-Deschutes-with-gentoo-2.7
KiB Mem:      374316 total,    193404 free
KiB Swap:   16777212 total,  16774140 free
Timestamp of repository gentoo: Sun, 01 Nov 2020 19:30:01 +0000
Head commit of repository gentoo: c76141a9ed378795deac9e8fdd8f9ff5e6b1b91b
sh bash 5.0_p18
ld GNU ld (Gentoo 2.33.1 p2) 2.33.1
ccache version 3.7.11 [enabled]
app-shells/bash:          5.0_p18::gentoo
dev-lang/perl:            5.30.3::gentoo
dev-lang/python:          2.7.18-r4::gentoo, 3.7.9::gentoo, 3.8.5::gentoo
dev-util/ccache:          3.7.11::gentoo
dev-util/cmake:           3.17.4-r1::gentoo
sys-apps/baselayout:      2.7::gentoo
sys-apps/openrc:          0.42.1::gentoo
sys-apps/sandbox:         2.20::gentoo
sys-devel/autoconf:       2.69-r5::gentoo
sys-devel/automake:       1.16.1-r1::gentoo
sys-devel/binutils:       2.33.1-r1::gentoo, 2.34-r2::gentoo
sys-devel/gcc:            9.3.0-r1::gentoo
sys-devel/gcc-config:     2.3.2::gentoo
sys-devel/libtool:        2.4.6-r6::gentoo
sys-devel/make:           4.2.1-r4::gentoo
sys-kernel/linux-headers: 5.4-r1::gentoo (virtual/os-headers)
sys-libs/glibc:           2.32-r2::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000
    sync-rsync-verify-max-age: 24
    sync-rsync-verify-metamanifest: yes
    sync-rsync-extra-opts: 
    sync-rsync-verify-jobs: 1

ACCEPT_KEYWORDS="x86"
ACCEPT_LICENSE="* -@EULA"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=native -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -march=native -pipe -fomit-frame-pointer"
DISTDIR="/var/cache/distfiles"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2 -march=native -pipe -fomit-frame-pointer"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs ccache config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -march=native -pipe -fomit-frame-pointer"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="C.UTF8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j2"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="acl alsa berkdb bzip2 cdda cdio cdrom cli crypt dri dvd emacs fbcon fortran gdbm gpm iconv jpeg libcaca libglvnd libtirpc mp3 ncurses nls nptl openmp pam pcre png readline seccomp split-usr ssl tcpd unicode vorbis x86 xattr zlib" ABI_X86="32" ADA_TARGET="gnat_2018" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-2 php7-3 php7-4" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_7" PYTHON_TARGETS="python2_7 python3_7" RUBY_TARGETS="ruby25 ruby26" USERLAND="GNU" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 1 Maiken Akka 2020-11-14 19:04:16 UTC
It was a bug in ffmpeg.

https://trac.ffmpeg.org/ticket/8986

The ffmpeg-developer was not a fan of the Gentoo generated configure options. I'm in over my head as it is. Check the link if you are interested.
Comment 2 Tiago Marques 2021-11-17 18:03:17 UTC
AFAIK, this is an ebuild bug:

X86_CPU_REQUIRED_USE="
	cpu_flags_x86_avx2? ( cpu_flags_x86_avx )
	cpu_flags_x86_fma4? ( cpu_flags_x86_avx )
	cpu_flags_x86_fma3? ( cpu_flags_x86_avx )
	cpu_flags_x86_xop?  ( cpu_flags_x86_avx )
	cpu_flags_x86_avx?  ( cpu_flags_x86_sse4_2 )
	cpu_flags_x86_aes? ( cpu_flags_x86_sse4_2 )
	cpu_flags_x86_sse4_2?  ( cpu_flags_x86_sse4_1 )
	cpu_flags_x86_sse4_1?  ( cpu_flags_x86_ssse3 )
	cpu_flags_x86_ssse3?  ( cpu_flags_x86_sse3 )
	cpu_flags_x86_sse3?  ( cpu_flags_x86_sse2 )
	cpu_flags_x86_sse2?  ( cpu_flags_x86_sse )
	cpu_flags_x86_sse?  ( cpu_flags_x86_mmxext )
	cpu_flags_x86_mmxext?  ( cpu_flags_x86_mmx )
	cpu_flags_x86_3dnowext?  ( cpu_flags_x86_3dnow )
	cpu_flags_x86_3dnow?  ( cpu_flags_x86_mmx )

There are a bunch of assumptions here which are wrong.

SSE does not require MMXext. In fact, MMXext is deprecated in recent chips like Apollo Lake/Goldmont.

FMA4 is AMD specific (Buldozzer) and also independent of AVX, though current architectures that support it will support AVX. It is deprecated in ZEN cores and later.

AES should not go with SSE4.2 (Nehalem+), as there are Pentium/Celeron from Core 3rd gen which support it but not AESNI.

I don't think a package should be doing this kind of checks on a user set variable. At most, it should filter down configurations ffmpeg itself doesn't support.
I haven't looked at the code but would be very surprised if MMXext is tied to SSE (and conversely up to SSE4.2).
Comment 3 Alexis Ballier gentoo-dev 2021-11-17 18:47:53 UTC
(In reply to Tiago Marques from comment #2)

> FMA4 is AMD specific (Buldozzer) and also independent of AVX, though current
> architectures that support it will support AVX. It is deprecated in ZEN
> cores and later.

It seems to me you are reading the constraints the opposite way. What should be read is "ffmpeg assumes an FMA4 capable cpu will have AVX" as you noted here, but the other comments are confusing to me.
Comment 4 Tiago Marques 2021-11-20 10:21:59 UTC
I think I'm reading it right but indeed FMA4 is really what doesn't matter.
AES is also a mistake on my part, as it is not a dependency, so it's not an issue.

The problem here is that SSE is requiring MMXext. You have recent cores without MMXext and only MMX. These cores support various versions of SSE and an example is Intel Goldmont and more recent. So the line:

	cpu_flags_x86_sse?  ( cpu_flags_x86_mmxext )

Is not valid nowadays. This causes a recent core not being able to enable SSE and later instructions.
Comment 5 Alexis Ballier gentoo-dev 2021-11-23 16:56:32 UTC
(In reply to Tiago Marques from comment #4)
> The problem here is that SSE is requiring MMXext. You have recent cores
> without MMXext and only MMX. These cores support various versions of SSE and
> an example is Intel Goldmont and more recent. So the line:
> 
> 	cpu_flags_x86_sse?  ( cpu_flags_x86_mmxext )


 + + cpu_flags_x86_mmxext   : Use the Extended MMX instruction set (a subset of SSE) ([mmxext] or [sse] in cpuinfo)


?
Comment 6 Tiago Marques 2021-11-24 09:12:18 UTC
(In reply to Alexis Ballier from comment #5)
> (In reply to Tiago Marques from comment #4)
> > The problem here is that SSE is requiring MMXext. You have recent cores
> > without MMXext and only MMX. These cores support various versions of SSE and
> > an example is Intel Goldmont and more recent. So the line:
> > 
> > 	cpu_flags_x86_sse?  ( cpu_flags_x86_mmxext )
> 
> 
>  + + cpu_flags_x86_mmxext   : Use the Extended MMX instruction set (a subset
> of SSE) ([mmxext] or [sse] in cpuinfo)
> 
> 
> ?

Ok. I looked at it and while some sources say the full MMXext isn't in SSE, it seems to be included.

Is it possible to remove it to follow CPUID flags correctly? As it is it will not compile on recent archs that don't announce MMXext and we auto create CPU_FLAGS_X86.
I imagine this bug will keep popping up.
Comment 7 Alexis Ballier gentoo-dev 2021-11-26 11:20:37 UTC
(In reply to Tiago Marques from comment #6)
> (In reply to Alexis Ballier from comment #5)
> > (In reply to Tiago Marques from comment #4)
> > > The problem here is that SSE is requiring MMXext. You have recent cores
> > > without MMXext and only MMX. These cores support various versions of SSE and
> > > an example is Intel Goldmont and more recent. So the line:
> > > 
> > > 	cpu_flags_x86_sse?  ( cpu_flags_x86_mmxext )
> > 
> > 
> >  + + cpu_flags_x86_mmxext   : Use the Extended MMX instruction set (a subset
> > of SSE) ([mmxext] or [sse] in cpuinfo)
> > 
> > 
> > ?
> 
> Ok. I looked at it and while some sources say the full MMXext isn't in SSE,
> it seems to be included.
> 
> Is it possible to remove it to follow CPUID flags correctly? As it is it
> will not compile on recent archs that don't announce MMXext and we auto
> create CPU_FLAGS_X86.
> I imagine this bug will keep popping up.

cpuid2cpuflags handles it properly here

$ cpuid2cpuflags
CPU_FLAGS_X86: aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sse sse2 sse3 sse4_1 sse4_2 ssse3

flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d