Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 739776 - >=media-libs/kvazaar-2.0.0 does not set -mlzcnt when compiling the avx2 version of the library
Summary: >=media-libs/kvazaar-2.0.0 does not set -mlzcnt when compiling the avx2 versi...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: High major (vote)
Assignee: Gentoo Media-video project
URL:
Whiteboard:
Keywords:
: 789267 (view as bug list)
Depends on:
Blocks:
 
Reported: 2020-08-31 19:01 UTC by Chris Smith
Modified: 2022-10-13 00:48 UTC (History)
8 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
kvazaar-2.0.0 build log (kvazaar-2.0.0.log,86.79 KB, text/plain)
2020-08-31 20:23 UTC, Chris Smith
Details
Add -mlzcnt to the GCC flags for AVX2 builds (kvazaar-fix-mlzcnt.patch,433 bytes, patch)
2021-05-16 17:03 UTC, Francisco Blas Izquierdo Riera
Details | Diff
git am able patch against gentoo.git (0001-media-libs-kvazaar-fix-AVX2-flags-on-GCC.patch,3.64 KB, patch)
2022-10-04 09:27 UTC, Francisco Blas Izquierdo Riera
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Chris Smith 2020-08-31 19:01:41 UTC
In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/include/immintrin.h:105,
                 from /var/tmp/portage/media-libs/kvazaar-2.0.0/work/kvazaar-2.0.0/src/strategies/avx2/quant-avx2.c:28:
/var/tmp/portage/media-libs/kvazaar-2.0.0/work/kvazaar-2.0.0/src/strategies/avx2/avx2_common_functions.h: In function ‘get_first_last_nz_int16’:
/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/include/lzcntintrin.h:51:1: error: inlining failed in call to ‘always_inline’ ‘_lzcnt_u32’: target specific option mismatch
   51 | _lzcnt_u32 (unsigned int __X)
      | ^~~~~~~~~~
In file included from /var/tmp/portage/media-libs/kvazaar-2.0.0/work/kvazaar-2.0.0/src/strategies/avx2/quant-avx2.c:31:
/var/tmp/portage/media-libs/kvazaar-2.0.0/work/kvazaar-2.0.0/src/strategies/avx2/avx2_common_functions.h:135:26: note: called from here
  135 |   *last = (31 - (int32_t)_lzcnt_u32(nonzero_bytes)) >> 1;
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/include/immintrin.h:105,
                 from /var/tmp/portage/media-libs/kvazaar-2.0.0/work/kvazaar-2.0.0/src/strategies/avx2/quant-avx2.c:28:
/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/include/lzcntintrin.h:51:1: error: inlining failed in call to ‘always_inline’ ‘_lzcnt_u32’: target specific option mismatch
   51 | _lzcnt_u32 (unsigned int __X)
      | ^~~~~~~~~~
In file included from /var/tmp/portage/media-libs/kvazaar-2.0.0/work/kvazaar-2.0.0/src/strategies/avx2/quant-avx2.c:31:
/var/tmp/portage/media-libs/kvazaar-2.0.0/work/kvazaar-2.0.0/src/strategies/avx2/avx2_common_functions.h:135:26: note: called from here
  135 |   *last = (31 - (int32_t)_lzcnt_u32(nonzero_bytes)) >> 1;
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~
make[2]: *** [Makefile:1166: strategies/avx2/libavx2_la-quant-avx2.lo] Error 1
make[2]: Leaving directory '/var/tmp/portage/media-libs/kvazaar-2.0.0/work/kvazaar-2.0.0-abi_x86_64.amd64/src'
make[1]: *** [Makefile:525: all-recursive] Error 1
make[1]: Leaving directory '/var/tmp/portage/media-libs/kvazaar-2.0.0/work/kvazaar-2.0.0-abi_x86_64.amd64'
make: *** [Makefile:395: all] Error 2
Comment 1 Jonas Stein gentoo-dev 2020-08-31 19:57:00 UTC
Thank you for the report. We need to have all information at hand before ticket assignment. That is why I ask you to 
* attach the logs and 
* paste the emerge info 
as described on https://wiki.gentoo.org/wiki/Attach_the_logs_to_the_bug_ticket
Please reopen this ticket (Status:UNCONFIRMED) afterwards.
Comment 2 Chris Smith 2020-08-31 20:22:03 UTC
emerge --info '=media-libs/kvazaar-2.0.0::gentoo'
Portage 3.0.5 (python 3.8.5-final-0, default/linux/amd64/17.1/desktop/plasma/systemd, gcc-10.2.0, glibc-2.32-r1, 5.7.19-gentoo x86_64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-5.7.19-gentoo-x86_64-Intel-R-_Core-TM-_i7-3770K_CPU_@_3.50GHz-with-glibc2.2.5
KiB Mem:    32837200 total,  11074684 free                                                                                                             KiB Swap:    6291452 total,   6290416 free                                                                                                             
Timestamp of repository gentoo: Mon, 31 Aug 2020 20:05:30 +0000
Head commit of repository gentoo: e8cc3718017815ef51c6351f8d922f6b451daf60                                                                                                                                                                                                                                    
Head commit of repository audio-overlay: 8de904174dd638feb3fc6f0cdc3272680a65bb32                                                                                                                                                                                                                             
sh bash 5.0_p18                                                                                                                                        
ld GNU ld (Gentoo 2.34 p6) 2.34.0                                                                                                                      app-shells/bash:          5.0_p18::gentoo                                                                                                              dev-java/java-config:     2.3.1::gentoo                                                                                                                dev-lang/perl:            5.30.3-r1::gentoo                                                                                                            
dev-lang/python:          2.7.18-r1::gentoo, 3.7.9::gentoo, 3.8.5::gentoo, 3.9.0_rc1::gentoo                                                           dev-util/cmake:           3.18.1::gentoo                                                                                                               dev-util/pkgconfig:       0.29.2::gentoo                                                                                                               
sys-apps/baselayout:      2.7::gentoo                                                                                                                  
sys-apps/sandbox:         2.20::gentoo                                                                                                                 
sys-devel/autoconf:       2.13-r1::gentoo, 2.69-r5::gentoo                                                                                             
sys-devel/automake:       1.16.2::gentoo                                                                                                               sys-devel/binutils:       2.34-r2::gentoo                                                                                                              
sys-devel/gcc:            10.2.0-r1::gentoo                                                                                                            
sys-devel/gcc-config:     2.3.1::gentoo                                                                                                                sys-devel/libtool:        2.4.6-r6::gentoo                                                                                                             sys-devel/make:           4.3::gentoo                                                                                                                  
sys-kernel/linux-headers: 5.8::gentoo (virtual/os-headers)
sys-libs/glibc:           2.32-r1::gentoo                                                                                                              Repositories:

gentoo
    location: /usr/portage
    sync-type: git
    sync-uri: https://anongit.gentoo.org/git/repo/sync/gentoo.git
    priority: -1000

audio-overlay
    location: /usr/local/portage/audio-overlay
    sync-type: git
    sync-uri: https://github.com/gentoo-audio/audio-overlay.git
    masters: gentoo

tcgOverlay
    location: /usr/local/portage
    masters: gentoo

ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CC="gcc"
CFLAGS="-O2 -march=native -fomit-frame-pointer -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sand
box.d /etc/terminfo"
CXX="g++"
CXXFLAGS="-O2 -march=native -fomit-frame-pointer -pipe"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--with-bdeps y --autounmask-keep-masks"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE
_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs clean-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandb
ox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms str
ict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://gentoo.osuosl.org/"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O2 -Wl,--as-needed"
LINGUAS="en_US en"
MAKEOPTS="V=1 -j1"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable
 --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="X a52 aac aalib acl acpi activities aes alsa amd64 amr apache2 audiofile avx bash-completion berkdb bluray branding btrfs bzip2 cairo caps cdaudio
 cdda cddb cdparanoia cdr cdrdao chm chromecast cleartype cli corefonts crypt cups curl dbus declarative djvu dri dssi dts dv dvb dvd dvdr ebook egl em
boss encode exif fam fdk ffmpeg flac fortran gdbm gif gles2 gmp google gphoto2 gpm gstreamer gtk gtk3 gui iconv icu imagemagick imap imlib introspectio
n ipv6 jack java jpeg jpeg2k kde kipi kpathsea kvm kwallet ladspa lame lash lcms libffi libglvnd libinput libnotify libtirpc linguas_en_US lv2 lzma mad
 marble midi mmap mmx mmxext mng mozilla mp3 mp4 mpeg mtp musicbrainz ncurses network nls nodrm nptl nsplugin nvenc nvidia ofx ogg opencl openexr openg
l openmp opus pam pango pcre pdf phonon plasma png policykit popcnt ppds pulseaudio qemu qml qt5 readline rtmp sasl scanner sdl seccomp semantic-deskto
p slp smp sndfile speex spell split-usr sse sse2 sse3 sse4_1 sse4_2 ssl ssse3 startup-notification subversion svg systemd theora threads tiff tk transp
arency truetype type1 udev udisks unicode unwind upower usb v4l vaapi vcd vdpau vim-syntax vorbis vpx vst wav wavpack wayland webp widgets wmf wxwidget
s x264 x265 xattr xcb xcomposite xine xkb xml xscreensaver xv xvid xvmc zlib" ABI_X86="64" ADA_TARGET="gnat_2018" ALSA_CARDS="hda-intel hdsp hpet hrtim
er rtctimer seqdummy virmidi" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm auth
n_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir
 disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speli
ng status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="author braindump flow karbon kexi krita sheets stage words" CAMERAS="*" COLLECTD_
PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GP
SD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore r
tcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64 pc" INPUT_DEVICES="libinput wacom" KERNEL="linux"
 LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" NGI
NX_MODULES_HTTP="autoindex fancyindex fastcgi gzip" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-2 php7-3" POSTGRES_TARGETS="postgres10 postgr
es11" PYTHON_SINGLE_TARGET="python3_8" PYTHON_TARGETS="(-python2_7) (-python3_6) python3_7 python3_8" QEMU_SOFTMMU_TARGETS="x86_64" QEMU_USER_TARGETS="
x86_64" RUBY_TARGETS="(-ruby23) (-ruby24) (-ruby25) (-ruby26) ruby27" SANE_BACKENDS="epson epson2" USERLAND="GNU" VIDEO_CARDS="nvidia nouveau" XTABLES_
ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude
 chaos account"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTR
A_OPTS
Comment 3 Chris Smith 2020-08-31 20:23:05 UTC
Created attachment 657662 [details]
kvazaar-2.0.0 build log
Comment 4 Chris Hall 2020-09-02 08:56:46 UTC
This seems to compile ok if I add -mlzcnt to $CFLAGS.  It also seems to run ok – though I've no idea if I've actually been exercising the code path in question with my minimal testing.

But if -mlzcnt isn't gcc's default, then I assume this is only safe to add on certain CPUs; I'm not sure which cpu flag would need to be checked for.
Comment 5 Chris Smith 2020-10-30 19:35:05 UTC
As this bug doesn't seem to be getting fixed I tried another workaround - using clang instead of gcc. It did compile fine (hopefully it works as well).

My clang env:
CHOST="x86_64-pc-linux-gnu"
CFLAGS="-O2 -march=native -fomit-frame-pointer -pipe"
CXXFLAGS="${CFLAGS}"
LDFLAGS="-Wl,-O2 -Wl,--as-needed"
CC="clang"
CXX="clang++"
AR="llvm-ar"
NM="llvm-nm"
RANLIB="llvm-ranlib"
Comment 6 Ionen Wolkens gentoo-dev 2021-05-10 06:53:43 UTC
*** Bug 789267 has been marked as a duplicate of this bug. ***
Comment 7 Bob Johnson 2021-05-12 15:32:18 UTC
It seems to be the avx2 cpu flag that is needed for this to compile. kvazaar-2.0.0 builds fine on my Ryzen 9 with avx2, but not on my Core i5 with avx. Mentioned by skeevy420 on the Arch package forum:

https://aur.archlinux.org/packages/kvazaar-git/

That references a bug report on the kvazaar github that was closed because no further information was given:

https://github.com/ultravideo/kvazaar/issues/228

This is blocking the now-stable ffmpeg-4.4 on older systems.
Comment 8 Tamas Jantvik 2021-05-14 09:02:57 UTC
Compiling with clang worked great on my x86_64 Intel(R) Core(TM) i3-3217U
Comment 9 Francisco Blas Izquierdo Riera gentoo-dev 2021-05-16 17:03:26 UTC
Created attachment 709161 [details, diff]
Add -mlzcnt to the GCC flags for AVX2 builds

The issue appears because the Makefile.am assumes that -mabm entails -mlzcnt but it does not. The attached patch addresses that.
Comment 10 Francisco Blas Izquierdo Riera gentoo-dev 2021-05-16 17:22:27 UTC
Also for documentation purposes, here is what is going on https://github.com/gcc-mirror/gcc/blob/16e2427f50c208dfe07d07f18009969502c25dc8/gcc/config/i386/i386-options.c#L2144 we may also need to add -mpopcnt too.
Comment 11 Francisco Blas Izquierdo Riera gentoo-dev 2021-05-16 18:40:07 UTC
Also for documentation purposes, here is what is going on https://github.com/gcc-mirror/gcc/blob/16e2427f50c208dfe07d07f18009969502c25dc8/gcc/config/i386/i386-options.c#L2144 we
Comment 12 Anthony Lieuallen 2022-02-24 21:05:15 UTC
I've been running into this with each ffmpeg release since 4.4 was stabilized.  To work around this I've (following https://wiki.gentoo.org/wiki/etc/portage/package.env and above comments) just:

created /etc/portage/env/lzcnt.conf file, contents:

CFLAGS="${CFLAGS} -mlzcnt"

created /etc/portage/package.env/kvazaar file, contents:

=media-libs/kvazaar-2.0.0 lzcnt.conf

Tried again.  Now it builds (and so does ffmpeg 4.4).  (Hopefully this is useful for someone else also waiting for the patch from last May to be added.  I have no idea if this is correct, but it builds.)
Comment 13 Anthony Lieuallen 2022-09-22 14:52:22 UTC
My earlier comment was true several months ago, but now (as of version 2.1.0) kvazaar fails to build, again.  Same error message ("error: inlining failed in call to 'always_inline' '_lzcnt_u32': target specific option mismatch") as above.
Comment 14 Francisco Blas Izquierdo Riera gentoo-dev 2022-10-04 07:28:07 UTC
This issue is still present on media-libs/kvazaar-2.1.0 the patch I submitted more than a year ago still fixes it.
Comment 15 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-10-04 07:32:13 UTC
(In reply to Francisco Blas Izquierdo Riera from comment #14)
> This issue is still present on media-libs/kvazaar-2.1.0 the patch I
> submitted more than a year ago still fixes it.

Please attach a git am-able patch to the gentoo.git repository.
Comment 16 Francisco Blas Izquierdo Riera gentoo-dev 2022-10-04 09:27:43 UTC
Created attachment 818485 [details, diff]
git am able patch against gentoo.git

Hi Sam!

Here is the patch, tell me if you have any issues with it.
Comment 17 Anthony Lieuallen 2022-10-05 17:56:27 UTC
Correction to my post from the 22nd, and as per immediately above comments:  The above posted fix does work, but I put "=...-2.0.0" in my configuration, so I thought earlier that it didn't fix 2.1.0, but now see/agree that it does.
Comment 18 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-10-13 00:46:58 UTC
Nice work, thanks!
Comment 19 Larry the Git Cow gentoo-dev 2022-10-13 00:48:43 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ef4eaa166e63164d648ec511c445f88b1c05d223

commit ef4eaa166e63164d648ec511c445f88b1c05d223
Author:     Francisco Blas (klondike) Izquierdo Riera <klondike@gentoo.org>
AuthorDate: 2022-10-04 09:17:25 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-10-13 00:47:59 +0000

    media-libs/kvazaar: fix AVX2 flags on GCC
    
    On older CPUS without support for popcnt or lzcnt compilation of
    kvazaar fails due to these instructions being used by the AVX2
    code but the relevant flags to support them not being set by
    the Makefile.
    
    Upstream fixed this in commit
    99423a825027c862bbfd50e112a1846a013b01a7
    
    Backport this patch so compilation works again on systems
    without CPU support for these instructions.
    
    Closes: https://bugs.gentoo.org/739776
    Bug: https://github.com/ultravideo/kvazaar/pull/301
    Signed-off-by: Francisco Blas (klondike) Izquierdo Riera <klondike@gentoo.org>
    Signed-off-by: Sam James <sam@gentoo.org>

 .../files/kvazaar-2.0.0-fix-avx2-flags.patch       | 33 ++++++++++++++++++++++
 media-libs/kvazaar/kvazaar-2.0.0.ebuild            |  6 +++-
 media-libs/kvazaar/kvazaar-2.1.0.ebuild            |  4 +++
 3 files changed, 42 insertions(+), 1 deletion(-)