Bug 941222 – media-libs/x264 runtime crashes with CPU_FLAGS_PPC=altivec on POWER9 ppc64le

• Product: Gentoo Linux
• Status: UNCONFIRMED ---

[Description Nick Bowler 2024-10-10 01:34:29 UTC]

Hi,

When x264 is built on ppc64le with CPU_FLAGS_PPC=altivec, encoding crashes at
runtime, at least with certain encoder settings:

  % ffmpeg -nostdin -y -f lavfi -i testsrc -c:v libx264 -preset ultrafast -threads 1 test.mkv
  [...]
  [libx264 @ 0x1000c10d0] using SAR=1/1
  [libx264 @ 0x1000c10d0] using cpu capabilities: Altivec
  [libx264 @ 0x1000c10d0] profile High 4:4:4 Predictive, level 1.3, 4:4:4, 8-bit
  [libx264 @ 0x1000c10d0] 264 - core 164 - H.264/MPEG-4 AVC codec - Copyleft 2003-2024 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=6 threads=1 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
  Output #0, matroska, to 'test.mkv':
    Metadata:
      encoder         : Lavf60.16.100
    Stream #0:0: Video: h264 (H264 / 0x34363248), yuv444p(tv, progressive), 320x240 [SAR 1:1 DAR 4:3], q=2-31, 25 fps, 1k tbn
      Metadata:
        encoder         : Lavc60.31.102 libx264
      Side data:
        cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
  ffmpeg: ../x264-4613ac3c15fd75cebc4b9f65b7fb95e70a3acce1/encoder/slicetype.c:1996: x264_8_rc_analyse_slice: Assertion `cost >= 0' failed.
  Aborted

Building x264 with CPU_FLAGS_PPC=-altivec yields a working encoder.

Portage 3.0.65 (python 3.12.6-final-0, default/linux/ppc64le/17.0, gcc-13, glibc-2.39-r6, 6.9.5 ppc64le)
=================================================================
                         System Settings
=================================================================
System uname: Linux-6.9.5-ppc64le-POWER9,_altivec_supported-with-glibc2.39
KiB Mem:    66547840 total,  25643264 free
KiB Swap:          0 total,         0 free
Timestamp of repository gentoo: Tue, 08 Oct 2024 00:45:00 +0000
Head commit of repository gentoo: 0de1abd7925a1e7ba01e7e0786caca3e5ca898f1
sh dash 0.5.12
ld GNU ld (Gentoo 2.42 p6) 2.42.0
ccache version 4.10.1 [enabled]
app-misc/pax-utils:        1.3.7::gentoo
app-shells/bash:           5.2_p37::gentoo
dev-build/autoconf:        2.13-r8::gentoo, 2.71-r7::gentoo
dev-build/automake:        1.16.5-r2::gentoo
dev-build/cmake:           3.30.2::gentoo
dev-build/libtool:         2.4.7-r4::gentoo
dev-build/make:            4.4.1-r1::gentoo
dev-build/meson:           1.5.1::gentoo
dev-lang/perl:             5.40.0::gentoo
dev-lang/python:           3.11.10_p1::gentoo, 3.12.6_p2::gentoo
dev-lang/rust:             1.80.1::gentoo
dev-util/ccache:           4.10.1::gentoo
sys-apps/baselayout:       2.15::gentoo
sys-apps/openrc:           0.54.2::gentoo
sys-apps/sandbox:          2.39::gentoo
sys-devel/binutils:        2.41-r5::gentoo, 2.42-r2::gentoo
sys-devel/binutils-config: 5.5.2::gentoo
sys-devel/clang:           16.0.6::gentoo, 17.0.6::gentoo, 18.1.8::gentoo
sys-devel/gcc:             9.5.0::gentoo, 13.3.1_p20240614::gentoo
sys-devel/gcc-config:      2.11::gentoo
sys-devel/llvm:            16.0.6::gentoo, 17.0.6::gentoo, 18.1.8-r1::gentoo
sys-kernel/linux-headers:  6.6-r1::gentoo (virtual/os-headers)
sys-libs/glibc:            2.39-r6::gentoo
Repositories:

gentoo
    location: /srv/repos/gentoo
    sync-type: rsync
    sync-uri: rsync://athena.draconx.ca/gentoo-portage
    priority: -1000
    volatile: True
    sync-rsync-verify-max-age: 3
    sync-rsync-verify-jobs: 1
    sync-rsync-extra-opts: 
    sync-rsync-verify-metamanifest: no

gentoo-draconx
    location: /srv/repos/gentoo-draconx
    masters: gentoo
    volatile: True

gentoo-fixes
    location: /srv/repos/gentoo-fixes
    masters: gentoo
    volatile: True

ACCEPT_KEYWORDS="ppc64"
ACCEPT_LICENSE="@FREE"
CBUILD="powerpc64le-unknown-linux-gnu"
CFLAGS="-O2 -pipe -mcpu=power9 -mtune=power9"
CHOST="powerpc64le-unknown-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -pipe -mcpu=power9 -mtune=power9"
DISTDIR="/var/cache/distfiles"
EMERGE_DEFAULT_OPTS="-j8 --keep-going --dynamic-deps=n --autounmask=n --autounmask-write=n --unordered-display --verbose-conflicts --binpkg-respect-use=y --with-bdeps-auto=n"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-O2 -pipe -mcpu=power9 -mtune=power9"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live ccache config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox lmirror merge-sync merge-wait mirror multilib-strict network-sandbox news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe -mcpu=power9 -mtune=power9"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
INSTALL_MASK="/usr/share/cursors/xorg-x11/default   /etc/alsa/conf.d/51-pulseaudio-probe.conf   /etc/portage/*postsync.d   /etc/gssproxy/??-*.conf"
LANG="C.UTF8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LEX="flex"
MAKEOPTS="-j32 -l40"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
SHELL="/bin/zsh"
USE="X acl bs2b bzip2 cairo cli crypt cups doc dri exif fbcon flac fortran gdbm iconv icu ipv6 jpeg jpeg2k kerberos lcms mp3 ncurses nls ogg opengl openmp opus pam pcre png ppc64 readline seccomp split-usr ssl svg test-rust threads tiff truetype unicode xattr xft xinerama zlib" ADA_TARGET="gcc_12" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_PPC="altivec vsx vsx2 vsx3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" GUILE_SINGLE_TARGET="3-0" GUILE_TARGETS="3-0" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-2" POSTGRES_TARGETS="postgres16" PYTHON_SINGLE_TARGET="python3_12" PYTHON_TARGETS="python3_12" RUBY_TARGETS="ruby32" VIDEO_CARDS="ast nouveau" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account"
Unset:  ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS

=================================================================
                        Package Settings
=================================================================

media-libs/x264-0.0.20240513::gentoo was built with the following:
USE="interlaced threads (-opencl) -static-libs" CPU_FLAGS_PPC="altivec"

[Comment 1 Nick Bowler 2024-10-10 03:21:22 UTC]

I tested a bit more:

- The crash occurs with all versions currently in the gentoo repo.

- The crash occurs only if x264 is built with -mcpu=power9 in CFLAGS.

- Even with -mcpu=power9, the crash does not occur if I remove -O2 from
  CFLAGS (causing the build to use the upstream default of -O3).

- The crash also does not occur if x264 is built with
  CFLAGS="-O2 -mcpu=power9 -mno-power9-vector".