Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 583076

Summary: media-video/x264-encoder-0.0.20151011: Configure fails on endian test with LTO
Product: Gentoo Linux Reporter: tokiclover <tokiclover>
Component: Current packagesAssignee: Gentoo Media-video project <media-video>
Status: RESOLVED OBSOLETE    
Severity: normal CC: logankaser, lynx1534
Priority: Normal Keywords: PATCH
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://bugs.gentoo.org/show_bug.cgi?id=497714
https://code.videolan.org/videolan/x264/-/merge_requests/45
https://bugs.gentoo.org/show_bug.cgi?id=853121
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 618550    
Attachments: Shell trace of configure script (with `set -x')
endian-test.patch
new endian test patch for media-video/x264-encoder-0.0.20170701

Description tokiclover 2016-05-15 09:15:32 UTC
Created attachment 434324 [details]
Shell trace of configure script (with `set -x')

I am getting this configure failure 'endian test failed' since January... but I did not have the time to report it back then. Just trying to update again and got the issue. Endian test failure? What? Attaching an extra log file when configure script has an extra `set -x' to get what's going on in there. I cannot spot anything but that the test fails. This machine has a Intel(R) Haswell CPU (little endian then).

${T}/build.log:
---
>>> Configuring source in /var/tmp/portage/media-video/x264-encoder-0.0.20151011/work/x264-snapshot-20151011-2245 ...
endian test failed
 ^[[31;01m*^[[0m ERROR: media-video/x264-encoder-0.0.20151011::gentoo failed (configure phase):
    (no error message)
 
 Call stack:
     ebuild.sh, line 133:  Called src_configure
   environment, line 2424:  Called die
 The specific snippet of code:
       ./configure --prefix="${EPREFIX}"/usr --libdir="${EPREFIX}"/usr/$(get_libdir) --system-libx264 --host="${CHOST}" --disable-lsmash $(usex 10bit "--bit-depth=10" "") $(usex avs "" "--disable-avs") $(usex ffmpeg "" "--disable-lavf --disable-swscale") $(usex ffmpegsource "" "--disable-ffms") $(usex interlaced "" "--disable-interlaced") $(usex mp4 "" "--disable-gpac") $(usex threads "" "--disable-thread") || die;

---

${S}/config.log:
---
x264 configure script
Command line options: "--prefix=/usr" "--libdir=/usr/lib64" "--system-libx264" "--host=x86_64-pc-linux-gnu" "--disable-lsmash" "

checking whether x86_64-pc-linux-gnu-gcc works... yes
checking whether x86_64-pc-linux-gnu-gcc supports for( int i = 0; i < 9; i++ ); with -std=gnu99... yes
checking for -mpreferred-stack-boundary=5... yes
checking whether yasm supports vpmovzxwd ymm0, xmm0... yes
checking whether x86_64-pc-linux-gnu-gcc supports __asm__("pabsw %xmm0, %xmm0");... yes
DIED: endian test failed
---

`emerge --info':
---
Portage 2.2.28 (python 2.7.11-final-0, hardened/linux/amd64, gcc-4.9.3, glibc-2.22-r3, 4.4.6-git x86_64)
=================================================================
System uname: Linux-4.4.6-git-x86_64-Intel-R-_Core-TM-_i7-4700HQ_CPU_@_2.40GHz-with-gentoo-2.2
KiB Mem:    16375692 total,   6061456 free
KiB Swap:     524284 total,    524284 free
Timestamp of repository gentoo: Mon, 04 Apr 2016 09:23:36 +0000
sh bash 4.4_rc1
ld GNU ld (Gentoo 2.25.1 p1.1) 2.25.1
app-shells/bash:          4.4_rc1::gentoo
dev-java/java-config:     2.2.0-r3::gentoo
dev-lang/perl:            5.22.1::gentoo
dev-lang/python:          2.7.11-r2::gentoo, 3.4.3-r7::gentoo, 3.5.1-r2::gentoo
dev-util/cmake:           3.5.1::gentoo
dev-util/pkgconfig:       0.29.1::gentoo
sys-apps/baselayout:      2.2::gentoo
sys-apps/openrc:          0.17::gentoo
sys-apps/sandbox:         2.10-r2::gentoo
sys-devel/autoconf:       2.13::gentoo, 2.69-r2::gentoo
sys-devel/automake:       1.11.6-r1::gentoo, 1.13.4::gentoo, 1.14.1-r1::gentoo, 1.15-r2::gentoo
sys-devel/binutils:       2.25.1-r1::gentoo
sys-devel/gcc:            4.9.3::gentoo, 5.3.0::gentoo
sys-devel/gcc-config:     1.8-r1::gentoo
sys-devel/libtool:        2.4.6-r2::gentoo
sys-devel/make:           4.1-r1::gentoo
sys-kernel/linux-headers: 4.4::gentoo (virtual/os-headers)
sys-libs/glibc:           2.22-r3::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: git
    sync-uri: git://github.com/gentoo-mirror/gentoo.git
    priority: -1000

bar
    location: /var/db/repos/bar
    sync-type: git
    sync-uri: git://github.com/tokiclover/bar-overlay.git
    masters: gentoo

mpd
    location: /var/db/repos/mpd
    sync-type: git
    sync-uri: git://git.musicpd.org/master/mpd-gentoo-overlay.git
    masters: gentoo

multimedia
    location: /var/db/repos/multimedia
    sync-type: git
    sync-uri: git://github.org/gentoo/multimedia.git
    masters: gentoo

x11
    location: /var/db/repos/x11
    sync-type: git
    sync-uri: git://git.overlays.gentoo.org/proj/x11.git
    masters: gentoo

Installed sets: @VapourSynth-Plugins, @X, @archive, @daw, @fonts, @gtk, @multimedia, @openbox, @portage, @sys, @themes, @vim
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -march=native -pipe -floop-interchange -ftree-loop-distribution -floop-strip-mine -floop-block -ftree-vectorize -flto=8 -fuse-linker-plugin"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/gnupg/qualified.txt /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -march=native -pipe -floop-interchange -ftree-loop-distribution -floop-strip-mine -floop-block -ftree-vectorize -flto=8 -fuse-linker-plugin"
DISTDIR="/var/src"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs buildpkg cgroup config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="ftp://ftp.free.fr/mirrors/ftp.gentoo.org/ ftp://mirror.ovh.net/gentoo-distfiles/ http://gentoo.mneisen.org/ ftp://mirror.mdfnet.se/gentoo"
INSTALL_MASK="/etc/systemd /lib*/systemd /usr/lib*/systemd"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -flto=8 -fuse-linker-plugin"
MAKEOPTS="-j8"
PKGDIR="/var/pkg"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="X a52 aac acl acpi alsa amd64 atlas audiofile avi berkdb blas bzip2 cairo caps cdda cddb cdio cdr cjk clang cleartype cli cracklib crypt css cups curl cxx dbus device-mapper directfb djvu dri dv dvd dvdr egl exif expat fbcon ffmpeg fftw flac fltk fontconfig fontforge ftp gd gdbm gif git glamor gles2 glut gpm graphite graphviz gtk gtk2 gzip hardened hddtemp iconv icu idn ieee1394 imagemagick imlib infinality ipv6 jack jbig jpeg jpeg2k justify kerberos ladspa lame lapack lash lcms libass libnotify libsamplerate lm_sensors lto lua luajit lz4 lzma lzo lzop mad matroska mbox mmx mmxext mng modplug modules mp3 mp4 mpeg multilib musepack ncurses nls nptl nsplugin ntp ogg opengl openldap openmp openssl oss pam pax_kernel pcre pdf perl pie png postscript python raw readline sasl savedconfig seccomp session smp sndfile snmp sockets sqlite sse sse2 ssl ssp startup-notification svg symlink tcpd theora threads tiff truetype udev unicode urandom usb vaapi vdpau vim vim-syntax vorbis wavpack wayland wifi x264 xattr xft xinerama xinetd xml xorg xtpax xv xvid xvmc xz zip zlib" ABI_X86="64" ALSA_CARDS="cmipci hda-intel usb-audio" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="avx avx2 mmx mmxext sse sse2 sse3 ssse3 sse4_1 sse4_2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="emu efi-32 efi-64 qemu pc" INPUT_DEVICES="evdev mouse keyboard synaptics vmmouse" KERNEL="linux" LCD_DEVICES="lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en fr" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_4" QEMU_SOFTMMU_TARGETS="aarch64 arm i386 ppc ppc64 sparc sparc64 s390x x86_64" QEMU_USER_TARGETS="aarch64 arm i386 ppc ppc64 sparc sparc64 s390x x86_64" RUBY_TARGETS="ruby22" SANE_BACKENDS="canon canon630u canon_pp canon_dr genesys net pixma" USERLAND="GNU" VIDEO_CARDS="intel i915 i965 nouveau vesa vmware" XTABLES_ADDONS="ipv4options ipset iface ipmark dhcpmac"
Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
---
Comment 1 tokiclover 2016-05-17 08:20:43 UTC
Oh write... no need to post the `strings' output do I? There is no `BIGE' nor `EGIB' string.

I found this in one search/try:

---
/* Ref: http://esr.ibiblio.org/?p=5095 */
#include <stdint.h>

#define IS_BIG_ENDIAN (*(uint16_t *)"\0\xff" < 0x100)

int main(void) { return IS_BIG_ENDIAN; }
---

works fine... but I don't know the magic behind it. So, this is not a proper proposal. Still, it's an elegant one-liner test.

Cheers.
Comment 2 tokiclover 2016-05-19 13:08:41 UTC
Created attachment 434716 [details, diff]
endian-test.patch

Little configure patch with the previous one-liner endian test.
Comment 3 Joe Harvell 2017-08-10 20:05:33 UTC
I get the same error.  It is caused by -flto.  But fundamentally the reason it fails is that grepping for a string in a binary relies on compiler implementation details of the compiler that it should not rely on.

The endian-test.patch from 2016-05-19 is the right way to fix this.  It has very low complexity and is easy to verify.  It has been here for over a year, yet there has been no action.  What is the reason nothing is happening on this bug?
Comment 4 Joe Harvell 2017-08-10 20:08:49 UTC
(In reply to tokiclover from comment #1)
> Oh write... no need to post the `strings' output do I? There is no `BIGE'
> nor `EGIB' string.
> 
> I found this in one search/try:
> 
> ---
> /* Ref: http://esr.ibiblio.org/?p=5095 */
> #include <stdint.h>
> 
> #define IS_BIG_ENDIAN (*(uint16_t *)"\0\xff" < 0x100)
> 
> int main(void) { return IS_BIG_ENDIAN; }
> ---
> 
> works fine... but I don't know the magic behind it. So, this is not a proper
> proposal. Still, it's an elegant one-liner test.
> 
> Cheers.

I understand how it works, and I see this as a proper proposal.
Comment 5 tokiclover 2018-05-14 11:15:57 UTC
Created attachment 531320 [details, diff]
new endian test patch for media-video/x264-encoder-0.0.20170701
Comment 6 Logan Kaser 2023-01-06 01:44:32 UTC
Bug still occurs for me with `amd64-llvm-openrc` with -flto, configure succeeds as soon as -flto is removed.
Comment 7 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-01-06 01:55:11 UTC
(In reply to Logan Kaser from comment #6)
> Bug still occurs for me with `amd64-llvm-openrc` with -flto, configure
> succeeds as soon as -flto is removed.

I'm surprised that https://code.videolan.org/videolan/x264/-/merge_requests/45 didn't help.
Comment 8 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-01-06 01:57:14 UTC
(In reply to Sam James from comment #7)
> (In reply to Logan Kaser from comment #6)
> > Bug still occurs for me with `amd64-llvm-openrc` with -flto, configure
> > succeeds as soon as -flto is removed.
> 
> I'm surprised that
> https://code.videolan.org/videolan/x264/-/merge_requests/45 didn't help.

I can't reproduce the failure with x264-encoder-0.0.20220222. Please include the full build.log.