Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 290741 - media-video/ffmpeg-0.5_p20373 fails with asm errors when building as PIC on x86 due to register pressure
Summary: media-video/ffmpeg-0.5_p20373 fails with asm errors when building as PIC on x...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: x86 Linux
: High normal (vote)
Assignee: media-video herd
URL:
Whiteboard:
Keywords:
: 300131 300427 304985 311389 313249 319005 (view as bug list)
Depends on:
Blocks:
 
Reported: 2009-10-27 13:13 UTC by Erik Zeek
Modified: 2013-02-08 14:16 UTC (History)
31 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build log (media-video:ffmpeg-0.5_p20373:20091027-130123.log,436.94 KB, text/plain)
2009-10-27 13:15 UTC, Erik Zeek
Details
reduced test case (casts.i,412 bytes, text/plain)
2009-10-27 17:42 UTC, Alexis Ballier
Details
another one (o0vs01.i,429 bytes, text/plain)
2009-10-27 23:03 UTC, Alexis Ballier
Details
Walkaround patch to compile media-video/ffmpeg-0.5_p22846 on x86 with pic (ffmpeg-pic-compile.patch,5.12 KB, patch)
2010-05-26 19:39 UTC, Maciej S. Szmigiero
Details | Diff
Possible modification of transpose4x4 (transpose4x4.diff,2.64 KB, patch)
2010-05-26 22:32 UTC, Reimar Döffinger
Details | Diff
ffmpeg compile patch - TEXTREL free version (ffmpeg-pic-compile.patch,4.52 KB, patch)
2010-05-28 14:56 UTC, Maciej S. Szmigiero
Details | Diff
patch against ebuild to always set pic for x86 ad allow mmx o amd64 (ffmpeg-0.6-ebuild.patch,1.39 KB, patch)
2010-08-13 22:31 UTC, Xake
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Erik Zeek 2009-10-27 13:13:30 UTC
media-video/ffmpeg-0.5_p20373 fails with asm errors about a ‘GENERAL_REGS’ class in lpc_mmx.c:

emerge -pqv =media-video/ffmpeg-0.5_p20373
[ebuild     U ] media-video/ffmpeg-0.5_p20373 [0.5_p19928] USE="X alsa dirac doc encode gsm hardcoded-tables ipv6 jack jpeg2k mmx mmxext mp3 network opencore-amr pic%* schroedinger sdl speex theora threads v4l v4l2 
vorbis x264 xvid zlib -3dnow* -3dnowext* (-altivec) -bindist -cpudetection -custom-cflags -debug -faac -faad -ieee1394 -oss -ssse3* -test -vdpau" VIDEO_CARDS="-nvidia"


i686-pc-linux-gnu-gcc -DHAVE_AV_CONFIG_H -I. -I"/gentoo/tmp/portage/media-video/ffmpeg-0.5_p20373/work/ffmpeg-0.5_p20373" -D_ISOC99_SOURCE -D_POSIX_C_SOURCE=200112 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -DPIC -O2 -g 
-mtune=prescott -march=prescott -pipe -mfpmath=sse -I/usr/include/gsm -fomit-frame-pointer  -march=prescott -std=c99 -fomit-frame-pointer -fPIC -pthread -I/usr/include/dirac -I/usr/include/schroedinger-1.0 
-I/usr/include/liboil-0.3 -Wdeclaration-after-statement -Wall -Wno-switch -Wdisabled-optimization -Wpointer-arith -Wredundant-decls -Wno-pointer-sign -Wcast-qual -Wwrite-strings -Wtype-limits -Wundef -O3 -fno-math-errno 
-fno-signed-zeros -fno-tree-vectorize         -MMD -MF libavcodec/x86/idct_mmx.d -MT libavcodec/x86/idct_mmx.o -c -o libavcodec/x86/idct_mmx.o libavcodec/x86/idct_mmx.c
In file included from libavcodec/x86/idct_mmx.c:26:
libavcodec/x86/mmx.h:24:2: warning: #warning Everything in this header is deprecated, use plain __asm__()! New code using this header will be rejected.
i686-pc-linux-gnu-gcc -DHAVE_AV_CONFIG_H -I. -I"/gentoo/tmp/portage/media-video/ffmpeg-0.5_p20373/work/ffmpeg-0.5_p20373" -D_ISOC99_SOURCE -D_POSIX_C_SOURCE=200112 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -DPIC -O2 -g 
-mtune=prescott -march=prescott -pipe -mfpmath=sse -I/usr/include/gsm -fomit-frame-pointer  -march=prescott -std=c99 -fomit-frame-pointer -fPIC -pthread -I/usr/include/dirac -I/usr/include/schroedinger-1.0 
-I/usr/include/liboil-0.3 -Wdeclaration-after-statement -Wall -Wno-switch -Wdisabled-optimization -Wpointer-arith -Wredundant-decls -Wno-pointer-sign -Wcast-qual -Wwrite-strings -Wtype-limits -Wundef -O3 -fno-math-errno 
-fno-signed-zeros -fno-tree-vectorize         -MMD -MF libavcodec/x86/lpc_mmx.d -MT libavcodec/x86/lpc_mmx.o -c -o libavcodec/x86/lpc_mmx.o libavcodec/x86/lpc_mmx.c
libavcodec/x86/lpc_mmx.c: In function ‘ff_lpc_compute_autocorr_sse2’:
libavcodec/x86/lpc_mmx.c:87: error: can't find a register in class ‘GENERAL_REGS’ while reloading ‘asm’
libavcodec/x86/lpc_mmx.c:62: error: ‘asm’ operand has impossible constraints
libavcodec/x86/lpc_mmx.c:64: error: ‘asm’ operand has impossible constraints
libavcodec/x86/lpc_mmx.c:87: error: ‘asm’ operand has impossible constraints
libavcodec/x86/lpc_mmx.c:116: error: ‘asm’ operand has impossible constraints
make: *** [libavcodec/x86/lpc_mmx.o] Error 1
make: *** Waiting for unfinished jobs....
In file included from libavcodec/x86/dsputil_mmx.c:1819:
libavcodec/x86/h264dsp_mmx.c: In function ‘put_h264_qpel4_h_lowpass_3dnow’:
libavcodec/x86/h264dsp_mmx.c:2058: error: can't find a register in class ‘GENERAL_REGS’ while reloading ‘asm’
libavcodec/x86/h264dsp_mmx.c:2058: error: ‘asm’ operand has impossible constraints
make: *** [libavcodec/x86/dsputil_mmx.o] Error 1
 * ERROR: media-video/ffmpeg-0.5_p20373 failed:
 *   make failed
 *
 * Call stack:
 *     ebuild.sh, line  49:  Called src_compile
 *   environment, line 2609:  Called die
 * The specific snippet of code:
 *       emake || die "make failed"
 *
 * If you need support, post the output of 'emerge --info =media-video/ffmpeg-0.5_p20373',
 * the complete build log and the output of 'emerge -pqv =media-video/ffmpeg-0.5_p20373'.
 * The complete build log is located at '/var/log/portage/media-video:ffmpeg-0.5_p20373:20091027-130123.log'.
 * The ebuild environment file is located at '/gentoo/tmp/portage/media-video/ffmpeg-0.5_p20373/temp/environment'.
 * S: '/gentoo/tmp/portage/media-video/ffmpeg-0.5_p20373/work/ffmpeg-0.5_p20373'



$ emerge --info =media-video/ffmpeg-0.5_p20373
Portage 2.2_rc46 (default/linux/x86/10.0/desktop, gcc-4.4.2, glibc-2.10.1-r0, 2.6.31-gentoo-r3 i686)
=================================================================
                        System Settings
=================================================================
System uname: Linux-2.6.31-gentoo-r3-i686-Intel-R-_Pentium-R-_4_CPU_3.00GHz-with-gentoo-2.0.1
Timestamp of tree: Tue, 27 Oct 2009 11:45:01 +0000
distcc 3.1 i686-pc-linux-gnu [disabled]
ccache version 2.4 [enabled]
app-shells/bash:     4.0_p35
dev-java/java-config: 2.1.9-r1
dev-lang/python:     2.6.3, 3.1.1-r1
dev-python/pycrypto: 2.0.1-r8
dev-util/ccache:     2.4-r8
dev-util/cmake:      2.6.4-r3
sys-apps/baselayout: 2.0.1
sys-apps/openrc:     0.5.2-r1
sys-apps/sandbox:    2.2
sys-devel/autoconf:  2.13, 2.63-r1
sys-devel/automake:  1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.2, 1.11
sys-devel/binutils:  2.20
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6a
virtual/os-headers:  2.6.30-r1
ACCEPT_KEYWORDS="x86 ~x86"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -g -mtune=prescott -march=prescott -pipe -mfpmath=sse"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/X11/xkb /usr/share/config /var/lib/hsqldb /var/spool/torque"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-O2 -g -mtune=prescott -march=prescott -pipe -mfpmath=sse"
DISTDIR="/gentoo/portage/distfiles"
FEATURES="assume-digests candy ccache collision-protect distlocks fixpackages news parallel-fetch preserve-libs protect-owned sandbox sfperms splitdebug strict unmerge-logs unmerge-orphans userfetch userpriv usersandbox"
FFLAGS="-O2 -g -mtune=prescott -march=prescott -pipe -mfpmath=sse"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo http://open-systems.ufl.edu/mirrors/gentoo http://gentoo.llarian.net/ http://mirror.datapipe.net/gentoo http://gentoo.cites.uiuc.edu/pub/gentoo/ http://gentoo.osuosl.org/ http://distfiles.gentoo.org  http://www.ibiblio.org/pub/Linux/distributions/gentoo"
LANG="en_US.UTF-8"
LC_ALL="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--sort-common"
LINGUAS="en en_US"
MAKEOPTS="-j2"
PKGDIR="/gentoo/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_EXTRA_OPTS="--timeout=20 --prune-empty-dirs -i"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/gentoo/tmp"
PORTDIR="/gentoo/portage"
PORTDIR_OVERLAY="/gentoo/local/portage /gentoo/local/layman/science /gentoo/local/layman/sunrise /gentoo/local/layman/kde-testing"
SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage"
USE="3ds R X X509 Xaw3d a52 aac accessibility ace acl acpi activefilter additions addressbook admin administrator aften agg aim aio akode akonadi alias alisp alsa amarok amarok2 amazon amr amrnb amrr animgif ansi antlr ao aotuv apache2 apidocs applet archive arpack artworkextra asm aspell aspnet2 assistant astr async asyncns atm audio audiofile audit auth authdaemond auto-completion automap automount autotrace avahi bash-completion bcmath bcp bdf berkdb bineditor binfilter bittorrent-external biysk bjam bl blas blender blender-game blksha1 bluetooth bonusscripts boo bookmarks boost bracketcompletion branding bugzilla builder bwscheduler bzip2 c++ cairo cal3d calendar canna captury cblas ccache cdaudio cdda cddax cddb cdf cdinstall cdparanoia cdr cdrkit cdrom cdsound cegui celt cern cgi cgns cgraph chaco chardet charmap checkpath chm chroot cipher-aes-128 cipher-aes-192 cipher-aes-256 cipher-blowfish cjk clamav cleartype cli client cln cmake colordiff colorpicker compress consolekit context contrast contrib corefonts cpio cppunit cracklib crypt css cups cupsddk cvs cvsgraph cxx d data dbus debugger dell demo deprecated designer-plugin desktopglobe detex development devhelp device-mapper dga dhcp dia dirac directfb discard-path discouraged disk-partition djvu dns doc docbook doom doom1 doom2 doomsday dosbox dot downloadorder dragonplayer dri dtmf dts dv dvb dvd dvdnav dvdr dvi dvipdfm ebook ecc editor eds eigen embedded emboss emf emovix enca encode enscript erandom evo examples excel exif expat extensions extra extra-algorithms extra-cardsets extra-ciphers extra-phrases extra-tools extrafilters extramodules extras faillog fam fame fastbuild fastcgi fat fax fbcon fbcondecor festival ffmpeg fftw filter finger firefox firefox3 fits flac flash flatfile flickr fltk fluidsynth fmod fontconfig fontforge foomaticdb force-cgi-redirect fortran fortran95 fortune fpx freetts frei0r ftp fuse gadu galago games gd gdal gdbm geoip geolocation geos gif gimp ginac git gkrellm glade glep glgd glib glibc-omitfp glitz glut gmail gmath gml gmp gmtfull gmthigh gmtsuppl gmttria gnome gnome-keyring gnome-print gnomecd gnuplot gnutls gocr google-gadgets gopher gpac gpg gpgme gphoto2 gpm grace grammar graphics graphite graphviz groupwise gs gsl gsm gstreamer gtk gtkhtml guile gzip h224 h281 h323 hal hash hbci hddtemp hdf hdf5 heterogeneous hexen hexenworld hfs high-ints highlight history hlapi hotpixels hou howl-compat html httpd humanities ical iconv icq icu id3 id3tag idb ide idea idn ignore-case image imagemagick imap imlib inifile inkjar inotify ipv6 irc isdnlog itcl ivr jabber jack jadetex jai java java5 java6 javacomm javascript jbig jce jimi jit jmf john jpeg jpeg2k kate kcal kde kdecards kdehiddenvisibility kerberos kexi kig-scripting kipi kontact kpathsea ktts ladspa lame lapack largefile largeterminal lash latex latex3 lcms ldap lensfun libass libburn libcaca libdsk libgda liblockfile libnotify libproxy libsamplerate libssh2 libtiger libv4l2 libwww lights live lm_sensors log4j log4p logitech-mouse logrotate logwatch lvm2 lzma lzo mad mail maildir mailwrapper maps math matroska mbrola md5sum mdnsresponder-compat melt mercurial mgetty mhash mikmod mime mixer mjpeg mkl mktemp mmap mmx mmxext mng mod_python modplug modules moonlight mopac7 motif mozdevelop mozdom moznocompose moznoirc moznomail mozsha1 mp2 mp3 mp3tunes mp4 mpeg mplayer mpqc mtp mudflap multiprocess munin-apache munin-dhcp musepack music musicbrainz mysql na_dd na_dtv na_icons nas ncurses neon net netcdf nethack netmeeting netpbm network network-cron networking new-clx new-login nfs nls nntp nocd nodrm nosource nowin nowlistening npp nptl nptlonly nsplugin nss ntfs ntlm ntp numarray numeric nut nuv objc ocr ocrad octave odk offensive ofx ogdi ogg ogm omega openal openbabel opencore-amr openexr opengl openmp openssl opensslcrypt openstreetmap optimize-cflags osc oscar osdmenu osgapps ots pager pam pango paranoidmsg parcheck parport parse-clocks pascal passwdqc patch patented pbs pch pcntl pcre pdf perl phonon php physfs pic pipe plasma player plib plotutils plugins plutonia pmount png pnm policykit pop posix postfix postproc postscript povray ppds pppd print proj psf pstricks psyco pth publishers pulseaudio python q32 q8 qhull qos qt-dbus qt-webkit qt3 qt3support qt4 qtdesigner qtscript quicktime quotas quotes radio rar raw rc5 rdesktop readline redeyes redland reflection regex remix remote remoteosd replytolist restrict-javascript rle rogue romio rpc rpm rrdcgi rrdtool rtc rtsp samba sametime sandbox sasl sbcl scanner schroedinger science screen script scripting sdk sdl sdl-image sdl-sound sdlaudio search semantic-desktop sensord session sftp sharedext sharedmem shm shmvideo shorten shout sigsegv sip sipim skins slp smbclient smime smp sms smtp sndfile sni soap sockets socks socks5 solver songs sou sound soundex sounds soup sox spamassassin sparse speech speex spell spl sql sqlite sqlite3 srt srtp sse sse2 ssl startup-notification statistics stats stlport stream subject-rewrite subtitles subversion suexec suid svg svgz swat symlink sysfs syslog sysvipc szip t1lib taglib tcl tcpd templates tesseract tex4ht text texteffect textures tga themes theora thesaurus threads threadsafe thumbnail thunar tidy tiff timezone timidity tk tntc tokenizer tomsfastmath toolbar tools totem transcode translator truetype tta tts tv_check tv_combiner tv_pick_cgi twolame type3 ucs2 udev umfpack unicode unzip upnp urandom usb userlocales userpriv utempter utils uudeview v4l v4l2 vcd vcdinfo vcdx vdr vhook vhosts video videos vidix vim vim-pager vim-syntax vim-with-x visualization vlm vnc voice vorbis vxml wav wavpack web webdav webinterface webkit webpresence win32codecs winpopup wma wma-fixed wmf wordperfect wxwidgets wxwindows x264 x86 xanim xattr xcap xcb xcomposite xerces-c xetex xface xfce xforms xft xhtml xindy xine xinerama xinetd xml xmlpatterns xmlreader xmlrpc xmlwriter xmp xmpi xorg xosd xplanet xpm xrandr xrender xscreensaver xsettings xsl xslt xtended xterm xtom xulrunner xv xvid xvmc yaepg yahoo yv12 zip zlib zoran zsh-completion zvbi" ALSA_CARDS="intel8x0 intel8x0m pxa2xx-ac97 pxa2xx-soc hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default  authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex  cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter  headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy  proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id  userdir usertrack vhost_alias" CAMERAS="canon" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en en_US" USERLAND="GNU" VIDEO_CARDS="i810 i830 intel vesa fbdev" 
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS
Comment 1 Erik Zeek 2009-10-27 13:15:25 UTC
Created attachment 208424 [details]
build log

Build log.
Comment 2 Helmut Auer 2009-10-27 13:18:14 UTC
I can confirm this one.
I guess the reason is the pic flag, with using USE="-pic" it should work.
Comment 3 Sergiy Borodych 2009-10-27 14:05:02 UTC
same error

PS.
IMHO And also "on" some flags (3dnow, 3dnowext, ssse3) by default - not good idea
Comment 4 Alexis Ballier gentoo-dev 2009-10-27 17:42:28 UTC
Created attachment 208443 [details]
reduced test case
Comment 5 Alexis Ballier gentoo-dev 2009-10-27 17:45:46 UTC
(In reply to comment #4)
> Created an attachment (id=208443) [details]
> reduced test case


# gcc -c casts.i 
casts.i: In function 't':
casts.i:4: error: can't find a register in class 'GENERAL_REGS' while reloading 'asm'
casts.i:4: error: 'asm' operand has impossible constraints

# gcc --version
gcc (Gentoo 4.4.2 p1.0) 4.4.2
Copyright (C) 2009 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


It seems gcc doesn't like the casts in the asm constraints.
If I remove one cast, this testcase builds. If I compile with -fPIC, it fails in the same way. If I remove two casts, it builds even with -fPIC.
My conclusion is that gcc needs (for no particular reason) one register to cast each variable. CC'ing toolchain, they may have different opinion or submit this to gcc upstream.
Comment 6 Harald van Dijk (RETIRED) gentoo-dev 2009-10-27 21:39:47 UTC
(In reply to comment #4)
> Created an attachment (id=208443) [details]
> reduced test case

This is too far reduced: it fails with -O0 only, and succeeds with -O1 or higher, with or without -fPIC.
Comment 7 Alexis Ballier gentoo-dev 2009-10-27 23:03:12 UTC
Created attachment 208465 [details]
another one
Comment 8 Alexis Ballier gentoo-dev 2009-10-27 23:07:54 UTC
(In reply to comment #6)
> (In reply to comment #4)
> > Created an attachment (id=208443) [details] [details]
> > reduced test case
> 
> This is too far reduced: it fails with -O0 only, and succeeds with -O1 or
> higher, with or without -fPIC.

Yeah, I suppose because gcc is clever enough to understand that it can use the same register for the same variables; nevertheless, I consider "m" constraints needing a register a bit weird. Here is another one that fails at -O1,2 but succeeds at -O0; all with -fPIC.
Comment 9 parafin 2009-10-29 14:26:41 UTC
(In reply to comment #2)
> I can confirm this one.
> I guess the reason is the pic flag, with using USE="-pic" it should work.
> 

Yes, I can confirm that disabling pic USE-flag helps.
Comment 10 Mark Loeser (RETIRED) gentoo-dev 2009-10-31 17:14:30 UTC
Not a gcc bug really, but broken asm or something along those lines.
Comment 11 Hubert Kowalski 2009-11-01 00:25:23 UTC
Well, it's more like throwing ball to ech other. Check: http://roundup.ffmpeg.org/roundup/ffmpeg/issue231 and http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11203
Comment 12 Cristi Magherusan 2009-11-02 18:46:59 UTC
Disabling fpic did it for me too, is there anyone for whom it doesn't?

Maybe we just filter it as a workaround, if it works for everyone...

Cristi
Comment 13 Alexis Ballier gentoo-dev 2009-11-03 01:34:44 UTC
(In reply to comment #10)
> Not a gcc bug really, but broken asm or something along those lines.

Then explain why and what is broken. We'll get nowhere with such vague comments.
If this is a dead end, mask the pic useflag on x86 and be done.
Comment 14 Rafał Mużyło 2009-11-19 17:17:00 UTC
(In reply to comment #13)
> Then explain why and what is broken. We'll get nowhere with such vague
> comments.
> If this is a dead end, mask the pic useflag on x86 and be done.
> 
Well, another approach would be making pic useflag force '--disable-mmx'
on x86 - that works too, bit extreme though.
Comment 15 Reimar Döffinger 2009-11-22 10:47:57 UTC
Unfortunately it is incredibly hard to create a nice test case, this issue seems to be caused by having multiple asm blocks.
However I claim it is a gcc bug, as evidence I have this patch that fixes the issue in dsputil_mmx and only changes a inline to attribute((__noinline__)) - I assume we all agree that code does not become invalid just due to inlining.
Patch:
Index: libavcodec/x86/dsputil_mmx.c
===================================================================
--- libavcodec/x86/dsputil_mmx.c        (revision 20575)
+++ libavcodec/x86/dsputil_mmx.c        (working copy)
@@ -723,7 +723,13 @@
     }
 }
 
-static inline void transpose4x4(uint8_t *dst, uint8_t *src, int dst_stride, int src_stride){
+// HACK gcc won't compile this otherwise
+#if ARCH_X86_32 && defined(PIC)
+static av_noinline
+#else
+static inline
+#endif
+void transpose4x4(uint8_t *dst, uint8_t *src, int dst_stride, int src_stride){
     __asm__ volatile( //FIXME could save 1 instruction if done as 8x4 ...
         "movd  %4, %%mm0                \n\t"
         "movd  %5, %%mm1                \n\t"
Comment 16 Lars Wendler (Polynomial-C) gentoo-dev 2010-01-10 14:23:07 UTC
*** Bug 300427 has been marked as a duplicate of this bug. ***
Comment 17 Jakub Paluszak 2010-01-11 08:11:30 UTC
I'm running a stable P4 system a I've got exactly the same error disabling pic USE-flag seems to fix the problem.

Portage 2.1.6.13 (default/linux/x86/10.0/desktop, gcc-4.3.4, glibc-2.9_p20081201-r2, 2.6.31-gentoo-r6 i686)
=================================================================
System uname: Linux-2.6.31-gentoo-r6-i686-Intel-R-_Pentium-R-_M_processor_1.73GHz-with-gentoo-1.12.13
Timestamp of tree: Mon, 11 Jan 2010 06:15:02 +0000
ccache version 2.4 [disabled]
app-shells/bash:     4.0_p35
dev-java/java-config: 2.1.9-r2
dev-lang/python:     2.6.4
dev-util/ccache:     2.4-r7
dev-util/cmake:      2.6.4-r3
sys-apps/baselayout: 1.12.13
sys-apps/sandbox:    1.6-r2
sys-devel/autoconf:  2.13, 2.63-r1
sys-devel/automake:  1.9.6-r2, 1.10.2
sys-devel/binutils:  2.18-r3
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6b
virtual/os-headers:  2.6.27-r2
ACCEPT_KEYWORDS="x86"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-Os -march=native -fomit-frame-pointer -fno-ident -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/X11/xkb /usr/share/config /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/udev/rules.d"
CXXFLAGS="-Os -march=native -fomit-frame-pointer -fno-ident -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="distlocks fixpackages metadata-transfer parallel-fetch protect-owned sandbox sfperms strict unmerge-orphans userfetch"
GENTOO_MIRRORS="http://gentoo.prz.rzeszow.pl"
LANG="en_GB.UTF-8"
LDFLAGS="-Wl,-O1"
LINGUAS="en en_GB en_US zh zh_CN zh_TW da da_DK pl pl_PL fr fr_FR de de_DE he he_IL vi vi_VN ru ru_RU ru_UA uk uk_UA"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage/layman/sunrise /usr/local/portage/layman/roslin /usr/local/portage/local"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="X a52 aac acpi alsa archive async bash-completion berkdb bidi branding bzip2 cairo cdda cddax cddb cdr cjk cleartype cli consolekit cracklib crypt cups cxx dbus dirac djvu dri dts dvd dvdr eds emboss emerald encode evo exchange exif fam ffmpeg firefox firefox3 flac fortran ftp fuse gdbm gif gnome gphoto2 gstreamer gtk hal hddtemp iconv idn imap immqt-bc inotify iproute2 ipv6 jpeg kdehiddenvisibility laptop libnotify live mad matroska mikmod mmap mmx mmxext mng modules mp3 mp4 mpeg mudflap musepack nautilus ncurses networks nls nptl nptlonly ogg opengl openmp pam pcre pdf perl pic pidgin png pop ppds pppd python qt3support qt4 quicktime readline reflection samba schroedinger sdl session sound speex spell spl sqlite sse sse2 ssh ssl startup-notification svg sysfs tcpd theora thunar tiff truetype udev unicode usb vorbis webkit win32codecs wma x264 x86 xattr xcb xml xmp xorg xulrunner xv xvid zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CAMERAS="canon ptp2" ELIBC="glibc" INPUT_DEVICES="evdev keyboard mouse synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en en_GB en_US zh zh_CN zh_TW da da_DK pl pl_PL fr fr_FR de de_DE he he_IL vi vi_VN ru ru_RU ru_UA uk uk_UA" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="radeon"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

However, I also have a hardened x86 with P4 and ffmpeg compiles fine there.

Portage 2.1.6.13 (hardened/linux/x86/10.0, gcc-4.3.4, glibc-2.9_p20081201-r2, 2.6.28-hardened-r9 i686)
=================================================================
System uname: Linux-2.6.28-hardened-r9-i686-Intel-R-_Pentium-R-_4_CPU_1.80GHz-with-gentoo-1.12.13
Timestamp of tree: Mon, 11 Jan 2010 04:45:01 +0000
ccache version 2.4 [enabled]
app-shells/bash:     4.0_p35
dev-lang/python:     2.6.4, 3.1.1-r1
dev-util/ccache:     2.4-r7
dev-util/cmake:      2.6.4-r3
sys-apps/baselayout: 1.12.13
sys-apps/sandbox:    1.6-r2
sys-devel/autoconf:  2.63-r1
sys-devel/automake:  1.8.5-r3, 1.9.6-r2, 1.10.2
sys-devel/binutils:  2.18-r3
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6b
virtual/os-headers:  2.6.27-r2
ACCEPT_KEYWORDS="x86"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=native -pipe -fomit-frame-pointer -fno-ident"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib/fax /var/spool/fax/etc"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/udev/rules.d"
CXXFLAGS="-O2 -march=native -pipe -fomit-frame-pointer -fno-ident"
DISTDIR="/mnt/lap4/distfiles"
FEATURES="ccache distlocks fixpackages metadata-transfer parallel-fetch protect-owned sandbox sfperms strict unmerge-orphans userfetch"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1"
LINGUAS="pl pl_PL"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage/layman/hanno /usr/local/portage/layman/berkano /usr/local/portage/layman/mpd /usr/local/portage/layman/sunrise /usr/local/local-overlay"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="a52 aac acpi alsa amr amrnb amrwb anonres async asyncns avahi bash-completion berkdb bluetooth bzip2 cdda cdparanoia cdr cjk cli cracklib crypt cups cxx dbus directfb dri dts dvd dvdr encode fam fbcon flac ftp fuse gdbm glibc-omitfp hal hardened hinotify iconv id3 idn iproute2 ipv6 lame lastfmradio live lm_sensors logrotate lzo magic matroska mmx mmxext modules mp2 mp3 mudflap musepack ncurses network nfs nfsexport nptl nptlonly ogg openmp optimisememory pam parport pcre perl pic ppds pppd pulseaudio python quicktime readline reflection replaygain rtsp samba sane scanner serial session speex spl sse sse2 ssl stream sysfs tcpd udev unicode urandom usb vhosts vorbis win32codecs x264 x86 xattr xorg zeroconf zlib" ALSA_CARDS="intel8x0" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias proxy proxy_ajp proxy_balancer proxy_connect proxy_ftp proxy_http" CAMERAS="canon ptp2" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="pl pl_PL" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="intel i830"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

Comment 18 Alexis Ballier gentoo-dev 2010-02-04 20:52:32 UTC
*** Bug 300131 has been marked as a duplicate of this bug. ***
Comment 19 Kevin O'Gorman 2010-02-06 16:24:49 UTC
(In reply to comment #2)
> I can confirm this one.
> I guess the reason is the pic flag, with using USE="-pic" it should work.
> 

flagedit describes "pic" as --- Local Flag: Force shared libraries to be built as PIC (this is slower) (media-video/ffmpeg)


I can confirm that -pic works, but I localize it to /etc/portage/package.use with
# See bug 290741
media-video/ffmpeg -pic
Comment 20 Helmut Auer 2010-02-14 11:09:44 UTC
*** Bug 304985 has been marked as a duplicate of this bug. ***
Comment 21 Kevin Lyles 2010-03-26 06:24:51 UTC
*** Bug 311389 has been marked as a duplicate of this bug. ***
Comment 22 Samuli Suominen gentoo-dev 2010-04-05 18:59:57 UTC
*** Bug 313249 has been marked as a duplicate of this bug. ***
Comment 23 Alexis Ballier gentoo-dev 2010-05-10 05:06:15 UTC
*** Bug 319005 has been marked as a duplicate of this bug. ***
Comment 24 William Waisse 2010-05-13 01:51:15 UTC
same here, same problem with media-video/ffmpeg-0.5_p20373

-pic worked here too

 for information I also had to first disable trusted path execution to have it built
( echo 0 >/proc/sys/kernel/grsecurity/tpe ) or it fails with :
Unable to create and execute files in /var/tmp/portage/media-video/ffmpeg-0.5_p20373/temp.  Set the TMPDIR environment
variable to another directory and make sure that it is not mounted noexec.
Sanity test failed.

Comment 25 SpanKY gentoo-dev 2010-05-20 03:19:42 UTC
*** Bug 319005 has been marked as a duplicate of this bug. ***
Comment 26 Maciej S. Szmigiero 2010-05-26 19:39:01 UTC
Created attachment 233045 [details, diff]
Walkaround patch to compile media-video/ffmpeg-0.5_p22846 on x86 with pic

Attached patch allows compilation of media-video/ffmpeg-0.5_p22846 on x86 with pic enabled.
Apply in /var/tmp/portage/media-video/ffmpeg-0.5_p22846/work/ffmpeg-0.5_p22846/ directory by patch -p0 <ffmpeg-pic-compile.patch .
Comment 27 Rafał Mużyło 2010-05-26 20:04:50 UTC
While we already have some strict aliasing warnings,
doesn't patch from comment 26 add more of them ?
Comment 28 Harald van Dijk (RETIRED) gentoo-dev 2010-05-26 21:30:51 UTC
> -        "movq %5, %%mm6             \n\t"\
> +        "movq "MANGLE(ff_pw_5) ", %%mm6\n\t"\

The point of USE=pic is to avoid the text relocations, and you're reintroducing them. If you don't care about that, just unset the pic flag.
Comment 29 Reimar Döffinger 2010-05-26 22:24:53 UTC
> The point of USE=pic is to avoid the text relocations, and you're reintroducing
> them. If you don't care about that, just unset the pic flag.

There's still a bit of a difference between avoiding relocations in generic (and usually not speed-critical code) or in general, if the point is to reduce the number of non-shared pages this kind of change should not be an issue.
The transpose4x4 change however will have a serious performance impact and sure isn't a good idea.
Passing dst, dst_stride, src and src_stride and calculating the addresses manually might be possible without a speed loss, however that requires some closer look to check.
Comment 30 Reimar Döffinger 2010-05-26 22:32:54 UTC
Created attachment 233055 [details, diff]
Possible modification of transpose4x4

This may work without costing performance, however I haven't even tested it, and this requires benchmarking and looking at the generated assembler code.
Comment 31 SpanKY gentoo-dev 2010-05-27 04:09:32 UTC
USE=pic should not produce textrels.  FIN.

hardened systems are often configured to even prevent loading of shared code that has textrels in it.
Comment 32 Reimar Döffinger 2010-05-27 19:10:47 UTC
> USE=pic should not produce textrels.  FIN.

Then you'd have to also add --disable-asm on (32-bit) x86.
I doubt this would generally be considered any better than just disabling the pic use on x86 though.
Nobody is going to rewrite a huge amount of x86 code to be textrel-free, if anyone cares so much about this it's not an unreasonable expectation that they use an architecture whose design works properly with PIC, e.g. x86_64.
Comment 33 SpanKY gentoo-dev 2010-05-27 22:12:08 UTC
hardened users would accept disabling of assembly in ffmpeg to get textrel PIC

we do this already with a few packages like gzip
Comment 34 Maciej S. Szmigiero 2010-05-28 14:56:25 UTC
Created attachment 233283 [details, diff]
ffmpeg compile patch - TEXTREL free version

Here is patch that allows compilation with pic enabled, but doesn't create TEXTRELs by itself.
However, there is still a lot of them remaining in libavcodec.
Comment 35 Reimar Döffinger 2010-05-30 11:52:24 UTC
This will needlessly slow-down non-PIC and x86_64, so it has no chance upstream.
In addition as said in bug 319005, the qpel8 functions are already "fixed" upstream by using MANGLE.
Unless the Gentoo developers seriously wish to carry their own PIC-patches for FFmpeg I strongly suggest discussing this on the upstream ffmpeg-devel mailing list.
And for any Gentoo developer (or anyone else) really caring about this I again strongly suggest to add a Gentoo hardend system to FFmpeg's automated testing: http://fate.multimedia.cx/
Comment 36 Xake 2010-08-13 22:31:12 UTC
Created attachment 242865 [details, diff]
patch against ebuild to always set pic for x86 ad allow mmx o amd64

This is for ffmpeg-0.6:

On an amd64 there is no problem what so ever to have --enable-mmx* as long as it goes hand in hand with --enable-pic, else runtime errors (picked up by the testsuit) follows. It is however about four times as fast on my machine so I really cannot understand why it was disabled for all uses of PIE to begin with.

So this patch against the ebuild does the following:
Removes USE="pic" and enforces --enable-pic for all people using PIE (i.e. hardened since why use PIE/hardened if you do not want pic and is rreally someone non-hardened having USE="pic"?).
Adds --disable-asm for x86 and only x86 since without it (both USE="pic/-pic") I get textrels that on my hardened x86 environments fails to run at all. On x86-64 ffmpeg runs fine (and around 4 times faster on my machine) with asm as long as --enable-pic is choosen, so no need at all to filter it.
Comment 37 Alexis Ballier gentoo-dev 2013-02-08 14:16:34 UTC
(In reply to comment #36)
> Created attachment 242865 [details, diff] [details, diff]
> patch against ebuild to always set pic for x86 ad allow mmx o amd64
> 
> This is for ffmpeg-0.6:
> 
> On an amd64 there is no problem what so ever to have --enable-mmx* as long
> as it goes hand in hand with --enable-pic, else runtime errors (picked up by
> the testsuit) follows. It is however about four times as fast on my machine
> so I really cannot understand why it was disabled for all uses of PIE to
> begin with.
> 
> So this patch against the ebuild does the following:
> Removes USE="pic" and enforces --enable-pic for all people using PIE (i.e.
> hardened since why use PIE/hardened if you do not want pic and is rreally
> someone non-hardened having USE="pic"?).
> Adds --disable-asm for x86 and only x86 since without it (both
> USE="pic/-pic") I get textrels that on my hardened x86 environments fails to
> run at all. On x86-64 ffmpeg runs fine (and around 4 times faster on my
> machine) with asm as long as --enable-pic is choosen, so no need at all to
> filter it.

the same logic is applied for the pic useflag nowadays, closing