It also segfaults when quantization type is set to mpeg. media-libs/xvid-1.1.3 doesn't show this behavior. Reproducible: Always Steps to Reproduce: 1.emerge =xvid-1.1.3-r2 2.encode something 3.watch the horrible quality 4.start wondering what the fudge is wrong with it 5.open it in avidemux and see that all frames are I-frames Actual Results: all frames all I-frames, segfaults with mpeg quantization Expected Results: I, P and B frames, no segfaults http://rapidshare.com/files/95389807/xvid.tar.bz2.html Two files, one encoded with 1.1.3-r2 and other with 1.1.3 with the same settings, xvid compilation time apart (I wasn't sure if I should attach big files to the bug itself, so I decided to post them elsewere) also: # emerge --info Portage 2.1.4.4 (default-linux/x86/2007.0/desktop, gcc-4.2.3, glibc-2.7-r1, 2.6.24-gentoo-r2 i686) ================================================================= System uname: 2.6.24-gentoo-r2 i686 AMD Athlon(TM) XP 2500+ Timestamp of tree: Tue, 26 Feb 2008 17:30:01 +0000 ccache version 2.4 [enabled] app-shells/bash: 3.2_p33 dev-java/java-config: 1.3.7, 2.1.4 dev-lang/python: 2.5.1-r5 dev-python/pycrypto: 2.0.1-r6 dev-util/ccache: 2.4-r7 sys-apps/baselayout: 2.0.0_rc6-r1 sys-apps/sandbox: 1.2.18.1-r2 sys-devel/autoconf: 2.13, 2.61-r1 sys-devel/automake: 1.4_p6, 1.5, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.1 sys-devel/binutils: 2.18-r1 sys-devel/gcc-config: 1.4.0-r4 sys-devel/libtool: 1.5.26 virtual/os-headers: 2.6.24 ACCEPT_KEYWORDS="x86 ~x86" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -pipe -march=athlon-xp -fomit-frame-pointer -mfpmath=sse" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/kde/4.0/env /usr/kde/4.0/share/config /usr/kde/4.0/shutdown /usr/share/config /var/lib/hsqldb" CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c /etc/udev/rules.d" CXXFLAGS="-O2 -pipe -march=athlon-xp -fomit-frame-pointer -mfpmath=sse" DISTDIR="/usr/portage/distfiles" FEATURES="ccache collision-protect distlocks fixpackages metadata-transfer parallel-fetch sandbox sfperms splitdebug strict unmerge-orphans userfetch" GENTOO_MIRRORS="ftp://mirror.icis.pcz.pl/gentoo/ ftp://pandemonium.tiscali.de/pub/gentoo/ http://mirrors.isc.org/pub/" LANG="pl_PL.utf8" LC_ALL="pl_PL.utf8" LINGUAS="pl en" MAKEOPTS="-j1" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp/portage" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/portage/local/layman/sunrise /usr/portage/local/layman/desktop-effects /usr/local/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="3dnow 3dnowext 7zip X a52 aac aalib acl acpi aiglx alsa amarok amr amrnb amrwb amuled asf atm audiofile bash-completion berkdb bitmap-fonts bittorrent bluetooth boost branding browserplugin bzip2 cairo cdaudio cdda cddb cdr cli clucene cracklib css cups curl cviewer dbus designer-plugin dga disk-partition divx double-precision dri dts dvd dvdnav dvdr dvdread eds emboss emovix enca encode evo exif extrafilters extraicons extramodules fam fame fat ffmpeg fftw firefox flac foomaticdb fortran gadu gd gdbm geoip gif gimp gimpprint glib glibc-omitfp glitz glut gmedia gphoto2 gpm gtk hal icc iconv imlib inkjar inotify insecure-savers ipod isdnlog java jpeg kde kdeenablefinal kdehiddenvisibility kdexdeltas kqemu kt400 lame lcms libcaca libnotify libsamplerate libvisual lm_sensors mad matroska midi mikmod mime mmap mmx mmxext mng mozbranding mozsvg mp3 mp4 mpeg mplayer mtp mudflap musepack musicbrainz ncurses network njb nls nptl nptlonly nsplugin ntfs nvidia offensive ogg openal openexr opengl openmp oss pam pch pcre pdf perl physfs pic plugin png povray ppds pppd psyco python qt-copy qt3 qt3support qt4 quicktime rar rdesktop readline real reflection reiserfs remote scanner sdl sensord session skins slang sndfile solver sound speex spell spl sqlite srt sse ssl stats stream subtitles suid svg svga swscaler symlink tcltk tcpd test theora threads threadsafe tiff timidity tk tordns truetype truetype-fonts type1-fonts unicode usb userlocales utempter vcd verse visualization vlm vorbis win32codecs wmp wxwindows x264 x86 xcb xcomposite xforms xim xine xml xorg xprint xscreensaver xv xvid xvmc zip zlib" ALSA_CARDS="via82xx emu10k1" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="pl en" USERLAND="GNU" VIDEO_CARDS="nvidia nv" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LDFLAGS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
confirmed here: gdb mencoder (gdb) run green.avi -o toto.avi -ovc xvid -oac lavc -xvidencopts bitrate=1000:qpel:quant_type=mpeg Starting program: /usr/bin/mencoder green.avi -o toto.avi -ovc xvid -oac lavc -xvidencopts bitrate=1000:qpel:quant_type=mpeg (no debugging symbols found) warning: Lowest section in system-supplied DSO at 0xffffe000 is .hash at ffffe0b4 (no debugging symbols found) (no debugging symbols found) [Thread debugging using libthread_db enabled] MEncoder dev-SVN-rUNKNOWN-4.2.3 (C) 2000-2008 MPlayer Team CPU: AMD Athlon(tm) 64 Processor 3200+ (Family: 15, Model: 31, Stepping: 0) CPUflags: Type: 15 MMX: 1 MMX2: 1 3DNow: 1 3DNow2: 1 SSE: 1 SSE2: 1 Compiled for x86 CPU with extensions: MMX MMX2 3DNow 3DNowEx SSE SSE2 success: format: 0 data: 0x0 - 0x32a632 AVI file format detected. [aviheader] Video stream found, -vid 0 AVI: No audio stream found -> no sound. VIDEO: [XVID] 704x304 24bpp 25.000 fps 2197.8 kbps (268.3 kbyte/s) [V] filefmt:3 fourcc:0x44495658 size:704x304 fps:25.00 ftime:=0.0400 xvid: using library version 1.1.3 (build xvid-1.1.3) Opening video filter: [expand osd=1] Expand: -1 x -1, -1 ; -1, osd: 1, aspect: 0.000000, round: 1 ========================================================================== Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family Selected video codec: [ffodivx] vfm: ffmpeg (FFmpeg MPEG-4) ========================================================================== VDec: vo config request - 704 x 304 (preferred colorspace: Planar YV12) VDec: using Planar YV12 as output csp (no 0) Movie-Aspect is 2.32:1 - prescaling to correct movie aspect. videocodec: XviD (704x304 fourcc=44495658 [XVID]) xvid: par=0/0 (vga11), displayed=704x304, sampled=704x304 xvid: CBR Rate Control -- bitrate=1000kbit/s Writing header... ODML: vprp aspect is 16384:7074. Writing header... ODML: vprp aspect is 16384:7074. [New Thread 0xf7c1a6c0 (LWP 25034)]Trem: 0min 3mb A-V:0.000 [0:0] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0xf7c1a6c0 (LWP 25034)] 0xf7d935ad in CodeCoeffInter_CalcBits (qcoeff=0xff9fde00, zigzag=0xf7e0ec80) at src/bitstream/mbcoding.c:420 420 src/bitstream/mbcoding.c: No such file or directory. in src/bitstream/mbcoding.c (gdb) bt #0 0xf7d935ad in CodeCoeffInter_CalcBits (qcoeff=0xff9fde00, zigzag=0xf7e0ec80) at src/bitstream/mbcoding.c:420 #1 0xf7dd2abc in CheckCandidateRD16 (x=-9, y=8, data=0xff9fe0cc, Direction=255) at src/motion/estimation_rd_based.c:74 #2 0xf7dd34f0 in findRD_inter (Data=0xff9fe0cc, x=0, y=0, pParam=0x87476c0, MotionFlags=13780) at src/motion/estimation_rd_based.c:311 #3 0xf7dd45cc in xvid_me_ModeDecision_RD (Data=0xff9fe0cc, pMB=0xf78f7040, pMBs=0xf78f7040, x=0, y=0, pParam=0x87476c0, MotionFlags=13780, VopFlags=<value optimized out>, VolFlags=5, pCurrent=0x8809ab8, pRef=0x8995978, vGMC=0x87477b4, coding_type=1) at src/motion/estimation_rd_based.c:662 #4 0xf7dcf142 in MotionEstimation (pParam=0x87476c0, current=0x8809a80, reference=0x8995940, pRefH=0x8747778, pRefV=0x8747784, pRefHV=0x874779c, pGMC=0x87477b4, iLimit=1048576) at src/motion/estimation_pvop.c:932 #5 0xf7d85aaf in FrameCodeP (pEnc=0x87476c0, bs=0xff9ff258) at src/encoder.c:1629 #6 0xf7d86e84 in enc_encode (pEnc=0x87476c0, xFrame=0x871be70, stats=0xff9ff2dc) at src/encoder.c:1347 #7 0xf7d888fd in xvid_encore (handle=0x87476c0, opt=-136252576, param1=0x871be70, param2=0xff9ff2dc) at src/xvid.c:852 #8 0x08071d47 in ?? () (gdb) x/8i $pc 0xf7d935ad <CodeCoeffInter_CalcBits+61>: movzwl (%ecx,%eax,2),%eax 0xf7d935b1 <CodeCoeffInter_CalcBits+65>: lea 0x1(%esi),%ecx 0xf7d935b4 <CodeCoeffInter_CalcBits+68>: test %ax,%ax 0xf7d935b7 <CodeCoeffInter_CalcBits+71>: movswl %ax,%edi 0xf7d935ba <CodeCoeffInter_CalcBits+74>: je 0xf7d935a0 <CodeCoeffInter_CalcBits+48> 0xf7d935bc <CodeCoeffInter_CalcBits+76>: cmp $0x3f,%ecx 0xf7d935bf <CodeCoeffInter_CalcBits+79>: mov %esi,0x4(%esp) 0xf7d935c3 <CodeCoeffInter_CalcBits+83>: mov %edi,%ebp (gdb) i r eax 0x4000 16384 ecx 0xff9fde00 -6300160 edx 0xf7e0f360 -136252576 ebx 0xf7e14ff4 -136228876 esp 0xff9fda2c 0xff9fda2c ebp 0x4 0x4 esi 0x370 880 edi 0x0 0 eip 0xf7d935ad 0xf7d935ad <CodeCoeffInter_CalcBits+61> eflags 0x210296 [ PF AF SF IF RF ID ] cs 0x23 35 ss 0x2b 43 ds 0x2b 43 es 0x2b 43 fs 0x0 0 gs 0x63 99 it doesnt segfault without the textrel patch and doesnt always segfault with it. However, if I encode a video with 1.1.3-2 with the following mencoder command: mencoder green.avi -o toto.avi -ovc xvid -oac lavc -xvidencopts bitrate=1000:qpel I do not get only I frames. @Piotr: could you please give us a mencoder command to trigger your problem ? @PaX team / hardened: could you please have a look at this as this seems to be caused by the textrel patch ?
Created attachment 144977 [details, diff] fixed segfault in xvid textrel patch what a silly bug, doh ;-). this fixes the segfault only, i have no idea about the frame type problem.
I get all I frames with just about the same command: mencoder a.avi -o b.avi -ovc xvid -oac mp3lame -xvidencopts bitrate=1000:qpel
Thanks, I've changed the patch to this one for 1.1.3-r3. However, while your videos are indeed very bad, I've tried this: mencoder good.avi -o shouldbebad.avi -ovc xvid -oac copy -xvidencopts bitrate=1000:qpel (from your good.avi file) And I get this: http://dev.gentoo.org/~aballier/shouldbebad.avi and there are frames that are not I frame and the quality isn't that bad :/ More importantly, it's in an athlon64 32bits chroot that I've made these videos, so it probably uses exactly the same asm as your athlon xp. Could you try if you get more luck with -r3 ? I doubt it will change something but as I cant reproduce your problem that's the best I can do.
Well, i tried 1.1.3-r3, and I still get all keyframes. If you want to dig into this, i can send you my libxvidcore.so. Personally, I don't mint sticking to 1.1.3. Thanks anyway ;)
(In reply to comment #5) > Well, i tried 1.1.3-r3, and I still get all keyframes. If you want to dig into > this, i can send you my libxvidcore.so. Personally, I don't mint sticking to > 1.1.3. Thanks anyway ;) do you see the problem if you leave out the textrel patch only?
It's caused by the textrel patch. (first I commented out only that patch, then everything except it - first time it worked, the second time i got all Iframes)
(In reply to comment #5) > Well, i tried 1.1.3-r3, and I still get all keyframes. If you want to dig into > this, i can send you my libxvidcore.so. Personally, I don't mint sticking to > 1.1.3. Thanks anyway ;) > bah we should really try to fix it. The idea is to provide only the version with the textrel patch, so at some point the 1.1.3 will be removed.
(In reply to comment #8) > (In reply to comment #5) > > Well, i tried 1.1.3-r3, and I still get all keyframes. If you want to dig into > > this, i can send you my libxvidcore.so. Personally, I don't mint sticking to > > 1.1.3. Thanks anyway ;) > > > > bah we should really try to fix it. The idea is to provide only the version > with the textrel patch, so at some point the 1.1.3 will be removed. > hmm sorry if I wasn't clear: please send me your libxvidcore.so or attach it here so that I can try to see what are the differences.
oops, sorry, I must have missed the mail :( I'm uploading it right away
Created attachment 145671 [details] the faulty libxvidcore with all the debug info in place (i think)
Please test =media-video/xvid-1.2.2-r1. If it is, please open a new bug against it.
