Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 135326 - media-libs/xvid has text relocations
Summary: media-libs/xvid has text relocations
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: x86 Linux
: High major (vote)
Assignee: Gentoo Media-video project
URL:
Whiteboard:
Keywords:
: 131422 147974 150861 151234 153304 159129 206982 (view as bug list)
Depends on:
Blocks: 149344
  Show dependency tree
 
Reported: 2006-06-02 15:21 UTC by Christoph Trassl
Modified: 2008-01-22 06:03 UTC (History)
13 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
fixed textrel patch for xvid-1.1.2 (xvid-1.1.2-textrel.patch,193.53 KB, patch)
2007-01-27 19:28 UTC, PaX Team
Details | Diff
/proc/pid/maps with 1 in /proc/sys/kernel/randomize_va_space (maps-random,5.13 KB, text/plain)
2007-05-25 16:55 UTC, Alexis Ballier
Details
/proc/pid/maps with 0 in /proc/sys/kernel/randomize_va_space (maps-non-random,5.13 KB, text/plain)
2007-05-25 17:00 UTC, Alexis Ballier
Details
textrel patch for xvid-1.1.3 (xvid-1.1.3-textrel.patch,193.56 KB, patch)
2007-10-05 15:53 UTC, PaX Team
Details | Diff
textrel patch for xvid-1.1.3 (xvid-1.1.3-textrel.patch,190.40 KB, patch)
2007-10-15 01:43 UTC, PaX Team
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Christoph Trassl 2006-06-02 15:21:39 UTC
I am using mencoder for video encoding with qpel enabled in xvidencopts.

xvid-1.1.0 w/  qpel worked fine.
xvid-1.1.0 w/o qpel worked fine.
xvid-1.1.0-r1 w/  qpel fails with SIGSEGV.
xvid-1.1.0-r1 w/o qpel fails with SIGSEGV.

I figured out that xvid-1.1.0-textrel-2 patch, that was included in xvid-1.1.0-r1, causes the segfault. 

Therefore I consider the patch as _harmful_ and not to work as intended. 

Please fix that issue - either 
* recheck the patch and rewrite the part that causes qpel to SIGSEGV or 
* leave the patch out in portage.

I don't think any further information could be helpful, but if anyone looking at this bugreport wants some evidence of the issue or any other information I of course will provide the information that is needed.

Thanks alot in advance for fixing this issue.
Comment 1 Christoph Trassl 2006-06-05 05:03:39 UTC
I was just overlooking my bugreport and noticed a copy'n'paste typo. The correct table is:

xvid-1.1.0 w/  qpel worked fine.
xvid-1.1.0 w/o qpel worked fine.
xvid-1.1.0-r1 w/  qpel fails with SIGSEGV.
xvid-1.1.0-r1 w/o qpel works fine

Sorry for that.
Comment 2 Bartosz Kulicki 2006-07-15 17:21:10 UTC
I have same observations. Tested with mencoder 1.0_pre8.
Comment 3 Roland Rasmussen 2006-08-13 14:06:37 UTC
Me too. I am also getting a SIGSEGV with the xvid-1.1.0-textrel-2 patch. Removing it from the ebuild solves (i.e. works around) the problem.
Comment 4 Michael 2006-08-24 05:28:44 UTC
This crashes mencoder on my athlon box with even without qpel.  I would also request that it me removed.
























































(In reply to comment #0)
> I am using mencoder for video encoding with qpel enabled in xvidencopts.
> 
> xvid-1.1.0 w/  qpel worked fine.
> xvid-1.1.0 w/o qpel worked fine.
> xvid-1.1.0-r1 w/  qpel fails with SIGSEGV.
> xvid-1.1.0-r1 w/o qpel fails with SIGSEGV.
> 
> I figured out that xvid-1.1.0-textrel-2 patch, that was included in
> xvid-1.1.0-r1, causes the segfault. 
> 
> Therefore I consider the patch as _harmful_ and not to work as intended. 
> 
> Please fix that issue - either 
> * recheck the patch and rewrite the part that causes qpel to SIGSEGV or 
> * leave the patch out in portage.
> 
> I don't think any further information could be helpful, but if anyone looking
> at this bugreport wants some evidence of the issue or any other information I
> of course will provide the information that is needed.
> 
> Thanks alot in advance for fixing this issue.
> 
Comment 5 Olivier Rolland 2006-09-11 14:43:31 UTC
mencoder crashes with quant_type=mpeg. Removing the textrel patch from the ebuild seems to solve the problem.
Comment 6 Martin Thierer 2006-09-20 07:41:49 UTC
*** Bug 147974 has been marked as a duplicate of this bug. ***
Comment 7 Diego Elio Pettenò (RETIRED) gentoo-dev 2006-10-02 01:12:00 UTC

*** This bug has been marked as a duplicate of 129022 ***
Comment 8 Olivier Rolland 2006-10-02 13:05:58 UTC
This bug has been marked a duplicate of bug #129022 and both have been marked resolved. However, I still suffer from the textrel issue with mencoder. Removing the textrel patch solves the problem. Please, reopen this bug.
Comment 9 Diego Elio Pettenò (RETIRED) gentoo-dev 2006-10-02 15:38:37 UTC
Okay, seems like there were two different problems with very different behaviour ... let's reopen this.
Comment 10 Diego Elio Pettenò (RETIRED) gentoo-dev 2006-10-02 15:39:20 UTC
*** Bug 131422 has been marked as a duplicate of this bug. ***
Comment 11 Diego Elio Pettenò (RETIRED) gentoo-dev 2006-10-02 15:40:35 UTC
Hardened, should I make the patch conditional pending futher examination?
Comment 12 Rene Schneider 2006-10-08 03:51:30 UTC
I have this problem as well, causing dvdrip, mplayer and transcode problems. CPU: Athlon-XP. As long as this is not fixed the ebuild should be masked. I did so on my system ("echo =media-libs/xvid-1.1.0-r1>>/etc/portage/package.mask"), downgraded to 1.0.3 ("emerge -uDa media-libs/xvid") and have a working system again.
Comment 13 Michael Gerecke 2006-10-08 08:12:49 UTC
Me too! xvid-1.1.0-r1 does not work with transcode 1.0.2-r3 on my Athlon XP 2700+.
Downgrade to xvid-1.1.0 fixed this.

So xvid-1.1.0-r1 is NOT stable on x86!
Comment 14 Matthias Langer 2006-10-10 04:24:38 UTC
it seems that this bug affects all xvid-encoding related stuff - including dvdrip-0.98.1:

[export_xvid4.so] Neither './xvid4.cfg' nor '~/.transcode/xvid4.cfg'
[export_xvid4.so] found. Default settings will be used instead.
[decode_mpeg2.c] libmpeg2 0.4.0b loop decoder
[decode_mpeg2.c] libmpeg2 acceleration: 3dnow
sh: line 1: 15727 Segmentation fault      execflow -n 15 transcode -H 10 -a 0 -x vob,null -i /home/antonio/rip/AOEO/vob/022/ -b 128,0,0 --a52_drc_off -f 25 -B 12,10,8 -R 1 -y xvid4,null -o /dev/null --print_status 25

Portage 2.1.1 (default-linux/x86/2006.1/desktop, gcc-4.1.1, glibc-2.4-r3, 2.6.16-gentoo-r13 i686)
=================================================================
System uname: 2.6.16-gentoo-r13 i686 AMD Athlon(tm) XP 2400+
Gentoo Base System version 1.12.5
Last Sync: Sun, 08 Oct 2006 09:50:01 +0000
distcc 2.18.3 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
app-admin/eselect-compiler: [Not Present]
dev-java/java-config: 1.3.7, 2.0.30
dev-lang/python:     2.4.3-r4
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     [Not Present]
dev-util/confcache:  [Not Present]
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.59-r7
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2
sys-devel/binutils:  2.16.1-r3
sys-devel/gcc-config: 1.3.13-r4
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.17-r1
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=athlon-xp -O2 -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/gconf /etc/java-config/vms/ /etc/revdep-rebuild /etc/terminfo"
CXXFLAGS="-march=athlon-xp -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig collision-protect distlocks metadata-transfer sandbox sfperms strict"
GENTOO_MIRRORS="http://gentoo.ynet.sk/pub "
LANG="en_US.utf8"
LC_ALL="en_US.utf8"
LINGUAS="en de"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://192.168.0.1/gentoo-portage"
USE="x86 3dnow 3dnowext X a52 aac acpi alsa audiofile beagle berkdb bitmap-fonts bzip2 cairo cdr cli crypt css cups dbus dlloader dri dvd dvdr dvdread eds elibc_glibc emboss encode evo exif fam fbcon ffmpeg firefox flac fortran gdbm gif ginac gmp gnome gphoto2 gpm gstreamer gtk gtk2 hal howl icq input_devices_keyboard input_devices_mouse ipv6 isdnlog java javascript jpeg jpeg2k kernel_linux lcms ldap libg++ linguas_de linguas_en mad mikmod mime mmx mmxext mozsvg mp3 mpeg msn nautilus ncurses nfs nls nptl nptlonly nsplugin nvidia offensive ogg oggvorbis opengl pam pcre perl plotutils png posix ppds pppd python qt3 qt4 quicktime readline real reflection ruby sdl session sockets spell spl sqlite3 sse ssl subtitles svg tcpd tetex theora tiff truetype truetype-fonts type1-fonts udev unicode usb userland_GNU vcd video_cards_nvidia vorbis win32codecs wma xine xml xorg xv xvid zlib"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LDFLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 15 Matthias Langer 2006-10-10 06:28:56 UTC
btw, xvid-1.1.0-r2 seems to be broken too ... at least dvdrip-0.98.1 doesn't like it that much:

[decode_mpeg2.c] libmpeg2 acceleration: 3dnow
failed to write Y plane of framesh: line 1: 15306 Segmentation fault      execflow -n 15 transcode -H 10 -a 0 -x vob,null -i /home/antonio/rip/AOEO/vob/022/ -b 128,0,0 --a52_drc_off -f 25 -Y 2,4,0,4 -Z 600x450 -R 1 -y xvid4,null -o /dev/null --print_status 25
(demuxer.c) write program stream packet: Broken pipe
Comment 16 Jakub Moc (RETIRED) gentoo-dev 2006-10-11 14:00:11 UTC
*** Bug 150861 has been marked as a duplicate of this bug. ***
Comment 17 Jakub Moc (RETIRED) gentoo-dev 2006-10-13 15:11:18 UTC
*** Bug 151234 has been marked as a duplicate of this bug. ***
Comment 18 Jakub Moc (RETIRED) gentoo-dev 2006-10-13 15:13:00 UTC
(In reply to comment #11)
> Hardened, should I make the patch conditional pending futher examination?

Maybe USE="segfault" ? :P 

Now seriously, can we nuke the patch that's breaking lots of stuff?
 
Comment 19 Norm Dugas 2006-10-16 21:42:52 UTC
I'm getting a segfault with xvid-1.1.0-r2 as well.  Took a while to track it down to xvid.  I thought it was mencoder choking on my firewire captured video.  Going down to xvid-1.0.3 worked fine.

equery -C list xvid
[ Searching for package 'xvid' in all categories among: ]
 * installed packages
[I--] [ ~] media-libs/xvid-1.1.0-r2 (1)

emerge --info
Portage 2.1.1 (default-linux/x86/2006.1/desktop, gcc-4.1.1, glibc-2.4-r3, 2.6.17-gentoo-r2 i686)
=================================================================
System uname: 2.6.17-gentoo-r2 i686 AMD Athlon(tm) XP  1800+
Gentoo Base System version 1.12.5
Last Sync: Mon, 16 Oct 2006 06:01:01 +0000
distcc 2.18.3 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
app-admin/eselect-compiler: [Not Present]
dev-java/java-config: 1.3.7, 2.0.30
dev-lang/python:     2.3.5-r2, 2.4.3-r4
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     [Not Present]
dev-util/confcache:  [Not Present]
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.59-r7
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2
sys-devel/binutils:  2.16.1-r3
sys-devel/gcc-config: 1.3.13-r4
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.17-r1
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=athlon-xp -O2 -pipe -D_FILE_OFFSET_BITS=64"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/gconf /etc/java-config/vms/ /etc/revdep-rebuild /etc/terminfo"
CXXFLAGS="-march=athlon-xp -O2 -pipe -D_FILE_OFFSET_BITS=64"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks metadata-transfer sandbox sfperms strict"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"
LINGUAS=""
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="x86 3dnow 3dnowext X a52 aac alsa amr apache2 arts berkdb bitmap-fonts cairo cdr clamav cli cpudetection crypt css cups dbus debug dlloader dri dvd dvdr dvdread eds elibc_glibc emboss encode esd examples fam ffmpeg firefox fortran gdbm gif gpm gstreamer gtk gtk2 hal input_devices_keyboard input_devices_mouse ipv6 isdnlog java javascript jpeg kde kernel_linux ldap libg++ mad madwifi mikmod mmx mmxext mp3 mpeg ncurses nfs nls nntp nptl nptlonly ogg opengl oss pam pcre perl png ppds pppd python qt qt3 qt4 quicktime readline reflection samba sdl session spell spl sse sse2 ssl tcpd threadsafe truetype truetype-fonts type1-fonts udev unicode userland_GNU video_cards_radeon vorbis wifi win32codecs xinetd xml xorg xv xvid zlib"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, PORTAGE_RSYNC_EXTRA_OPTS

Comment 20 Mario Bachmann 2006-10-21 05:09:18 UTC
On AMD XP 2000+ and on AMD Duron 800 MHz I noticed the same Bug:

the command
"mencoder original.avi -o encoded.avi -ovc xvid -xvidencopts bitrate=1200:me_quality=6:vhq=4:hq_ac -oac copy"
shows a segfault when using the xvid-1.1.0-r2.ebuild.

My solution is to use a modified xvid-1.1.0-r1.ebuild.
I deleted the following lines:
#       mirror://gentoo/${P}-textrel-2.patch.bz2"
#       epatch "${DISTDIR}/${P}-textrel-2.patch.bz2"
(and made "ebuild xvid-1.1.0-r1.ebuild digest" and so on).

on the AMD Duron: ------------------------------------------------ 
grep flags /proc/cpuinfo
flags           : fpu vme de pse tsc msr pae mce cx8 sep mtrr pge mca cmov pat pse36 mmx fxsr syscall mmxext 3dnowext 3dnow up

mencoder
MEncoder 1.0pre8-4.1.1 (C) 2000-2006 MPlayer Team
CPU: AMD Duron(tm) Processor (Family: 6, Model: 3, Stepping: 1)
CPUflags: Type: 6 MMX: 1 MMX2: 1 3DNow: 1 3DNow2: 1 SSE: 0 SSE2: 0
Kompiliert f
Comment 21 Mario Bachmann 2006-10-21 05:09:18 UTC
On AMD XP 2000+ and on AMD Duron 800 MHz I noticed the same Bug:

the command
"mencoder original.avi -o encoded.avi -ovc xvid -xvidencopts bitrate=1200:me_quality=6:vhq=4:hq_ac -oac copy"
shows a segfault when using the xvid-1.1.0-r2.ebuild.

My solution is to use a modified xvid-1.1.0-r1.ebuild.
I deleted the following lines:
#       mirror://gentoo/${P}-textrel-2.patch.bz2"
#       epatch "${DISTDIR}/${P}-textrel-2.patch.bz2"
(and made "ebuild xvid-1.1.0-r1.ebuild digest" and so on).

on the AMD Duron: ------------------------------------------------ 
grep flags /proc/cpuinfo
flags           : fpu vme de pse tsc msr pae mce cx8 sep mtrr pge mca cmov pat pse36 mmx fxsr syscall mmxext 3dnowext 3dnow up

mencoder
MEncoder 1.0pre8-4.1.1 (C) 2000-2006 MPlayer Team
CPU: AMD Duron(tm) Processor (Family: 6, Model: 3, Stepping: 1)
CPUflags: Type: 6 MMX: 1 MMX2: 1 3DNow: 1 3DNow2: 1 SSE: 0 SSE2: 0
Kompiliert für x86 CPU mit folgenden Erweiterungen: MMX MMX2 3DNow 3DNowEx


on the AMD XP 2000+: ------------------------------------------------
grep flags /proc/cpuinfo
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr sse syscall mmxext 3dnowext 3dnow up ts

mencoder
MEncoder 1.0pre8-4.1.1 (C) 2000-2006 MPlayer Team
CPU: AMD Athlon(tm) XP 2100+ (Family: 6, Model: 6, Stepping: 2)
CPUflags: Type: 6 MMX: 1 MMX2: 1 3DNow: 1 3DNow2: 1 SSE: 1 SSE2: 0
Kompiliert für x86 CPU mit folgenden Erweiterungen: MMX MMX2 3DNow 3DNowEx SSE
Comment 22 Matthias Langer 2006-10-27 21:03:33 UTC
well, i've just started encoding with

media-libs/xvid-1.1.0-r3  USE="(-altivec) -doc"

using

media-video/dvdrip-0.98.1  USE="hal ogg subtitles vcd vorbis xine xvid -fping -mplayer"

(see bug 149344) on x86 and got no segfaults so far (they appeared immediatly with -r2). encoding should be finished in about 6h ... if nothing fancy happens we may consider reopening bug 149344 and stable dvdrip-0.98.1 on x86 - at least in my opinion ...
Comment 23 Matthias Langer 2006-10-27 21:06:43 UTC
arg ... wrong bug; sorry for the spam
Comment 24 Jakub Moc (RETIRED) gentoo-dev 2006-10-29 09:17:17 UTC
*** Bug 153304 has been marked as a duplicate of this bug. ***
Comment 25 Mario Bachmann 2006-11-02 00:29:43 UTC
The media-libs/xvid-1.1.0-r3.ebuild looks same (same patches are included) as my working modified "-r1" (see comment #20). So it should work now on AMD XP 2000+ and so on. Thanks.
[ As I see it, "textrel-2.patch" caused the segfaults. ]

But now I will wait for a xvid-1.1.1.ebuild ;-)
Comment 26 solar (RETIRED) gentoo-dev 2006-11-30 13:54:38 UTC
(In reply to comment #11)
> Hardened, should I make the patch conditional pending futher examination?

Correct. And bug #153014 should of never happened. Vs dropping support for somethign that's supposed to fix stuff we should of CC: the author of that 
patch (which we failed to do) so it could be debuged and corrected. 
Comment 27 Jakub Moc (RETIRED) gentoo-dev 2006-12-26 05:50:42 UTC
*** Bug 159129 has been marked as a duplicate of this bug. ***
Comment 28 Alexis Ballier gentoo-dev 2007-01-24 22:19:33 UTC
media-libs/xvid-1.1.0-r1: xvid-1.1.0-textrel-2 patch causes SIGSEGV when using qpel



we might need some help there to have a working patch for this because it still applies to 1.1.2 that I just bumped. I'm a n00b with asm so that's probably better that I don't try to fix it myself ;)
Comment 29 PaX Team 2007-01-27 19:28:11 UTC
Created attachment 108302 [details, diff]
fixed textrel patch for xvid-1.1.2

here's the new patch that i hope will fix the segfaults, please test it with the workload that reproduced the crashes for you and report back (if it still crashes, i'll need gdb output, stuff like 'bt', 'x/8i $pc', 'i r', etc).

there's still one problem with the current handling of nasm/yasm though, the configure script will prefer the latter even on i386 therefore it will fail to build if you have it installed. see bug #90287 about the details.

last but not least... this whole thing originated in bug #90287 and the problems should have been address there (that i was subscribed to already) vs. a new bug (that noone bothered to cc me, i learned about it the hard way when the textrels showed up again). please try to make a minimum effort to search bugzilla for related issues next time, these segfaults could have been resolved a long time ago.
Comment 30 Alexis Ballier gentoo-dev 2007-05-24 07:13:30 UTC
(In reply to comment #29)
> here's the new patch that i hope will fix the segfaults, please test it with
> the workload that reproduced the crashes for you and report back (if it still
> crashes, i'll need gdb output, stuff like 'bt', 'x/8i $pc', 'i r', etc).

I'm having a segfault with that patch applied to xvid : 

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1210434880 (LWP 21822)]
0xb56ba01f in enc_encode (pEnc=0x85fb340, xFrame=0xb5dfd74c, stats=0xbff2e974) at ../../src/encoder.c:1467
1467    ../../src/encoder.c: No such file or directory.
        in ../../src/encoder.c


(gdb) bt
#0  0xb56ba01f in enc_encode (pEnc=0x85fb340, xFrame=0xb5dfd74c, stats=0xbff2e974) at ../../src/encoder.c:1467
#1  0xb56bb3ea in xvid_encore (handle=0x85fb340, opt=-1521309356, param1=0xb5dfd74c, param2=0xbff2e974)
    at ../../src/xvid.c:852
#2  0xb5df0028 in tc_export () from /usr/lib/transcode/export_xvid4.so
#3  0x0805fbef in tcv_export ()
#4  0x0805f231 in encoder ()
#5  0x080588dd in main ()


(gdb) bt full
#0  0xb56ba01f in enc_encode (pEnc=0x85fb340, xFrame=0xb5dfd74c, stats=0xbff2e974) at ../../src/encoder.c:1467
        q = <value optimized out>
        type = 0
        bs = {bufa = 0, bufb = 0, buf = 130637824, pos = 19, tail = 0x85fb53c, start = 0x85bb720, length = 0, initpos = 0}
#1  0xb56bb3ea in xvid_encore (handle=0x85fb340, opt=-1521309356, param1=0xb5dfd74c, param2=0xbff2e974)
    at ../../src/xvid.c:852
No locals.
#2  0xb5df0028 in tc_export () from /usr/lib/transcode/export_xvid4.so
No symbol table info available.
#3  0x0805fbef in tcv_export ()
No symbol table info available.
#4  0x0805f231 in encoder ()
No symbol table info available.
#5  0x080588dd in main ()
No symbol table info available.


(gdb) x/8i $pc
0xb56ba01f <enc_encode+2847>:   add    0x44(%edx),%esi
0xb56ba022 <enc_encode+2850>:   mov    %edx,0xfffff838(%ebp)
0xb56ba028 <enc_encode+2856>:   mov    0x158(%esi),%eax
0xb56ba02e <enc_encode+2862>:   movl   $0x3,0xec(%esi)
0xb56ba038 <enc_encode+2872>:   movl   $0x0,0x18(%esi)
0xb56ba03f <enc_encode+2879>:   movl   $0x0,0x10(%esi)
0xb56ba046 <enc_encode+2886>:   test   %eax,%eax
0xb56ba048 <enc_encode+2888>:   movl   $0x0,0x8(%esi)

(gdb) i r
eax            0x9add596c       -1696769684
ecx            0x85fb340        140489536
edx            0xa552a954       -1521309356
ebx            0xb5741ff4       -1250680844
esp            0xbff29be0       0xbff29be0
ebp            0xbff2a3f8       0xbff2a3f8
esi            0xc0fc4bf0       -1057207312
edi            0xc      12
eip            0xb56ba01f       0xb56ba01f <enc_encode+2847>
eflags         0x10a13  [ CF AF IF OF RF ]
cs             0x73     115
ss             0x7b     123
ds             0x7b     123
es             0x7b     123
fs             0x0      0
gs             0x33     51



I hope this will be useful, I only compiled xvid with debug infos, please tell me if you need more debugger outputs or a simple way to reproduce it.
Comment 31 PaX Team 2007-05-25 10:35:30 UTC
could you please post the commandline to reproduce this and your USE flags for xvid and transcode? also, could you post a /proc/pid/maps file corresponding to a segfault (and disable randomization in /proc/sys/kernel/randomize_va_space and post the same again)?
Comment 32 Alexis Ballier gentoo-dev 2007-05-25 16:55:12 UTC
Created attachment 120299 [details]
 /proc/pid/maps with 1 in  /proc/sys/kernel/randomize_va_space

#echo 1 > /proc/sys/kernel/randomize_va_space

$ transcode -x vob -i VTS_06_0.VOB -y xvid4,null -o /dev/null

[...]

[export_xvid4.so] Neither './xvid4.cfg' nor '~/.transcode/xvid4.cfg'
[export_xvid4.so] found. Default settings will be used instead.
tc_memcpy: using sse for memcpy
No accelerated IMDCT transform found
tc_memcpy: using sse for memcpy
[decode_mpeg2.c] libmpeg2 acceleration: 3dnowSegmentation fault
Comment 33 Alexis Ballier gentoo-dev 2007-05-25 17:00:16 UTC
Created attachment 120301 [details]
 /proc/pid/maps with 0 in /proc/sys/kernel/randomize_va_space

# echo "0" >  /proc/sys/kernel/randomize_va_space

[attached maps]


emerge -pv xvid transcode : 
[ebuild   R   ] media-libs/xvid-1.1.2-r1  USE="(-altivec) -doc" 0 kB [3] 
[ebuild   R   ] media-video/transcode-1.0.3  USE="3dnow X a52 dv dvdread iconv imagemagick jpeg lzo mjpeg mmx mp3 ogg quicktime sdl sse theora truetype v4l2 vorbis xml xvid (-altivec) -extrafilters -fame -gtk -mpeg -network -sse2" 0 kB [2] 


The vob file has been taken here : http://samples.mplayerhq.hu/MPEG-VOB/VTS_06_0.VOB

Perhaps a more useful bt : 

(gdb) bt  
#0  MBCoding (frame=0x80a8fc0, pMB=0xb3015980, qcoeff=0xbfff9c40, bs=0xbfffa318, pStat=0x80a90dc)
    at ../../src/bitstream/bitstream.h:392
#1  0xb527214d in enc_encode (pEnc=0x80a8a40, xFrame=0xb59b574c, stats=0xbfffe8c4) at ../../src/encoder.c:1479
#2  0xb52733ea in xvid_encore (handle=0x80a8a40, opt=31468455, param1=0xb59b574c, param2=0xbfffe8c4)
    at ../../src/xvid.c:852
#3  0xb59a7f23 in tc_export (opt=13, para1=0xbfffe964, para2=0x80a4008) at export_xvid4.c:374
#4  0x0805fddf in tcv_export (opt=13, para1=0xbfffe964, para2=0x80a4008) at dl_loader.c:66
#5  0x0805f3f1 in encoder (vob=0x80a4008, frame_a=0, frame_b=2147483647) at encoder.c:734
#6  0x0805923e in main (argc=9, argv=0xbffff554) at transcode.c:4014

(gdb) x/8i $pc
0xb5280a17 <MBCoding+5639>:     mov    %eax,(%edi)
0xb5280a19 <MBCoding+5641>:     add    $0x4,%edi
0xb5280a1c <MBCoding+5644>:     subl   $0x20,0xc(%ecx)
0xb5280a20 <MBCoding+5648>:     mov    %edi,0xffffff68(%ebp)
0xb5280a26 <MBCoding+5654>:     mov    %edi,0x10(%ecx)
0xb5280a29 <MBCoding+5657>:     movl   $0x0,0x8(%ecx)
0xb5280a30 <MBCoding+5664>:     mov    0x14(%ebp),%eax
0xb5280a33 <MBCoding+5667>:     mov    $0x20,%ecx
Comment 34 Alexis Ballier gentoo-dev 2007-06-19 17:56:27 UTC
rhaaa you should stab me... on every box I had tested yasm was installed and xvid defaults to it for compiling asm... I didn't pay attention to that, even if you said that nasm is required to compile it, it doesn't fail to compile, but it seems that yasm computes wrong value for addresses... I encountered that in x264 aswell.

If I remove yasm from the box and recompile xvid, it seems to work fine.

Does anybody has a clue if that's a yasm bug or something else ? 
Comment 35 Alexis Ballier gentoo-dev 2007-09-23 18:24:51 UTC
most likely to be a yasm bug :
http://www.tortall.net/projects/yasm/ticket/114

will test again asap
Comment 36 PaX Team 2007-10-05 15:53:46 UTC
Created attachment 132669 [details, diff]
textrel patch for xvid-1.1.3

just a simple forward port, no changes were needed. the transcode test from comment #32 ran fine here. can this go into portage?
Comment 37 Alexis Ballier gentoo-dev 2007-10-14 16:06:42 UTC
(In reply to comment #36)
> Created an attachment (id=132669) [edit]
> textrel patch for xvid-1.1.3
> 
> just a simple forward port, no changes were needed. the transcode test from
> comment #32 ran fine here. can this go into portage?

thanks, it's fine with yasm 0.6.2 and transcode, but I still get the segfault with qpel:

mencoder foo.avi -o toto.avi -ovc xvid -oac lavc -xvidencopts bitrate=1000:qpel
-> seg fault


mencoder foo.avi -o toto.avi -ovc xvid -oac lavc -xvidencopts bitrate=1000 -> video is fine


some debugging:
Program received signal SIGSEGV, Segmentation fault.
0xf7e74ff4 in ?? () from /usr/lib/libxvidcore.so.4
(gdb) bt full
#0  0xf7e74ff4 in ?? () from /usr/lib/libxvidcore.so.4
No symbol table info available.
#1  0xf7e25609 in xvid_me_interpolate16x16qpel (x=-134516368, y=-136816742, dir=4156896560, 
    data=0xf7d857ba) at ../../src/motion/estimation_common.c:188
        Reference = (
    uint8_t *) 0xf7edf1b4 "\n+���_���\r������Zp��kS��\220\027��\220����4���:���\b��@����D���8������\220\034��07��8\037���6��P���\200���\224Z��p>�
        iEdgedWidth = 4158930000
        rounding = 4158323644
        halfpel_x = -135897344
        halfpel_y = -136717808
        ref1 = (const uint8_t *) 0xf7e9f104 "�S��\020?�
        ref2 = (const uint8_t *) 0xf7d9ec1e "U\211�WV\203�0\213E$@\203��\211E�\213E�\215\024\205"
        ref3 = (const uint8_t *) 0xf7edf208 "\224Z��p>�
        ref4 = (const uint8_t *) 0xf7e6d100 " "
Backtrace stopped: previous frame inner to this frame (corrupt stack?)



(gdb) x/8i $pc
0xf7e74ff4:     pop    %eax
0xf7e74ff5:     or     %ecx,(%edi)
0xf7e74ff7:     add    %dh,%al
0xf7e74ff9:     int3   
0xf7e74ffa:     stc    
0xf7e74ffb:     divl   0x71(%eax)
0xf7e74ffe:     sti    
0xf7e74fff:     negl   0x30f7d857(%edx)

(gdb) i r
eax            0xf762ef00       -144511232
ecx            0x8c6e740        147253056
edx            0x350    848
ebx            0xffa423e8       -6020120
esp            0xffa4236c       0xffa4236c
ebp            0xf7e74ff4       0xf7e74ff4
esi            0x0      0
edi            0x0      0
eip            0xf7e74ff4       0xf7e74ff4
eflags         0x210246 [ PF ZF IF RF ID ]
cs             0x23     35
ss             0x2b     43
ds             0x2b     43
es             0x2b     43
fs             0x0      0
gs             0x63     99



without the patch everything is fine
Comment 38 PaX Team 2007-10-15 01:43:46 UTC
Created attachment 133487 [details, diff]
textrel patch for xvid-1.1.3

thanks for the report, i fixed a few bugs i could reproduce.
Comment 39 PaX Team 2007-11-01 00:32:15 UTC
can anyone else try it/take a look please?
Comment 40 Alexis Ballier gentoo-dev 2007-12-02 17:40:45 UTC
I've added the patch to xvid 1.1.3-r1; big thanks for the huge work on it.

Could you please try to send it upstream also ? I doubt I'll be able to port it if the asm part changes in future versions
Comment 41 PaX Team 2007-12-03 14:01:45 UTC
(In reply to comment #40)
> Could you please try to send it upstream also ?

http://list.xvid.org/pipermail/xvid-devel/2007-December/005717.html

> I doubt I'll be able to port it if the asm part changes in future versions

i'll do it as time permits, no worries (maybe email me before a release as i'm not following this and other similar packages on a day-to-day basis).
Comment 42 solar (RETIRED) gentoo-dev 2008-01-22 06:03:14 UTC
*** Bug 206982 has been marked as a duplicate of this bug. ***