Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 445590 - media-libs/mesa-9.0[g3dvl,vdpau]: Segmentation fault at surface.c:366 with media-video/mplayer
Summary: media-libs/mesa-9.0[g3dvl,vdpau]: Segmentation fault at surface.c:366 with me...
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Library (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo X packagers
: 476120 (view as bug list)
Depends on:
Reported: 2012-12-02 10:12 UTC by Walther
Modified: 2015-02-21 19:05 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---

Sample patch that shows one way around the crash. (9.0.1-surfaces.patch,1.15 KB, patch)
2012-12-08 16:33 UTC, Walther
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Walther 2012-12-02 10:12:07 UTC
Since last week's world update, mplayer2 no longer works and crashes upon attempting to play a video. Recompiling a few packages with debug settings I've pinned it down to mesa-9.0.

In file src/gallium/state_trackers/vdpau/surface.c:366 (function vlVdpVideoSurfaceClear) there is a crash when attempting to access surface[4]:

 gdb --args mplayer2 <some film>
GNU gdb (Gentoo 7.3.1 p2) 7.3.1
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu".
For bug reporting instructions, please see:
Reading symbols from /usr/bin/mplayer2...done.
(gdb) run
Starting program: /usr/bin/mplayer2 <some film>
warning: Could not load shared library symbols for
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
MPlayer2 2.0-467-ga816810 (C) 2000-2012 MPlayer Team

Playing <some film>
[mkv] Track ID 1: video (V_THEORA), -vid 0
[mkv] Track ID 2: audio (A_VORBIS), -aid 0, -alang eng
[mkv] Track ID 3: subtitles (S_VOBSUB), -sid 0, -slang eng
[mkv] Will play video track 1.
Detected file format: Matroska
VIDEO:  [theo]  720x336  24bpp  25.000 fps    0.0 kbps ( 0.0 kbyte/s)
Load subtitles in <home dir>
couldn't open, software DXTn compression/decompression unavailable
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
Asking decoder to use 2 threads if supported.
[New Thread 0xf4863b40 (LWP 15747)]
[New Thread 0xf4062b40 (LWP 15748)]
Selected video codec: [fftheora] vfm: ffmpeg (FFmpeg Theora)
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
AUDIO: 48000 Hz, 2 ch, s16le, 0.0 kbit/0.00% (ratio: 0->192000)
Selected audio codec: [ffvorbis] afm: ffmpeg (FFmpeg Vorbis)
AO: [alsa] 48000Hz 2ch s16le (2 bytes per sample)
Starting playback...
A:   0.0 V:   0.0 A-V:  0.000 ct:  0.000   0/  0 ??% ??% ??,?% 0 0 
Movie-Aspect is 3.05:1 - prescaling to correct movie aspect.
VO: [vdpau] 720x336 => 1026x336 Planar YV12 
[vdpau] Got display refresh rate 59.997 Hz.
[vdpau] If that value looks wrong give the -vo vdpau:fps=X suboption manually.

Program received signal SIGSEGV, Segmentation fault.
vlVdpVideoSurfaceClear (vlsurf=0x85d8fd0) at surface.c:366
366	      pipe->clear_render_target(pipe, surfaces[i], &c, 0, 0,
(gdb) p surfaces[i]
$1 = (struct pipe_surface *) 0x61
(gdb) p i
$2 = 4
(gdb) p surfaces[0]
$3 = (struct pipe_surface *) 0x85d4198
(gdb) p surfaces[1]
$4 = (struct pipe_surface *) 0x85d90e0
(gdb) p surfaces[2]
$5 = (struct pipe_surface *) 0x0
(gdb) p surfaces[3]
$6 = (struct pipe_surface *) 0x0

It seems the problem is that the loop in this function goes until VL_MAX_SURFACES, but get_surfaces() is returning an array with only four elements defined (VL_MAX_SURFACES is 6).

If in this function I replace the continue for a break:
     if (!surfaces[i])

mplayer2 works, but it only displays a black screen, so this probably isn't the right solution. I tried changing the get_surfaces method (nouveau_video_buffer_surfaces at src/gallium/drivers/nouveau/nouveau_video.c) to fill in the rest of the array with NULL, but that gave me instead an assertion failure elsewhere... I don't know enough about Mesa to fix this, but at least I know where the bug is.

Reproducible: Always

Steps to Reproduce:
1. play a video
2. crash

Actual Results:  
Segmentation fault.

Expected Results:  
Play a video.

> emerge --info
Portage (default/linux/x86/10.0/desktop, gcc-4.5.4, glibc-2.15-r3, 3.5.7-gentoo-64 x86_64)
System uname: Linux-3.5.7-gentoo-64-x86_64-Intel-R-_Core-TM-2_Duo_CPU_T9550_@_2.66GHz-with-gentoo-2.1
Timestamp of tree: Sat, 01 Dec 2012 04:15:01 +0000
ld GNU ld (GNU Binutils) 2.22
app-shells/bash:          4.2_p37
dev-java/java-config:     2.1.11-r3
dev-lang/python:          2.7.3-r2, 3.2.3
dev-util/cmake:           2.8.9
dev-util/pkgconfig:       0.27.1
sys-apps/baselayout:      2.1-r1
sys-apps/openrc:          0.11.6
sys-apps/sandbox:         2.5
sys-devel/autoconf:       2.13, 2.68
sys-devel/automake:       1.10.3, 1.11.6
sys-devel/binutils:       2.22-r1
sys-devel/gcc:            4.5.4
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4-r1
sys-devel/make:           3.82-r3
sys-kernel/linux-headers: 3.6 (virtual/os-headers)
sys-libs/glibc:           2.15-r3
Repositories: gentoo Local
CFLAGS="-march=native -O0 -ggdb"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-march=native -O0 -ggdb"
FCFLAGS="-O2 -march=i686 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news nostrip parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox"
FFLAGS="-O2 -march=i686 -pipe"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="fr_FR fr en_GB en es_MX es"
MAKEOPTS="-j3 -s"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
USE="3dnow X a52 aac acpi alsa avi berkdb boundschecking bzip2 cairo canna cdda cdr cjk cli cracklib crypt cscope cups curl cxx dbus dedicated dga divx divx4linux dlloader dri dts dvd dvdr dvdread emboss encode exif fam fbcon fbsplash ffmpeg fftw flac foomaticdb fortran freewnn gd gdbm gif gimp ginac gmedia gpm gstreamer gtk gtk2 gtkhtml hal hddtemp howl iconv icu idn imap imlib innodb inotify ipv6 ithreads jabber java java6 jikes joystick jpeg lame lcms libnotify libsamplerate libwww lm_sensors lzma mad madwifi matroska mbox mmx mng modplug modules mp3 mp4 mpeg mplayer mudflap musepack ncurses nls nptl nsplugin ogg opencl opengl openmp pam pango pcre pdf pdflib perl png ppds pppd python quicktime readline realmedia scanner schroedinger sdl session speex spell sse sse2 ssl ssse3 svg svga tcpd tetex theora threads tiff timidity truetype udev udisks unicode upower usb v4l v4l2 vaapi vdpau vidix vim-syntax vorbis webkit win32codecs wma wmf wmp wxwidgets x264 x86 xcb xcomposite xface xft xine xml xml2 xscreensaver xv xvid zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev keyboard mouse synaptics wacom joystick" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="fr_FR fr en_GB en es_MX es" PHP_TARGETS="php5-3" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_2" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby18 ruby19" SANE_BACKENDS="plustek pixma" USERLAND="GNU" VIDEO_CARDS="nouveau vesa" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Comment 1 Walther 2012-12-08 16:31:00 UTC
Upon further inspection, the crash is caused because vlVdpVideoSurfaceClear assumes that the surfaces pointer array returned by get_surfaces() has VL_MAX_SURFACES elements, whereas the surfaces array returned by nouveau_video_buffer_surfaces only has 3 elements (as is defined in src/gallium/drivers/nouveau/nouveau_video.h, "struct nouveau_video_buffer").

If I change surfaces[3] to surfaces[VL_MAX_SURFACES], and initialize these to null, then the crash goes away, and the video plays. However, from time to time the video stalls for several seconds before updating. With my current knowledge of the software, I cannot guess at what might be causing that problem.
Comment 2 Walther 2012-12-08 16:33:09 UTC
Created attachment 331814 [details, diff]
Sample patch that shows one way around the crash.
Comment 3 Matt Turner gentoo-dev 2015-02-21 18:58:59 UTC
Is this still a problem with a more recent Mesa (try >=10.5.0_rc1)? If so, please let us know and then report to upstream.
Comment 4 Matt Turner gentoo-dev 2015-02-21 19:05:07 UTC
*** Bug 476120 has been marked as a duplicate of this bug. ***