Python and/or matplotlib and/or pygtk (I'll need help to figure it out) segfaults using threads. I've got a Python program using pygtk, matplotlib and python threading that fails on my system, but not on other distributions such as Ubuntu. See steps to reproduce to see if you can reproduce this behavior... Reproducible: Always Steps to Reproduce: 1. set backend to GtkAgg in /etc/matplotlib/matplotlibrc 2. run "ipython -pylab" (to create an ipython session using interactive mode of matplotlib + threads) 3. plot([1,2,3]) Actual Results: Segmentation fault Expected Results: No Segmentation fault This behavior has been seen in another python program (not using ipython) that creates a thread that avoids "pyplot.show()" to be blocking. This program uses pygtk and fails even if another Agg is used (Qt4Agg for instance). This program runs fine as well in Ubuntu. That fact, along with the fact that "ipython -pylab" doesn't fail suggests that there is no bug in the program. I'll need your help guys to figure out what exact package is broken. # emerge --info Portage 2.2_rc67-r2 (default/linux/amd64/2008.0, gcc-4.4.3, glibc-2.11-r1, 2.6.35-gentoo-r7 x86_64) ================================================================= System uname: Linux-2.6.35-gentoo-r7-x86_64-Intel-R-_Core-TM-2_CPU_T5600_@_1.83GHz-with-gentoo-2.1.8 Timestamp of tree: Wed, 22 Sep 2010 05:30:01 +0000 app-shells/bash: 4.1_p7 dev-java/java-config: 2.1.11 dev-lang/python: 2.6.5-r3, 3.1.2-r4 dev-util/cmake: 2.8.1-r2 sys-apps/baselayout: 2.1.8 sys-apps/openrc: 0.6.1-r3 sys-apps/sandbox: 2.3-r1 sys-devel/autoconf: 2.13, 2.67 sys-devel/automake: 1.8.5-r4, 1.9.6-r3, 1.10.3, 1.11.1 sys-devel/binutils: 2.20.1 sys-devel/gcc: 4.4.3 sys-devel/gcc-config: 1.4.1 sys-devel/libtool: 2.2.10 sys-devel/make: 3.81-r2 virtual/os-headers: 2.6.32 ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=core2 -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/X11/xkb /usr/share/config" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /etc/udev/rules.d" CXXFLAGS="-march=core2 -O2 -pipe" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--jobs=2 --load-average=2.0 --with-bdeps y" FEATURES="assume-digests distlocks fixpackages mini-manifest news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unmerge-logs unmerge-orphans userfetch" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--sort-common -Wl,--as-needed" LINGUAS="fr en" MAKEOPTS="-j3 -l3" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/var/lib/layman/sunrise /usr/local/portage/local-overlay" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="64bit X a52 aac acl acpi alac alsa amd64 berkdb bluetooth branding bzip2 cairo cblas cddb cdr cleartype cli consolekit corefonts cracklib crypt cups cxx dbus dri dts dvd dvdr dvdread eds encode exif faac fam ffmpeg flac fortran gdbm gif gpm gsl gstreamer gtk iconv ipv6 irc jabber jpeg kde lame lapack latex lcms ldap libnotify mad mikmod mmx mng modules mp3 mpeg msn mudflap multilib ncurses networkmanager nls nptl nptlonly ogg opengl openmp pam pango pcre pdf perl png pppd pulseaudio python qt3support qt4 quicktime readline reflection sdl semantic-desktop session sip spell sse sse2 ssl startup-notification subversion svg sysfs tcpd tiff truetype type1 unicode usb vorbis wavpack webkit x264 xcb xcomposite xml xorg xscreensaver xulrunner xv xvid zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="fr en" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, FFLAGS, INSTALL_MASK, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Update : I've been able find out what segfaults exactly. See the *very* simple program attached. When importing "gtk", it fails (event if I'm not using it). Comment the import and voilà, it works.
Created attachment 248489 [details] The program segfaulting
(In reply to comment #2) > Created an attachment (id=248489) [details] > The program segfaulting > Not sure if this will help but here is a backtrace from running this test program: (gdb) run test.py Starting program: /usr/bin/python2.7 test.py [Thread debugging using libthread_db enabled] Program received signal SIGSEGV, Segmentation fault. __cxxabiv1::__cxa_allocate_exception (thrown_size=136) at /tmp/portage/sys-devel/gcc-4.4.3/work/gcc-4.4.3/libstdc++-v3/libsupc++/eh_alloc.cc:133 133 /tmp/portage/sys-devel/gcc-4.4.3/work/gcc-4.4.3/libstdc++-v3/libsupc++/eh_alloc.cc: No such file or directory. in /tmp/portage/sys-devel/gcc-4.4.3/work/gcc-4.4.3/libstdc++-v3/libsupc++/eh_alloc.cc (gdb) bt #0 __cxxabiv1::__cxa_allocate_exception (thrown_size=136) at /tmp/portage/sys-devel/gcc-4.4.3/work/gcc-4.4.3/libstdc++-v3/libsupc++/eh_alloc.cc:133 #1 0x00007fffe82a45b4 in py_to_agg_transformation_matrix (obj=0x0, errors=5) at src/agg_py_transforms.cpp:20 #2 0x00007fffe82af98b in _path_module::update_path_extents ( this=<value optimized out>, args=...) at src/path.cpp:374 #3 0x00007fffe82b65a8 in Py::ExtensionModule<_path_module>::invoke_method_varargs (this=<value optimized out>, method_def=<value optimized out>, args=...) at /usr/include/python2.7/CXX/ExtensionModule.hxx:184 #4 0x00007fffe82a107d in Py::method_varargs_call_handler ( _self_and_name_tuple=<value optimized out>, _args=<value optimized out>) at CXX/cxx_extensions.cxx:1714 #5 0x00007ffff7b0cd7e in call_function (f=0x1cfc150, throwflag=<value optimized out>) at Python/ceval.c:4012 #6 PyEval_EvalFrameEx (f=0x1cfc150, throwflag=<value optimized out>) at Python/ceval.c:2665 #7 0x00007ffff7b0e8cf in PyEval_EvalCodeEx (co=0x13a44b0, globals=<value optimized out>, locals=<value optimized out>, args=0x5, argcount=9, kws=<value optimized out>, kwcount=2, defs=0x13e06a8, defcount=3, closure=0x0) at Python/ceval.c:3252 #8 0x00007ffff7b0ca22 in fast_function (f=0x1cfae40, throwflag=<value optimized out>) at Python/ceval.c:4108
I can reproduce it here. Change your default backend from GTKAgg to TkAgg (which is ubuntu default afaik) in your /etc/matplotlib/matplotlibrc meanwhile we are working on it. Thanks.
Yep that's the workaround I'm using (well, I'm using QT4Agg), but I'm glad you're able to reproduce it. However, I had this problem in another context (a pygtk application that uses matplotlib), but we decided switch to a CLI based on ipython. Thanks :-)
Good news! I'm unable to reproduce the problem today. There is no segfault running this little test, and I can use GTKAgg without any issue in ipython/pylab. Can somebody else that had this problem confirm that it's working well now? I will then close the bug.
it is indeed gone whether bug was in pygtk, python or somewhere else. reopen if it does reappear.
I can still reproduce segmentation fault.
For reference: similar issue with sage from the sage-on-gentoo overlay: https://github.com/cschwan/sage-on-gentoo/issues/issue/40/ I've tried to investigate this in some depth, and have discovered a problem with the thread local store as the immediate cause of the segfault. Not yet sure where this comes from. It doesn't seem related to matplotlib, but might be python or pygtk. If someone understands these generations in the TLS implementation of glibc, that might help in deciding how things SHOULD work. http://tinyurl.com/27p7mjd#l749 is the glibc function __tls_get_addr which returns NULL although it shouldn't.
This is still a problem. I just installed ipython-0.10.1 and it crashes when I try to plot. Plotting from a normal python shell works. Please note that changing the backend DOES NOT fix the problem for me. All backends crash when a plot command is issued. $ emerge --info Portage 2.1.9.42 (default/linux/amd64/10.0, gcc-4.4.5, glibc-2.11.3-r0, 2.6.36-gentoo-r5 x86_64) ================================================================= System uname: Linux-2.6.36-gentoo-r5-x86_64-Intel-R-_Core-TM-_i7_CPU_920_@_2.67GHz-with-gentoo-1.12.14 Timestamp of tree: Wed, 23 Mar 2011 23:15:01 +0000 app-shells/bash: 4.1_p9 dev-java/java-config: 2.1.11-r3 dev-lang/python: 2.6.6-r2, 3.1.3-r1 dev-util/cmake: 2.8.4 sys-apps/baselayout: 1.12.14-r1 sys-apps/sandbox: 2.4 sys-devel/autoconf: 2.13, 2.65-r1 sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.1 sys-devel/binutils: 2.20.1-r1 sys-devel/gcc: 4.4.5 sys-devel/gcc-config: 1.4.1 sys-devel/libtool: 2.2.10 sys-devel/make: 3.81-r2 virtual/os-headers: 2.6.36.1 (sys-kernel/linux-headers) ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -march=core2 -msse4 -msse4.1 -msse4.2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /var/lib/hsqldb" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-O2 -march=core2 -msse4 -msse4.1 -msse4.2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="assume-digests binpkg-logs distlocks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch" FFLAGS="" GENTOO_MIRRORS="http://gentoo.chem.wisc.edu/gentoo http://distfiles.gentoo.org http://www.ibiblio.org/pub/Linux/distributions/gentoo" LANG="C" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LINGUAS="en" MAKEOPTS="-j9" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage" USE="X a52 acl acpi alsa amd64 bash-completion berkdb blas branding bzip2 bzlib cairo caps cblas cdda cddb cdparanoia cdr cli consolekit cracklib crypt cups cxx dbus device-mapper dri dvb dvd dvdr dvdread eds emacs emboss encode esd evo extras fam ffmpeg fftw firefox flac fltk fortran fuse gcj gd gdbm gdu gif gimp gnome gnome-keyring gnomecd gnuplot gnutls gpm gstreamer gtk guile hal iconv id3tag ipv6 jack java jpeg kde lame lapack latex libnotify libsamplerate mad mikmod mmx modules mono mp3 mpeg mudflap multilib mysql nautilus ncurses networking networkmanager nls nptl nptlonly nsplugin ogg opengl openmp pam pcre pdf perl plotutils png policykit postscript ppds pppd python qt3 qt3support qt4 readline samba sasl sdl session smp spell sqlite sse sse2 ssl svg sysfs tcpd theora threads tiff tk truetype twolame udev unicode usb v4l v4l2 vorbis wxwidgets wxwindows x264 xml xorg xulrunner xv xvid zlib" ALSA_CARDS="emu10k1x" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY
I just checked, and changing the backend does not avoid the problem anymore for me too.
(In reply to comment #10) > This is still a problem. I just installed ipython-0.10.1 and it crashes when I > try to plot. Plotting from a normal python shell works. > > Please note that changing the backend DOES NOT fix the problem for me. All > backends crash when a plot command is issued. i suspect the crash you are experiencing is different than the one reported. could you backtrace your crash and compare it to comment #3? and also could you try with ipython-0.10? thanks
Forget my previous message, this still fails even in the plain python interpreter. Mark, please try to import gtk before using pyplot, and post here if it worked for you. Commenting the import is still a work-around (that could be problematic in some programs) The work-around of changing the Agg is no more a work-around. Here is a fresh backtrace : (gdb) run test.py Starting program: /usr/bin/python test.py process 1247 is executing new program: /usr/bin/python2.6 [Thread debugging using libthread_db enabled] [New Thread 0x7fffdc000710 (LWP 1250)] Program received signal SIGSEGV, Segmentation fault. 0x00007fffed8e4594 in __cxa_allocate_exception () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.4.5/libstdc++.so.6 (gdb) bt #0 0x00007fffed8e4594 in __cxa_allocate_exception () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.4.5/libstdc++.so.6 #1 0x00007fffe783e7f4 in py_to_agg_transformation_matrix(_object*, bool) () from /usr/lib64/python2.6/site-packages/matplotlib/_path.so #2 0x00007fffe784946b in _path_module::update_path_extents(Py::Tuple const&) () from /usr/lib64/python2.6/site-packages/matplotlib/_path.so #3 0x00007fffe7850168 in Py::ExtensionModule<_path_module>::invoke_method_varargs(void*, Py::Tuple const&) () from /usr/lib64/python2.6/site-packages/matplotlib/_path.so #4 0x00007fffe7838b1d in method_varargs_call_handler () from /usr/lib64/python2.6/site-packages/matplotlib/_path.so #5 0x00007ffff7b158e5 in PyEval_EvalFrameEx () from /usr/lib/libpython2.6.so.1.0 #6 0x00007ffff7b17727 in PyEval_EvalCodeEx () from /usr/lib/libpython2.6.so.1.0 #7 0x00007ffff7b15af1 in PyEval_EvalFrameEx () from /usr/lib/libpython2.6.so.1.0 #8 0x00007ffff7b17727 in PyEval_EvalCodeEx () from /usr/lib/libpython2.6.so.1.0 #9 0x00007ffff7b15af1 in PyEval_EvalFrameEx () from /usr/lib/libpython2.6.so.1.0 #10 0x00007ffff7b17727 in PyEval_EvalCodeEx () from /usr/lib/libpython2.6.so.1.0 #11 0x00007ffff7b15af1 in PyEval_EvalFrameEx () from /usr/lib/libpython2.6.so.1.0 #12 0x00007ffff7b17727 in PyEval_EvalCodeEx () from /usr/lib/libpython2.6.so.1.0 #13 0x00007ffff7aaadbd in ?? () from /usr/lib/libpython2.6.so.1.0 #14 0x00007ffff7a83102 in PyObject_Call () from /usr/lib/libpython2.6.so.1.0 #15 0x00007ffff7b144eb in PyEval_EvalFrameEx () from /usr/lib/libpython2.6.so.1.0 #16 0x00007ffff7b17727 in PyEval_EvalCodeEx () from /usr/lib/libpython2.6.so.1.0 #17 0x00007ffff7b15af1 in PyEval_EvalFrameEx () from /usr/lib/libpython2.6.so.1.0 #18 0x00007ffff7b17727 in PyEval_EvalCodeEx () from /usr/lib/libpython2.6.so.1.0 #19 0x00007ffff7b17802 in PyEval_EvalCode () from /usr/lib/libpython2.6.so.1.0 #20 0x00007ffff7b318dc in ?? () from /usr/lib/libpython2.6.so.1.0 #21 0x00007ffff7b319b2 in PyRun_FileExFlags () from /usr/lib/libpython2.6.so.1.0 #22 0x00007ffff7b32e8c in PyRun_SimpleFileExFlags () from /usr/lib/libpython2.6.so.1.0 #23 0x00007ffff7b3ef1c in Py_Main () from /usr/lib/libpython2.6.so.1.0 #24 0x00007ffff74e6bbd in __libc_start_main () from /lib/libc.so.6 #25 0x00000000004008c9 in _start ()
Could you try the glibc patch of Martin (discussed in comment #9). You can find instructions on how to apply it here: https://github.com/cschwan/sage-on-gentoo/issues/#issue/40/comment/722867
(In reply to comment #14) > Could you try the glibc patch of Martin (discussed in comment #9). Discussed in more detail in bug #353224, on which this one here depends.
Sorry for the delay. I confirm that the discussed patch resolves this issue for me.
Also have the same problem. I need matplotlib and don't know how to use the patch. Is there any estimation when the patch would hit stable?
(In reply to comment #17) > don't know how to use the patch. It seems the link from comment #14 doesn't point to the proper comment on github any more. Apparently the correct path and anchor would now be this: https://github.com/cschwan/sage-on-gentoo/issues/40#issuecomment-722867 In any case, I've written a beter instruction about how to use the patch in bug #353224 comment #18. It will include the patch in all upgrades as well, so you won't accidentially loose the fix when your glibc is upgraded. > Is there any estimation when the patch would hit stable? I guess having it in stable will need some time in testing first, but I'm wondering when that's going to happen, too. You could vote for bug #353224 to express your desire for a fix there.
x86 arch team doesn't belong here. Don't add us to the CC list if you don't know what we do.
closing since patch has been applied. re-open if still a problem. thanks. *** This bug has been marked as a duplicate of bug 353224 ***
