$ emerge python (...) test_frozen test_funcattrs test_future test_gc test_gdbm test_generators test_getargs test_getargs2 test_getopt test_gettext test_gl test_gl skipped -- No module named gl test_glob test_global python: stack smashing attack in function symtable_node() make: *** [test] Aborted !!! ERROR: dev-lang/python-2.3.5 failed. !!! Function src_test, Line 134, Exitcode 2 !!! make test failed !!! If you need support, post the topmost build error, NOT this status message. $ emerge infoPortage 2.0.51-r15 (default-linux/x86/2004.3, gcc-3.4.3, glibc-2.3.4.20050125-r0, 2.6.10-as3 i686) ================================================================= System uname: 2.6.10-as3 i686 Intel(R) Pentium(R) 4 CPU 3.40GHz Gentoo Base System version 1.6.9 Python: dev-lang/python-2.3.4-r1 [2.3.4 (#1, Feb 8 2005, 01:30:18)] ccache version 2.3 [enabled] dev-lang/python: 2.3.4-r1 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.9.4, 1.5, 1.8.5-r3, 1.6.3, 1.7.9-r1, 1.4_p6 sys-devel/binutils: 2.15.92.0.2-r2 sys-devel/libtool: 1.5.10-r4 virtual/os-headers: 2.6.8.1-r2 ACCEPT_KEYWORDS="x86 ~x86" AUTOCLEAN="yes" CFLAGS="-O2 -march=pentium4 -fomit-frame-pointer -fstack-protector -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/lib/X11/xkb /usr/lib/mozilla/defaults/pref /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-O2 -march=pentium4 -fomit-frame-pointer -fstack-protector -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs autoconfig ccache distlocks maketest sandbox sfperms test userpriv usersandbox" GENTOO_MIRRORS="ftp://mirror.switch.ch/mirror/gentoo/ http://mirror.switch.ch/mirror/gentoo/ http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/" LANG="de_DE.utf-8" LC_ALL="de_DE.utf-8" MAKEOPTS="-j1 -s" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="x86 X aalib acl acpi alsa apache2 avi bash-completion bitmap-fonts bzlib caca calendar cdparanoia cdr cpdflib crypt cups divx4linux dlloader dvd eds encode esd evo exif fam fbcon flac ftp gd gd-external gif gimpprint gnome gphoto2 gpm gstreamer gtk2 gtkhtml hal hbci howl iconv imlib2 ipv6 ithreads jpeg mad mmx mng mono moznocompose moznoirc moznomail mozsvg mpeg mpeg4 mysql ncurses nls nptl nptlonly oggvorbis opengl openssh pam pcre pic png python quicktime real rtc samba session simplexml sockets spell sqlite sse sse2 ssl svg svga theora threads tiff truetype truetype-fonts type1-fonts unicode usb userlocales vim-pager wmf xanim xmlrpc xv zlib linguas_GER" Unset: ASFLAGS, CBUILD, CTARGET, LDFLAGS
I've checked python.org for a bug report on this, but there are none. Before we look any further into this, Lars, could you please cd /usr/lib/python2.3/test; python test_global.py and let us know if it fails? (We've had tests that work fine from the shell but always fail when run from emerge)
Yes, it failes. $ python test_global.py python: stack smashing attack in function symtable_node() Aborted $ python -v test_global.py # installing zipimport hook import zipimport # builtin # installed zipimport hook # /usr/lib/python2.3/site.pyc matches /usr/lib/python2.3/site.py import site # precompiled from /usr/lib/python2.3/site.pyc # /usr/lib/python2.3/os.pyc matches /usr/lib/python2.3/os.py import os # precompiled from /usr/lib/python2.3/os.pyc import posix # builtin # /usr/lib/python2.3/posixpath.pyc matches /usr/lib/python2.3/posixpath.py import posixpath # precompiled from /usr/lib/python2.3/posixpath.pyc # /usr/lib/python2.3/stat.pyc matches /usr/lib/python2.3/stat.py import stat # precompiled from /usr/lib/python2.3/stat.pyc # /usr/lib/python2.3/UserDict.pyc matches /usr/lib/python2.3/UserDict.py import UserDict # precompiled from /usr/lib/python2.3/UserDict.pyc # /usr/lib/python2.3/copy_reg.pyc matches /usr/lib/python2.3/copy_reg.py import copy_reg # precompiled from /usr/lib/python2.3/copy_reg.pyc # /usr/lib/python2.3/types.pyc matches /usr/lib/python2.3/types.py import types # precompiled from /usr/lib/python2.3/types.pyc # /usr/lib/python2.3/warnings.pyc matches /usr/lib/python2.3/warnings.py import warnings # precompiled from /usr/lib/python2.3/warnings.pyc # /usr/lib/python2.3/linecache.pyc matches /usr/lib/python2.3/linecache.py import linecache # precompiled from /usr/lib/python2.3/linecache.pyc import encodings # directory /usr/lib/python2.3/encodings # /usr/lib/python2.3/encodings/__init__.pyc matches /usr/lib/python2.3/encodings/__init__.py import encodings # precompiled from /usr/lib/python2.3/encodings/__init__.pyc # /usr/lib/python2.3/codecs.pyc matches /usr/lib/python2.3/codecs.py import codecs # precompiled from /usr/lib/python2.3/codecs.pyc import _codecs # builtin # /usr/lib/python2.3/encodings/aliases.pyc matches /usr/lib/python2.3/encodings/aliases.pyimport encodings.aliases # precompiled from /usr/lib/python2.3/encodings/aliases.pyc # /usr/lib/python2.3/encodings/ascii.pyc matches /usr/lib/python2.3/encodings/ascii.py import encodings.ascii # precompiled from /usr/lib/python2.3/encodings/ascii.pyc Python 2.3.5 (#1, Feb 18 2005, 18:03:38) [GCC 3.4.3 20050110 (Gentoo Linux 3.4.3.20050110, ssp-3.4.3.20050110-0, pie-8.7 on linux2 Type "help", "copyright", "credits" or "license" for more information. import test # directory /usr/lib/python2.3/test # /usr/lib/python2.3/test/__init__.pyc matches /usr/lib/python2.3/test/__init__.py import test # precompiled from /usr/lib/python2.3/test/__init__.pyc # /usr/lib/python2.3/test/test_support.pyc matches /usr/lib/python2.3/test/test_support.pyimport test.test_support # precompiled from /usr/lib/python2.3/test/test_support.pyc # /usr/lib/python2.3/unittest.pyc matches /usr/lib/python2.3/unittest.py import unittest # precompiled from /usr/lib/python2.3/unittest.pyc dlopen("/usr/lib/python2.3/lib-dynload/time.so", 2); import time # dynamically loaded from /usr/lib/python2.3/lib-dynload/time.so # /usr/lib/python2.3/traceback.pyc matches /usr/lib/python2.3/traceback.py import traceback # precompiled from /usr/lib/python2.3/traceback.pyc # /usr/lib/python2.3/string.pyc matches /usr/lib/python2.3/string.py import string # precompiled from /usr/lib/python2.3/string.pyc dlopen("/usr/lib/python2.3/lib-dynload/strop.so", 2); import strop # dynamically loaded from /usr/lib/python2.3/lib-dynload/strop.so # /usr/lib/python2.3/re.pyc matches /usr/lib/python2.3/re.py import re # precompiled from /usr/lib/python2.3/re.pyc # /usr/lib/python2.3/sre.pyc matches /usr/lib/python2.3/sre.py import sre # precompiled from /usr/lib/python2.3/sre.pyc # /usr/lib/python2.3/sre_compile.pyc matches /usr/lib/python2.3/sre_compile.py import sre_compile # precompiled from /usr/lib/python2.3/sre_compile.pyc import _sre # builtin # /usr/lib/python2.3/sre_constants.pyc matches /usr/lib/python2.3/sre_constants.py import sre_constants # precompiled from /usr/lib/python2.3/sre_constants.pyc # /usr/lib/python2.3/sre_parse.pyc matches /usr/lib/python2.3/sre_parse.py import sre_parse # precompiled from /usr/lib/python2.3/sre_parse.pyc python: stack smashing attack in function symtable_node() Aborted
same here on ppc. test_syntax.py fails with the same error. Python will be killed when you use a variable before you global it. Python traps this kind of error, but with ssp gets killed for the effort. Basically if you write your code properly this should never bite you in the first place. Reproduced interactively: bash-2.05b$ python Python 2.3.4 (#1, Mar 25 2005, 18:29:53) [GCC 3.4.3 20041125 (Gentoo Hardened Linux 3.4.3-r1, ssp-3.4.3-0, pie-8.7.7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> def foo(): ... print x ... global x ... <stdin>:1: SyntaxWarning: name 'x' is used prior to global declaration python: stack smashing attack in function symtable_node() Aborted
i686-pc-linux-gnu-gcc: glibc: No such file or directory ar cr libpython2.3.a Objects/abstract.o Objects/boolobject.o Objects/bufferobject.o Objects/cellobject.o Objects/classobject.o Objects/cobject.o Objects/complexobject.o Objects/descrobject.o Objects/enumobject.o Objects/fileobject.o Objects/floatobject.o Objects/frameobject.o Objects/funcobject.o Objects/intobject.o Objects/iterobject.o Objects/listobject.o Objects/longobject.o Objects/dictobject.o Objects/methodobject.o Objects/moduleobject.o Objects/object.o Objects/obmalloc.o Objects/rangeobject.o Objects/sliceobject.o Objects/stringobject.o Objects/structseq.o Objects/tupleobject.o Objects/typeobject.o Objects/weakrefobject.o Objects/unicodeobject.o Objects/unicodectype.o ar cr libpython2.3.a Python/bltinmodule.o Python/exceptions.o Python/ceval.o Python/compile.o Python/codecs.o Python/errors.o Python/frozen.o Python/frozenmain.o Python/future.o Python/getargs.o Python/getcompiler.o Python/getcopyright.o Python/getmtime.o Python/getplatform.o Python/getversion.o Python/graminit.o Python/import.o Python/importdl.o Python/marshal.o Python/modsupport.o Python/mystrtoul.o Python/mysnprintf.o Python/pyfpe.o Python/pystate.o Python/pythonrun.o Python/structmember.o Python/symtable.o Python/sysmodule.o Python/traceback.o Python/getopt.o Python/dynload_shlib.o Python/thread.o ln: Zugriff auf ,,libpython2.3.so.1.0": No such file or directory make: *** [libpython2.3.so] Fehler 1 make: *** Warte auf noch nicht beendete Prozesse... ar cr libpython2.3.a Modules/config.o Modules/getpath.o Modules/main.o Modules/gcmodule.o make: *** Warte auf noch nicht beendete Prozesse... ar cr libpython2.3.a Modules/threadmodule.o Modules/signalmodule.o Modules/posixmodule.o Modules/errnomodule.o Modules/_sre.o Modules/_codecsmodule.o Modules/zipimport.o Modules/symtablemodule.o Modules/xxsubtype.o make: *** Warte auf noch nicht beendete Prozesse... i686-pc-linux-gnu-ranlib libpython2.3.a make: *** Warte auf noch nicht beendete Prozesse...
Comment #4 concerns bug 91024, which has been fixed
*** Bug 85224 has been marked as a duplicate of this bug. ***
Same on AMD64. Portage 2.0.51.22-r2 (default-linux/amd64/2005.0, gcc-3.4.4, glibc-2.3.5-r1, 2.6.12-gentoo-r4 x86_64) ================================================================= System uname: 2.6.12-gentoo-r4 x86_64 AMD Athlon(tm) 64 Processor 3000+ Gentoo Base System version 1.6.13 dev-lang/python: 2.3.5 sys-apps/sandbox: 1.2.12 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6 sys-devel/binutils: 2.15.92.0.2-r10 sys-devel/libtool: 1.5.18-r1 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="amd64" AUTOCLEAN="yes" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=athlon64 -O3 -pipe -fomit-frame-pointer" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /usr/X11R6/bin/startx /etc/env.d" CXXFLAGS="-march=athlon64 -O3 -pipe -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig candy distlocks sandbox severe sfperms strict test" GENTOO_MIRRORS="http://pandemonium.tiscali.de/pub/gentoo/ ftp://ftp.tu-clausthal.de/pub/linux/gentoo/ ftp://ftp.gentoo.mesh-solutions.com/gentoo/ http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/" LANG="de_DE@euro" LC_ALL="de_DE@euro" LINGUAS="de" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://rsync.de.gentoo.org/gentoo-portage" USE="X a52 aac acpi alsa amd64 audiofile avi berkdb bitmap-fonts bzlib cdr crypt cups curl dbus dga directfb dlloader dmalloc dpms dts dvd dvdr dvdread eds emacs emboss encode esd exif fam fame fat fbcon ffmpeg firefox flac foomaticdb fortran gdbm gif gimpprint gnome gpm gstreamer gtk gtk2 hal hardened howl imagemagick imlib ipv6 ithreads javascript jpeg jpeg2k kde lcms libsamplerate libwww lzo lzw lzw-tiff mad matroska memlimit mikmod mjpeg mmap mng mozcalendar mozdevelop mozsvg mp3 mpeg mpi ncurses nls nptl nptlonly ntfs nvidia ogg oggvorbis on-the-fly-crypt openal openexr opengl pam pda pdflib perl png ppds python qt quicktime readline real reiserfs sdl slang sndfile spell ssl svg symlink tcltk tcpd test tetex tga theora threads tidy tiff toolbar truetype truetype-fonts type1-fonts unicode usb userlocales videos vorbis wmf xine xml2 xmms xpm xprint xtermtoolbar xv xvid xvmc yv12 zlib linguas_de userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LDFLAGS, PORTDIR_OVERLAY
Disabled broken tests.
Tripped this on test_syntax.py, with python 2.4.1 I don't think it's a good idea to consider it as a "broken test". The stack protector is highlighting what it thinks is a buffer overrun on the stack. If there is indeed a buffer overrun, it needs to be found and fixed, not brushed under the carpet. Investigating.
(In reply to comment #8) > Disabled broken tests. The test is not broken, python is. See comment #3
still occurs with python 2.4.2 in test_syntax (bootstraping from Knoppix) Portage 2.0.51.22-r3 (default-linux/x86/2005.1, gcc-3.4.4, glibc-2.3.4.20041102-r1, 2.6.9 i686) ================================================================= System uname: 2.6.9 i686 AMD Athlon(tm) XP 1800+ Gentoo Base System version 1.6.13 ccache version 2.3 [disabled] dev-lang/python: 2.3.5-r2, 2.4.2 sys-apps/sandbox: 1.2.11 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1 sys-devel/binutils: 2.16.1 sys-devel/libtool: 1.5.20 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -march=athlon-xp -mfpmath=sse -fomit-frame-pointer -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/env.d" CXXFLAGS="-O2 -march=athlon-xp -mfpmath=sse -fomit-frame-pointer -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig distlocks maketest nostrip sandbox sfperms strict test" GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/Linux/distributions/gentoo" LANG="C" LC_ALL="C" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="x86 3dnow X a52 aac acpi aim alsa audiofile avi bcmath berkdb bitmap-fonts bzip2 caps cdb cdparanoia cdr cpdflib crypt cscope ctype cups curl dbus dbx debug dedicated dga dio doc dri dts dv dvb dvd dvdr dvdread eds emboss encode ethereal examples exif expat fam fbcon ffmpeg flac flash flatfile foomaticdb fortran ftp gb gcj gd gdbm ggi gif ginac glut gmp gnome gnutls gphoto2 gpm gstreamer gtk gtk2 gtkhtml hal hardened howl iconv icq idn ieee1394 imagemagick imap imlib inifile ipv6 jabber javascript jpeg kdexdeltas kerberos lcms ldap libg++ libwww lm_sensors lua mad maildir matroska matrox mbox mikmod mime ming mmap mmx mng motif mozilla mp3 mpeg msn nas ncurses nls nptl nsplugin offensive ogg oggvorbis openal opengl osc oscar oss pam pcntl pcre pdflib perl png ppds python quicktime readline recode ruby samba sasl sdl session sharedmem shorten skey slp sndfile snmp sockets sox speex spell sqlite sse ssl svg sysvipc tcpd test tetextheora threads tidy tiff truetype truetype-fonts type1-fonts udev unicode usb v4l vcd videos vorbis wifi wmf xface xml xml2 xmms xosd xpm xprint xv xvid yahoo zlib userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LDFLAGS, LINGUAS, PORTDIR_OVERLAY
Created attachment 74813 [details] smaller testcase This is derived from the code in python that crashes. If I compile this with -O2 -fno-strict-aliasing -fstack-protector using gcc 3.4.4 it dies as python does. Without -fno-strict-aliasing everything works fine. Now I know next to nothing about asm, but as far as I can tell the code generated for -fno-strict-aliasing -fstack-protector here is actually incorrect (only 32 bytes of "room" for the 500 bytes buffer) (checked by compiling with -save-temps and comparing the .s files). So the stack is smashed, but the c code seems to be ok, the generated asm isn't. With -fnostack-protector or without -fno-strict-aliasing there is no problem. Can someone who knows more about asm than me please verify this?
*** Bug 118136 has been marked as a duplicate of this bug. ***
*** Bug 119823 has been marked as a duplicate of this bug. ***
*** Bug 129233 has been marked as a duplicate of this bug. ***
Created attachment 111400 [details, diff] Remove stack smash (off-by-one error) Running the tests with gcc-4.1.2, the originally reported stack smash does not occur (so we could put that down to the gcc-3 SSP implementation). However, it did find a stack smash in Modules/socketmodule.c - an obvious off-by-one error in a memset() call. Attached patch fixes that - after which python-2.3.5-r3 passes its tests. The bug probably doesn't cause any actual effect - the overflowed byte on the buffer doesn't go anywhere and is unconditionally overwritten the first time another function is called. If upstream are still maintaining python-2.3.5, could push the fix upstream as well.
(In reply to comment #16) > If upstream are still maintaining python-2.3.5, could push the fix upstream as > well. > Upstreamed, see http://sourceforge.net/tracker/index.php?func=detail&aid=1713797&group_id=5470&atid=305470.
The bug is fixed in the 2.5 and 2.6 branches, so I closed the report in the Python tracker. The 2.3 (and 2.4 too) branches are not maintained anymore. Only critical security fixes can cause a new release; these should be reported via security at python org.
Fixed in python-2.3.6-r1.
