63762 2004-09-12 03:37 0000 (toolchain) egenix-mx-base won't compile when python has -fstack-protector enabled. 2005-03-15 08:08:42 0000 1 1 1 Unclassified Gentoo Linux Hardened unspecified All Linux RESOLVED FIXED P2 blocker --- 1 kl@vsen.dk hardened@gentoo.org askwar@digitalprojects.com merlin@gentoo.org python@gentoo.org rich0@gentoo.org sandino@sandino.net kl@vsen.dk 2004-09-12 03:37:13 0000 Hi guys, Just installed a new gcc with stack-smashing protection - and while recompiling modules with stack-smashing protection, I got this message: -- copying build/lib.linux-i686-2.3/mx/Misc/__init__.py -> /var/tmp/portage/egenix-mx-base-2.0.5/image/usr/l ib/python2.3/site-packages/mx/Misc copying build/lib.linux-i686-2.3/mx/Misc/Cache.py -> /var/tmp/portage/egenix-mx-base-2.0.5/image/usr/lib/ python2.3/site-packages/mx/Misc writing byte-compilation script '/var/tmp/portage/egenix-mx-base-2.0.5/temp/tmpTQ2rvx.py' /usr/bin/python -O /var/tmp/portage/egenix-mx-base-2.0.5/temp/tmpTQ2rvx.py usr/lib/python2.3/site-packages/mx/TextTools/Examples/pytag.py:47: SyntaxWarning: name 'debugging' is use d prior to global declaration python: stack smashing attack in function symtable_node() error: command '/usr/bin/python' terminated by signal 6 !!! ERROR: dev-python/egenix-mx-base-2.0.5 failed. !!! Function distutils_src_install, Line 43, Exitcode 1 !!! (no error message) so it seems egenix-mx-base causes a bufferoverflow in python which is caught by the stack-protector. p.s. it's python-2.3.3-r1. p.s. I'm using gcc-3.4.1 with USE flags: hardened pie pic and -fstack-protector Reproducible: Always Steps to Reproduce: 1.emerge python with -fstac-protector in CFLAGS 2.emerge egenix-mx-base. Actual Results: emerge failed. Expected Results: finished emerging :) kl@vsen.dk 2004-09-12 03:38:37 0000 I reported it to egenix - and here's what they said: http://lists.egenix.com/mailman-archives/egenix-users/2004-September/000476.html solar@gentoo.org 2004-09-28 18:52:52 0000 Please report this to the upstream maintainers. I don't think anybody on the hardened team uses that software. Get the author to audit said function. But also be sure your python is not compiled with -O3 and -fstack-protector-all. (python does not like -O3 and -all) nigelenki@comcast.net 2004-09-29 22:44:12 0000 /usr/bin/python -O /var/tmp/portage/egenix-mx-base-2.0.5/temp/tmpqVXq7h.py usr/lib/python2.3/site-packages/mx/TextTools/Examples/pytag.py:47: SyntaxWarning: name 'debugging' is used prior to global declaration >>>python<<<: stack smashing attack in function symtable_node() error: command '/usr/bin/python' terminated by signal 6 !!! ERROR: dev-python/egenix-mx-base-2.0.5 failed. This is dying in *python* mind you (note the >>><<< emphasising the program). Is it to be blamed on egenix for causing an overflow, or on python for having a condition in which a buffer overflows? this is needed for postgresql, btw. kl@vsen.dk 2004-09-29 23:45:13 0000 I agree it's a bug in python (as i did state - bufferoverflow in Python) - triggered by egenix. The CFLAGS I've compiled python with are these: CFLAGS="-march=i586 -m3dnow -O2 -pipe -fomit-frame-pointer -ffast-math -falign-functions=4 -falign-jumps=4 -fstack-protector -ftracer -fweb" danorris@gmail.com 2004-12-03 07:37:12 0000 I get the same problem in amd64 non-hardened, with egenix-mx-base-2.0.5 and python-2.3.4. Here's my emerge info: Portage 2.0.51-r3 (default-linux/amd64/2004.3, gcc-3.4.3, glibc-2.3.4.20040808-r1, 2.6.9-gentoo-r6 x86_64) ================================================================= System uname: 2.6.9-gentoo-r6 x86_64 AMD Athlon(tm) 64 Processor 3000+ Gentoo Base System version 1.4.16 Autoconf: sys-devel/autoconf-2.59-r5 Automake: sys-devel/automake-1.8.5-r1 Binutils: sys-devel/binutils-2.15.90.0.1.1-r3,sys-devel/binutils-2.15.92.0.2-r1 Headers: sys-kernel/linux26-headers-2.6.8.1-r1 Libtools: sys-devel/libtool-1.5.2-r7 ACCEPT_KEYWORDS="amd64" AUTOCLEAN="yes" CFLAGS="-march=athlon64 -O3 -pipe -funroll-loops -fomit-frame-pointer" CHOST="x86_64-pc-linux-gnu" COMPILER="" CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-march=athlon64 -O3 -pipe -funroll-loops -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs autoconfig ccache distlocks sandbox" GENTOO_MIRRORS="http://mirrors.tds.net/gentoo ftp://ftp6.uni-erlangen.de/pub/mirrors/gentoo http://vlaai.snt.ipv6.utwente.nl/pub/os/linux/gentoo/ ftp://ftp.ipv6.uni-muenster.de/pub/linux/distributions/gentoo http://ftp6.uni-erlangen.de/pub/mirrors/gentoo" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="X acpi aim alsa amd64 apache2 apm avi bash-completion berkdb bitmap-fonts bzlib cdr crypt ctype cups directfb divx4linux doc dvd dvdr encode esd ethereal f77 fftw flac fortran ftp gd gif gnome gpm gtk gtk2 gtkhtml hardened icq innodb ipv6 java jikes joystick jp2 jpeg junit lzw lzw-tiff mad mpeg multilib mysql mysqli ncurses nls nocd offensive oggvorbis openal opengl oscar oss pam pcre pdflib perl png posix postgres python qt quicktime readline samba sdl shared sharedmem soap sockets spell ssl svg tcpd tidy tiff truetype unicode usb userlocales vhosts videos wxwindows xine xml2 xmms xpm xrandr xv xvid yahoo zlib" mystilleef@gmail.com 2004-12-16 14:49:45 0000 Same problem here. How is this issue being resolved? Portage 2.0.51-r8 (hardened/x86, gcc-3.4.3, glibc-2.3.4.20041102-r0, 2.6.7-hardened-r17 i686) ================================================================= System uname: 2.6.7-hardened-r17 i686 AMD Athlon(tm) Processor Gentoo Base System version 1.6.8 Python: dev-lang/python-2.3.4 [2.3.4 (#1, Dec 15 2004, 08:16:14)] ccache version 2.3 [enabled] dev-lang/python: 2.3.4 sys-devel/autoconf: 2.59-r6, 2.13 sys-devel/automake: 1.8.5-r2, 1.5, 1.4_p6, 1.6.3, 1.7.9, 1.9.3 sys-devel/binutils: 2.15.92.0.2-r2 sys-devel/libtool: 1.5.10-r1 virtual/os-headers: 2.6.8.1-r1 ACCEPT_KEYWORDS="x86 ~x86" AUTOCLEAN="yes" CFLAGS="-w -pipe -funit-at-a-time -ftracer -frename-registers -fweb -march=athlon -Os -fomit-frame-pointer -fforce-addr -s -fPIC -fpic" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/lib/X11/xkb /usr/lib/mozilla/defaults/pref /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-w -pipe -funit-at-a-time -ftracer -frename-registers -fweb -march=athlon -Os -fomit-frame-pointer -fforce-addr -s -fPIC -fpic" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs ccache digest distlocks fixpackages sandbox sfperms strict" GENTOO_MIRRORS="http://mirror.datapipe.net/gentoo http://mirror.datapipe.net/gentoo http://open-systems.ufl.edu/mirrors/gentoo" MAKEOPTS="-j2" PKGDIR="/usr/portage//packages/x86/" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage /usr/local/bmg-main" SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage" USE="3dnow 3dnowex X alsa avi bindist bonobo cdparanoia cdr crypt curl dlloader dvd encode erandom esd faad fam flac gif gnome gnutls gstreamer gtk gtk2 hal hardened howl imagemagick imlib ithreads jpeg libwww live mad mmx mmx2 mng mpeg ncurses network nls nptl nptlonly oggvorbis opengl oss pam pic pie png quicktime readline real rtc speex spell ssl svg tcpd tetex theora threads tiff truetype truetype-fonts unicode usb userlocales wmf x86 xine xml xml2 xv zlib" dario@ronzani.it 2004-12-20 04:46:16 0000 Helo, I have "bypass" the problem by deleting the file pytag.py from the tarball, because in the link http://lists.egenix.com/mailman-archives/egenix-users/2004-September/000476.html posted by klavs klavsen the developer of this program say "pytag.py is no longer used or maintained, so you can safely delete it" also I had do this command to recreate the diggest information: ebuild egenix-mx-base-2.0.5.ebuild digest also with the version 2.0.6 find in the http://www.egenix.com/ the problem persist. Hope to be usefull, By Dario askwar@digitalprojects.com 2004-12-31 01:33:42 0000 Is -fstack-protector automatically enabled when a hardened toolchain is used? I'm also seeing this issue: writing byte-compilation script '/var/tmp/portage/egenix-mx-base-2.0.6/temp/tmpu2q9MO.py' /usr/bin/python -O /var/tmp/portage/egenix-mx-base-2.0.6/temp/tmpu2q9MO.py usr/lib/python2.3/site-packages/mx/TextTools/Examples/pytag.py:47: SyntaxWarning: name 'debugging' is used prior to global declaration python: stack smashing attack in function symtable_node() error: command '/usr/bin/python' terminated by signal 6 !!! ERROR: dev-python/egenix-mx-base-2.0.6 failed. However, in my CFLAGS, there's no -fstack-protector specified by me. But I did compile python with a hardened gcc: 10:29:26 askwar@alturo:/tmp $ gcc --version gcc (GCC) 3.4.3 20041125 (Gentoo Hardened Linux 3.4.3-r1, ssp-3.4.3-0, pie-8.7.7) 10:32:08 askwar@alturo:/tmp $ emerge info Portage 2.0.51-r8 (default-linux/x86/2004.3, gcc-3.4.3, glibc-2.3.4.20041102-r0, 2.6.10-gentoo-r1.PHP2.06 i686) ================================================================= System uname: 2.6.10-gentoo-r1.PHP2.06 i686 Intel(R) Celeron(R) CPU 2.00GHz Gentoo Base System version 1.6.8 Python: dev-lang/python-2.3.4 [2.3.4 (#1, Dec 31 2004, 10:25:08)] ccache version 2.3 [enabled] dev-lang/python: 2.3.4 sys-devel/autoconf: 2.59-r6, 2.13 sys-devel/automake: 1.8.5-r2, 1.5, 1.4_p6, 1.6.3, 1.7.9, 1.9.3 sys-devel/binutils: 2.15.92.0.2-r2 sys-devel/libtool: 1.5.10-r2 virtual/os-headers: 2.6.8.1-r1 ACCEPT_KEYWORDS="x86 ~x86" AUTOCLEAN="yes" CFLAGS="-march=pentium4 -O2 -pipe -fomit-frame-pointer" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-fomit-frame-pointer -pipe -O2 -march=pentium4" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs autoconfig ccache collision-protect distlocks sandbox sfperms" GENTOO_MIRRORS="http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/ ftp://mirrors.sec.informatik.tu-darmstadt.de/gentoo/ ftp://sunsite.informatik.rwth-aachen.de/pub/Linux/gentoo ftp://ftp.wh2.tu-dresden.de/pub/mirrors/gentoo http://194.117.158.29/" LDFLAGS="-O1" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="x86 acl apache2 apm arts async avi bzlib crypt cscope cups curl curlwrappers diet djbfft encode exif fam flash foomaticdb ftp fwdzone gd gdbm gif hal hardened iconv imagemagick imap immqt-bc ithreads javascript jpeg libedit libwww lzo lzw lzw-tiff mad maildir mailwrapper mbox mmap mmx mng mpeg mysql ncurses nls no-old-linux noantlr nobcel nobeanutils nobsh nocommonslogging nocommonsnet nojdepend nojsch nojython nolog4j nooro noregexp norhino noxalan noxerces nptl offensive oggvorbis opengl oss pam parse-clocks pcntl pcre pdflib perl pic pie png posix postgres pwdb python qt quicktime readline recode sasl sdl shared slang sockets spell sse ssl sysvipc tcpd tiff truetype unicode vhosts xfs xml2 xmms xv zlib linguas_de" askwar@digitalprojects.com 2004-12-31 01:33:48 0000 *** Bug 76215 has been marked as a duplicate of this bug. *** evilshenaniganz@yahoo.com 2005-01-06 23:44:03 0000 I am having the same problem with this egenix-mx-base. All this stuff floating around about -O2, -O3, -fstack-protector-all seems to yield the same problem for me. First, here is my gcc info: Reading specs from /usr/lib/gcc/i686-pc-linux-gnu/3.4.3/specs Configured with: /var/tmp/portage/gcc-3.4.3-r1/work/gcc-3.4.3/configure --enable-version-specific-runtime-libs --prefix=/usr --bindir=/usr/i686-pc-linux-gnu/gcc-bin/3.4.3 --includedir=/usr/lib/gcc/i686-pc-linux-gnu/3.4.3/include --datadir=/usr/share/gcc-data/i686-pc-linux-gnu/3.4.3 --mandir=/usr/share/gcc-data/i686-pc-linux-gnu/3.4.3/man --infodir=/usr/share/gcc-data/i686-pc-linux-gnu/3.4.3/info --with-gxx-include-dir=/usr/lib/gcc/i686-pc-linux-gnu/3.4.3/include/g++-v3 --host=i686-pc-linux-gnu --disable-altivec --enable-nls --without-included-gettext --enable-__cxa_atexit --enable-clocale=gnu --with-system-zlib --disable-checking --disable-werror --disable-libunwind-exceptions --enable-shared --enable-threads=posix --disable-multilib --enable-java-awt=gtk --enable-languages=c,c++,f77,objc,java Thread model: posix gcc version 3.4.3 20041125 (Gentoo Hardened Linux 3.4.3-r1, ssp-3.4.3-0, pie-8.7.7) Secondly, here is my emerge info: Gentoo Base System version 1.6.8 Portage 2.0.51-r8 (default-linux/x86/2004.3, gcc-3.4.3, glibc-2.3.4.20041102-r0, 2.6.9-nitro4 i686) ================================================================= System uname: 2.6.9-nitro4 i686 Intel(R) Pentium(R) M processor 1300MHz Python: dev-lang/python-2.3.4 [2.3.4 (#1, Jan 6 2005, 23:36:22)] dev-lang/python: 2.3.4 sys-devel/autoconf: 2.59-r6, 2.13 sys-devel/automake: 1.8.5-r2, 1.5, 1.4_p6, 1.6.3, 1.7.9, 1.9.3 sys-devel/binutils: 2.15.92.0.2-r2 sys-devel/libtool: 1.5.10-r2 virtual/os-headers: 2.6.8.1-r1 ACCEPT_KEYWORDS="x86 ~x86" AUTOCLEAN="yes" CFLAGS="-march=pentium4 -O2 -pipe -mfpmath=sse -mmmx -msse -msse2 -fomit-frame-pointer" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /opt/openjms/config /usr/kde/2/share/config /usr/kde/3.3/env /usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-march=pentium4 -O2 -pipe -mfpmath=sse -mmmx -msse -msse2 -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs autoconfig ccache distlocks sandbox sfperms" GENTOO_MIRRORS="http://192.168.0.104/gentoo http://gentoo.osuosl.org http://www.ibiblio.org/pub/Linux/distributions/gentoo" LDFLAGS="" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="" SYNC="rsync://rsync.gentoo.org/gentoo-portage" ###USE FLAGS WERE CUT DOWN TO RELEVANT USE FLAGS TO SAVE SPACE, THERE ARE MANY MORE IN MY /etc/make.conf### USE="X berkdb doc gdbm ipv6 ncurses readline ssl tcltk" I ran six cases, each time recompiling python with different CFLAGS/CXXFLAGS. The cases are: 1. -march=pentium4 -O2 -pipe -mfpmath=sse -mmmx -msse -msse2 -fomit-frame-pointer 2. -march=pentium4 -O2 -fstack-protector -pipe -mfpmath=sse -mmmx -msse -msse2 -fomit-frame-pointer 3. -march=pentium4 -O2 -fstack-protector-all -pipe -mfpmath=sse -mmmx -msse -msse2 -fomit-frame-pointer 4. -march=pentium4 -O3 -pipe -mfpmath=sse -mmmx -msse -msse2 -fomit-frame-pointer 5. -march=pentium4 -O3 -fstack-protector -pipe -mfpmath=sse -mmmx -msse -msse2 -fomit-frame-pointer 6. -march=pentium4 -O3 -fstack-protector-all -pipe -mfpmath=sse -mmmx -msse -msse2 -fomit-frame-pointer Taking excerpts of successfull building up to the point of failure from each attempt to build egenix-mx-base yielded the following results for every case: copying build/lib.linux-i686-2.3/mx/DateTime/Parser.py -> /var/tmp/portage/egenix-mx-base-2.0.6/image/usr/lib/python2.3/site-packages/mx/DateTime copying build/lib.linux-i686-2.3/mx/DateTime/__init__.py -> /var/tmp/portage/egenix-mx-base-2.0.6/image/usr/lib/python2.3/site-packages/mx/DateTime copying build/lib.linux-i686-2.3/mx/DateTime/Feasts.py -> /var/tmp/portage/egenix-mx-base-2.0.6/image/usr/lib/python2.3/site-packages/mx/DateTime copying build/lib.linux-i686-2.3/mx/__init__.py -> /var/tmp/portage/egenix-mx-base-2.0.6/image/usr/lib/python2.3/site-packages/mx writing byte-compilation script '/var/tmp/portage/egenix-mx-base-2.0.6/temp/tmppzCfWB.py' /usr/bin/python -O /var/tmp/portage/egenix-mx-base-2.0.6/temp/tmppzCfWB.py usr/lib/python2.3/site-packages/mx/TextTools/Examples/pytag.py:47: SyntaxWarning: name 'debugging' is used prior to global declaration python: stack smashing attack in function symtable_node() error: command '/usr/bin/python' terminated by signal 6 !!! ERROR: dev-python/egenix-mx-base-2.0.6 failed. !!! Function distutils_src_install, Line 43, Exitcode 1 !!! (no error message) !!! If you need support, post the topmost build error, NOT this status message. --- to be sure the results were identical i ran each excerpt through a word processors and each excerpt had the same characteristics: Words: 76 Characters (no spaces): 1118 Characters (with spaces): 1181 Paragraphs: 13 Lines: 20 --- Hope this helps in tracking down this bug. merlin@gentoo.org 2005-01-09 22:31:57 0000 I'm able to get this to build nicely by simply removing pytag.py during the unpack stage. Does anyone have any objections to resolving the bug in this manner? evilshenaniganz@yahoo.com 2005-01-10 11:03:26 0000 I successfully built egenix-mx-base-2.0.6 as well by removing /var/tmp/portage/egenix-mx-base-2.0.6/work/egenix-mx-base-2.0.6/mx/TextTools/Examples/pytag.py I forget where I read it, but isn't pytag.py considered to be deprecated? Can anyone verify this? liquidx@gentoo.org 2005-01-13 09:17:29 0000 removed pytag.py from egenix-mx-base .. hardened please verify? solar@gentoo.org 2005-01-22 10:44:04 0000 Alastair nobody @hardened sems to use or run this program. I think we will have to wait for the userbase to confirm. liquidx@gentoo.org 2005-03-15 08:08:42 0000 assuming fixed. reopen if found otherwise.