Home | Docs | Forums | Lists | Bugs | Planet | Store | GMN | Get Gentoo!
Not eligible to see or edit group visibility for this bug.
View Bug Activity | Format For Printing | XML | Clone This Bug
Python crashs with -fstack-protector-all compiled on making an instance of smtplib. Please change filter-flags -malign-double to filter-flags -malign-double -fstack-protector-all Reproducible: Always Steps to Reproduce: 1. CFLAGS="-O3 -fstack-protector-all" emerge python 2. python Python 2.3.3 (#1, May 7 2004, 05:28:30) [GCC 3.3.3 20040412 (Gentoo Linux 3.3.3-r3, ssp-3.3-7, pie-8.5.3)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import smtplib >>> server = smtplib.SMTP('localhost') Actual Results: python: stack smashing attack in function fast_function() Segmentation fault Expected Results: creating an instance of smtplib that I can use to spam myself to death ;) Kernel is with grsec patch, PaX enabled. cat root # emerge info Portage 2.0.50-r6 (default-x86-1.4, gcc-3.3.3, glibc-2.3.3_pre20040420-r0, 2.6.5-grsec) ================================================================= System uname: 2.6.5-grsec i686 AMD Athlon(tm) prosussor Gentoo Base System version 1.4.10 Autoconf: sys-devel/autoconf-2.59-r3 Automake: sys-devel/automake-1.8.3 ACCEPT_KEYWORDS="x86 ~x86" AUTOCLEAN="yes" CFLAGS="-O3" CHOST="i686-pc-linux-gnu" COMPILER="gcc3" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-O3" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs ccache sandbox" GENTOO_MIRRORS="http://ftp.easynet.nl/mirror/gentoo/ ftp://ftp.join.uni-muenster.de/pub/linux/distributions/gentoo ftp://ftp.tu-clausthal.de/pub/linux/gentoo/ http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/ http://ftp.uni-erlangen.de/pub/mirrors/gentoo" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/root/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="X509 acl apache2 berkdb crypt curl flash gd gdbm ipv6 jpeg ldap maildir mysql ncurses nls pdflib perl png python readline skey ssl tiff truetype x86 xml2"
I can confirm that "-fstack-protector" compiler flag seems to hurt the python executable extremely. I have my own python program which started to crash the python interpreter after i compiled python with "-fstack-protector". I tried some combinations, and finally the program is working fine again after recompiling python (2.3.3) without "-fstack-protector". So it should be better to remove all stack-protector flags when compiling python to protect the users from itself. But because if someone is playing with these compiler flags he should know what he is doing, i would rate this bug minor :-) For completeness: Portage 2.0.50-r6 (default-x86-1.4, gcc-3.3.2, glibc-2.3.2-r9, 2.4.25-gentoo-r2) ================================================================= System uname: 2.4.25-gentoo-r2 i686 AMD Athlon(TM) XP 1900+ Gentoo Base System version 1.4.10 distcc 2.13 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [enabled] Autoconf: sys-devel/autoconf-2.59-r3 Automake: sys-devel/automake-1.8.3 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CFLAGS="-mcpu=athlon-xp -O3 -pipe -fomit-frame-pointer" CHOST="i686-pc-linux-gnu" COMPILER="gcc3" CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3.2/share/config /usr/kde/3/share/config /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-mcpu=athlon-xp -O3 -pipe -fomit-frame-pointer" DISTDIR="/alderan/distfiles" FEATURES="autoaddcvs ccache distcc fixpackages sandbox" GENTOO_MIRRORS="ftp://ftp.freenet.de/pub/ftp.snt.utwente.nl/pub/os/linux/gentoo http://ftp.easynet.nl/mirror/gentoo/ http://www.mirror.ac.uk/sites/www.ibiblio.org/gentoo/" MAKEOPTS="-j4" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.ohlmeier.home/gentoo-portage" USE="3dnow X X509 aalib acpi acpi4linux adns alsa arts artswrappersuid avi berkdb cdr crypt cscope cups dga directfb doc dvb dvd encode ethereal fbcon flash foomaticdb gdbm gif gimpprint gphoto2 gpm gstreamer gtk gtk2 guile hardened hbci idea imap imlib innodb irda java jpeg kde libg++ libwww lirc mad maildir matroska matrox mbox mikmod mmx motif mozilla mpeg mysql nas ncurses nls odbc ofx oggvorbis opengl oss pam pda pdflib perl pic png ppds python qt qtmt quicktime readline samba sasl scanner sdl skey slang slp speex spell sse ssl svga tcltk tcpd tetex theora tiff truetype usb v4l v4l2 wmf x86 xinerama xml xml2 xmms xv zlib"
is the stack protector stuff managed by hardended? i don't have a pax enabled kernel so i can't reproduce. what negative affects are there for filtering this cflag?
Hey.... python works just fine with -fstack-protector and -fstack-protector-all Don't filter it! The problem is that function has an array that can be overflowed. Fix the python core code instead of filtering it.
good, as long as someone knows whats going on here. anyone with a patch?
*** Bug 59603 has been marked as a duplicate of this bug. ***
*** Bug 59604 has been marked as a duplicate of this bug. ***
The gentoo python herd/maintainer should take care of patching her/his package and/or make arrangements/get advice with/from upstream as/if needed.
well, what are the requirements to reproduce this? i have no hardened stuff running at all, so i'm not even sure what it involves? the closest thing i've seen is this openbsd bug where they use stack protector, but they seem to have figured it out? http://cvs.openbsd.org/cgi-bin/query-pr-wrapper?full=yes&numbers=3622
For whoever's still looking into this: I believe the simplest example of something pythonic that doesn't work with -fstack-protector* is "idle," the basic python development environment that comes with python (tcltk must be in USE, of course). I am running no grsec, no pax, no hardened kernel. I don't know if this is going to be considered an upstream problem or not, but as it stands, even some things that come with python and are integral to it fail. IMHO, -fstack-protector should be filtered at least for the time-being. # emerge info Portage 2.0.50-r9 (default-x86-2004.2, gcc-3.3.3, glibc-2.3.3.20040420-r1, 2.6.8-gentoo) ================================================================= System uname: 2.6.8-gentoo i686 Pentium III (Coppermine) Gentoo Base System version 1.4.16 distcc 2.13 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [enabled] Autoconf: sys-devel/autoconf-2.59-r4 Automake: sys-devel/automake-1.8.3 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CFLAGS="-O3 -march=pentium3 -pipe -fomit-frame-pointer -fstack-protector" CHOST="i686-pc-linux-gnu" COMPILER="gcc3" CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3/share/config /usr/lib/mozilla/defaults/pref /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-O3 -march=pentium3 -pipe -fomit-frame-pointer -fstack-protector" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs ccache distcc fixpackages sandbox" GENTOO_MIRRORS="http://mirror.datapipe.net/gentoo http://mirrors.tds.net/gentoo ftp://linux.thai.net/pub/mirrors/gentoo" MAKEOPTS="-j4" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="X aalib alsa apache2 apm avi crypt cups encode flash foomaticdb gd gnome gtk gtk2 imagemagick imap imlib jpeg lcms libg++ libwww mad mcal memlimit mikmod mmap mmx motif mozilla mpeg mysql ncurses nls nntp oggvorbis opengl pam pdflib perl png postgres ppds python quicktime readline samba sasl sdl slang slp spell sse ssl svga tcltk threads tiff truetype usb vhosts x86 xml xml2 xmms xv zlib"
Test box #1 (solar@toucan:~)$ python Python 2.3.3 (#1, May 15 2004, 21:11:00) [GCC 3.3.2 20031218 (Gentoo Linux 3.3.2-r5, propolice-3.3-7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import smtplib; >>> server = smtplib.SMTP("localhost"); >>> -------------------------------------------------------------- Test Box #2 echo 'import smtplib; server = smtplib.SMTP("smtp.gentoo.org");' | python - ; echo $? 0 Python 2.3.3 (#1, Jan 14 2004, 15:32:05) [GCC 3.3.2 20031218 (Gentoo Linux 3.3.2-r5, propolice-3.3-7)] on linux2 ---------------------------------------------------------------------- Test Box #3 echo 'import smtplib; server = smtplib.SMTP("smtp.gentoo.org");' | python - ; echo $? 0 Python 2.3.4 (#1, Aug 17 2004, 05:55:33) [GCC 3.4.1 20040803 (Gentoo Hardened Linux 3.4.1-r2, ssp-3.4-2, pie-8.7.6.5)] on linux2 Type "help", "copyright", "credits" or "license" for more information. ----------------------------------------------------------------------------- Test Box #4 echo 'import smtplib; server = smtplib.SMTP("smtp.gentoo.org");' | python - ; echo $? 0 Python 2.3.3 (#1, Apr 18 2004, 02:20:33) [GCC 3.3.3 20040217 (Gentoo PaX Linux 3.3.3-r1, ssp-3.3-7, pie-0.8.5.3)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> ----------------------------------------------------------------------------- I can't even reproduce this on the 4 boxes I've tested on. All of them use the hardened toolchain and or -fstack-protector{,-all} via CFLAGS.
I've talked with some others. It does seem that this might only happen when -O3 is in use. Testing
Yes, I can confirm that. Just tried recompiling python with -O2 instead of -O3 (all other options the same), and the problem dissapears. So it seems it might be a bug in GCC. Either it's a problem of the stack protector patches, or the bug is already there and stack protector only makes it visible.
Created an attachment (id=37582) [details] python-2.3.4.ebuild.diff Yep.. Just confirmed this only happens at O3 ------------------------------------------------------------------- echo 'import smtplib; server = smtplib.SMTP("localhost");' | python - ; echo $? python: stack smashing attack in function call_function() Aborted 134 Python 2.3.4 (#1, Aug 17 2004, 07:05:07) [GCC 3.3.3 20040412 (Gentoo Hardened Linux 3.3.3-r5, ssp-3.3-7, pie-8.7.5.3)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> I'll attach a patch shortly for your testing pleasure.
Confirmed for the "idle" case as well. The bug is only triggered with -O3. Note: Bug #54856 (an evms bug) appears to be the same bug (same behavior). Reassign to toolchain?
No need to reassign to toolchain@ The bug is with python, and python maintainer is given a chance to update his ebuild. If no objections or if Alastair is fine with it I can add the work around it to the ebuild.
I don't understand why you consider this a python bug. It is triggered by a compiler optimization option and affects at least two completely different packages (python and evms - see comment #14). It seems to me that this is a compiler issue. Of course, I've now learned that using any optimizations higher than -O2 with ProPolice is not recommended anyway, apparently because it can optimize away some of the protection.
15:57:09 [/space/chroots/chroot001:24976.pts-49.papillon]papillon ~ # rm -rf /var/tmp/portage/python-2.3.*; FEATURES="debug nostrip keeptemp keepwork" ACCEPT_KEYWORDS="~x86" CFLAGS="-g -ggdb -O3" emerge -v python 2>&1 | tee /tmp/python.txt # cat /root/bug.py #!/usr/bin/python import smtplib; smtplib.SMTP("smtp.gentoo.org"); 15:59:38 [/space/chroots/chroot001:24976.pts-49.papillon]papillon ~ # ./bug.py Killed Aug 18 15:59:55 papillon PAX: execution attempt in: /usr/lib/libpython2.3.so.1.0, 2156c000-21664000 00000000 Aug 18 15:59:55 papillon PAX: terminating task: /usr/bin/python2.3(bug.py):5860, uid/euid: 0/0, PC: 21630b58, SP: 581e1a50 Aug 18 15:59:55 papillon PAX: bytes at PC: 8b 72 0c 89 72 08 8b 12 39 fa 75 f4 39 f9 89 ce 74 2a 8d 8b Aug 18 15:59:55 papillon PAX: bytes at SP: 215850f6 00000000 0000001a 2167fe00 215c068e 153e82ec 00000000 581e1918 21847000 153e82ec 21823060 21812824 215b851a 2167fe28 00000000 00000000 624a7f12 2165dda9 581e1ab8 2162a383 Aug 18 15:59:55 papillon grsec: attempted resource overstep by requesting 4096 for RLIMIT_CORE against limit 0 by /space/chroots/chroot001/root/bug.py[bug.py:5860] uid/euid:0/0 gid/egid:0/0, parent /space/chroots/chroot001/bin/bash[bash:29065] uid/euid:0/0 gid/egid:0/0 does not happen with -O2 or leaving out optimization. -O3 seems to have more issues with python than you expected, and apparently my pax kernel stops the show way before your problems with SSP would make it bite the dust. and no, i will not chpax and paxctl it, because i hate overoptimizing security- oriented technology. And please, next time you try to embarrass devs with making them bounce bugs back and forth between teams: dont do it. Test with -O2 and do not use -O3 and SSP together. TIA; Alex
I still say that since the bug only occurs with -O3, as you've proven, it is an optimization issue, not a bug in python. If it is an unavoidable tradeoff due to overoptimization, then I can accept that. I wasn't attempting to embarrass anybody (unlike you). I merely wanted to have an opposing viewpoint explained to me. As for not using -O3, I believe I said that in my previous comment.
other than that, the ebuild sucks hard. # rm -rf /var/tmp/portage/python-2.3.*; FEATURES="debug nostrip keeptemp keepwork" ACCEPT_KEYWORDS="~x86" CFLAGS="-O3" emerge -v python 2>&1 | tee /tmp/python.txt; python-updater; paxctl -pemrxs -v $(which python2.3); paxctl -pemrxs -v /usr/lib/libpython2.3.so.1.0 gives an other result as # rm -rf /var/tmp/portage/python-2.3.*; FEATURES="debug nostrip keeptemp keepwork" ACCEPT_KEYWORDS="~x86" CC="gcc -O3" emerge -v python 2>&1 | tee /tmp/python.txt; python-updater; paxctl -pemrxs -v $(which python2.3); paxctl -pemrxs -v /usr/lib/libpython2.3.so.1.0 You can try building python with CC="gcc -O3 -fstack-protector -fstack-protector-all" and see what it yields. there is only one more thing i have to say: this would have not happened with a CFLAGS wrapper. have a nice day! Alex
Tested with a hardened toolchain (emits -fstack-protector-all by default) Portage 2.0.51_pre18 (hardened/x86, gcc-3.4.1, glibc-2.3.4.20040619-r1, ... CC="gcc -O3" CFLAGS="-pipe -fforce-addr -fomit-frame-pointer" Package works as pappy has described. echo 'import smtplib; server = smtplib.SMTP("smtp.gentoo.org");' | python - ; echo $? 0
Again... this is a python and not a toolchain bug.
solar, can you please motivate *why* you think this is not a toolchain issue but a python bug?
1) I'm on the toolchain herd and I know when something is and is not a toolchain bug and who should handle said bugs for pkgs. I'm also aware of the fundamentals around ssp. 2) This problem only happens with python. 3) Forcing the -O3 logic via CC= (ie force it across the board for all compile object files) fixes the problem at hand.
i'm fine with applying this to python, if no one else does it, i'll apply it. but it would be interesting to know which bit of code is triggering the -O3 problem, i still reckon its a toolchain bug, but I agree we should put a workaround in python for this.
in my opinion, rather than just adding a "bandaid", you should find out where the reason is. i wrote a simple DEBUG wrapper for gcc-config that lists "malformed" CFLAGS when the gcc is run. solar: please test and paste output here. my chroots are out of order because i am preparing a new version of dev-util/devel-chroots thanks in advance, Alex
Well great that you wrote a wrapper but I'm not a mind reader I have nfc how to use your tool. How about you use it and paste the output here.
emerge -gK ./gcc-3.3.3-r6.tbz2 # cat /tmp/build-python.sh #!/bin/bash cd /root CC="gcc -g3 -ggdb3" CFLAGS="-O3 -fstack-protector-all" ACCEPT_KEYWORDS="x86" FEATURES="keeptemp keepwork nostrip debug" emerge -v python 2>&1 | tee /tmp/python.txt python-updater ldconfig this produces: Python 2.3.3 (#1, Aug 20 2004, 12:42:50) [GCC 3.3.3 20040412 (Gentoo Linux 3.3.3-r6, ssp-3.3.2-2, pie-8.7.6)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> after running the test script, the output is okay # /tmp/bug.py; echo $? 0 17:59:17 [/space/chroots/chroot001:2332.pts-4.papillon]papillon ~ # cat /tmp/bug.py #!/usr/bin/python ## http://bugs.gentoo.org/show_bug.cgi?id=50309 import smtplib; server = smtplib.SMTP("smtp.gentoo.org"); but after doing this: # cat /tmp/bug.py #!/usr/bin/python -i ## http://bugs.gentoo.org/show_bug.cgi?id=50309 import smtplib; server = smtplib.SMTP("smtp.gentoo.org"); (remember the -i is interactive mode) starting the script, attaching with gdb and exiting python, it gives the following results: 18:00:03 [/space/chroots/chroot001:2332.pts-4.papillon]papillon ~ # /tmp/bug.py >>> papillon root # gdb -q /space/chroots/chroot001/usr/bin/python2.3 $(ps axufwww | grep python | grep "bug.py" | awk '{ print $2 }') Loaded symbols for /lib/libresolv.so.2 0x40141af8 in thread_self () at descr.h:260 260 return (pthread_descr)(((unsigned long)sp | (STACK_SIZE-1))+1) - 1; (gdb) c Continuing. # /tmp/bug.py >>> ((pressing CTRL-D)) the gdb says: Program received signal SIGSEGV, Segmentation fault. 0x400db904 in eval_doc () from /usr/lib/libpython2.3.so.1.0 (gdb) where #0 0x400db904 in eval_doc () from /usr/lib/libpython2.3.so.1.0 #1 0x40131d7c in ?? () #2 0x0804a258 in ?? () #3 0x0d409294 in ?? () #4 0x4012ae00 in ?? () #5 0x40016000 in ?? () #6 0x400217f6 in ?? () from /usr/lib/libpython2.3.so.1.0 (gdb) quit The program is running. Quit anyway (and detach it)? (y or n) y Detaching from program: /space/chroots/chroot001/usr/bin/python2.3 Unfortunately i could not reproduce the SSP error, but maybe you can give me the information if the problem goes away if you would use gcc-3.3.3-r6. thanks, Alex
additionally, these last tests have been done on a nonPAX kernel (happens on pax kernel too though) 18:04:16 [/space/chroots/chroot001:2332.pts-4.papillon]papillon ~ # uname -a Linux papillon 2.6.7 #1 Thu Aug 19 19:10:57 CEST 2004 i686 Mobile Intel(R) Pentium(R) 4 - M CPU 1.80GHz GenuineIntel GNU/Linux
The patch mitigates the problem while further investigation can be done by pappy and or others.
Alistair: please add this peace to all pythons "in the wild". + if is-flag -O3 ; then + is-flag -fstack-protector-all && replace-flags -O3 -O2 + use hardened && replace-flags -O3 -O2 TIA, Alex (closing bug)
committed for all 2.3* pythons, thanks alex!
I think this can be closed now.
closing
I think the fix is not complete. The mentioned problem still exists when compiling python with -fstack-protector -O3 and that will not be catched by the modified ebuild. Steps to reproduce: 1. Compile python with -fstack-protector -O3 2. Start pydoc -p 8080 3. Open a web browser at http://localhost:8080/ Actual results: henryk@gleam henryk $ pydoc -p 8080 pydoc server ready at http://localhost:8080/ python: stack smashing attack in function fast_function() Aborted Expected results: Shouldn't abort. Simple fix: Change + if is-flag -O3 ; then + is-flag -fstack-protector-all && replace-flags -O3 -O2 + use hardened && replace-flags -O3 -O2 to + if is-flag -O3 ; then + is-flag -fstack-protector-all && replace-flags -O3 -O2 + is-flag -fstack-protector && replace-flags -O3 -O2 + use hardened && replace-flags -O3 -O2
tip. Don't use -O3 with ssp.
That is a good tip. Too many people are unaware of this ssp/-O3 incompatibility (as you can see above, I was bitten by this about eight months ago). Perhaps portage should be made to automatically warn about this situation when it sees it?
