Due to the recent change in PaX behaviour, which does not allow for a 'rwx' mappings to be created (even if the can be reduced to 'r-x' only by PaX, which was previously accepted) it is not possible to load the ctypes module. This has been confirmed for hardened-sources-2.6.32-r11 as well as hand-patched kernel 2.6.34.1 with grsecurity-2.2.0-2.6.34.1-201007162107.patch Steps to reproduce: 1. emerge and boot hardened-sources-2.6.32-r11 (or use hand patched 2.6.34.1 as described above) 2. make sure using paxctl that mprotect is not disabled for python 3. run: python -c "import modules" Results: # python -c "import ctypes" Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/lib64/python2.6/ctypes/__init__.py", line 546, in <module> CFUNCTYPE(c_int)(lambda: None) MemoryError Relevant strace snippet: open("/usr/lib64/python2.6/ctypes/_endian.pyc", O_RDONLY) = 5 fstat(5, {st_mode=S_IFREG|0644, st_size=2286, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x31b9d421000 read(5, "\321\362\r\n3r+Lc\0\0\0\0\0\0\0\0\5\0\0\0@\0\0\0s\307\0\0\0d\0"..., 4096) = 2286 fstat(5, {st_mode=S_IFREG|0644, st_size=2286, ...}) = 0 read(5, "", 4096) = 0 close(5) = 0 munmap(0x31b9d421000, 4096) = 0 mmap(NULL, 266240, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x31b9d2b5000 close(4) = 0 mmap(NULL, 4080, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 EPERM (Operation not permitted) close(3) = 0 write(2, "Traceback (most recent call last"..., 35Traceback (most recent call last): ) = 35 Expected result: successful import of the ctypes module. Additional info: In Python source under Modules/_ctypes/malloc_closure.c there's a following code snippet: #ifdef MS_WIN32 item = (ITEM *)VirtualAlloc(NULL, count * sizeof(ITEM), MEM_COMMIT, PAGE_EXECUTE_READWRITE); if (item == NULL) return; #else item = (ITEM *)mmap(NULL, count * sizeof(ITEM), PROT_READ | PROT_WRITE | PROT_EXEC, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); if (item == (void *)MAP_FAILED) return; #endif Losing the 'PROT_EXEC' seems to solve the issue which has also been confirmed by running the ctypes tests. strace output for the patched version: ... fstat(4, {st_mode=S_IFREG|0644, st_size=2041, ...}) = 0 open("/usr/lib64/python2.6/ctypes/_endian.pyc", O_RDONLY) = 5 fstat(5, {st_mode=S_IFREG|0644, st_size=2286, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x26fa4a7f000 read(5, "\321\362\r\nz\275HLc\0\0\0\0\0\0\0\0\5\0\0\0@\0\0\0s\307\0\0\0d\0"..., 4096) = 2286 fstat(5, {st_mode=S_IFREG|0644, st_size=2286, ...}) = 0 read(5, "", 4096) = 0 mmap(NULL, 266240, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x26fa4913000 close(5) = 0 munmap(0x26fa4a7f000, 4096) = 0 close(4) = 0 mmap(NULL, 4080, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x26fa4a7f000 close(3) = 0 rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x26fa42a8110}, {0x26fa45c5975, [], SA_RESTORER, 0x26fa42a8110}, 8) = 0 munmap(0x26fa4913000, 266240) = 0 exit_group(0)
Created attachment 239871 [details, diff] proposed patch
Created attachment 239875 [details] emerge --info
Sorry, I made few mistakes: - mappings were obviously reduced to 'rw-' not 'r-x' - steps to reproduce: point 3, should be: python -c "import ctypes" this affects python 2.6 and 3.1 (2.6.5-r3 and 3.1.2-r3). The proposed patch has only been tested with 2.6
We won't include this experimental patch until upstream approves it.
(In reply to comment #4) > We won't include this experimental patch until upstream approves it. why? it fixes a bug and is needed for hardened users right now. if you're unable to verify its correctness (a simple removal of PROT_EXEC) then maybe you shouldn't be sticking your nose into things you don't understand.
Report the patch to upstream [1] instead of complaining. [1] http://bugs.python.org/
(In reply to comment #4) > We won't include this experimental patch until upstream approves it. > From what I recall, that code consistently (ab)used PROT_EXEC where unnecessary and potentially dangerous. I would put some pressure on upstream to explain why all their mmap()'s use it.
(In reply to comment #4) > We won't include this experimental patch until upstream approves it. If you don't we will. python has to be patched one way or another.
This has been fixed in the tree 2.6.x, 2.7.x, 3.x. I used a sed statement vs patching so it gets carried onto version bumps w/o being lost. Radoslaw Madej please also file a bug with the upstream python team.
Upstream bug logged: http://bugs.python.org/issue9385
Upstream has better patch, which will be applied.
The current fix (sed in the ebuild) makes test_ctypes segv on hppa using vanilla kernel, gcc 4.2.4 or 4.4.4-r1 and python 2.6.5-r2.
The usage of sed in the ebuilds has been reverted. HPPA maintainers: Please test the patches from ctypes maintainer. If the patches work correctly, then I will include them in the new patchsets for new revisions of dev-lang/python.
Created attachment 240887 [details, diff] Patch for 2.6.5-r3
Created attachment 240889 [details, diff] Patch for 2.7
Created attachment 240891 [details, diff] Patch for 3.1.2-r4
Alpha / HPPA maintainers: Has the reversion of wrong changes fixed bug #330323 for you? Alpha maintainers: Please check if these patches work for you.
I've now tested the most recent versions of Python as mentioned in bugs 14-16, with and without the patches of said comments. They all pass the ctypes test now.
The test_ctypes passes fine as well on hppa with the patch for 2.6.5-r3.
Fixed in r365.
Fixed in 2.6.5_p20100801, 2.7_p20100801, 3.1.2_p20100801 and 3.2_pre20100801.
(In reply to comment #21) > Fixed in 2.6.5_p20100801, 2.7_p20100801, 3.1.2_p20100801 and 3.2_pre20100801. Afrever, as these versions had to be dropped - for unrelated reasons as I understand - would there be any chance of getting the patch folded in for a simple revision bump of the existing instances? Naturally, 2.6.5 is of particular importance, being the current stable version.
(In reply to comment #22) > (In reply to comment #21) > > Fixed in 2.6.5_p20100801, 2.7_p20100801, 3.1.2_p20100801 and 3.2_pre20100801. > > Afrever, as these versions had to be dropped - for unrelated reasons as I > understand - would there be any chance of getting the patch folded in for a > simple revision bump of the existing instances? Naturally, 2.6.5 is of > particular importance, being the current stable version. Yes, it'll be in.
*** Bug 332419 has been marked as a duplicate of this bug. ***
I have tested the 2.6.5 patch on AMD64 gentoo hardened. uname -r 2.6.34-hardened-r1-UL30VT and it works like a charm, strace confirms that it doesnt attempt any RWX callings, just RX or RW.
*** Bug 336722 has been marked as a duplicate of this bug. ***
Not to sound like a stuck record, but will the patch be committed any time soon? I realise that it serves a limited purpose for those not using Hardened Gentoo but I, for one, would greatly appreciate a 2.6.5-r4 bump to this end. Are there other bugs that need to be addressed for a new revision to be issued?
(In reply to comment #27) > Not to sound like a stuck record, but will the patch be committed any time > soon? I realise that it serves a limited purpose for those not using Hardened > Gentoo but I, for one, would greatly appreciate a 2.6.5-r4 bump to this end. > Are there other bugs that need to be addressed for a new revision to be issued? > +1, needed for iotop
*** Bug 339152 has been marked as a duplicate of this bug. ***
+1. Currently cannot use coherence. Will appreciated patched source.
(In reply to comment #28) > (In reply to comment #27) > > Not to sound like a stuck record, but will the patch be committed any time > > soon? I realise that it serves a limited purpose for those not using Hardened > > Gentoo but I, for one, would greatly appreciate a 2.6.5-r4 bump to this end. > > Are there other bugs that need to be addressed for a new revision to be issued? > > > > +1, needed for iotop > +1 issue exists on 2.6.35-r1 as well...
(In reply to comment #31) > (In reply to comment #28) > > (In reply to comment #27) > > > Not to sound like a stuck record, but will the patch be committed any time > > > soon? I realise that it serves a limited purpose for those not using Hardened > > > Gentoo but I, for one, would greatly appreciate a 2.6.5-r4 bump to this end. > > > Are there other bugs that need to be addressed for a new revision to be issued? > > > > > > > +1, needed for iotop > > > > +1 issue exists on > 2.6.35-r1 > as well... > This issue will continue on all hardened-sources from now on because of the improved restrictions on mmap in grsec-2.2.0. The last hardened-sources where this is not an issue is 2.6.32-r9 which was based on the last grsec-2.1.14 patch. However, since certain configurations of that kernel are exploitable on amd64 (see bug #338273), this bug has become critical. I know some users that are now resorting to overlays to get past this. Been there done that --- not good from QA point of view etc. So ... adding to the above +1's.
gnuplot-py-1.8 is marked stable but can't build because of this bug. I tested the 2.6 patch and it fixed the issue. I put it in my overlay so other people can have it but as several people pointed out, this is a horrible solution. Please don't let this desperately needed and confirmed working fix rot in bug tracker. Thanks
python herd (other than Arfrever): can we please get this included ASAP? With the stabilization of new kernels for the recent IA32 exploits, it's starting to bite everybody. If there no response by the end of Oct 9th, I'll look at bumping them myself, because this will break Python for infra as well.
Thanks for your intervention, Robin.
python-2.6.6, including a patch to fix this issue, has been released in unstable. I presume that means this bug can be closed, though feel free to reopen if I've missed something.
djc: so we're fast-tracking 2.6.6 to stable for this?
jasmin ~ # python -c "import ctypes" Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/lib64/python2.6/ctypes/__init__.py", line 546, in <module> CFUNCTYPE(c_int)(lambda: None) MemoryError jasmin ~ # jasmin ~ # eselect python list Available Python interpreters: [1] python2.6 * [2] python3.1 jasmin ~ # [ebuild R ] dev-lang/python-2.6.6 USE="berkdb gdbm ncurses readline ssl threads (wide-unicode) xml -build -doc -examples -ipv6 -sqlite -tk -wininst" 0 kB
python-2.6.6 doesn't include the patch ...
Yeah, I failed there. Please give 2.6.6-r1 a shot.
Robin, I'd be happy to fast-track 2.6.6 to stable for this.
Sorry to be spammy: the reason I wanted 2.6.6 instead of updating 2.6.5 is that 2.6.5 isn't actually upstream 2.6.5, it has a huge amount of backports, and it turned out people (including me, btw) didn't really like that way of doing it, so I figured that we should get a quick 2.6.6 instead of another broken 2.6.5 revision with this patch included.
The -r1 works for me thank you for fixing it.
(In reply to comment #43) > The -r1 works for me thank you for fixing it. > Same here. Thanks.
I encountered another rwx mapping with 2.6.6-r1 inside read-only chroot on a Hardened system: It Oct 24 05:57:44 [kernel] grsec: denied RWX mmap of <anonymous mapping> by /tmp/readonly_chroot/usr/bin/python2.6[python2.6:31335] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[bash:20184] uid/euid:0/0 gid/egid:0/0 Oct 24 05:57:44 [kernel] grsec: Segmentation fault occurred at 7465725f in /tmp/readonly_chroot/usr/bin/python2.6[python2.6:31335] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[bash:20184] uid/euid:0/0 gid/egid:0/0 And another issue is due to TPE restrictions when python tries to map executable some untrusted temporary .so file generated by libffi: testing@worm3 ~ $ id uid=1219(testing) gid=1221(testing) groups=1221(testing) testing@worm3 ~ $ python -c "import ctypes" Segmentation fault Oct 24 06:01:48 [kernel] grsec: denied untrusted exec of /home/testing/ffiCigB2E by /usr/bin/python2.6[python2.6:31420] uid/euid:1219/1219 gid/egid:1221/1221, parent /bin/bash[bash:31406] uid/euid:1219/1219 gid/egid:1221/1221 Oct 24 06:01:48 [kernel] grsec: Segmentation fault occurred at 7465725f in /usr/bin/python2.6[python2.6:31420] uid/euid:1219/1219 gid/egid:1221/1221, parent /bin/bash[bash:31406] uid/euid:1219/1219 gid/egid:1221/1221 Workaround is to add user testing to trusted TPE group: $ id uid=1219(testing) gid=1221(testing) groups=1221(testing),500(tpe) $ python -c "import ctypes" $
(In reply to comment #45) > I encountered another rwx mapping with 2.6.6-r1 inside read-only chroot on a > Hardened system: To reproduce run python -c "import python" inside read-only chroot.
> To reproduce run python -c "import python" inside read-only chroot. Sorry, it's python -c "import ctypes"
python-3.2_pre20101010, unlike 2.6.6-r1, doesn't do any rwx mappings inside read-only chroot, but just segfaults instead: # python3.2 -c "import ctypes" Segmentation fault Oct 24 06:33:09 [kernel] grsec: Segmentation fault occurred at 00000074 in /home/root/python3.2_readonly_chroot/usr/bin/python3.2[python3.2:31705] uid/euid:0/0 gid/egid:0/0, parent /home/root/python3.2_readonly_chroot/bin/bash[bash:31192] uid/euid:0/0 gid/egid:0/0 The TPE issue is the same for 3.2_pre20101010 as for 2.6.6-r1: $ id uid=1219(testing) gid=1221(testing) groups=1221(testing) $ python3.2 -c "import ctypes" Segmentation fault Oct 24 06:41:12 [kernel] grsec: denied untrusted exec of /home/root/python3.2_chroot/home/testing/ffixA4YwV by /home/root/python3.2_chroot/usr/bin/python3.2[python3.2:31801] uid/euid:1219/1219 gid/egid:1221/1221, parent /home/root/python3.2_chroot/bin/bash[bash:31790] uid/euid:1219/1219 gid/egid:1221/1221 Oct 24 06:41:12 [kernel] grsec: Segmentation fault occurred at 00000074 in /home/root/python3.2_chroot/usr/bin/python3.2[python3.2:31801] uid/euid:1219/1219 gid/egid:1221/1221, parent /home/root/python3.2_chroot/bin/bash[bash:31790] uid/euid:1219/1219 gid/egid:1221/1221 And with the same workaround: $ id uid=1219(testing) gid=1221(testing) groups=1221(testing),500(tpe) $ python3.2 -c "import ctypes" $
2.6.6-r1 stabilization requested in bug 342927.
*** Bug 345591 has been marked as a duplicate of this bug. ***
This bug was fixed long time ago.
(In reply to comment #51) > This bug was fixed long time ago. Are you sure? # python -c "import ctypes" Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/lib/python2.6/ctypes/__init__.py", line 546, in <module> CFUNCTYPE(c_int)(lambda: None) MemoryError # python --version Python 2.6.5 [I] dev-lang/python (2.6.5-r3(2.6)@14/10/10 3.1.2-r4(3.1)@14/10/10) The installed version 2.6.5 is the latest stable. Requesting REOPEN.
Bugs should be closed after they have been fixed in the newest (potentially not yet stable) versions.
> Bugs should be closed after they have been fixed in the newest (potentially not > yet stable) versions. Well, it depends. I don't mean to sound ungrateful for your help in finding the correct solution (I'm aware of the effort that you put into python maintenance at large) but the trouble is this: from the hardened user's perspective, this bug has now been in existence for over four months and there is still no solution in the stable branch. The purpose of distinguishing between stable and testing is, quite naturally, to provide a reasonable assurance to any user that chooses the stable branch that what they install will actually be as stable and bug-free as is possible. By closing this bug as it stands, the implication is that affected users should have to upgrade to an ~arch package in order to resolve the issue, which I find odd because inserting the patch into the stable branch in this case would have been trivial. Granted, I can see that a stable request exists for python-2.6.6 but there is no timeframe (will it go stable in a week, a month or longer?). That is why I asked back in Comment #22 whether we could simply fold in the patch to the *current* stable branch, revision bump and be done with it. The bug could have been closed at the time under the assumption that the patch would be retained in future ebuilds for as long as upstream haven't fixed it and then we could have stopped nagging about it. I upgraded to python-2.6.6 as soon as I knew it was carrying the patch but my point is that not everyone should have to if they would rather stick to the stable branch. It's not as though it needs testing either; we know the patch fixes the problem.
I switched to ~ia64 for dev-lang/python-2.6.6, but the issue seems not to be fixed: # python --version Python 2.6.6 # python -c "import ctypes" <no output, good so far> # /etc/init.d/pyicq-t restart * Starting ICQ Jabber Transport ... Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/pyicq-t/pyicq-t.py", line 11, in <module> import main File "/usr/lib/python2.6/site-packages/pyicq-t/src/main.py", line 129, in <module> from twisted.words.protocols.jabber import component File "/usr/lib/python2.6/site-packages/twisted/words/protocols/jabber/component.py", line 23, in <module> from twisted.application import service File "/usr/lib/python2.6/site-packages/twisted/application/service.py", line 23, in <module> from twisted.plugin import IPlugin File "/usr/lib/python2.6/site-packages/twisted/plugin.py", line 34, in <module> from twisted.python.modules import getModule File "/usr/lib/python2.6/site-packages/twisted/python/modules.py", line 68, in <module> from twisted.python.filepath import FilePath, UnlistableError File "/usr/lib/python2.6/site-packages/twisted/python/filepath.py", line 29, in <module> from twisted.python.win32 import ERROR_FILE_NOT_FOUND, ERROR_PATH_NOT_FOUND File "/usr/lib/python2.6/site-packages/twisted/python/win32.py", line 168, in <module> formatError = _ErrorFormatter.fromEnvironment().formatError File "/usr/lib/python2.6/site-packages/twisted/python/win32.py", line 128, in fromEnvironment from ctypes import WinError File "/usr/lib/python2.6/ctypes/__init__.py", line 546, in <module> CFUNCTYPE(c_int)(lambda: None) MemoryError * start-stop-daemon: failed to start `/usr/bin/python' [ !! ] * ERROR: pyicq-t failed to start I do not understand what is happening here, but it seems that there are some conditions under which it is still not possible to import ctypes. Again I request to reopen this bug. [I] dev-lang/python (2.6.6-r1(2.6)@28/11/10 3.1.2-r4(3.1)@28/11/10): Python is an interpreted, interactive, object-oriented programming language. [I] net-im/pyicq-t (0.8.1.5@29/11/10): Python based jabber transport for ICQ A similar error happens when starting pymsn-t: # /etc/init.d/pymsn-t restart * Starting MSN Jabber Transport ... ImportError! You probably forgot to install Twisted Words or Web. Have a look at the docs. You may also be using an unsupported version of Twisted. * start-stop-daemon: failed to start `/usr/bin/python' [ !! ] * ERROR: pymsn-t failed to start It is at 0.11.3-r2 + more patches from upstream: [I] net-im/pymsn-t (0.11.3_p1[1]@29/11/10): Python based jabber transport for MSN
Re Comment 55 - I'm unable to reproduce this on amd64 with: dev-lang/python-2.6.6-r1 dev-python/twisted-web-10.1.0 dev-python/twisted-words-10.1.0 dev-python/imaging-1.1.7 dev-python/pycrypto-2.3 dev-python/pyopenssl-0.10-r1 net-im/pymsn-t-0.11.3-r2 Perhaps the issue concerns ia64 specifically.
(In reply to comment #55) > I switched to ~ia64 for dev-lang/python-2.6.6, but the issue seems not to be > fixed: can you "strace -f" these failing apps and provide the logs here please?
(In reply to comment #57) > (In reply to comment #55) > > I switched to ~ia64 for dev-lang/python-2.6.6, but the issue seems not to be > > fixed: > > can you "strace -f" these failing apps and provide the logs here please? Seems fixed with sys-kernel/hardened-sources-2.6.32-r42 and dev-lang/python-2.7.1-r1.
Created attachment 288671 [details] strace output from twistd I can reproduce this problem on amd64 while starting twistd for a buildbot master. The strace output indicates that it tries to create a temporary file and mmap() it with PROT_EXEC in /tmp, /var/tmp and /dev/shm. All of these are mounted with the noexec flag on my system. I can start the server after removing this flag from one of these mount points. 7885 open("/tmp/ffiS3iyaG", O_RDWR|O_CREAT|O_EXCL, 0600) = 13 7885 unlink("/tmp/ffiS3iyaG") = 0 7885 ftruncate(13, 4096) = 0 7885 mmap(NULL, 4096, PROT_READ|PROT_EXEC, MAP_SHARED, 13, 0) = -1 EPERM (Operation not permitted) 7885 close(13) = 0 7885 open("/var/tmp/ffi5ASZqt", O_RDWR|O_CREAT|O_EXCL, 0600) = 13 7885 unlink("/var/tmp/ffi5ASZqt") = 0 7885 ftruncate(13, 4096) = 0 7885 mmap(NULL, 4096, PROT_READ|PROT_EXEC, MAP_SHARED, 13, 0) = -1 EPERM (Operation not permitted) 7885 close(13) = 0 7885 open("/dev/shm/ffiANqtHg", O_RDWR|O_CREAT|O_EXCL, 0600) = 13 7885 unlink("/dev/shm/ffiANqtHg") = 0 7885 ftruncate(13, 4096) = 0 7885 mmap(NULL, 4096, PROT_READ|PROT_EXEC, MAP_SHARED, 13, 0) = -1 EPERM (Operation not permitted) 7885 close(13) = 0
Re-opened. I see in my grsec.log: Oct 14 04:58:37 devil kernel: grsec: denied RWX mmap of <anonymous mapping> by /usr/bin/python2.7[python2.7:18717] uid/euid:250/250 gid/egid:250/250, parent /usr/bin/python2.7[python2.7:18716] uid/euid:250/250 gid/egid:250/250 Feel free to ask other info if are needed.
(In reply to comment #60) > Re-opened. I see in my grsec.log: > > Oct 14 04:58:37 devil kernel: grsec: denied RWX mmap of <anonymous mapping> by > /usr/bin/python2.7[python2.7:18717] uid/euid:250/250 gid/egid:250/250, parent > /usr/bin/python2.7[python2.7:18716] uid/euid:250/250 gid/egid:250/250 > > Feel free to ask other info if are needed. I have seen this problem "randomly" pop up on hardened systems and I don't know why. It is not fixable by the original patch so it is not the same bug, but it is an rwx mmap killed by pax and it does come from "import ctypes". Ago, can you test python -c "import ctypes" and see if it gets killed in comment 1. We may want to open other bug if we isolate this to some other issue.
(In reply to comment #61) > I have seen this problem "randomly" pop up on hardened systems and I don't know > why. It is not fixable by the original patch so it is not the same bug, but it > is an rwx mmap killed by pax and it does come from "import ctypes". if the culprit process get a SIGKILL from PaX then all you need to do is enable coredumping (ulimit -c unlimited) in your shell and run it again and you'll get a nice coredump where you can look up the backtrace, etc and figure out which code tried to execute runtime generated code.
*** Bug 369789 has been marked as a duplicate of this bug. ***
We have confirmed this with mod_wsgi and CPython 2.7.2-r3 and a third-party is also having trouble. libffi's closures are to blame for the /tmp/ffiXXXXX RX/RWX mmap behavior. The result is a SIGSEGV in the Apache process in some dubious PyMalloc call which clearly isn't the culprit. Someone please give Drepper incentives to take some long vacations as far as possible from a networked computer.
(In reply to comment #48) > python-3.2_pre20101010, unlike 2.6.6-r1, doesn't do any rwx mappings inside > read-only chroot, but just segfaults instead: > > # python3.2 -c "import ctypes" > Segmentation fault > > Oct 24 06:33:09 [kernel] grsec: Segmentation fault occurred at 00000074 in > /home/root/python3.2_readonly_chroot/usr/bin/python3.2[python3.2:31705] > uid/euid:0/0 gid/egid:0/0, parent > /home/root/python3.2_readonly_chroot/bin/bash[bash:31192] uid/euid:0/0 > gid/egid:0/0 > > > The TPE issue is the same for 3.2_pre20101010 as for 2.6.6-r1: > > $ id > uid=1219(testing) gid=1221(testing) groups=1221(testing) > $ python3.2 -c "import ctypes" > Segmentation fault > > Oct 24 06:41:12 [kernel] grsec: denied untrusted exec of > /home/root/python3.2_chroot/home/testing/ffixA4YwV by > /home/root/python3.2_chroot/usr/bin/python3.2[python3.2:31801] > uid/euid:1219/1219 gid/egid:1221/1221, parent > /home/root/python3.2_chroot/bin/bash[bash:31790] uid/euid:1219/1219 > gid/egid:1221/1221 > Oct 24 06:41:12 [kernel] grsec: Segmentation fault occurred at 00000074 in > /home/root/python3.2_chroot/usr/bin/python3.2[python3.2:31801] > uid/euid:1219/1219 gid/egid:1221/1221, parent > /home/root/python3.2_chroot/bin/bash[bash:31790] uid/euid:1219/1219 > gid/egid:1221/1221 > > And with the same workaround: > > $ id > uid=1219(testing) gid=1221(testing) groups=1221(testing),500(tpe) > $ python3.2 -c "import ctypes" > $ This is exactly the current situation on amd64 with 2.7.2-r3.
(In reply to comment #65) > (In reply to comment #48) > > python-3.2_pre20101010, unlike 2.6.6-r1, doesn't do any rwx mappings inside > > read-only chroot, but just segfaults instead: > > > > # python3.2 -c "import ctypes" > > Segmentation fault > > > > Oct 24 06:33:09 [kernel] grsec: Segmentation fault occurred at 00000074 in > > /home/root/python3.2_readonly_chroot/usr/bin/python3.2[python3.2:31705] > > uid/euid:0/0 gid/egid:0/0, parent > > /home/root/python3.2_readonly_chroot/bin/bash[bash:31192] uid/euid:0/0 > > gid/egid:0/0 > > > > > > The TPE issue is the same for 3.2_pre20101010 as for 2.6.6-r1: > > > > $ id > > uid=1219(testing) gid=1221(testing) groups=1221(testing) > > $ python3.2 -c "import ctypes" > > Segmentation fault > > > > Oct 24 06:41:12 [kernel] grsec: denied untrusted exec of > > /home/root/python3.2_chroot/home/testing/ffixA4YwV by > > /home/root/python3.2_chroot/usr/bin/python3.2[python3.2:31801] > > uid/euid:1219/1219 gid/egid:1221/1221, parent > > /home/root/python3.2_chroot/bin/bash[bash:31790] uid/euid:1219/1219 > > gid/egid:1221/1221 > > Oct 24 06:41:12 [kernel] grsec: Segmentation fault occurred at 00000074 in > > /home/root/python3.2_chroot/usr/bin/python3.2[python3.2:31801] > > uid/euid:1219/1219 gid/egid:1221/1221, parent > > /home/root/python3.2_chroot/bin/bash[bash:31790] uid/euid:1219/1219 > > gid/egid:1221/1221 > > > > And with the same workaround: > > > > $ id > > uid=1219(testing) gid=1221(testing) groups=1221(testing),500(tpe) > > $ python3.2 -c "import ctypes" > > $ > > > This is exactly the current situation on amd64 with 2.7.2-r3. This is the undead bug that just won't die. Try enabling PAX_MPROTECT_COMPAT. It is less secure but should work.
Created attachment 297183 [details, diff] python part of the fix (2.7.2)
Created attachment 297185 [details, diff] libffi part of the fix (3.0.10)
(In reply to comment #67 and comment #68) Please send patches to upstreams.
These patches implement workaround for several basic ctypes use cases, so they won't be accepted by the upstreams. The solution is to disable mprotect restrictions for python executable files by default.
(In reply to comment #70) > These patches implement workaround for several basic ctypes use cases, so they > won't be accepted by the upstreams. The solution is to disable mprotect > restrictions for python executable files by default. The solution is to get upstream to implement mmap'd functions correctly, however they seem to be reluctant for reasons I cannot fathom. Please use PAX_MPROTECT_COMPAT=y as it will work, albeit with a bit less robustness in protection.
(In reply to comment #71) > The solution is to get upstream to implement mmap'd functions correctly, > however they seem to be reluctant for reasons I cannot fathom. Maybe the reason is some parts of the code do need W|X mappings. > Please use PAX_MPROTECT_COMPAT=y as it will work, albeit with a bit less > robustness in protection. The following ctypes test cases will fail regardless: test_libc test_functions test_as_parameter test_random_things test_pointers test_refcounts test_callbacks test_simplesubclasses test_funcptr PAX_MPROTECT_COMPAT=y and my patches basically do the same thing: make common ctypes use cases not to crash python. Sad, but they are just partial workarounds. Btw, PAX_MPROTECT_COMPAT=y might deceive some apps into believing they got executable memory and consequentially fail instead of interpreting EACCES/EPERM and act accordingly.
(In reply to comment #72) > PAX_MPROTECT_COMPAT=y and my patches basically do the same thing: make common > ctypes use cases not to crash python. Sad, but they are just partial > workarounds. Let me understand this clearly, are you saying that with your patches you can retain the more restrictive PAX_MPROTECT and still have python work? Or do you need both PAX_MPROTECT_COMPAT=y and your patches. Looking over what you did, it looks like the former, but I have not tested. > Btw, PAX_MPROTECT_COMPAT=y might deceive some apps into believing they got > executable memory and consequentially fail instead of interpreting EACCES/EPERM > and act accordingly. Yeah there are issues in this direction. I'm not opposed to asking the python folks to add these patches conditional on USE="pax_kernel" but I'm afraid this is going to become a nightmare. Upstream is showing no signs of moving in a direction where W|X is avoided. If this gets worse then I'm afraid we may have to move in the direction of pax marking python like we do with other misbehaving interpreters. I hate the idea but what other options do we have?
(In reply to comment #73) > Let me understand this clearly, are you saying that with your patches you can > retain the more restrictive PAX_MPROTECT and still have python work? Yes. Some of the basic ctypes use cases work. Examples: $ python -c 'import uuid; uuid.uuid4()' $ python -c 'import ctypes; ctypes.CDLL("libc.so.6")["getuid"]()' That's with a small overhead of one more syscall per (infrequent) closure allocation when MPROTECT is on and with no such overhead when it's off. > I'm not opposed to asking the python folks to add these patches conditional on > USE="pax_kernel" but I'm afraid this is going to become a nightmare. Even if you decide to supply the workaround, it should be documented very clearly that in some use cases ctypes *will* crash the interpreter. Maybe something like USE="ctypes_pax_workaround" with a clear description will be more appropriate than USE="pax_kernel". > python like we do with other misbehaving interpreters. I hate the idea but > what other options do we have? MPROTECT doesn't prevent python bytecode injection, so maybe native code injection shouldn't be much of a concern. Guess we should ask pipacs or somebody experienced in exploitation.
(In reply to comment #74) > Even if you decide to supply the workaround, it should be documented very > clearly that in some use cases ctypes *will* crash the interpreter. what i'd like to know is why ctypes has to resort to runtime codegen, is that some fundamental need or just a programmer's choice that could be changed? > MPROTECT doesn't prevent python bytecode injection, so maybe native code > injection shouldn't be much of a concern. Guess we should ask pipacs or > somebody experienced in exploitation. bytecode injection is definitely a possible way of exploiting a python process but at the same time we'd also have to worry about the more mundane ret2libc style attacks as well (as there's no defense against either out there yet). with all that said, after discussion with Pavel i implemented EMUTRAMP emulation for the libffi trampolines in the latest patches, so just have the ebuild enable it on python and everything should work fine.
There are minor issues. 1. Kernel should be compiled with PAX_MPROTECT_COMPAT or python should be compiled with this patch: https://bugs.gentoo.org/attachment.cgi?id=297183 - or else the mmap in Modules/_ctypes/malloc_closure.c:more_core will fail because of MPROTECT and ctypes won't work. 2. To apply grsec TPE restrictions to python users and/or to run python ctypes code in a read-only chroot, one should compile libffi with this patch: https://bugs.gentoo.org/attachment.cgi?id=297185 - or else libffi will try to avoid MPROTECT restrictions by creating a temporary file like /tmp/ffiS3iyaG, mmap it RW and RX and execute trampoline code from the RX mapping (MPROTECT will deny that trick). The patches make python and libffi fall back to RW mappings when RWX ones fail. This behavior is compatible: one can disable MPROTECT restrictions for python and get vanilla ctypes behavior and performance (the overhead of EMUTRAMP is about 25% here, but YMMV).
Created attachment 298441 [details, diff] kernel EMUTRAMP patch (tested on 3.1.8-grsec-201201062207)
The patches also seem to be working as expected on a x64 system: 1) I've tested the patches provided in c76 and c77, applied to python-2.7.2-r3, libffi-3.0.10 and hardened-sources-3.0.7. Grsec config was the gentoo-virtualization profile with emutramp enabled (without TPE though) 2) Ran: python2.7 /usr/lib/python2.7/ctypes/test/runtests.py -v All tests finished fine :)
Created attachment 298841 [details] ctypes test results Test results attached. Grsec denies RWX mapping but the tests finish fine, from the kern.log: grsec: denied RWX mmap of <anonymous mapping> by /usr/bin/python2.7[python2.7:9030] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[bash:8048] uid/euid:0/0 gid/egid:0/0
Would the python team consider adding Pavel's patches conditional to USE="pax_kernel"?
Maybe I should make it clearer. Now, with libffi trampolines support in EMUTRAMP, the proposed patches are not workaround anymore - they are a part of the solution, but only for hardened-sources (because of dependency on EMUTRAMP) and thus cannot be upstreamed. Please, note that the issue is in the stable python and, imho, it should be fixed asap, as it keeps impacting more and more people. Please, if you could, don't delay resolution.
Firstly, you should discuss changes in dev-libs/libffi with maintainers of dev-libs/libffi (toolchain team). Secondly, in patch for dev-lang/python please use '#ifdef some_definition_defined_only_on_systems_which_need_this_patch'.
(In reply to comment #82) > Firstly, you should discuss changes in dev-libs/libffi with maintainers of > dev-libs/libffi (toolchain team). > > Secondly, in patch for dev-lang/python please use '#ifdef > some_definition_defined_only_on_systems_which_need_this_patch'. I can push this through, but there is just one important difference between this and other #ifdefs which are only defined on systems that need it. Our criterion does NOT live in userland. So its not something that a ./configure script should automagically define/undefine based on the current running kernel. While you can tell if the *current* kernel is PaX enabled, cat /proc/self/status | grep -q PaX && echo "PaX" || echo "No PaX" the user can easily get into a situation where, they build on a system where PaX is not running and don't get the patch. Then boot into a PaX kernel and fail! I recommend a ./configure flag called --with-pax-compatibility (as you suggested), which is disabled by default, will only be enabled by the user adding the flag, and comes with a help that says, enable this if you *ever* anticipate running python under a PaX enabled kernel. @Pavel. They're your patches, do you want to integrate them into the build system as Arfrever suggests, or shall I?
(In reply to comment #83) > I recommend a ./configure flag called --with-pax-compatibility (as you > suggested), which is disabled by default, will only be enabled by the user > adding the flag, and comes with a help that says, enable this if you *ever* > anticipate running python under a PaX enabled kernel. Ain't that what IUSE=pax_kernel is for? Has there been any recent progress with this issue?
before you dig too deep into trying to make all this depend on something: what's the whole point of that? non-PaX kernel users won't be executing this code at all, and PaX kernel users need this now unconditionally. so why go all the trouble to finding yet another superfluous switch to toggle?
Created attachment 304207 [details, diff] python3 patch
*** Bug 407681 has been marked as a duplicate of this bug. ***
I am having this problem on amd64, kernel version 3.2.2-hardened-r1 (Mar 19). The problem exists under python2.7.2, python3.1.4 and python 3.2.2. These are the only versions I have installed. Pavel's patch above (from 4th of March) fixes the issue. Can someone please integrate it with python + pax_kernel?
Pavel's patch is a nice implementation of solar's sed from the end of july, 2010. I include it here for reference; sed -i -e s/'PROT_READ | PROT_WRITE | PROT_EXEC'/'PROT_READ | PROT_WRITE'/g Modules/_ctypes/malloc_closure.c Is there no way to get this patch upstream?
I have applied this patch to all of my deployments now using the following in /etc/portage/env/dev-lang/python (make a new file, no need to mark it +x): post_src_prepare (){ sed -i -e s/'PROT_WRITE\s*|\s*PROT_EXEC'/'PROT_WRITE'/g Modules/_ctypes/mallo*.c } Thanks to everyone for identifying the issue and to Matt for isolating the code and giving me the idea how to fix this. Since upstream Python people apparently don't care about -hardened users, or the security of their other users, I think this a decent solution if you need a usable Python installation. The fact that this is issue has not been solved after 20 months is sad.
I've added the proposed fix to the python 2.7.3 ebuild, hitting the tree shortly.
(In reply to comment #91) In-tree python uses system libffi that needs to be patched, or else the python part of the fix won't work.
This is the "libffi part of the fix (3.0.10)"? Has that been discussed with libffi upstream at all?
(In reply to comment #93) > This is the "libffi part of the fix (3.0.10)"? That depends on which behavior you choose. "libffi part of the fix (3.0.10)" is a dumb compatible fix that causes annoying "denied RWX mmap" kernel log messages per every closure allocation on kernels compiled with CONFIG_GRKERNSEC_RWXMAP_LOG. "python3 patch" contains a better fix: mmap RW by default unless the FFI_DISABLE_EMUTRAMP environment variable does exist (empty or not). > Has that been discussed with libffi upstream at all? No.
(In reply to comment #94) > "libffi part of the fix (3.0.10)" is a dumb compatible fix that causes > annoying "denied RWX mmap" kernel log messages per every closure allocation > on kernels compiled with CONFIG_GRKERNSEC_RWXMAP_LOG. > > "python3 patch" contains a better fix: mmap RW by default unless the > FFI_DISABLE_EMUTRAMP environment variable does exist (empty or not). err, somewhere above i questioned this 'better' fix already and got no response so here it is again: what's the point of the runtime detection of EMUTRAMP again? and why would anyone have to set a *userland* variable for it when it's a *kernel* feature? (if you really want this kind of detection then check for 'E' in /proc/self/status:PaX)
(In reply to comment #95) > err, somewhere above i questioned this 'better' fix already and got no It's better in comparison with "libffi part of the fix (3.0.10)". It's no way better than the simple conditional patches I use on my systems. Upstreaming this wasn't my idea, and I don't like it, if you ask me. > response so here it is again: what's the point of the runtime detection of > EMUTRAMP again? and why would anyone have to set a *userland* variable for FFI_DISABLE_EMUTRAMP *disables* EMUTRAMP support (read: enables PROT_EXEC use) in _ctypes and libffi. I tried to predict what upstream would want, keeping in mind their usual ways to resolve such issues, and at the same time keep the fix secure by default yet tunable at runtime, for performance and compatibility. > it when it's a *kernel* feature? (if you really want this kind of detection > then check for 'E' in /proc/self/status:PaX) I refused this idea because /proc isn't always available. For example, I don't mount it inside chroot environments yet I do run chrooted python apps. And someone on #grsecurity have the entire /proc hidden using RBAC. So I had to keep that in mind.
Created attachment 308707 [details, diff] simple conditional libffi fix
Created attachment 308711 [details, diff] simple conditional python fix
This bug is still not resolved. Please, change the status.
(In reply to comment #99) > This bug is still not resolved. Please, change the status. This bug was fixed, but hardened kernel some months later invented a new way of breaking ctypes.
A simpler solution was implemented and backported to dev-lang/python-{2.6.8,2.7.3-r1,3.1.5,3.2.3}::gentoo. If somebody wants to use a different solution, then he should send patches to libffi/CPython upstreams. When patches are accepted by upstreams, then a separate bug for libffi and a separate bug for CPython can be created in Gentoo bugzilla.
Updated to libffi-3.0.13-r1 and recompiled python, now dmesg prints 'grsec: denied RWX mmap of <anonymous mapping> by' every time i run emerge or equery: [169075.733809] grsec: denied RWX mmap of <anonymous mapping> by /usr/lib64/portage/bin/emerge[emerge:24515] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[bash:18955] uid/euid:0/0 gid/egid:0/0 [169076.268528] grsec: denied RWX mmap of <anonymous mapping> by /usr/lib64/portage/bin/portageq[portageq:24524] uid/euid:0/0 gid/egid:0/0, parent /usr/bin/gcc-config[gcc-config:24523] uid/euid:0/0 gid/egid:0/0 [169081.002196] grsec: denied RWX mmap of <anonymous mapping> by /usr/lib64/python-exec/python2.7/equery[equery:24542] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[bash:18955] uid/euid:0/0 gid/egid:0/0 emerge and equery do not crash and are not killed. # paxctl-ng -v /usr/bin/python2.7 /usr/bin/python2.7: open(O_RDWR) failed: cannot change PT_PAX flags PT_PAX : -e--- XATTR_PAX : -E--- [ebuild R ] sys-apps/portage-2.2.8-r1 USE="(ipc) python2 (xattr) -build -doc -epydoc (-pypy2_0) -python3 (-selinux)" LINGUAS="ru" PYTHON_TARGETS="python2_7 (-pypy2_0) -python2_6 -python3_2 -python3_3 (-python3_4)" 0 kB [ebuild NS ] dev-lang/python-3.3.3:3.3 [2.7.5-r3:2.7] USE="hardened ipv6 ncurses readline sqlite ssl threads xml -build -doc -examples -gdbm -tk -wininst" 0 kB [ebuild R ] dev-libs/libffi-3.0.13-r1 USE="pax_kernel {test} -debug -static-libs" ABI_X86="(64) (-32) (-x32)" 0 kB
I don't see any more problems, reopen if.