Created attachment 389182 [details] emerge --info '=dev-python/pyasn1-modules-0.0.5-r1::gentoo' =dev-python/pyasn1-modules-0.0.5-r1::gentoo failed to ebuild for the reason I can't understand.
Created attachment 389184 [details] build.log
$ emerge -pqv '=dev-python/pyasn1-modules-0.0.5-r1::gentoo' [ebuild N ] dev-python/pyasn1-modules-0.0.5-r1 PYTHON_TARGETS="python2_7 python3_3 python3_4 -pypy (-python3_2)"
i give hint. Traceback (most recent call last): File "setup.py", line 53, in <module> from setuptools import setup File "/usr/lib64/python2.7/site-packages/setuptools/__init__.py", line 12, in <module> from setuptools.extension import Extension File "/usr/lib64/python2.7/site-packages/setuptools/extension.py", line 8, in <module> from .dist import _get_unpatched File "/usr/lib64/python2.7/site-packages/setuptools/dist.py", line 18, in <module> from setuptools import windows_support File "/usr/lib64/python2.7/site-packages/setuptools/windows_support.py", line 2, in <module> import ctypes File "/usr/lib64/python2.7/ctypes/__init__.py", line 555, in <module> _reset_cache() File "/usr/lib64/python2.7/ctypes/__init__.py", line 279, in _reset_cache CFUNCTYPE(c_int)(lambda: None) MemoryError
I don't really understand why did you close the bug. The problem was in dev-libs/libffi-3.2 As it's also reported here: http://forums.gentoo.org/viewtopic-t-1004050.html I emerged dev-libs/libffi-3.1-r3 and masked version 3.2. Everything works fine now but the problem in libffi stays.
Toolchain/selinux: Any ideas here? The forums seem to indicate that disabling selinux works around this.
*** Bug 529266 has been marked as a duplicate of this bug. ***
*** Bug 529118 has been marked as a duplicate of this bug. ***
i'll de-KEYWORD it for now, but it is a bit odd, and i can't reproduce can someone with the failure see if this works: python2.7 -c 'import ctypes' if it does, what about: tar xf youtube-dl-2014.11.14.tar.gz cd youtube-dl python2.7 setup.py build if any of those commands fail, please run it through strace: strace -f -s 4096 -v -o log python2.7 ... and then attach that log file here
A user in IRC said that this problem was occurring for him only for processes run inside emerge session and not outside, and also only with FEATURES="userpriv" (i.e. not with FEATURES="-userpriv"). (I cannot reproduce this problem.)
(In reply to SpanKY from comment #8) I can reproduce the problem consistently; however, it indeed disappears if I disable userpriv; if I run these commands manually with an unprivilegied user I don't see the memoryerror. FEATURES="userpriv -usersandbox" also works fine, so I'd say it's related to usersandbox. what would portage do with usersandbox ?
This sounds like a repeat of bug #436626 with selinux denying a PROT_WRITE | PROT_EXEC mmaping. Try this: Make sure you're hitting the bug, then strace -f python -c 'import ctypes' and see if its dying on some mmapping. If you were using a pax kernel, then I would recommend configuring your kernel with EMUTRAMP and doing paxctl-ng -E on your python executables. This won't work on a non-hardened kernel because you need the kernel's help to emulate the trampolines.
(In reply to Anthony Basile from comment #11) > This sounds like a repeat of bug #436626 with selinux denying a PROT_WRITE | > PROT_EXEC mmaping. > > Try this: Make sure you're hitting the bug, then strace -f python -c 'import > ctypes' and see if its dying on some mmapping. Sorry I should read these reports more carefully. See comment #8 for the strace instructions.
Created attachment 389724 [details] strace log okey, so I've been able to reproduce it with "python2.7 -c 'import ctypes'" from the sandbox shell as a user. here is the strace log.
Are any of you on the SELinux/Hardened profile or is SELinux enabled in the kernel? (ie does /sys/fs/selinux/ exist?) If you do have selinux enabled, can you attach the relevant part of the audit logs. If I update to libffi-3.2 to try and replicate the issue will it completely hose my system or can i just downgrade right after and there will be no lasting problems? @aballier: what exactly did you run to replicate it in the sandbox? I dont see any PROT_WRITE|PROT_EXEC mmap lines in the strace log tho so it doesnt seem like that would be the issue
(In reply to Jason Zaman from comment #14) > Are any of you on the SELinux/Hardened profile or is SELinux enabled in the > kernel? (ie does /sys/fs/selinux/ exist?) > If you do have selinux enabled, can you attach the relevant part of the > audit logs. > > If I update to libffi-3.2 to try and replicate the issue will it completely > hose my system or can i just downgrade right after and there will be no > lasting problems? > > @aballier: what exactly did you run to replicate it in the sandbox? > > I dont see any PROT_WRITE|PROT_EXEC mmap lines in the strace log tho so it > doesnt seem like that would be the issue There isn't any. This not the same bug as the earlier one on pax. I suspect that selinux is denying the following: 19464 open("/tmp", O_RDWR|O_EXCL|O_DIRECTORY|O_CLOEXEC|0x400000) = -1 EACCES (Permission denied) 19464 open("/tmp", O_RDWR|O_EXCL|O_DIRECTORY|O_CLOEXEC|0x400000) = -1 EACCES (Permission denied) 19464 open("/var/tmp", O_RDWR|O_EXCL|O_DIRECTORY|O_CLOEXEC|0x400000) = -1 EACCES (Permission denied) and that libffi is trying to write its snippets of code there. I'm not running selinux, but take a look at SwifT's blog on the subject as it might shed some light: http://blog.siphos.be/2013/04/securely-handling-libffi/
I have selinux enabled (I don't know why, seriously, maybe I should disable it) and I'm able to reproduce failing emerge pyasn1-modules but have no errors doing (In reply to SpanKY from comment #8) > can someone with the failure see if this works: > python2.7 -c 'import ctypes' > > if it does, what about: > tar xf youtube-dl-2014.11.14.tar.gz > cd youtube-dl > python2.7 setup.py build
(In reply to Anthony Basile from comment #15) 0x400000 is O_TMPFILE. that fails on my system as well: $ uname -r $ cat test.c main(int argc, char *argv[]) { open(argv[1], O_RDWR|O_EXCL|O_DIRECTORY|O_CLOEXEC|O_TMPFILE); } $ gcc test.c $ strace -e open ./a.out /tmp open("/tmp", O_RDWR|O_EXCL|O_DIRECTORY|O_CLOEXEC|0x400000) = -1 EACCES (Permission denied) $ strace -e open ./a.out ~/ open("/home/vapier/", O_RDWR|O_EXCL|O_DIRECTORY|O_CLOEXEC|0x400000) = -1 EACCES (Permission denied) not that on my system it triggers that execution code path in libffi, but i'm thinking they didn't quite test it out. would be easy to change the lib to also use EACCES as a fallback case in the library while we chase down why it's returning that in the first place from the kernel.
Created attachment 389792 [details, diff] libffi-EACCES.patch can someone who is seeing this problem see if this patch helps ?
looks like the kernel has already fixed the bug on their side: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=69a91c237ab0ebe4e9fdeaf6d0090c85275594ec so we do need libffi to check for EACCES and fall back when it fails
(In reply to SpanKY from comment #19) glibc behavior might also be related. if i adjust my testcase to call open() with explicit mode flags, it works. can someone who is seeing this bug try upgrading to linux-3.17.3 ? the latest 3.16.x patchset doesn't include the fix. probably the reason it fails as non-root but works as root is that root gets a free pass on a lot of permission checks in the kernel.
Looking at sandbox's open() wrapper, it doesn't handle O_TMPFILE correctly - it should treat O_TMPFILE the same way as it does O_CREAT and pass the mode argument through to open(), but it actually ignores the mode specified by the caller and uses 0 instead, triggering the underlying kernel issue. (Also, patching libffi may not be enough. I've been having a weird build error in Firefox that I suspect might be related - make exits with a mysterious "Out of memory" error that goes away if I disable usersandbox and appears to have stopped now I've upgraded to 3.17.3)
(In reply to SpanKY from comment #20) > (In reply to SpanKY from comment #19) > > glibc behavior might also be related. if i adjust my testcase to call > open() with explicit mode flags, it works. > > can someone who is seeing this bug try upgrading to linux-3.17.3 ? the > latest 3.16.x patchset doesn't include the fix. > > probably the reason it fails as non-root but works as root is that root gets > a free pass on a lot of permission checks in the kernel. Yes, the problem has gone on 3.17.3
(In reply to Aidan Thornton from comment #21) that probably is true, but i don't think it matters in this case. libffi is passing O_CREAT in addition to O_TMPFILE, and libsandbox loads the mode flag whenever O_CREAT is specified ... it doesn't filter out the flags. you can see this in strace: $ sandbox $ strace -e open ./a.out /tmp ... open("/tmp", O_RDWR|O_CREAT|O_DIRECTORY|O_TMPFILE, 0700) = -1 EINVAL (Invalid argument)
(In reply to SpanKY from comment #23) If I'm following the strace log that was posted correctly, the calls that are failing with EACCES *don't* have O_CREAT set, e.g.: 19464 open("/tmp", O_RDWR|O_EXCL|O_DIRECTORY|O_CLOEXEC|0x400000) = -1 EACCES (Permission denied) Notice that the example you give with O_CREAT set in addition to O_TMPFILE ends up passing a valid mode flag to open, which means it doesn't hit the kernel bug and fails with EINVAL rather than EACCES, and that error causes libffi to retry without O_TMPFILE even without the patch. So far as I can tell, the only reason libffi is receiving EACCES here and breaking is because sandbox is buggy.
Please test 3.2.1. Samuli tells me that we should be able to restore keywords in this version.
(In reply to Aidan Thornton from comment #24) i must have been confusing code bases. libffi isn't using O_CREAT anywhere. it is using O_EXCL|O_TMPFILE. (In reply to William Hubbs from comment #25) the only diff between 3.2.1 and 3.2 is a build fix for iOS. the problem doesn't lie with libffi, but the C library, sandbox, and the kernel. largely if you upgrade past the broken kernel version, things work. i've deployed my EACCES workaround in 3.2.1: http://sources.gentoo.org/dev-libs/libffi/files/libffi-3.2.1-o-tmpfile-eacces.patch?rev=1.1 lets leave this open until sandbox & glibc also get updated.
sandbox-2.7 should be fixed: http://gitweb.gentoo.org/proj/sandbox.git/commit/?id=775921d02b2d1fea48e6232520497c2ad7431975
i'm not sure there's anything left to fix here ...
