The following message appeared a couple dozen times in my dmesg during the day: [33547.145599] PAX: size overflow detected in function ptr_to_compat /usr/src/linux-3.8.6-hardened/arch/x86/include/asm/compat.h:293 cicus.55_4 min, count: 4 [33547.145604] Pid: 27295, comm: ld-linux.so.2 Not tainted 3.8.6 #1 [33547.145605] Call Trace: [33547.145610] [<ffffffff810ff7a5>] ? 0xffffffff810ff7a5 [33547.145612] [<ffffffff81041c0b>] ? 0xffffffff81041c0b [33547.145614] [<ffffffff81042702>] ? 0xffffffff81042702 [33547.145616] [<ffffffff8100242b>] ? 0xffffffff8100242b [33547.145619] [<ffffffff810029d2>] ? 0xffffffff810029d2 [33547.145621] [<ffffffff815f66c0>] ? 0xffffffff815f66c0 According to https://forums.grsecurity.net/viewtopic.php?f=7&t=3043 this led to the discovery of `cat CVE-2013-0914`. That CVE was fixed in 3.8.4. I run 3.8.6-hardened and the very same PAX report pops up. Is the bug not completely gone? Please suggest further action. I'll try out the latest ~arch hardened-sources (or the latest vanilla+pax, anyway) tomorrow, but I've run 3.8.6 for quite some time now and never seen the error before, despite checking dmesg now and then (albeit not daily, that's for sure). It could be a few weeks before I run into the bug again, if the latest kernel has it too. If you happen to know which particular kernel option deals with the symbol resolution, you can save some of my time.
Created attachment 348572 [details] emerge --info So why don't I look into the older logs? - The report is found today and yesterday, that's all. I played with coccinelle and tried 'make coccicheck' for the kernel sources these two days. Could that at all be related? I though coccinelle was just a parser, not a tool that triggers errors on purpose.
Could you attach your full dmesg, the bzImage of your kernel, and the System.map of your kernel?
Created attachment 348588 [details] kern.log (registering kern.*), 3.8.6-hardened
Created attachment 348590 [details] System.map, 3.8.6-hardened
Created attachment 348592 [details] .config, 3.8.6-hardened
http://www.arkhalia.cz/temp/vmlinuz-3.8.6 SHA256: 981db254f6a0577fa422adb35a627595c2d41d8cea009b4fa8abc0229a49c0e5
(In reply to comment #6) > http://www.arkhalia.cz/temp/vmlinuz-3.8.6 > SHA256: 981db254f6a0577fa422adb35a627595c2d41d8cea009b4fa8abc0229a49c0e5 Its `uname -r` is 3.8.6, but it's a regular hardened-sources product.
After a week spent mostly with 3.8.12-hardened I have no new overflow reports by PAX. I had coccinelle running throughout the day yesterday and today, too.
Looks like the 32bit analogs might not have received the prior fix?
1. events triggered by the size overflow plugin should be reported directly to Emese and me, we're in the best position to track these down (especially false positives), not to mention that as the discoverer of real bugs you could have earned some cash from google's bug bounty program where we would have redirected you after the initial analysis ;). 2. a quick look at the backtrace suggests that it's a similar issue as the already fixed CVE, but i can't tell for sure until i can look at the disasm so please post vmlinux (from the build dir), bzImage is not good enough for this (vmlinux also has symbol info, so you won't need to bother with recompilation unless of course you already lost vmlinux ;). 3. for the future, it's CONFIG_GRKERNSEC_HIDESYM that you want to disable for getting symbol names in backtraces but the kernel will also then leak address information in other ways, so be careful where you run such a kernel. 4. as for 3.8.12, did you since enable STRUCTLEAK? if my guess is right, it'd fix the underlying problem which seems to be a kernel stack leak via sys_timer_create and its event.sigev_value which may not be fully initialized on 64 bit archs. this field is a union between an int and void *, so writing to the int field won't fully set the whole structure whereas on copying out to userland, the entire structure is copied. this is where the overflow plugin's check triggers (if the leaked high 32 bits are not 0 then the resulting 64 bit pointer cannot be represented as a 32 bit one), but i'll be able to tell for sure from the disasm. the sequence of relevant events is this: event.sigev_value.sival_int = new_timer->it_id; new_timer->sigq->info.si_value = event.sigev_value; put_user_ex(ptr_to_compat(from->si_ptr), &to->si_ptr); so we set the int field, then copy the entire union (with the high 32 bits leaking kernel stack) then again copy the ptr field of the union (instead of the int field). one way to fix this would be to copy out only the int field in copy_siginfo_to_user32 (i.e., use si_int instead of si_ptr) but this would then have to be double-checked on other archs as well, so the more robust fix is to initialize event or its event.sigev_value field completely in sys_timer_create. so Roman, if you can reproduce this at will (after disabling STRUCTLEAK if you have it enabled) then you can test the above suggested fixes and report this to the kernel devs and oss-security as well to get a CVE.
(In reply to PaX Team from comment #10) > 1. events triggered by the size overflow plugin should be reported directly > to Emese and me Please, include this in the log warning. > 2. a quick look at the backtrace suggests that it's a similar issue as the > already fixed CVE, but i can't tell for sure until i can look at the disasm > so please post vmlinux I don't have the original anymore, so I'm attaching a freshly compiled vmlinux, using the same sources, gcc, binutils and .config: http://www.arkhalia.cz/temp/vmlinux.xz SHA256: 6dcdb241d9d15195725094338a131d560723ba1942b3d43fefd27d5e9c0187bf > 3. for the future, it's CONFIG_GRKERNSEC_HIDESYM that you want to disable > for getting symbol names in backtraces but the kernel will also then leak > address information in other ways, so be careful where you run such a kernel. I'm not following you. The kernel itself is not limited in access to compiled-in symbol addresses (with KALLSYMS), right? Provided that root can view dmesg, I can get the resolved symbols from the log, can't I? > 4. as for 3.8.12, did you since enable STRUCTLEAK? That one is still off. > so Roman, if you can reproduce this at will (after > disabling STRUCTLEAK if you have it enabled) then you can test the above > suggested fixes and report this to the kernel devs and oss-security as well > to get a CVE. Thank you for the suggestions. Sadly, I haven't hit the itchy spot since. I don't remember / cannot think of any unusual work I did with the computer when the error got triggered. Coccinelle was the only idea, but it seems to be a dry source after repeated runs. Unless the problem is fixed in 3.8.12. I'll keep trying for a couple more days and then go back to 3.8.6 and try to trigger the error with coccinelle there. If it occurs, I'll sort of know that 3.8.12 has the bug fixed. Sort of.:)
so a few more notes after i could spend some time on reproducing this. 1. of course i'm dumb and the leak doesn't happen with a 32 bit process, that's only where the overflow plugin detects the problem, but for actually leaking something you'll need a 64 bit process. that also means that the fix should be in copy_siginfo_to_user (to write si_int under __SI_TIMER if that field is always interpreted as an int, and also take care of ia64 that has its own implementation of this function) or fully initialize event.sigev_value in sys_timer_create. the rest of the analysis stands (and no, 3.8.12 didn't fix this, i have a PoC working under 3.9.3 just fine). 2. Roman, i was wrong about HIDESYM (it used to be true many years ago as spender kindly reminded me ;P), what you need is KALLSYMS and you had it disabled in your config. 3. for reproducing it i think you ran something other than cocci as well. i assume your userland is all 64 bit (at least the bits you compile yourself) and the comm (ld-linux) in the backtrace suggests that you were probably running acrobat reader or similar that tries to be LSB compliant and uses an ELF interpreter that doesn't exist on gentoo and then falls back to executing its main binary via the glibc ld.so (look at the hack in /opt/Adobe/Reader9/bin/acroread and LaunchBinary). if it's not acrobat then it's still some binary only 32 bit app, i assume you don't have many around so you can figure out which one triggered this.
(In reply to PaX Team from comment #12) > 3. for reproducing it i think you ran something other than cocci as well. Acroread is clean, so is an old 32b game I keep around. I strongly suspect glibc's build - the 'make check' phase, to be concrete. I ran it during the incriminated period. This time I only got this during 'make check': May 27 22:59:54 palantir kernel: PAX: execution attempt in: <anonymous mapping>, 00000000-00041000 00000000 May 27 22:59:54 palantir kernel: PAX: terminating task: /boot/tmp/portage/sys-libs/glibc-2.15-r3/work/build-x86-x86_64-pc-linux-gnu-nptl/elf/ld.so(ld-linux.so.2):1422, uid/euid: 250/250, PC: (nil), SP: 00000000fe2e1b7c May 27 22:59:54 palantir kernel: PAX: bytes at PC: ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? May 27 22:59:54 palantir kernel: PAX: bytes at SP-8: fe2e5fc7 fe2e1c28 5a69a607 ffffffff 5a69a6a2 5a660004 bd51a800 5a8409e0 5a69a5e6 5a65ff4c 5a65e255 5a65e460 00000000 fe2e1c28 fe2e1c28 5a65e247 5a65ff4c fe2e1c00 5a65e0d0 00000000 00000000 00000002 (...) May 27 23:15:28 palantir kernel: PAX: execution attempt in: (null), 00000000-00000000 00000000 May 27 23:15:28 palantir kernel: PAX: terminating task: /boot/tmp/portage/sys-libs/glibc-2.15-r3/work/build-amd64-x86_64-pc-linux-gnu-nptl/elf/ld.so(ld-linux-x86-64):8170, uid/euid: 250/250, PC: (nil), SP: 000003aa8ca8bce8 May 27 23:15:28 palantir kernel: PAX: bytes at PC: ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? May 27 23:15:28 palantir kernel: PAX: bytes at SP-8: 000003aa8ca8be30 000002681b69e706 000003aa8ca8be30 000002681b69e5f1 0000000000000000 0000000000000000 0000000004000000 ffffffffffffffff 000003aa8ca905da 000002681b69e8f0 0000000000000000 Plus a couple resource limit overflows and segfaults in ld.so in between. These are nearby in the kernel log from when the original ptr_to_compat bug happened. I will re-run the glibc build additional couple times tomorrow.
Sorry, false alarm, turns out glibc is innocent. (Nevertheless, the 'make check' process displays some unseemly behavior there. Is that important/dangerous? If so, we may need to do something about that Gentoo-wide.) I'm looking at the old log once again. ============================================================================== May 17 10:18:42 palantir kernel: grsec: mount of proc to /proc by /bin/mount[mount:423] uid/euid:0/0 gid/egid:0/0, parent /sbin/rc[fstabinfo:422] uid/euid:0/0 gid/egid:0/0 May 17 10:18:43 palantir kernel: e1000e 0000:00:19.0: irq 47 for MSI/MSI-X May 17 10:18:43 palantir kernel: e1000e 0000:00:19.0: irq 47 for MSI/MSI-X May 17 10:18:45 palantir kernel: e1000e: enp0s25 NIC Link is Up 100 Mbps Full Duplex, Flow Control: Rx/Tx May 17 10:18:45 palantir kernel: e1000e 0000:00:19.0 enp0s25: 10/100 speed: disabling TSO May 17 12:59:12 palantir kernel: grsec: failed fork with errno ENOMEM by /usr/bin/python2.7[equery:19837] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[bash:19506] uid/euid:0/0 gid/egid:0/0 May 17 19:38:19 palantir kernel: PAX: size overflow detected in function ptr_to_compat /usr/src/linux-3.8.6-hardened/arch/x86/include/asm/compat.h:293 cicus.55_4 min, count: 4 May 17 19:38:19 palantir kernel: Pid: 27295, comm: ld-linux.so.2 Not tainted 3.8.6 #1 May 17 19:38:19 palantir kernel: Call Trace: May 17 19:38:19 palantir kernel: [<ffffffff810ff7a5>] ? 0xffffffff810ff7a5 May 17 19:38:19 palantir kernel: [<ffffffff81041c0b>] ? 0xffffffff81041c0b May 17 19:38:19 palantir kernel: [<ffffffff81042702>] ? 0xffffffff81042702 May 17 19:38:19 palantir kernel: [<ffffffff8100242b>] ? 0xffffffff8100242b May 17 19:38:19 palantir kernel: [<ffffffff810029d2>] ? 0xffffffff810029d2 May 17 19:38:19 palantir kernel: [<ffffffff815f66c0>] ? 0xffffffff815f66c0 ============================================================================== 10:18am: system boot. Then equery fell into an infinite fork loop, apparently. Could this be significant wrt the subsequent size overflow? I have an idea of what might have caused the ENOMEM, but I'm not able to replicate that error now. I'll try again tomorrow and if I'm still unsuccessful I'll try my own fork bomb. FWIW, I use no swap. Can you think of a way of finding out what command spawned that PID there (besides /var/log/*)? It's not glibc compilation, it's not acroread, wine, my old game or flash. Hmm...
I ran a custom basic fork bomb (32b and 64b), hit ENOMEM, then ran all kinds of stuff for a day. No gain.
What shall I do with this bug? Are you doing something with the data or do I report the matter elsewhere? Or is the whole thing useless when I can't replicate the bug?
This really might be just fixed in 3.8.12. I went back to 3.8.6 today and launched a 32b game under wine. It started. So I quit, began compiling glibc and then tried the game again. It didn't start and PAX again prints: [ 1271.025399] PAX: size overflow detected in function ptr_to_compat /usr/src/linux-3.8.6-hardened/arch/x86/include/asm/compat.h:293 cicus.55_4 min, count: 4 [ 1271.025404] Pid: 1772, comm: Game.exe Not tainted 3.8.6 #1 [ 1271.025405] Call Trace: [ 1271.025410] [<ffffffff810ff7a5>] ? 0xffffffff810ff7a5 [ 1271.025412] [<ffffffff81041c0b>] ? 0xffffffff81041c0b [ 1271.025414] [<ffffffff81042702>] ? 0xffffffff81042702 [ 1271.025416] [<ffffffff8100242b>] ? 0xffffffff8100242b [ 1271.025418] [<ffffffff81001012>] ? 0xffffffff81001012 [ 1271.025420] [<ffffffff810029d2>] ? 0xffffffff810029d2 [ 1271.025422] [<ffffffff815f66c0>] ? 0xffffffff815f66c0 [ 1271.025424] [<ffffffff815f7853>] ? 0xffffffff815f7853 Are you or the upstream interested in bugs in 3.8.6? I could recompile it with KALLSYMS to get the symbols resolved.
like i already said, there's a real infoleak bug here and it affects newer kernels as well, it's definitely not been fixed. if you give me the backtrace with symbols i can tell you if you triggered the same problem or not. as for passing it upstream, since you're the lucky finder, feel free to take credit for it otherwise the gentoo folks probably have some process to report this.
just a heads up, since there seems to be no progress on this, i'm going to fix this explicitly in the next PaX patch (STRUCTLEAK already provided protection).
Ouch, this slipped my mind. I'm absolutely out of time and other resources here. Could anyone please kindly forward this report to the kernel bugzilla, so that a fix is made in the vanilla tree as well? I don't have any new findings regarding this issue and haven't hit the bug with 3.8.12 or 3.9.5 (or 3.9.9 - not well tested, though).
(In reply to PaX Team from comment #19) > just a heads up, since there seems to be no progress on this, i'm going to > fix this explicitly in the next PaX patch (STRUCTLEAK already provided > protection). Or perhaps this can be supplied as a finished fix for the issue and merged with the vanilla kernel...?
spender, can you confirm that you requested on IRC that this bug be made public?
Yes. -Brad
3.8.x is no more in the Gentoo repository, so i think we can close this bug.
There are no longer any 2.x or <3.8.x kernels available in the repository with the exception of sys-kernel/xbox-sources which is unsupported by security.
