Summary: | =dev-java/icedtea-7.2.2.1 does not build on a PaX-enabled kernel | ||
---|---|---|---|
Product: | Gentoo Linux | Reporter: | Maciej S. Szmigiero <mail> |
Component: | [OLD] Java | Assignee: | Java team <java> |
Status: | RESOLVED FIXED | ||
Severity: | normal | CC: | alon.barlev, andrej.gelenberg, hardened, lejonet, powerman-asdf |
Priority: | Normal | ||
Version: | unspecified | ||
Hardware: | x86 | ||
OS: | Linux | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Attachments: |
Patch to skip the test_gamma script that makes the build fail
strace of java call with -Xms600m -Xmx600m flags analyse of mmap calls |
Description
Maciej S. Szmigiero
2011-11-06 19:32:02 UTC
I can confirm part 2 of this bug on gentoo ~amd64 no-multilib hardened with a pax-enabled kernel 3.0.8-hardened. Suggested workaround works too. For part 1, I probably was lucky my javac already had appropiate pax flags. Part 1 should not be an issue on amd64 as it has a native NX bit support without tricks such as SEGMEXEC or PAGEEXEC. MPROTECT still does apply, but javac has this disabled by ebuild when installed. The icedtea-7 missing the no-test_gamma.patch Created attachment 292281 [details, diff] Patch to skip the test_gamma script that makes the build fail After looking at the icedtea-6 ebuilds we found that this issue is a old one and a patch for it had been submitted before, see bug 244901, and after looking in the patches dir for icedtea-7 the patch wasn't found there. http://icedtea.classpath.org/hg/icedtea6/rev/cb463b94b82d is the commit which fixed this issue for IcedTea6. Committed a fix replicating the above for IcedTea7. Works for me on a PaX enabled kernel on amd64. Thanks to Daniel Kuehn and Magnus Granberg for their findings. The patch only fixes non bootstrap builds. I changed the ebuild to use the non bootstrap path whenever possible, which means no PaX if using gcj-jdk for now. As for issue 1) is building icedtea-7 the only time this is triggered and other java applications run just fine? If using icedtea-bin-7 do you see the same happen? (In reply to comment #6) > The patch only fixes non bootstrap builds. I changed the ebuild to use the non > bootstrap path whenever possible, which means no PaX if using gcj-jdk for now. > > As for issue 1) is building icedtea-7 the only time this is triggered and other > java applications run just fine? If using icedtea-bin-7 do you see the same > happen? Well if I try to build hello world using "-J-Xmx1024m" it exits with the same error. Compiling it (hello world) without this switch or with object heap size of 700m or less works. For reference malloc(1024*1024*1024) from C program fails too when SEGMEXEC is enabled. I can try to install icedtea-bin, but I am pretty sure the effect will be the same if it does not have SEGMEXEC disabled. (In reply to comment #7) > Well if I try to build hello world using "-J-Xmx1024m" it exits with the same > error. > Compiling it (hello world) without this switch or with object heap size of 700m > or less works. > For reference malloc(1024*1024*1024) from C program fails too when SEGMEXEC is > enabled. Thanks for the explanation, just a bit surprising that this only surfaced now. All 1.6/1.7 JDKs/JREs in tree have now the required PaX markings. You might have to reinstall the latest unstable version of each slot. Builds fine now, thanks. @Maciej Szmigiero no, malloc(1024*1024*1024) did not fail. Code: #include <stdlib.h> #include <stdio.h> int main() { void * p = malloc(1024 * 1024 * 1024); if ( p ) { free(p); return 0; } else { printf("malloc failed: %m\n"); return 1; } } # gcc test.c # pax -S a.out # a.out a.out return 0 It seems, what java do overlapping mmaps calls. Here strace linsting: [pid 9533] munmap(0x7f34644f4000, 4096) = 0 [pid 9533] mmap(NULL, 1073741824, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x7f3422196000 [pid 9533] munmap(0x7f3422196000, 1073741824) = 0 [pid 9533] mmap(NULL, 1075838976, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x7f3421f96000 [pid 9533] munmap(0x7f3421f96000, 1075838976) = 0 [pid 9533] mmap(0x7f3422000000, 1073741824, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x7f3422000000 [pid 9533] mmap(0x7f3422000000, 4194304, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|0x40000, -1, 0) = 0x7f3422000000 [pid 9533] mmap(NULL, 16777216, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x7f3421000000 [pid 9533] mmap(0x7f3421000000, 2097152, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|0x40000, -1, 0) = 0x7f3421000000 [pid 9533] mmap(0xca800000, 897581056, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xca800000 [pid 9533] mmap(NULL, 1757184, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x7f3420e53000 [pid 9533] mmap(0x7f3420fff000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f3420fff000 [pid 9533] mmap(0xf3800000, 209715200, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|0x40000, -1, 0) = 0xf3800000 [pid 9533] mmap(0x7f3420f9b000, 409600, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f3420f9b000 [pid 9533] mmap(0xda800000, 419430400, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|0x40000, -1, 0) = -1 ENOMEM (Cannot allocate memory) [pid 9533] write(1, "Error occurred during initializa"..., 43Error occurred during initialization of VM ) = 43 [pid 9533] write(1, "Could not reserve enough space f"..., 46Could not reserve enough space for object heap) = 46 [pid 9533] write(1, "\n", 1 ) = 1 Suspicious are: [pid 9533] mmap(0xca800000, 897581056, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xca800000 and pid 9533] mmap(0xda800000, 419430400, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|0x40000, -1, 0) = -1 ENOMEM (Cannot allocate memory) The first mmap would alloc memory until 0x000100000000, and 0xda800000 is before that. Then it fail with MAP_FIXED set. Created attachment 294193 [details]
strace of java call with -Xms600m -Xmx600m flags
Are you sure you are testing this on 32-bit system? # cat test.c #include <malloc.h> #include <stdio.h> int main(void) { void *p = malloc(1024*1024*1024); printf("%p\n", p); return 0; } # gcc -o test test.c # paxctl -v test PaX control v0.5 Copyright 2004,2005,2006,2007 PaX Team <pageexec@freemail.hu> - PaX flags: -------x-e-- [test] RANDEXEC is disabled EMUTRAMP is disabled # strace ./test execve("./test", ["./test"], [/* 35 vars */]) = 0 brk(0) = 0x13988370 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x51281000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) open("/etc/ld.so.cache", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=58282, ...}) = 0 mmap2(NULL, 58282, PROT_READ, MAP_PRIVATE, 3, 0) = 0x51272000 close(3) = 0 open("/lib/libc.so.6", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0@o\1\0004\0\0\0\374"..., 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=1417548, ...}) = 0 mmap2(NULL, 1428672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x51115000 mprotect(0x5126b000, 4096, PROT_NONE) = 0 mmap2(0x5126c000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x156) = 0x5126c000 mmap2(0x5126f000, 11456, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x5126f000 close(3) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x51114000 set_thread_area({entry_number:-1 -> 6, base_addr:0x511146c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 mprotect(0x5126c000, 8192, PROT_READ) = 0 mprotect(0x13985000, 4096, PROT_READ) = 0 mprotect(0x5129f000, 4096, PROT_READ) = 0 munmap(0x51272000, 58282) = 0 mmap2(NULL, 1073745920, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory) brk(0) = 0x13988370 brk(0x539a9370) = 0x13988370 mmap2(NULL, 1073876992, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory) mmap2(NULL, 2097152, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x50f14000 munmap(0x50f14000, 966656) = 0 munmap(0x51100000, 81920) = 0 mprotect(0x51000000, 135168, PROT_READ|PROT_WRITE) = 0 mmap2(NULL, 1073745920, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory) fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 6), ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x51280000 write(1, "(nil)\n"..., 6(nil) ) = 6 exit_group(0) = ? Created attachment 294195 [details]
analyse of mmap calls
This file has form:
<start address> <end address> <size> [*]
With * marked lines mean, what mmap call failed.
*** Bug 344135 has been marked as a duplicate of this bug. *** it's 64-bit system, but i get same error $ /usr/bin/java -Xms600m -Xmx600m Error occurred during initialization of VM Could not reserve enough space for object heap (In reply to comment #15) > it's 64-bit system, but i get same error > $ /usr/bin/java -Xms600m -Xmx600m > Error occurred during initialization of VM > Could not reserve enough space for object heap Andrej, I think you are having an different issue here, as SEGMEXEC is only used on 32-bit x86 architecture. 64-bit x86 has native NX bit support and does not require hacks like split code/data vmas (SEGMEXEC). Unfortunately, I do not have an 64-bit PaX-enabled box to check what is wrong there. Fixed bootstrap builds in icedtea-6.1.10.4-r3 and icedtea-7.2.0-r3 by building and pax marking the intermediate jdk. Thanks for the report and the support from various sides in this matter. @Andrej Gelenberg: That looks like a sufficiently different issue, please open a separate bug. Hello, icedtea-7.2.2.1 has this again. --- # /var/tmp/portage/dev-java/icedtea-7.2.2.1/work/icedtea-2.2.1/openjdk.build-boot/bin/java -XX:-PrintVMOptions -XX:+UnlockDiagnosticVMOptions -XX:-LogVMOutput -Xmx512m -Xms512m -XX:PermSize=32m -XX:MaxPermSize=160m -cp /var/tmp/portage/dev-java/icedtea-7.2.2.1/work/icedtea-2.2.1/openjdk.build-boot/classes sun.rmi.rmic.Main -classpath "/var/tmp/portage/dev-java/icedtea-7.2.2.1/work/icedtea-2.2.1/openjdk.build-boot/classes" -d /var/tmp/portage/dev-java/icedtea-7.2.2.1/work/icedtea-2.2.1/openjdk.build-boot/classes -v1.2 -keepgenerated javax.management.remote.rmi.RMIConnectionImpl Error occurred during initialization of VM Could not reserve enough space for code cache --- --- # paxctl -v /var/tmp/portage/dev-java/icedtea-7.2.2.1/work/icedtea-2.2.1/openjdk.build-boot/bin/java PaX control v0.7 Copyright 2004,2005,2006,2007,2009,2010,2011,2012 PaX Team <pageexec@freemail.hu> - PaX flags: -------x-e-- [/var/tmp/portage/dev-java/icedtea-7.2.2.1/work/icedtea-2.2.1/openjdk.build-boot/bin/java] RANDEXEC is disabled EMUTRAMP is disabled --- Adding MPROTECT solves the issue. Do you want to reopen or should I open a new bug? (In reply to comment #18) > Hello, > icedtea-7.2.2.1 has this again. Guys, Please reopen, or response... I haven't tested, but somehow I don't doubt it. I'm opening unconfirmed. Thanks to Andrew John Hughes the PaX patches from back then were added to upstream icedtea 7. This time, the reason is a security patch which changed where a freshly built jdk is used first. Either way, it's fixed now. 24 Jun 2012; Ralph Sennhauser <sera@gentoo.org> icedtea-6.1.11.3.ebuild, files/icedtea-6_pax_kernel_support.patch, icedtea-7.2.1.1.ebuild, +files/icedtea-7.2.1.1-pax_mark_rmic_java.patch, icedtea-7.2.2.1.ebuild, +files/icedtea-7.2.2.1-pax_mark_rmic_java.patch: Fix building with PaX enabled kernels. #422525 |