Bug 386647 – dev-java/jamvm-1.5.4-r1 segfaults with hardened gcc-4.5.3-r1 (any profile), unless -Xnoinlining is specified

• Product: Gentoo Linux
• Status: RESOLVED TEST-REQUEST

[Description Faustus 2011-10-10 12:10:37 UTC]

Here is a summary, as far as I can tell after some testing:

1. The problem is with using a hardened compiler, not with hardened kernel. JamVM segfaults when executed under regular kernel, too.
2. JamVM worked fine when compiled with hardened gcc-4.4
3. JamVM, when compiled with *any* profile of hardened gcc-4.5.3-r1 (including vanilla) segfaults on any class file (e.g., empty main(String[] args)).
4. Compiling jamvm on a non-hardened x86 host (same gcc version), and then running the resulting /usr/bin/jamvm in the hardened environment (regardless of the kernel), does not segfault.
5. Compiling jamvm with a hardened gcc, and running the resulting /usr/bin/jamvm on a regular x86 host (no hardened libraries or kernel) produces a segfault. So the issue appears to be limited to code produced by gcc for /usr/bin/jamvm (i.e., not any of the other hardened libraries, or gnu-classpath, etc.).
5. The problem persists with any CFLAGS, LDFLAGS and USE flags used for jamvm, e.g.: CFLAGS=-O0 LDFLAGS= emerge -q1 jamvm - still segfaults.
6. With -Xnoinlining, jamvm runs normally, even with complex long-running applications.

I did the testing on x86 only, as I have no hardened amd64. With regular amd64 gcc, jamvm does not segfault.

gcc USE-flags: hardened nls nptl openmp
jamvm USE-flags: (none)
gnu-classpath USE-flags: alsa gconf gmp gstreamer gtk xml

[Comment 1 Faustus 2012-02-07 19:43:55 UTC]

The problem persists with hardened gcc-4.5.3-r2 (piepatchset 0.4.7).

[Comment 2 Faustus 2012-07-11 11:36:26 UTC]

Hi, is anyone else able to confirm? This seems like a really strange issue: caused by any profile of hardened gcc, easy to reproduce.

[Comment 3 Magnus Granberg 2014-07-08 22:58:40 UTC]

Is it still there with gcc 4.8.x or newer