Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 567890

Summary: =dev-java/icedtea-bin-3.0.0_pre09-r1 - Stabilization request
Product: Gentoo Linux Reporter: James Le Cuirot <chewi>
Component: [OLD] Keywording and StabilizationAssignee: Java team <java>
Status: RESOLVED FIXED    
Severity: normal CC: ppc64
Priority: Normal Keywords: STABLEREQ
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://bugs.gentoo.org/show_bug.cgi?id=567850
Whiteboard:
Package list:
Runtime testing required: ---

Description James Le Cuirot gentoo-dev 2015-12-09 20:33:53 UTC
Removing Java 6 is now high on the agenda so we need to get Java 7 stable on ppc64.

Despite ppc64 HotSpot support in icedtea for some time now, we haven't been able to keyword it because I have not been able to find the cause of race conditions observed on timberdoodle. Luckily icedtea 3 (Java 8) seems to work so we do have an eventual escape route.

Because of this issue, I am providing a CACAO-based icedtea-bin for ppc64. 7.2.6.2 has been in the tree since 22nd Nov and 7.2.6.3 has just been added to deal with a vulnerability. 

I know it hasn't had much air time but my feeling is that we should not delay the Java 6 removal any longer as it should have gone ages ago. It's becoming increasingly fragile as package dependencies built for newer Java versions cause unexpected breakages.

So the package to stabilize is:
dev-java/icedtea-bin-7.2.6.3

Thanks!
Comment 1 Agostino Sarubbo gentoo-dev 2015-12-10 14:30:34 UTC
  dependency.bad [fatal]        14                                                                                                                                                                                                                                             
   dev-java/icedtea-bin/icedtea-bin-7.2.6.3.ebuild: PDEPEND: ppc64(default/linux/powerpc/ppc64/13.0/64bit-userland)                                                                                                                                                            
[     'dev-java/icedtea-web:0[icedtea7]',                                                                                                                                                                                                                                      
      'dev-java/icedtea-web:0[icedtea7,nsplugin]',                                                                                                                                                                                                                             
      'dev-java/icedtea-sound']
Comment 2 Agostino Sarubbo gentoo-dev 2015-12-10 14:43:19 UTC
This is the complete list I need from repoman side:

=dev-java/icedtea-bin-7.2.6.3
=dev-util/patchelf-0.8
=dev-java/icedtea-web-1.5.1-r1
=dev-java/icedtea-sound-1.0.1
=media-fonts/lklug-0.6
Comment 3 James Le Cuirot gentoo-dev 2015-12-10 14:47:54 UTC
(In reply to Agostino Sarubbo from comment #2)
> This is the complete list I need from repoman side:
> 
> =dev-java/icedtea-bin-7.2.6.3
> =dev-util/patchelf-0.8
> =dev-java/icedtea-web-1.5.1-r1
> =dev-java/icedtea-sound-1.0.1
> =media-fonts/lklug-0.6

Sorry, I forgot that I'd only masked those flags against icedtea and not icedtea-bin and I also missed those additional two packages. I was about to post the same list myself so please go ahead. I'll check more carefully in future.
Comment 4 Jeroen Roovers (RETIRED) gentoo-dev 2015-12-20 07:49:13 UTC
I am trying to build dev-java/icedtea against dev-java/icedtea-bin by way of having something to directly test with. It keeps failing on running out of memory. This is because it self-imposes some memory constraints:

-J-Xmx1792m -J-Xms1792m -J-XX:PermSize=32m -J-XX:MaxPermSize=160m

when it could be using all 8 gigabytes of RAM. Is there some way of raising these constraints to fit the system icedtea is being built on?
Comment 5 James Le Cuirot gentoo-dev 2015-12-20 13:09:38 UTC
(In reply to Jeroen Roovers from comment #4)
> I am trying to build dev-java/icedtea against dev-java/icedtea-bin by way of
> having something to directly test with. It keeps failing on running out of
> memory. This is because it self-imposes some memory constraints:
> 
> -J-Xmx1792m -J-Xms1792m -J-XX:PermSize=32m -J-XX:MaxPermSize=160m
> 
> when it could be using all 8 gigabytes of RAM. Is there some way of raising
> these constraints to fit the system icedtea is being built on?

I'm guessing you built it with USE=doc. Why do you think icedtea isn't keyworded for ppc64? :P That isn't the main reason, otherwise I would have patched around it, but I have raised this issue with upstream already. Also note that by the time it gets to building the docs, it is using the newly-built icedtea instead of the one used to build initially, unless you disabled jbootstrap.

The requirements tend to gradually rise over time and while upstream does test these things, they are using HotSpot rather than the lesser-tested CACAO, so they rely on us to let them know if it doesn't work any more.

If you don't specify any limit then HotSpot will allocate 1/4 of the total RAM, up to a maximum of 2GB. Limits are therefore specified here to aid people with say 2GB, which would be sufficient if HotSpot didn't only use 512MB of it.

In this particular case, upstream CACAO actually defaults to a max heap of 128MB! I patched this to give similar behaviour to HotSpot as too many ebuilds were dying as a result but on testing it just now, I have found that manually requesting more than 2GB causes the VM to die horribly. I'm pretty certain I tested this thoroughly so maybe something has changed. I will investigate and report back ASAP.

Out of interest, do you have your own ppc64 hardware? You mentioned 8GB but timberdoodle has 16GB.
Comment 6 James Le Cuirot gentoo-dev 2015-12-20 15:29:32 UTC
(In reply to James Le Cuirot from comment #5)
> In this particular case, upstream CACAO actually defaults to a max heap of
> 128MB! I patched this to give similar behaviour to HotSpot as too many
> ebuilds were dying as a result but on testing it just now, I have found that
> manually requesting more than 2GB causes the VM to die horribly. I'm pretty
> certain I tested this thoroughly so maybe something has changed. I will
> investigate and report back ASAP.

Hmm I didn't have that quite right. It dies if you specify any letter for -Xmx other than "m" or no letter at all, with or without my patch. Not my fault then! I'll look closer later.
Comment 7 Jeroen Roovers (RETIRED) gentoo-dev 2015-12-21 07:02:16 UTC
(In reply to James Le Cuirot from comment #5)
> I'm guessing you built it with USE=doc.

You would be wrong.

> Out of interest, do you have your own ppc64 hardware? You mentioned 8GB but
> timberdoodle has 16GB.

Yes. Or else, why would I be on that architecture team?
Comment 8 Jeroen Roovers (RETIRED) gentoo-dev 2015-12-21 09:24:36 UTC
It appears to happen especially often with C*FLAGS=".. -g", where the memory ceiling is hit more quickly.
Comment 9 James Le Cuirot gentoo-dev 2015-12-22 16:52:14 UTC
Okay, my investigation found that CACAO doesn't support specifying the heap size in gigabytes like HotSpot does. It also uses a signed 32-bit integer to deal with this, resulting in horrendous overflows that aren't immediately apparent. If you ask for 4097MB, it actually allocates 1MB! I have created patches for both these issues, which I will submit upstream, but I'd like to try them out locally a bit more before I include them in a new stable release here.

jer, your issues with building icedtea are more to do with icedtea's build process than icedtea-bin. Keep in mind that icedtea:7 isn't even keyworded for ppc64 and we never mark it stable anyway. I know you also filed bug #569152 but the fact is that CACAO isn't guaranteed to work with everything.

With all that in mind, I'd like to now proceed with getting this stable, especially as I'm keen to press on with the Java 6 removal.
Comment 10 James Le Cuirot gentoo-dev 2016-01-23 22:50:58 UTC
icedtea-bin-7.2.6.3 is vulnerable so this request is now for icedtea-bin-7.2.6.4.
Comment 11 James Le Cuirot gentoo-dev 2016-02-18 11:21:15 UTC
It seems that I wasn't able to convince you to stabilize Java 7 on ppc64 and that's perfectly understandable, CACAO evidently wasn't working as well as any of us wanted it to.

I've therefore changed tack and focused my efforts on Java 8 instead. icedtea-bin-3.0.0_pre08 has been available to ~ppc64 for 3 weeks now. As soon as it was added, I built almost every Java package keyworded for ppc64 or ~ppc64, over 200 of them, to really put it through its paces. Only a small handful failed and some of those were simply broken regardless of Java 8. I've already fixed all but one of these problems. The remaining one (qdox) cannot be fully resolved until after this stabilisation happens.

Please don't be put off by the fact that this is a prerelease. Remember that icedtea is primarily a build process and that the OpenJDK being built is more or less the same OpenJDK that Oracle has based their releases on for ages. Since this is icedtea-bin, the build process isn't even a factor here so this should not be a cause for concern.

As far as runtime behaviour goes, it is certainly not plagued by any of the issues we saw with CACAO or the race condition I saw with Java 7's HotSpot. As stated earlier, I am very keen to close this out because it will allow us to finally remove Java 6.

If you're wondering about the version number, upstream were keen to correct our divergence from their scheme. icedtea 3 is for Java 8 and we have hinted at that in the SLOT, which is still 8. Given that almost all packages depend on virtual/jdk rather than icedtea directly, we don't see this as a problem.

So ppc64 team, please stabilize:
=app-eselect/eselect-java-0.2.0
=dev-java/icedtea-bin-3.0.0_pre08
=dev-java/icedtea-sound-1.0.1
=dev-java/icedtea-web-1.6.1-r1
=dev-util/patchelf-0.8
=media-fonts/lklug-0.6
Comment 12 Andrew John Hughes 2016-02-19 02:46:18 UTC
It might be a good idea to wait for pre09 which will have the latest security updates, matching the current proprietary Oracle JDK in Gentoo (u72). This is nearly done. I'm just doing some comparisons to weed out unnecessary differences between the IcedTea trees and the upstream ones.
Comment 13 James Le Cuirot gentoo-dev 2016-02-25 13:21:47 UTC
(In reply to Andrew John Hughes from comment #12)
> It might be a good idea to wait for pre09 which will have the latest
> security updates, matching the current proprietary Oracle JDK in Gentoo
> (u72).

This has now been done so ppc64 team, please proceed:

=app-eselect/eselect-java-0.2.0
=dev-java/icedtea-bin-3.0.0_pre09
=dev-java/icedtea-sound-1.0.1
=dev-java/icedtea-web-1.6.1-r1
=dev-util/patchelf-0.8
=media-fonts/lklug-0.6
Comment 14 James Le Cuirot gentoo-dev 2016-03-02 11:28:04 UTC
There's been a revision bump to icedtea-bin-3.0.0_pre09-r1 because I forgot to remove the dependency on libgif.so.4, which is a problem for unstable users on giflib-5.

I have added the other architectures now but I do not wish to stabilize these yet. ppc64 team, please proceed.
Comment 15 James Le Cuirot gentoo-dev 2016-03-14 22:12:56 UTC
For the record, ago says we need these as well, which is fine.

=virtual/jdk-1.8.0-r2
=virtual/jre-1.8.0-r1
Comment 16 James Le Cuirot gentoo-dev 2016-03-15 07:10:24 UTC
Thanks for sorting this, ago!