Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 208351 - dev-java/lucene-2.3.0 out of memory during src_test
Summary: dev-java/lucene-2.3.0 out of memory during src_test
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Java (show other bugs)
Hardware: AMD64 Linux
: High normal (vote)
Assignee: Java team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-01-31 21:18 UTC by Michael Baer
Modified: 2008-11-01 10:13 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Baer 2008-01-31 21:18:34 UTC
emerge of dev-java/lucene-2.3.0 fails to test. IMHO this is a emerge only problem. If I'm running "ant common.compile-test" within WORKDIR, everything works as expected.

My system is running junit 3.8.2-r1, sun-jdk 1.6.0_04 and javacc 4.0-r4.

java-check-environment finishes with Java environment is sane.

USE="-test" doesn't has any effect.

So, I'm running out of ideas, what I could do to fix this error.

Reproducible: Always

Steps to Reproduce:
1. emerge =dev-java/lucene-2.3.0

Actual Results:  
    [javac] /var/tmp/portage/dev-java/lucene-2.3.0/work/lucene-2.3.0/contrib/analyzers/src/test/org/apache/lucene/analysis/ngram/NGramTokenizerTest.java:93: cannot find symbol
    [javac] symbol  : method assertEquals(int,int)
    [javac] location: class org.apache.lucene.analysis.ngram.NGramTokenizerTest
    [javac]         assertEquals(4, tokens.size());
    [javac]         ^
    [javac] /var/tmp/portage/dev-java/lucene-2.3.0/work/lucene-2.3.0/contrib/analyzers/src/test/org/apache/lucene/analysis/ngram/NGramTokenizerTest.java:96: cannot find symbol
    [javac] symbol  : method assertEquals(java.util.ArrayList,java.util.ArrayList)
    [javac] location: class org.apache.lucene.analysis.ngram.NGramTokenizerTest
    [javac]         assertEquals(exp, tokens);
    [javac]         ^
    [javac] /var/tmp/portage/dev-java/lucene-2.3.0/work/lucene-2.3.0/contrib/analyzers/src/test/org/apache/lucene/analysis/ngram/NGramTokenizerTest.java:113: cannot find symbol
    [javac] symbol  : method assertEquals(int,int)
    [javac] location: class org.apache.lucene.analysis.ngram.NGramTokenizerTest
    [javac]         assertEquals(12, tokens.size());
    [javac]         ^
    [javac] 100 errors

BUILD FAILED
/var/tmp/portage/dev-java/lucene-2.3.0/work/lucene-2.3.0/build.xml:467: The following error occurred while executing this line:
/var/tmp/portage/dev-java/lucene-2.3.0/work/lucene-2.3.0/build.xml:460: The following error occurred while executing this line:
/var/tmp/portage/dev-java/lucene-2.3.0/work/lucene-2.3.0/contrib/contrib-build.xml:65: The following error occurred while executing this line:
/var/tmp/portage/dev-java/lucene-2.3.0/work/lucene-2.3.0/common-build.xml:270: The following error occurred while executing this line:
/var/tmp/portage/dev-java/lucene-2.3.0/work/lucene-2.3.0/common-build.xml:399: Compile failed; see the compiler error output for details.

Total time: 8 seconds
 * 
 * ERROR: dev-java/lucene-2.3.0 failed.
 * Call stack:
 *               ebuild.sh, line   46:  Called src_compile
 *             environment, line 3908:  Called eant 'src_compile' 'src_compile' '-Dversion=2.3.0' 'jar-core'
 *             environment, line  879:  Called die
 * The specific snippet of code:
 *       ant ${antflags} "${@}" || diefunc "$FUNCNAME" "$LINENO" "$?" "eant failed"
 *  The die message:
 *   eant failed
 * 
 * If you need support, post the topmost build error, and the call stack if relevant.
 * A complete build log is located at '/var/tmp/portage/dev-java/lucene-2.3.0/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/dev-java/lucene-2.3.0/temp/environment'.
 * 
!!! When you file a bug report, please include the following information:
GENTOO_VM=sun-jdk-1.6  CLASSPATH="" JAVA_HOME="/opt/sun-jdk-1.6.0.04"
JAVACFLAGS="-source 1.4 -target 1.4" COMPILER="javac"
and of course, the output of emerge --info

Expected Results:  
lucene will be emerged successfully


Portage 2.1.4 (default-linux/amd64/2007.0, gcc-4.2.2, glibc-2.7-r1, 2.6.24-gentoo x86_64)
=================================================================
System uname: 2.6.24-gentoo x86_64 AMD Athlon(tm) 64 X2 Dual Core Processor 6000+
Timestamp of tree: Thu, 31 Jan 2008 13:16:01 +0000
distcc 2.18.3 x86_64-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
ccache version 2.4 [enabled]
app-shells/bash:     3.2_p33
dev-java/java-config: 1.3.7, 2.1.4
dev-lang/python:     2.4.4-r6, 2.5.1-r5
dev-python/pycrypto: 2.0.1-r6
dev-util/ccache:     2.4-r7
sys-apps/baselayout: 1.12.11.1
sys-apps/sandbox:    1.2.18.1-r2
sys-devel/autoconf:  2.13, 2.61-r1
sys-devel/automake:  1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.1
sys-devel/binutils:  2.18-r1
sys-devel/gcc-config: 1.4.0-r4
sys-devel/libtool:   1.5.24
virtual/os-headers:  2.6.24
ACCEPT_KEYWORDS="amd64 ~amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=k8"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-O2 -pipe -march=k8"
DISTDIR="/usr/portage/distfiles"
FEATURES="ccache collision-protect distlocks metadata-transfer parallel-fetch sandbox sfperms strict unmerge-orphans userfetch"
GENTOO_MIRRORS="ftp://ftp.tu-clausthal.de/pub/linux/gentoo/ ftp://sunsite.informatik.rwth-aachen.de/pub/Linux/gentoo http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/ ftp://linux.rz.ruhr-uni-bochum.de/gentoo-mirror/ http://ftp.uni-erlangen.de/pub/mirrors/gentoo ftp://ftp.uni-erlangen.de/pub/mirrors/gentoo ftp://ftp.join.uni-muenster.de/pub/linux/distributions/gentoo ftp://ftp.wh2.tu-dresden.de/pub/mirrors/gentoo ftp://ftp.join.uni-muenster.de/pub/linux/distributions/gentoo http://mirrors.sec.informatik.tu-darmstadt.de/gentoo/ http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/ ftp://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/ "
LANG="de_DE@euro"
LINGUAS="de"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --filter=H_**/files/digest-*"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/portage/local/layman/java-overlay /usr/portage/local/layman/php-testing /usr/portage/local/layman/sunrise /usr/portage/local/my-ebuilds"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="3dnow 3dnowext X a52 aac aalib acl acpi alsa amd64 apache2 apic arts asf avi bash-completion bcmath berkdb bitmap-fonts bzip2 calendar caps cgi cli cracklib crypt css cups curl dba dbus devmap dga doc dri drm dvd dvdr dvdread emovix encode examples ffmpeg flac flatfile fortran freetype gd gdbm gif glitz gnome gpm graphviz gs gstreamer gtk gtk2 guile hal iconv icu imagemagick imap ipv6 isdnlog java java5 jpeg jpeg2k kde kerberos latex ldap libnotify lm_sensors mad maildir mhash midi mmx motif mozdevelop mp3 mp4 mpeg mplayer msn mudflap musepack musicbrainz mysql ncurses nfs nls nptl nptlonly ntfs objc offensive ogg opengl openmp oss pam pcre pdf pear perl png postscript pppd python qt qt3 qt4 readline reflection samba scanner sdl session slang smp sndfile soap sockets socks5 spl sse sse2 ssl svg tcl tcltk tcpd themes theora threads tiff tk truetype truetype-fonts type1-fonts unicode usb v4l2 vcd vhosts vorbis wmf x264 xcomposite xfs xine xinerama xml xml2 xorg xplanet xrandr xsettings xsl xv xvid xvmc zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="de" USERLAND="GNU" VIDEO_CARDS="fglrx ati vesa radeon"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LDFLAGS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 1 Vlastimil Babka (Caster) (RETIRED) gentoo-dev 2008-02-01 10:21:40 UTC
(In reply to comment #0)
> emerge of dev-java/lucene-2.3.0 fails to test. IMHO this is a emerge only
> problem. If I'm running "ant common.compile-test" within WORKDIR, everything
> works as expected.

Yeah, running ant manually puts its own classpath including junit on the classpath for compilation, while in emerge we tell it not to, for good reasons. We set the classpath via -Dgentoo.classpath and there's "java-ant_rewrite-classpath common-build.xml" command which rewrites the build.xml to respect this setting. And I guess this is what fails for you for some reason. It works for me.

> My system is running junit 3.8.2-r1, sun-jdk 1.6.0_04 and javacc 4.0-r4.

What about javatoolkit version.

> USE="-test" doesn't has any effect.

Yeah, it should be FEATURES="-test", the useflag should not be changed alone. BTW your emerge --info doesn't say you have FEATURES=test set?
 
> So, I'm running out of ideas, what I could do to fix this error.

Please try running emerge/ebuild with JAVA_PKG_DEBUG=1 exported, and attach the build log (its path is in the error message).

Meanwhile I can figure why the hugeram test fails for me (that's different problem than yours :)
Comment 2 Vlastimil Babka (Caster) (RETIRED) gentoo-dev 2008-02-01 10:32:47 UTC
    [junit] Testcase: testHugeFile(org.apache.lucene.store.TestHugeRamFile):    Caused an ERROR
    [junit] Java heap space
    [junit] java.lang.OutOfMemoryError: Java heap space
    [junit]     at java.util.Arrays.copyOf(Arrays.java:2772)
    [junit]     at java.util.Arrays.copyOf(Arrays.java:2746)
    [junit]     at java.util.ArrayList.ensureCapacity(ArrayList.java:187)
    [junit]     at java.util.ArrayList.add(ArrayList.java:378)
    [junit]     at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:69)
    [junit]     at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:128)
    [junit]     at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:114)
    [junit]     at org.apache.lucene.store.TestHugeRamFile.testHugeFile(TestHugeRamFile.java:68)

On amd64 with sun-jdk-1.7. Will try to check for some balanced heap limit setting parameter later.
Comment 3 Michael Baer 2008-02-02 18:44:17 UTC
(In reply to comment #1)
> (In reply to comment #0)
> > emerge of dev-java/lucene-2.3.0 fails to test. IMHO this is a emerge only
> > problem. If I'm running "ant common.compile-test" within WORKDIR, everything
> > works as expected.
> 
> Yeah, running ant manually puts its own classpath including junit on the
> classpath for compilation, while in emerge we tell it not to, for good reasons.
> We set the classpath via -Dgentoo.classpath and there's
> "java-ant_rewrite-classpath common-build.xml" command which rewrites the
> build.xml to respect this setting. And I guess this is what fails for you for
> some reason. It works for me.

Ah, I see. I assumed something like that.

> 
> > My system is running junit 3.8.2-r1, sun-jdk 1.6.0_04 and javacc 4.0-r4.
> 
> What about javatoolkit version.

0.2.0-r4 from java-overlay. But 0.2.0-r1 also has the same problem.

> 
> > USE="-test" doesn't has any effect.
> 
> Yeah, it should be FEATURES="-test", the useflag should not be changed alone.
> BTW your emerge --info doesn't say you have FEATURES=test set?

That's right. I don't want emerge to test the programms. So neither FEATURE nor USE contains "test". That's why I'm astonished, that lucene is testing the compiled classes.

> 
> > So, I'm running out of ideas, what I could do to fix this error.
> 
> Please try running emerge/ebuild with JAVA_PKG_DEBUG=1 exported, and attach the
> build log (its path is in the error message).

Done. 

> 
> Meanwhile I can figure why the hugeram test fails for me (that's different
> problem than yours :)
> 

Good luck...and much fun ;-)
Comment 4 Michael Baer 2008-02-02 18:51:33 UTC
f...k

I can't upload the build.log as attachment.

You can find it here:
http://www.beekay.de/build.log
Comment 5 Vlastimil Babka (Caster) (RETIRED) gentoo-dev 2008-02-02 19:17:26 UTC
(In reply to comment #3)
> That's right. I don't want emerge to test the programms. So neither FEATURE nor
> USE contains "test". That's why I'm astonished, that lucene is testing the
> compiled classes.

Ah I see now, it was running the tests in src_compile, because the target somehow was dependency of javadocs target. Fixed in CVS by running only javadocs-core and javadocs-demo targets. Thanks for the report and help. I'll leave this open for the out of memory bug.
Comment 6 Michael Baer 2008-02-02 22:20:52 UTC
Now it works.

Thanks
Comment 7 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2008-11-01 07:18:52 UTC
Is there anything you still need me on this bug for?
I know that Lucene itself can eat a lot of memory if you have an index with many documents.
Comment 8 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2008-11-01 10:13:59 UTC
Ok, i'm pretty sure this is solved by my other src_test cleanups for Lucene now anyway.