Bug 801694 – java-utils-2.eclass: JAVA_TESTING_FRAMEWORKS="testng" does not print test failures

• Product: Gentoo Linux
• Status: RESOLVED FIXED

[Description Volkmar W. Pogatzki 2021-07-12 08:23:56 UTC]

testng used as JAVA_TESTING_FRAMEWORKS, does not print test failures to the build log.  According to upstream it should produce separate output index.html[1] or testng-failed.xml[2].  None of them was produced for dev-java/dom4j-2.1.3 while test failures were unsolved.[4]

Only by switching "-usedefaultlisteners" from false to true in java-utils-2.eclass it would produce an test.html showing the test failures.
However "-usedefaultlisteners true" leads to other errors and seems not useful as a default.

Another "-listener"[3] ( Lets you specify your own test listeners. The classes need to implement org.testng.ITestListener ) set in the eclasses might be very helpful for package maintainers  and testers.

[1]https://testng.org/doc/documentation-main.html#logging
[2]https://testng.org/doc/documentation-main.html#rerunning
[3]https://testng.org/doc/documentation-main.html#running-testng
[4]https://github.com/gentoo/gentoo/pull/21319#issuecomment-877825113

[Comment 1 Florian Schmaus 2022-04-19 07:28:48 UTC]

Created attachment 771719 [details, diff]
java-util-2-runner-args.patch

Attached fordfrog's patch for java-util-2.eclass from https://github.com/gentoo/gentoo/pull/21319#issuecomment-878128343.

[Comment 2 Larry the Git Cow 2022-04-27 08:45:50 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ebf1f3cb5fa8551465f263780719d90400cb5daf

commit ebf1f3cb5fa8551465f263780719d90400cb5daf
Author:     Florian Schmaus <[e-mail]>
AuthorDate: 2022-04-20 08:28:51 +0000
Commit:     Florian Schmaus <[e-mail]>
CommitDate: 2022-04-27 08:44:39 +0000

    java-utils-2.eclass: introduce JAVA_TEST_RUNNER_EXTRA_ARGS
    
    Also add special handling wrt -usedfaultlisteners for TestNG, see
    bug #801694.
    
    Co-authored-by: Miroslav Šulc <[e-mail]>
    Bug: https://bugs.gentoo.org/801694
    Closes: https://github.com/gentoo/gentoo/pull/2512
    Signed-off-by: Florian Schmaus <[e-mail]>

 eclass/java-utils-2.eclass | 49 ++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 41 insertions(+), 8 deletions(-)

[Comment 3 Volkmar W. Pogatzki 2022-06-21 12:02:09 UTC]

Created attachment 786581 [details]
missing details of test failures in dev-java/cdi-api (upcoming new ebuild)

It still does not work as it should. It only prints:

[TestNG] Running:
  Command line suite


===============================================
Command line suite
Total tests run: 56, Failures: 7, Skips: 0
===============================================

 * ERROR: dev-java/cdi-api-4.0.1::gentoo failed (test phase):
 *   Running TestNG failed.

[Comment 4 Volkmar W. Pogatzki 2022-06-21 12:29:42 UTC]

Created attachment 786584 [details, diff]
patch for java-utils2.eclass fixing testng defaultlisteners

This change in the eclass will produce useful output without extra arguments in the ebuild.

[Comment 5 Volkmar W. Pogatzki 2022-06-22 11:11:32 UTC]

A testcase in cdi-api-4.0.1 from https://github.com/gentoo/gentoo/pull/26035.

[Comment 6 Larry the Git Cow 2022-07-04 13:20:31 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=a4440928b24f7a765fa7c9427261fa4ca722ea51

commit a4440928b24f7a765fa7c9427261fa4ca722ea51
Author:     Volkmar W. Pogatzki <[e-mail]>
AuthorDate: 2022-06-28 16:02:45 +0000
Commit:     Florian Schmaus <[e-mail]>
CommitDate: 2022-07-04 13:15:16 +0000

    eclass/java-utils-2.eclass: make testng output more verbose
    
    Bug: https://bugs.gentoo.org/801694
    Signed-off-by: Volkmar W. Pogatzki <[e-mail]>
    Closes: https://github.com/gentoo/gentoo/pull/26118
    Signed-off-by: Florian Schmaus <[e-mail]>

 eclass/java-utils-2.eclass | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

[Comment 7 Volkmar W. Pogatzki 2022-07-04 15:31:29 UTC]

Re-tested. Works. Closing.