| Summary: | dev-cpp/doctest: tests fail intermittently on alpha, hppa (timeout-related) | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | matoro <matoro_gentoo> |
| Component: | Current packages | Assignee: | Petr Vaněk <arkamar> |
| Status: | CONFIRMED --- | ||
| Severity: | normal | CC: | alpha, hppa, matoro_gentoo, mgorny, proxy-maint |
| Priority: | Normal | Keywords: | PullRequest |
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| URL: | https://github.com/doctest/doctest/issues/711 | ||
| See Also: | https://github.com/gentoo/gentoo/pull/30607 | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | |||
| Bug Blocks: | 837098, 873814 | ||
| Attachments: |
LastTest.log
build.log |
||
Created attachment 816124 [details]
LastTest.log
Created attachment 816127 [details]
build.log
It also requires dev-vcs/git in BDEPEND, but that's a moot point until we can get the tests passing. This is a true horror — and the output is completely unreadable. (In reply to matoro from comment #3) > It also requires dev-vcs/git in BDEPEND, but that's a moot point until we > can get the tests passing. Actually git is optional, it is used only if it is found only for showing difference in test results, see [1]. I decided to not put it to the BDEPEND list because it is useful only for debugging purposes. [1] https://github.com/doctest/doctest/blob/86892fc480f80fb57d9a3926cb506c0e974489d8/scripts/cmake/exec_test.cmake#L42-L47 Could you try ignoring the problem and seeing if revdeps work? I suppose they may not be affected by this. (In reply to Michał Górny from comment #6) > Could you try ignoring the problem and seeing if revdeps work? I suppose > they may not be affected by this. They almost certainly will since this pretty much only affects the timeout decorator feature, but signing off on a keyword when tests are explicitly not passing kind of defeats the point of the tests. This is on my todo list to go back and re-investigate, and of course if anyone else would like to take a crack at it, just let me know for hardware access. This blocks two of the three remaining items left to transition alpha from exp -> dev. This also affects hppa apparently. https://github.com/doctest/doctest/pull/770 This fixes HPPA, and partially fixes alpha (remaining items are segfaults, not test failures). The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=b88bedd14cd548acfdbabb96a4300522db9f2109 commit b88bedd14cd548acfdbabb96a4300522db9f2109 Author: Matoro Mahri <matoro@users.noreply.github.com> AuthorDate: 2023-04-15 19:33:59 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-04-26 04:02:59 +0000 dev-cpp/doctest: add patch for tests with clock resolution problems See: https://github.com/doctest/doctest/pull/770 See: https://github.com/doctest/doctest/issues/711 Bug: https://bugs.gentoo.org/874345 Signed-off-by: Matoro Mahri <matoro@users.noreply.github.com> Closes: https://github.com/gentoo/gentoo/pull/30607 Signed-off-by: Sam James <sam@gentoo.org> dev-cpp/doctest/doctest-2.4.11.ebuild | 1 + .../files/doctest-2.4.11-backport-pr770.patch | 72 ++++++++++++++++++++++ 2 files changed, 73 insertions(+) |
On alpha, tests for dev-cpp/doctest fail with timeout-related errors, but the exact number of failures varies intermittently, usually between 5 and 10. Possibly related to clock resolution? Here's an example of one of the failing cases: 35/155 Testing: test_cases_and_suites.cpp_xml 35/155 Test: test_cases_and_suites.cpp_xml Command: "/usr/bin/cmake" "-DCOMMAND=/var/tmp/portage/dev-cpp/doctest-2.4.9/work/doctest-2.4.9_build/examples/all_featur es/all_features --no-skipped-summary --no-version -sf=*test_cases_and_suites.cpp --reporters=xml --dt-no-path-filenames= 1 --dt-no-line-numbers=1 --dt-no-exitcode=1 --dt-gnu-file-line=0 --dt-no-time-in-output=1" "-DTEST_OUTPUT_FILE=/var/tmp/ portage/dev-cpp/doctest-2.4.9/work/doctest-2.4.9/examples/all_features/test_output/test_cases_and_suites.cpp_xml.txt" "- DTEST_TEMP_FILE=/var/tmp/portage/dev-cpp/doctest-2.4.9/work/doctest-2.4.9_build/examples/all_features/CMakeFiles/temp_te st_output_test_cases_and_suites.cpp_xml.txt" "-DTEST_MODE=COMPARE" "-P" "/var/tmp/portage/dev-cpp/doctest-2.4.9/work/doc test-2.4.9/scripts/cmake/exec_test.cmake" Directory: /var/tmp/portage/dev-cpp/doctest-2.4.9/work/doctest-2.4.9_build/examples/all_features "test_cases_and_suites.cpp_xml" start time: Oct 03 04:09 UTC Output: ---------------------------------------------------------- -- Found Git: /usr/bin/git (found version "2.37.3") diff --git a/var/tmp/portage/dev-cpp/doctest-2.4.9/work/doctest-2.4.9/examples/all_features/test_output/test_cases_and_s uites.cpp_xml.txt b/var/tmp/portage/dev-cpp/doctest-2.4.9/work/doctest-2.4.9_build/examples/all_features/CMakeFiles/temp _test_output_test_cases_and_suites.cpp_xml.txt index 6fc245a..e9d1e79 100644 --- a/var/tmp/portage/dev-cpp/doctest-2.4.9/work/doctest-2.4.9/examples/all_features/test_output/test_cases_and_suites.c pp_xml.txt +++ b/var/tmp/portage/dev-cpp/doctest-2.4.9/work/doctest-2.4.9_build/examples/all_features/CMakeFiles/temp_test_output_t est_cases_and_suites.cpp_xml.txt @@ -54,7 +54,7 @@ failing because of the timeout decorator! </Text> </Message> - <OverallResultsAsserts successes="0" failures="0" test_case_success="false"/> + <OverallResultsAsserts successes="0" failures="0" test_case_success="true"/> </TestCase> </TestSuite> <TestSuite name="skipped test cases"> @@ -109,6 +109,6 @@ </TestCase> </TestSuite> <OverallResultsAsserts successes="1" failures="11"/> - <OverallResultsTestCases successes="6" failures="9"/> + <OverallResultsTestCases successes="7" failures="8"/> </doctest> Program code. CMake Error at /var/tmp/portage/dev-cpp/doctest-2.4.9/work/doctest-2.4.9/scripts/cmake/exec_test.cmake:69 (message): Running '/var/tmp/portage/dev-cpp/doctest-2.4.9/work/doctest-2.4.9_build/examples/all_features/all_features --no-skipped-summary --no-version -sf=*test_cases_and_suites.cpp --reporters=xml --dt-no-path-filenames=1 --dt-no-line-numbers=1 --dt-no-exitcode=1 --dt-gnu-file-line=0 --dt-no-time-in-output=1' ended with code 'Output is different from reference file!' <end of output> Test time = 0.22 sec ---------------------------------------------------------- Test Failed. "test_cases_and_suites.cpp_xml" end time: Oct 03 04:09 UTC "test_cases_and_suites.cpp_xml" time elapsed: 00:00:00 ---------------------------------------------------------- Reproducible: Always