| Summary: | dev-python/testresources-0.2.7-r2 fails tests | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Patrick Lauer <patrick> |
| Component: | [OLD] Unspecified | Assignee: | Python Gentoo Team <python> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | erikdenstore+gbugs, paolo.pedroni |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
|
Description
Patrick Lauer
2015-01-24 11:09:37 UTC
not here ~/cvsPortage/gentoo-x86/dev-python/testresources $ ebuild testresources-0.2.7-r2.ebuild clean test >>> Source compiled. >>> Test phase: dev-python/testresources-0.2.7-r2 * python3_3: running distutils-r1_run_phase python_test .................................................................................................... ---------------------------------------------------------------------- Ran 100 tests in 0.272s OK * python3_4: running distutils-r1_run_phase python_test .................................................................................................... ---------------------------------------------------------------------- Ran 100 tests in 0.326s OK * python2_7: running distutils-r1_run_phase python_test .................................................................................................... ---------------------------------------------------------------------- Ran 100 tests in 0.228s OK >>> Completed testing dev-python/testresources-0.2.7-r2 I ran into this, the failure happens *sometimes*. Thus far, I've seen two failures from 5 tries. Both failures were with python3_4. Intermittent failures like this are sometimes cause by leftover files from a previous run. Do a clean or even a logout login and re-run and it passes. Whatever, intermittent still means it could be anything, however passing at all proves the test is passable and not flawed. The intermittency is the problem I've seen it today, several times. Sometimes it fails with pypy3, sometimes with python3.4, sometimes with python2.7, but it's always the same test. It completes successfully maybe one time out of five. IMHO, tests that are so flaky should be restricted. (In reply to Paolo Pedroni from comment #4) > I've seen it today, several times. Sometimes it fails with pypy3, sometimes > with python3.4, sometimes with python2.7, but it's always the same test. It > completes successfully maybe one time out of five. > > IMHO, tests that are so flaky should be restricted. Which failures do you see? The dict/list comparesion ones? (In reply to Justin Lecher from comment #5) > Which failures do you see? The dict/list comparesion ones? The same reported by Patrick Lauer in the original bug report: * python2_7: running distutils-r1_run_phase python_test ..........F................................................................................................ ====================================================================== FAIL: testresources.tests.test_optimising_test_suite.TestGraphStuff.testBasicSortTests ---------------------------------------------------------------------- _StringException: Traceback (most recent call last): File "/var/tmp/portage/dev-python/testresources-0.2.7-r2/work/testresources-0.2.7/lib/testresources/tests/test_optimising_test_suite.py", line 513, in testBasicSortTests [self.case3, self.case2, self.case1, self.case4]]) File "/usr/lib64/python2.7/site-packages/testtools/testcase.py", line 356, in assertIn self.assertThat(haystack, Contains(needle), message) File "/usr/lib64/python2.7/site-packages/testtools/testcase.py", line 435, in assertThat raise mismatch_error testtools.matchers._impl.MismatchError: [test_three, test_one, test_two, test_four] not in [[test_one, test_two, test_three, test_four], [test_three, test_two, test_one, test_four]] ---------------------------------------------------------------------- Ran 107 tests in 0.403s I've seen it happen roughly 9 times out of 10 with "ebuild /usr/portage/dev-python/testresources/testresources-0.2.7-r2.ebuild clean test": sometimes with python2.7, sometimes with pypy3, sometimes with python3.4, oddly never with pypy. # emerge -1pqv =dev-python/testresources-0.2.7-r2 [ebuild N ] dev-python/testresources-0.2.7-r2 USE="{test}" PYTHON_TARGETS="pypy pypy3 python2_7 python3_4 -python3_3 (-python3_5)" commit fd56767c19be5d180511e9331929c9bc6a9b558c Author: Justin Lecher <jlec@gentoo.org> Date: Tue Nov 3 11:38:01 2015 +0100 dev-python/testresources: Skip flanky tests Gentoo-Bug: https://bugs.gentoo.org/show_bug.cgi?id=537534 Package-Manager: portage-2.2.23 Signed-off-by: Justin Lecher <jlec@gentoo.org> https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=fd56767c19be5d180511e9331929c9bc6a9b558c |