Summary: | dev-python/nose-1.2.1_p2012 fails tests (failing to count to one instead of two) | ||
---|---|---|---|
Product: | Gentoo Linux | Reporter: | Markus Walter <gentoo> |
Component: | [OLD] Development | Assignee: | Python Gentoo Team <python> |
Status: | RESOLVED FIXED | ||
Severity: | normal | ||
Priority: | Normal | ||
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
URL: | https://github.com/nose-devs/nose/issues/636 | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Attachments: |
build log
emerge --info |
Created attachment 337658 [details]
emerge --info
Markus, by the code coverage is not high enough do you mean the coverage version is not high enough as in a recent version is > prior version as in the style of a literal int or float? I mean the following error message which occurs in the test phase: ===== nose.plugins.cover: ERROR: TOTAL Coverage did not reach minimum required: 100% ===== which repeats quite a lot and later on we have many of the following ===== nose.plugins.cover: ERROR: No total percentage was found in coverage output, something went wrong. ===== thus I think, that the nose test suite tries to parse the coverage output, which probably changed at some point. The coverage messages are irrelevant. I think that's nose testing if coverage reports errors correctly. Markus, can you re-run and describe what happens now with recent changes to the nose-1.2.1_p2012 ebuild and a snapshot in place? With all changes I still get these many dozens of lines of that error message. However that's non fatal and thus a triviality since they don't actually cause the testsuite to abort. However, I am getting ====================================================================== ERROR: test_error (test_twisted.TestTwisted) test_error ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py", line 138, in maybeDeferred result = f(*args, **kw) File "/usr/lib64/python2.7/site-packages/twisted/internet/_utilspy3.py", line 41, in runWithWarningsSuppressed reraise(exc_info[1], exc_info[2]) File "/usr/lib64/python2.7/site-packages/twisted/internet/_utilspy3.py", line 37, in runWithWarningsSuppressed result = f(*a, **kw) File "/mnt/gen2/TmpDir/portage/dev-python/nose-1.2.1_p2012/work/nose-1.2.1_p2012-python2_7/functional_tests/support/twist/test_twisted.py", line 12, in test_error raise TypeError("oops, wrong type") TypeError: oops, wrong type ====================================================================== FAIL: test_fail (test_twisted.TestTwisted) test_fail ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py", line 138, in maybeDeferred result = f(*args, **kw) File "/usr/lib64/python2.7/site-packages/twisted/internet/_utilspy3.py", line 41, in runWithWarningsSuppressed reraise(exc_info[1], exc_info[2]) File "/usr/lib64/python2.7/site-packages/twisted/internet/_utilspy3.py", line 37, in runWithWarningsSuppressed result = f(*a, **kw) File "/mnt/gen2/TmpDir/portage/dev-python/nose-1.2.1_p2012/work/nose-1.2.1_p2012-python2_7/functional_tests/support/twist/test_twisted.py", line 9, in test_fail self.fail("I failed") File "/usr/lib64/python2.7/site-packages/twisted/trial/_synctest.py", line 286, in fail raise self.failureException(msg) FailTest: I failed ---------------------------------------------------------------------- Ran 5 tests in 0.022s FAILED (failures=1, errors=1, skipped=1) from $ PYTHON_TARGETS=python2_7 ebuild nose-1.2.1_p2012.ebuild clean test Do you? I still see the coverage message, but as already mentioned they are mostly noise. I see the same error as you for python-2.7, whereas for python-3.2 the test-suite runs fine. However there is a bit more output before the lines you pasted, namely ====================================================================== FAIL: Collect and run tests in functional/support/twist ---------------------------------------------------------------------- Traceback (most recent call last): File "/var/tmp/portage/dev-python/nose-1.2.1_p2012/work/nose-1.2.1_p2012-python2_7/functional_tests/test_program.py", line 126, in test_run_support_twist assert len(res.failures) == 2 AssertionError: -------------------- >> begin captured stdout << --------------------- runTest (test_twisted.TestTwisted) If no C{methodName} argument is passed to the constructor, L{run} will ... ok test (test_twisted.TestTwisted) test ... ok test_error (test_twisted.TestTwisted) test_error ... ERROR test_fail (test_twisted.TestTwisted) test_fail ... FAIL test_skip (test_twisted.TestTwisted) test_skip ... skipped Exception('skip me',) ====================================================================== ERROR: test_error (test_twisted.TestTwisted) test_error ---------------------------------------------------------------------- which suggests, that there should be two failures, but what we get are one failure and one error (which is quite logical with tests named test_error and test_fail) ... I'm a bit confused, what's wrong here. <I'm a bit confused, what's wrong here. I'm not surprised!! I have figured it, and thanks for confirming the error state. It goes like this. The nose testsuite writers in their 'wisdom' have set functional_tests/test_program.py in around line 126 to call the tests in functional_tests/support/twist/test_twisted.py. THESE ARE COMPULSORY FAILS!!!!! Yes you read right. They not only wrote test_error ...to exit with ERROR and test_fail to exit with FAIL, they then wrote lines of code in test_run_support_twist in test_program.py around line 126 to REPORT on the SUCCESS of the error and fail(!!!!!), thereby celebrating failure by outputting it to console, and they didn't even do that right!!!!!!!! assert len(res.failures) == 2 should be 1 not 2. The tests of test_twisted.py import twisted.trial.unittest, namely using twisted's own tester 'trial' on a test that is testing twisted specifically. It seems the tests are 'bullet proof', i.e. they cannot be made to expected Failure. Believe me I tried. They raise fail or error which is then invoked by a system installed file which you therefore can't 'attack'. It passes, or doesn't fail, with py3 because twisted doesn't support py3 and therefore with python3_? these tests are harmlessly skipped. So here we have a testsuite that by these 'tests' basically turns the norm of returning a passed test on its head, turning failure / error into success. I am utterly gobsmacked. Fortunately it can be testskipped in test_program.py Markus, the title of this Bug refers to the coverage noise causing test failure. I leave it to you, if you can, to change it at yr discretion. 02 Mar 2013; Ian Delaney <idella4@gentoo.org> +files/nose-1.2.1-skiptest.patch, nose-1.1.2-r1.ebuild, nose-1.2.1.ebuild, nose-1.2.1_p2012.ebuild: Added skiptest.patch for 1.2.1 versions skipping tests set to fail, wrt Bug #455104 hmm nice touch. see why I left it to you. Against ALL expectations, upstream ack'd the bug and made a fix. I am utterly gobsmacked. Re-opening as confirmed, and I suggest mark this for closure on release of 1.3.0. *nose-1.3.0 (16 Apr 2013) 16 Apr 2013; Patrick Lauer <patrick@gentoo.org> +nose-1.3.0.ebuild: Bump |
Created attachment 337656 [details] build log On my hardened ~amd64 machine nose fails it's test. It fails, because the code coverage is not high enough and complains, that it was unable to parse the coverage output. For reference: I have coverage-3.6 installed.