dev-python/tox-1.8.1 and dev-python/tox-1.8.0 fail their test-suite with [...] tests/test_config.py ....................................................................................................FFFFFF............s tests/test_interpreters.py s........ tests/test_quickstart.py ............. tests/test_result.py ... tests/test_venv.py ..................s................... tests/test_z_cmdline.py ........FFFFFFFFFFFFFFFFFFFF..F ======================================== short test summary info ======================================== SKIP [1] tests/test_interpreters.py:10: condition: sys.platform != 'win32' SKIP [1] /var/tmp/portage/dev-python/tox-1.8.1/work/tox-1.8.1/tests/test_venv.py:318: needs python3.3 SKIP [1] tests/test_config.py:1445: condition: sys.platform != 'win32' FAIL tests/test_config.py::TestCmdInvocation::()::test_help FAIL tests/test_config.py::TestCmdInvocation::()::test_version FAIL tests/test_config.py::TestCmdInvocation::()::test_listenvs FAIL tests/test_config.py::TestCmdInvocation::()::test_config_specific_ini FAIL tests/test_config.py::TestCmdInvocation::()::test_no_tox_ini FAIL tests/test_config.py::TestCmdInvocation::()::test_showconfig_with_force_dep_version FAIL tests/test_z_cmdline.py::test_minversion FAIL tests/test_z_cmdline.py::test_run_custom_install_command_error FAIL tests/test_z_cmdline.py::test_unknown_interpreter_and_env FAIL tests/test_z_cmdline.py::test_unknown_interpreter FAIL tests/test_z_cmdline.py::test_skip_unknown_interpreter FAIL tests/test_z_cmdline.py::test_unknown_dep FAIL tests/test_z_cmdline.py::test_unknown_environment FAIL tests/test_z_cmdline.py::test_skip_sdist FAIL tests/test_z_cmdline.py::test_minimal_setup_py_empty FAIL tests/test_z_cmdline.py::test_minimal_setup_py_comment_only FAIL tests/test_z_cmdline.py::test_minimal_setup_py_non_functional FAIL tests/test_z_cmdline.py::test_sdist_fails FAIL tests/test_z_cmdline.py::TestToxRun::()::test_toxuone_env FAIL tests/test_z_cmdline.py::TestToxRun::()::test_json FAIL tests/test_z_cmdline.py::test_test_piphelp FAIL tests/test_z_cmdline.py::test_notest FAIL tests/test_z_cmdline.py::test_PYC FAIL tests/test_z_cmdline.py::test_sdistonly FAIL tests/test_z_cmdline.py::test_separate_sdist_no_sdistfile FAIL tests/test_z_cmdline.py::test_separate_sdist FAIL tests/test_z_cmdline.py::test_envsitepackagesdir [...] ================================================== FAILURES ================================================== __________________________________________________ TestCmdInvocation.test_help __________________________________________________ self = <test_config.TestCmdInvocation instance at 0x3a27e083200>, cmd = <tox._pytestplugin.Cmd instance at 0x3a27e0832d8> def test_help(self, cmd): > result = cmd.run("tox", "-h") ../../../../../work/tox-1.8.1/tests/test_config.py:1293: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <tox._pytestplugin.Cmd instance at 0x3a27e0832d8>, argv = ['tox', '-h'], x = '-h' def run(self, *argv): argv = [str(x) for x in argv] > assert py.path.local.sysfind(str(argv[0])), argv[0] E AssertionError: tox ../../../../../work/tox-1.8.1-python2_7/lib/tox/_pytestplugin.py:183: AssertionError [...] and many more with the same assertion error. The reason for the failure is that the executable "tox" (that's the value of argv[0]) cannot be found by py.path.local.sysfind since "tox" it is in the process of being installed. Note that for this to happen no old installation of tox must be present on the system, otherwise the old executable will be found and tested (which is kind of pointless for the test-suite). A way to reproduce the issue with a working tox installed is to create a fake tox executable some where and appending the path to the fake tox at the start of PATH, so the fake tox gets found before the real tox. The same failure happens when running the tests directly from the extracted tar-ball with PYTHONPATH=. py.test tests . When using the setup.py script of the tox source-tarball, then the executable gets build (and installed in a temporary location) first and afterwards the tests are run.
and what about 2.4.1?
Still the same failure for 2.4.1: All tests pass except the tests which call the executable 'tox' (which is then not found).
Created attachment 464200 [details, diff] Skip test which need the 'tox' executable installed
Created attachment 464202 [details, diff] Patch for the tox ebuild to apply above patch
Above a patch which skips the tests which need the 'tox' executable installed. Note: That there is some collateral damage since the patch skips all tests in tests/test_z_cmdline.py. However, there are a few (~5) which do not depend on the 'tox' executable. However, it would be to cumbersome to mark every other test to skip except those few.
I played around with 'distutils_install_for_testing', sadly the tests need the executable and still cannot find it. The simple trick with python_test() { distutils_install_for_testing PATH="${TEST_DIR}/scripts:${PATH}" py.test -v -v tests || die "Testsuite failed under ${EPYTHON}" } Does not work. The test suite now finds the fitting executable 'tox'. But executing this file yields the error pkg_resources.DistributionNotFound: The 'tox==2.4.1' distribution was not found and is required by the application So it seems that the executable cannot find its correct python environment. Feel free to investigate further, since I have no clue how to continue from here and how Python packages and in particular Python packages in portage/ebuilds work.
Still an issue with dev-python/tox-2.4.1
*** Bug 623294 has been marked as a duplicate of this bug. ***
Since the filing of this bug, time passed and tests were RESTRICTed in dev-python/tox. In the bump I've just made to v3.2.1, I re-enabled tests and made sure that they passed on a network-sandboxed machine that didn't have tox installed. There's 8 tests I needed to exclude, although I'm not sure that the root cause it the same for all of them.
Hi Virgil Thanks for your efforts!
