Hi, Not so long ago I merged dev-python/pytest-relaxed ((indirect)dependency of dev-vcs/bzr). Since then many python package are failing their tests suites with errors like <pre> ============================= test session starts ============================== platform linux2 -- Python 2.7.14, pytest-3.2.2, py-1.4.34, pluggy-0.4.0 -- /usr/bin/python2.7 cachedir: .cache rootdir: /var/tmp/portage/dev-python/bleach-1.5.0/work/bleach-1.5.0, inifile: plugins: xdist-1.15.0, virtualenv-1.2.11, shutil-1.2.11, relaxed-1.1.0, cov-2.3.1, hypothesis-3.6.0, flaky-3.1.1, backports.unittest-mock-1.3 collecting ... collected 155 items / 1 errors ===Flaky Test Report=== ===End Flaky Test Report=== ==================================== ERRORS ==================================== __________________________ ERROR collecting setup.py ___________________________ setup.py:82: in <module> 'Topic :: Software Development :: Libraries :: Python Modules', /usr/lib64/python2.7/site-packages/setuptools/__init__.py:129: in setup return distutils.core.setup(**attrs) /usr/lib64/python2.7/distutils/core.py:139: in setup raise SystemExit, gen_usage(dist.script_name) + "\nerror: %s" % msg E SystemExit: usage: py.test [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] E or: py.test --help [cmd1 cmd2 ...] E or: py.test --help-commands E or: py.test cmd --help E E error: no commands supplied </pre> (the last line can read differently, e.g. "E error: option -x not recognized", ..) or <pre> ============================= test session starts ============================== platform linux2 -- Python 2.7.14, pytest-3.2.2, py-1.4.34, pluggy-0.4.0 rootdir: /var/tmp/portage/dev-python/path-py-8.1.2/work/path.py-8.1.2, inifile: pytest.ini plugins: xdist-1.15.0, virtualenv-1.2.11, shutil-1.2.11, relaxed-1.1.0, cov-2.3.1, hypothesis-3.6.0, flaky-3.1.1, backp orts.unittest-mock-1.3 collected 73 items path.py .EFEEE... test_path.py ................................................................ ===Flaky Test Report=== ===End Flaky Test Report=== ==================================== ERRORS ==================================== ______________________ ERROR at setup of surrogate_escape ______________________ file /var/tmp/portage/dev-python/path-py-8.1.2/work/path.py-8.1.2/path.py, line 80 def surrogate_escape(error): E fixture 'error' not found > available fixtures: cache, capfd, capsys, cov, doctest_namespace, environ, monkeypatch, pytestconfig, record_xm l_property, recwarn, tmpdir, tmpdir_factory, virtualenv, worker_id, workspace > use 'pytest --fixtures [testpath]' for help on them. /var/tmp/portage/dev-python/path-py-8.1.2/work/path.py-8.1.2/path.py:80 </pre> or <pre> ============================= test session starts ============================== platform linux2 -- Python 2.7.14, pytest-3.2.2, py-1.4.34, pluggy-0.4.0 -- /usr/bin/python2.7 cachedir: .cache rootdir: /var/tmp/portage/dev-python/sqlalchemy-1.2.4/work/SQLAlchemy-1.2.4-python2_7, inifile: setup.cfg plugins: xdist-1.15.0, virtualenv-1.2.11, shutil-1.2.11, relaxed-1.1.0, cov-2.3.1, hypothesis-3.6.0, flaky-3.1.1, backp orts.unittest-mock-1.3 collecting ... collected 8552 items / 1 errors ===Flaky Test Report=== ===End Flaky Test Report=== ==================================== ERRORS ==================================== _____________ ERROR collecting test/perf/invalidate_stresstest.py ______________ ImportError while importing test module '/var/tmp/portage/dev-python/sqlalchemy-1.2.4/work/SQLAlchemy-1.2.4-python2_7/t est/perf/invalidate_stresstest.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: /usr/lib64/python2.7/site-packages/_pytest/python.py:395: in _importtestmodule mod = self.fspath.pyimport(ensuresyspath=importmode) /usr/lib64/python2.7/site-packages/py/_path/local.py:662: in pyimport __import__(modname) test/perf/invalidate_stresstest.py:1: in <module> import gevent.monkey E ImportError: No module named gevent.monkey !!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!! =========================== 1 error in 16.23 seconds =========================== </pre> I suspect the culprit to be pytest-relaxed and indeed uninstalling pytest-relaxed lets the test suites succeed again. Packages failing with pytest-relaxed include for example dev-python/Babel-2.5.3-r1 dev-python/bcrypt-3.1.3 dev-python/cryptography-2.1.4-r1 dev-python/pyopenssl-17.5.0 dev-python/pytest-3.2.2 dev-python/six-1.11.0 dev-python/sphinx-1.6.5-r1 dev-python/setuptools-36.7.2 dev-python/virtualenv-15.1.0 ... Side note: Currently only dev-python/paramiko depends dev-python/pytest-relaxed. pytest-relaxed is a pytest plugin and gets automatically loaded every time as it is installed globally and has a setuptools entry point. Skimming through the web I found no way to tell pytest to not load any plugins or ignore entry points. Individual plugins, however, can be disabled with @-p no:PLUGINNAME@. How to incorporate this argument into the test suite depends on the package as different packages call pytest differently (directly, via tox, in setup.py, custom script, ...). I also saw that upstream of some packages use virtualenv to install a pytest version without plugins to run their test suite. This makes the package robust against globally installed python packages. Were to go from here. 1. Should such packages get a !dev-python/pytest-relaxed entry in their package DEPEND? 2. Is there a way to disable setuptools entry points (during merge) maybe with pyvenv or similar? 3. Should each package pass @-p no:relaxed@ in its test suite (patching the source if needed)? 4. What else can be done about this? 5. Did I miss something? Bob
I've already reported this upstream and I'm waiting for their reply. Preferably we'd find a way to make this plugin less horrible. If we can't, we'd either end up patching it for sanity (= off by default) or patching it out of paramiko.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ac9e67fa461c484fd751040bb6b58c258b6d601c commit ac9e67fa461c484fd751040bb6b58c258b6d601c Author: Virgil Dupras <vdupras@gentoo.org> AuthorDate: 2018-08-01 00:47:31 +0000 Commit: Virgil Dupras <vdupras@gentoo.org> CommitDate: 2018-08-01 12:09:58 +0000 dev-python/pytest-relaxed: disable plugin autoload When installed, this plugins autoload everywhere pytest runs, breaking pretty much everything in its wake. We patch out its entry point to avoid autoloading. Packages using this plugin may load it manually by adding "-p pytest_relaxed.plugin" to their pytest invocation. Closes: https://bugs.gentoo.org/661218 Package-Manager: Portage-2.3.44, Repoman-2.3.10 dev-python/pytest-relaxed/Manifest | 1 + .../files/pytest-relaxed-1.1.4-no-autoload.patch | 12 ++++++++ .../pytest-relaxed/pytest-relaxed-1.1.4.ebuild | 35 ++++++++++++++++++++++ 3 files changed, 48 insertions(+) Additionally, it has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=e647b476018a2f9e16c9c507ffd02fad8121aa45 commit e647b476018a2f9e16c9c507ffd02fad8121aa45 Author: Virgil Dupras <vdupras@gentoo.org> AuthorDate: 2018-08-01 00:50:07 +0000 Commit: Virgil Dupras <vdupras@gentoo.org> CommitDate: 2018-08-01 12:09:59 +0000 dev-python/fabric: load pytest-relaxed plugin explicitly Bug: https://bugs.gentoo.org/661218 Package-Manager: Portage-2.3.44, Repoman-2.3.10 dev-python/fabric/fabric-2.2.1.ebuild | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=cfdccf48a2b1651a903f67bae2fc91dc132b9ed3 commit cfdccf48a2b1651a903f67bae2fc91dc132b9ed3 Author: Virgil Dupras <vdupras@gentoo.org> AuthorDate: 2018-08-01 00:48:54 +0000 Commit: Virgil Dupras <vdupras@gentoo.org> CommitDate: 2018-08-01 12:09:58 +0000 dev-python/invoke: explicitly load pytest-relaxed plugin Bug: https://bugs.gentoo.org/661218 Package-Manager: Portage-2.3.44, Repoman-2.3.10 dev-python/invoke/invoke-1.1.0.ebuild | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
