Created attachment 450900 [details] build.log =================================== FAILURES =================================== ____________________ TestDistribution.test_n1_import_error _____________________ self = <acceptance_test.TestDistribution instance at 0x7f059e143c20> testdir = <Testdir local('/var/tmp/portage/dev-python/pytest-xdist-1.15.0/temp/pytest-of-portage/pytest-0/testdir/test_n1_import_error0')> def test_n1_import_error(self, testdir): p1 = testdir.makepyfile(""" import __import_of_missing_module def test_import(): pass """) result = testdir.runpytest(p1, "-n1") assert result.ret == 1 result.stdout.fnmatch_lines([ > "E ImportError: *__import_of_missing_module*", ]) E Failed: nomatch: 'E ImportError: *__import_of_missing_module*' E and: u'============================= test session starts ==============================' E and: u'platform linux2 -- Python 2.7.10, pytest-3.0.3, py-1.4.30, pluggy-0.4.0' E and: u'rootdir: /var/tmp/portage/dev-python/pytest-xdist-1.15.0/temp/pytest-of-portage/pytest-0/testdir/test_n1_import_error0, inifile: ' E and: u'plugins: xdist-1.15.0' E and: u'gw0 I' E and: u'gw0 [0]' E and: u'' E and: u'scheduling tests via LoadScheduling' E and: u'' E and: u'==================================== ERRORS ====================================' E and: u'___________________ ERROR collecting test_n1_import_error.py ___________________' E and: u"ImportError while importing test module '/var/tmp/portage/dev-python/pytest-xdist-1.15.0/temp/pytest-of-portage/pytest-0/testdir/test_n1_import_error0/test_n1_import_error.py'." E and: u'Original error message:' E and: u"'No module named __import_of_missing_module'" E and: u'Make sure your test modules/packages have valid Python names.' E and: u'=========================== 1 error in 0.32 seconds ============================' E and: u'' E remains unmatched: 'E ImportError: *__import_of_missing_module*' ../../../../../work/pytest-xdist-1.15.0/testing/acceptance_test.py:33: Failed ----------------------------- Captured stdout call ----------------------------- ============================= test session starts ============================== platform linux2 -- Python 2.7.10, pytest-3.0.3, py-1.4.30, pluggy-0.4.0 rootdir: /var/tmp/portage/dev-python/pytest-xdist-1.15.0/temp/pytest-of-portage/pytest-0/testdir/test_n1_import_error0, inifile: plugins: xdist-1.15.0 gw0 I gw0 [0] scheduling tests via LoadScheduling ==================================== ERRORS ==================================== ___________________ ERROR collecting test_n1_import_error.py ___________________ ImportError while importing test module '/var/tmp/portage/dev-python/pytest-xdist-1.15.0/temp/pytest-of-portage/pytest-0/testdir/test_n1_import_error0/test_n1_import_error.py'. Original error message: 'No module named __import_of_missing_module' Make sure your test modules/packages have valid Python names. =========================== 1 error in 0.32 seconds ============================ !!!!!!!!!!!!!!!!!!!! Interrupted: stopping after 1 failures !!!!!!!!!!!!!!!!!!!! ====================== 1 failed, 2 passed in 1.60 seconds ====================== * ERROR: dev-python/pytest-xdist-1.15.0::gentoo failed (test phase): * Tests failed under python2.7 * * Call stack: * ebuild.sh, line 115: Called src_test * environment, line 3649: Called distutils-r1_src_test * environment, line 974: Called _distutils-r1_run_foreach_impl 'python_test' * environment, line 311: Called python_foreach_impl 'distutils-r1_run_phase' 'python_test' * environment, line 3167: Called multibuild_foreach_variant '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'python_test' * environment, line 2276: Called _multibuild_run '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'python_test' * environment, line 2274: Called _python_multibuild_wrapper 'distutils-r1_run_phase' 'python_test' * environment, line 582: Called distutils-r1_run_phase 'python_test' * environment, line 906: Called python_test * environment, line 3528: Called die * The specific snippet of code: * py.test -vv -x || die "Tests failed under ${EPYTHON}"
localhost ~ # emerge --info '=dev-python/pytest-xdist-1.15.0::gentoo' Portage 2.3.0 (python 3.4.3-final-0, default/linux/amd64/13.0, gcc-4.9.3, glibc-2.22-r4, 4.4.6-gentoo x86_64) ================================================================= System Settings ================================================================= System uname: Linux-4.4.6-gentoo-x86_64-Intel-R-_Core-TM-_i5-2520M_CPU_@_2.50GHz-with-gentoo-2.2 KiB Mem: 4043380 total, 3772336 free KiB Swap: 0 total, 0 free Timestamp of repository gentoo: Fri, 21 Oct 2016 00:45:01 +0000 sh bash 4.3_p48 ld GNU ld (Gentoo 2.25.1 p1.1) 2.25.1 app-shells/bash: 4.3_p48::gentoo dev-lang/perl: 5.22.2::gentoo dev-lang/python: 2.7.10-r1::gentoo, 3.4.3-r1::gentoo dev-util/pkgconfig: 0.28-r2::gentoo sys-apps/baselayout: 2.2::gentoo sys-apps/openrc: 0.21.7::gentoo sys-apps/sandbox: 2.10-r1::gentoo sys-devel/autoconf: 2.69::gentoo sys-devel/automake: 1.14.1::gentoo, 1.15::gentoo sys-devel/binutils: 2.25.1-r1::gentoo sys-devel/gcc: 4.9.3::gentoo sys-devel/gcc-config: 1.7.3::gentoo sys-devel/libtool: 2.4.6::gentoo sys-devel/make: 4.1-r1::gentoo sys-kernel/linux-headers: 4.3::gentoo (virtual/os-headers) sys-libs/glibc: 2.22-r4::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -pipe" DISTDIR="/usr/portage/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="acl amd64 berkdb bindist bzip2 cli cracklib crypt cxx dri fortran gdbm iconv ipv6 mmx mmxext modules multilib ncurses nls nptl openmp pam pcre readline seccomp session sse sse2 ssl tcpd unicode xattr zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_4" RUBY_TARGETS="ruby20 ruby21" USERLAND="GNU" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON localhost ~ # emerge -pqv '=dev-python/pytest-xdist-1.15.0::gentoo' [ebuild N ] dev-python/pytest-xdist-1.15.0 USE="{test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -python3_5" * IMPORTANT: 11 news items need reading for repository 'gentoo'. * Use eselect news read to view new items.
Created attachment 462928 [details] other test failures on mips for pytest-xdist-1.15.0 Failures on mips (not sure if mips specific or just with the used other packages versions resulting these different test failures=
This seems to be a typical case of "testing the version installed on the system instead of testing the version which is going to be installed". All the test cases which fail call "runpytest" with arguments which are only supported after pytest-xdist is installed: Test which fail due to -n argument: * testing/acceptance_test.py::TestDistribution::()::test_n1_import_error * testing/acceptance_test.py::TestDistribution::()::test_basetemp_in_subprocesses * testing/acceptance_test.py::TestDistribution::()::test_keyboardinterrupt_hooks_issue79 * testing/acceptance_test.py::TestDistribution::()::test_keyboard_interrupt_dist * testing/acceptance_test.py::test_issue34_pluginloading_in_subprocess * testing/acceptance_test.py::test_internal_error_with_maxfail * testing/acceptance_test.py::test_worker_id_fixture2 Test which fail due to -f argument * testing/test_looponfail.py::TestFunctional::()::test_fail_to_ok * testing/test_looponfail.py::TestFunctional::()::test_xfail_passes Test which fail due to --dist argument * testing/acceptance_test.py::TestDistEach::()::test_simple Note that some test fail quite explicitly complaining not to understand the argument given, e.g. testing/acceptance_test.py::TestDistribution::()::test_basetemp_in_subprocesses: pytest.py: error: unrecognized arguments: -n1 whilst others fail quite strangely... (so my diagnose should be taken with a grain of salt). Also note that Debian runs the test suite only after they are installed (from their rules [0] file): override_dh_auto_install: dh_auto_install PYBUILD_SYSTEM=custom PYBUILD_TEST_ARGS="{interpreter} -m pytest -x -v" dh_auto_test # run tests after install bcs the plugin needs to get registered by setup.py/entry_points [0] https://anonscm.debian.org/cgit/collab-maint/pytest-xdist.git/tree/debian/rules?h=debian/1.15.0-3&id=c09d6308a01864fe247d998205a0ffb241980015 Workaround: Install dev-python/pytest-xdist without feature "test". Then re-emerge it with feature "test" enabled. Possibly solutions: 1. Patch the test suite to skip the failing tests, or 2. Mark the ebuild "RESTRICT=test", or 3. Patch the test suite not to need the pytest-xdist installed (this should probably done by upstream).
The test failure of testing/acceptance_test.py::TestDistribution::test_manytests_to_one_import_error is known as upstream issue #99 [0] It is fixed in the upstream commit e5d80645347fc7e67efdceb97df8e95d140b283d [1], but squashed together with other changes as well. Debian has it extracted and packed as patch [2]. [0] https://github.com/pytest-dev/pytest-xdist/issues/99 [1] https://github.com/pytest-dev/pytest-xdist/pull/100/commits/e5d80645347fc7e67efdceb97df8e95d140b283d [2] https://anonscm.debian.org/cgit/collab-maint/pytest-xdist.git/tree/debian/patches/fix-acceptance_test-for-pytest3.patch?h=debian/1.15.0-3&id=c09d6308a01864fe247d998205a0ffb241980015
https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0d0f8ba34d9809e980807c9bd76d46c5c00a97a1 commit 0d0f8ba34d9809e980807c9bd76d46c5c00a97a1 Author: Mike Gilbert <floppym@gentoo.org> Date: Sat Feb 18 12:12:54 2017 -0500 dev-python/pytest-xdist: fix test_manytests_to_one_import_error Bug: https://bugs.gentoo.org/597700 Package-Manager: Portage-2.3.3_p56, Repoman-2.3.1_p49 ...1.15.0-test_manytests_to_one_import_error.patch | 27 ++++++++++++++++++++++ dev-python/pytest-xdist/pytest-xdist-1.15.0.ebuild | 4 ++++ 2 files changed, 31 insertions(+)
https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=54ee58613d5d4f744f062fed0fcf5e0ff8862604 commit 54ee58613d5d4f744f062fed0fcf5e0ff8862604 Author: Mike Gilbert <floppym@gentoo.org> Date: Sat Feb 18 12:38:55 2017 -0500 dev-python/pytest-xdist: install package for testing pytest doesn't load the xdist plugin unless xdist has already been installed. Bug: https://bugs.gentoo.org/597698 Bug: https://bugs.gentoo.org/597700 Bug: https://bugs.gentoo.org/609516 Package-Manager: Portage-2.3.3_p56, Repoman-2.3.1_p49 dev-python/pytest-xdist/pytest-xdist-1.15.0.ebuild | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Created attachment 464320 [details, diff] Patch to adapt to new output format of pytest >= 3.0.0 Thanks Mike for the quick response and fix. While these solves nearly all of the test failures the original one (TestDistribution.test_n1_import_error as well as TestDistribution.test_manytests_to_one_import_error) still fails. The reason is that in commit [0] of pytest the output format of error messages is changed slightly. The pytest-xdist tests however never where adapted to this. The timeline is: * commit [0] is dated April 2016 * Release of pytest-xdist 12 August 2016 * Release of pytest 18 August 2016 containing the changed output format * No new release of pytest-xdist since then Attached a patch which fixes the test patterns to match the new (pytest >= 3.0.0, tested) and old (pytest < 3.0.0, untested) output format. [0] https://github.com/pytest-dev/pytest/commit/56855893ca21e57b644387b3d6ada05c3448440c
(In reply to maram from comment #7) I'm not getting any failures currently with pytest-3.0.6 and pytest-xdist-1.15.0.
I can confirm that the test suite passes with pytest-3.0.6 (latest unstable), but still fails with pytest-3.0.3 (latest stable). I think I also found the culprit. It is commit [0], which introduces printing the traceback for ImportError and it happens to be that it contains the line E ImportError: No module named __import_of_missing_module which the test matches against. The commit [0] is also contained in any pytest version > 3.0.3. [0] https://github.com/pytest-dev/pytest/commit/7d66e4eae1cc5989df07e140f2ff6949416c2c18
