I've been working on the test suite for a python-xdist ebuild [1]. However, many of the tests fail because py.test doesn't detect the plugin, which should be loaded via Setuptools' entry points [2]. Reproducible: Always Steps to Reproduce: You can see that the entry-point is not successfully installed by distutils_install_for_testing if you insert: die $("${EPYTHON}" -c 'import pkg_resources; print("entry points: {}".format(list(pkg_resources.iter_entry_points("pytest11"))))') between the distutils_install_for_testing and py.test calls, which will give you: * ERROR: dev-python/pytest-xdist-9999::wtk failed (test phase): * entry points: [] An additional: rsync -av "${BUILD_DIR}/pytest_xdist.egg-info/" "${TEST_DIR}/lib/pytest_xdist-1.11-py2.7.egg/EGG-INFO/" between the distutils_install_for_testing and the iter_entry_points call will copy over PKG-INFO, SOURCES.txt, dependency_links.txt, entry_points.txt, …. The key for this particular package is entry_points.txt, but I expect the others also have jobs ;). With the rsync call, the iter_entry_points dies with: * ERROR: dev-python/pytest-xdist-9999::wtk failed (test phase): * entry points: [EntryPoint.parse('xdist = xdist.plugin')] which shows that we've succesfully installed the egg metadata with the rsync call. The source of the problem is that distutils-r1 has (in one form or another since 2013-02-10): [egg_info] egg-base = ${BUILD_DIR} which is causing egg_info to only match files prefixed with that (absolute) path in manifest_maker.add_default's: self.filelist.include_pattern("*", prefix=ei_cmd.egg_info) But the allfiles list comes from setuptools.findall, which returns paths relative to the searched directory. Looking among a set of relative paths for something starting with that absolute prefix gives no matches, so the pytest_xdist.egg-info/ content is not getting added to the manifest. Which is why it's not getting installed (without my manual rsync). Commenting out: [egg_info] egg-base = ${BUILD_DIR} in distutils-r1.eclass gets things working without the rsync. Looking into why the install phase *does* work despite an absolute egg-base, I believe it's because _distutils-r1_copy_egg_info is the same manual-copy-fixup that my rsync line is doing. We've had distutils-r1_copy_egg_info since 2013-08-25, and the commit message (Copy bundled egg-info files for reuse in python_compile(). This solves issues that caused some of the files not to be installed in broken packages) sounds suspiciously like the problem I'm having here. Potential fixes: a. Use something like _distutils-r1_copy_egg_info to patch over this in distutils_install_for_testing. b. Patch Setuptools' manifest_maker.add_default to convert ei_cmd.egg_info into a relative path before passing it to filelist.include_pattern. It might be harder to land (b) upstream, but if it did land, we could probably drop _distutils-r1_copy_egg_info locally. [1]: http://git.tremily.us/?p=wtk-overlay.git;a=blob;f=pytest-xdist-9999.ebuild;h=944b118571d013c1fbf14c0f461ee528a447e39f;hb=faf3f3dd21e680b0461f3bdfd3e5fe9606e70bfb [2]: https://pythonhosted.org/setuptools/setuptools.html#dynamic-discovery-of-services-and-plugins
I can confirm that this problem also exists with =dev-python/setuptools-9999 (which installed 312a67d000cb, Remove Python 2.5 compatibility methods for zipfiles, 2014-09-29). Looking into a Setuptools patch now.
I see the same problem in a checkout of the current Setuptools source (removing the pytest-xdist and ebuild portions of my original situation) with: $ rm -rf /tmp/xyz *.egg-info/ build/ dist/ $ mkdir -p /tmp/xyz/{home,lib,scripts,data,egg} $ echo -e '[egg_info]\negg-base = /tmp/xyz/egg' > /tmp/xyz/home/.pydistutils.cfg $ PYTHONPATH=/tmp/xyz/lib HOME=/tmp/xyz/home python2.7 setup.py install --home=/tmp/xyz/home --install-lib=/tmp/xyz/lib --install-scripts=/tmp/xyz/scripts --install-data=/tmp/xyz/data which installs a zipped tmp/xyz/lib/setuptools-6.0.3dev-py2.7.egg with only SOURCES.txt and zip-safe in EGG-INFO.
I've submitted a patch fixing this issue to Setuptools [1]. [1]: https://bitbucket.org/pypa/setuptools/pull-request/85/egg_info-search-egg-base-for-files-to-add/diff
I added the recommended setup.py egg_info from that upstream issue in this PR: https://github.com/gentoo/gentoo/pull/2746
Could you please send your diff to gentoo-dev? There's a unwritten rule for patches modifying eclasses and this rule says those patches have to be sent to gentoo-dev for approval/discussion. Thanks!
This has been merged https://github.com/gentoo/gentoo/commit/733fbcdcf08b74153bb6c3786e8f79ba031cb339
The merge of https://github.com/gentoo/gentoo/commit/733fbcdcf08b74153bb6c3786e8f79ba031cb339 breaks testing for all jupyter related ebuilds. Also other packages seem affected, see https://bugs.gentoo.org/show_bug.cgi?id=600222. Reverting the commit, tests can be executed again on my machine.
(In reply to Marius Brehler from comment #7) > The merge of > https://github.com/gentoo/gentoo/commit/ > 733fbcdcf08b74153bb6c3786e8f79ba031cb339 breaks testing for all jupyter > related ebuilds. The problem with that patch is that it requires an egg_info command for all distutils tests, but that command is specific to setuptools. We're already configuring setuputils via [1]. If we're using a version of setuptools that include the patch (8.2+) [2], I'm not clear on why we need 733fbcdc at all. Probably need more digging into the #595272 failure. [1]: https://github.com/gentoo/gentoo/blob/d642644bfbd0ba605c561fefcbf6df58f30b6a84/eclass/distutils-r1.eclass#L397-L398 [2]: https://bitbucket.org/pypa/setuptools/pull-requests/85/egg_info-search-egg-base-for-files-to-add/diff
