Created attachment 411038 [details, diff] distutils-r1.eclass.patch The distutils-r1.eclass sets PYTHONPATH in distutils-r1_run_phase() before running setup.py scripts. Using PYTHONPATH when the ${BUILD_DIR}/lib is not created yet might lead to problems in setup.py scripts that try to import modules/packages from that path during the build process (Python at startup evaluates PYTHONPATH and associate a NullImporter object to ${BUILD_DIR}/lib in the sys.path_importer_cache). The fix is really simple, a mkdir is enough, proposed patch is attached.
Can you provide an example of a package that actually requires this?
(In reply to Mike Gilbert from comment #1) > Can you provide an example of a package that actually requires this? I am writing an ebuild for PyTango which is not in portage. The building system for it is based on distutils and its setup.py builds a PyTango package in ${BUILD_DIR}/lib and later tries to import this freshly built package to write documentation with Sphinx. The problem arises with an import statement as python doesn't find the package even if it's in the proper place and in the path. After investigating a little bit I have discovered how the import mechanism works and the problem with PYTHONPATH in the distutils-r1.eclass (there was no problem with distutil.eclass because PYTHONPATH is not set during compilation phase). Basically indicating a not exisisting path in PYTHONPATH has the effect to mark that path as invalid because it's inserted in sys.path_importer_cache as a NullImporter object. This means that PYTHONPATH is useless in distutils-r1.eclass because it masks *always* the ${BUILD_DIR}/lib directory as it's not present when python is executed to run setup.py. Moreover an ebuild wanting to import something freshly built from that path will *always* fail (unless there is something in the site-packages path satisfying the import but this is not the desired result). I have written a demo program to show what happens, you can run it with $ PYTHONPATH=dir python test.py Moreover I'd like to point out that the import mechanism behaves differently if the dir exists when python is executed but there are no modules or packages inside. In this case the import fails only if later, when the effective import statement is executed, there are no required files in place.
Created attachment 411234 [details] test.zip
I want to add that with python2 the missing dir is cached as imp.NullImporter object while with python 3 is cached as None, the final effect described above is the same
Thanks for your detailed explanation. If your patch indeed fixes the issue, I'm all for it as a clean solution.
(In reply to Michał Górny from comment #5) > Thanks for your detailed explanation. If your patch indeed fixes the issue, > I'm all for it as a clean solution. Yes, it fixes the issue but, most important, makes the PYTHONPATH effective. From the point of view of distutils there are no problems if ${BUILD_DIR}/lib is already in place because the distutils package uses dir_util.mkpath which doesn't fail.
Thanks for the patch. commit de65dc9655b2a783fd5340a7af2e455ea215c5db Author: Mike Gilbert <floppym@gentoo.org> Date: Sat Sep 12 19:59:55 2015 -0400 distutils-r1.eclass: mkdir ${BUILD_DIR}/lib Bug: https://bugs.gentoo.org/559644
