Tracker bug for packages that don't use $(get_libdir) where they should.
Isn't it about time to provide $PYTHON_LIBDIR and maybe for convenience $PYTHON_SITE globally via python.eclass, instead reiterating /usr/$(get_libdir)/python${PYVER} in all and every ebuild using Python, again and again!?
Created attachment 161110 [details, diff] python.eclass (In reply to comment #1) > Isn't it about time to provide $PYTHON_LIBDIR and maybe for convenience > $PYTHON_SITE globally via python.eclass, instead reiterating > /usr/$(get_libdir)/python${PYVER} in all and every ebuild using Python, again > and again!? We agreed to do this a few years ago at a Python project meeting but never took action. We'll do it today unless someone speaks up with. I'd suggest PYTHON_LIBDIR and PYTHON_SITEDIR as names (see patch). Note the variables are exported in python_version() because we need $PYVER, so you'd use it like: python_version doins ${PYTHON_SITEDIR}/foo
After talk in #gentoo-python the latest suggestion is to use functions instead of variables: # @FUNCTION: get_python_libdir # @RETURN: Returns the python library dir get_python_libdir() { python_version echo "/usr/$(get_libdir)/python${PYVER}" } # @FUNCTION: get_python_sitedir # @RETURN: Returns the python site-packages dir get_python_sitedir() { echo "$(get_python_libdir)/site-packages" }
Should all packages always use $(get_libdir), or only in specific cases? I have a multilib amd64 system and have come across the lib/lib64 question before. Notably, in dev-python/astng, where a fix to prevent a file collision with another ebuild assumes the duplicate file is being installed to /usr/$(get_libdir), when in fact it's being installed to /usr/lib. This causes the fix to fail on systems where get_libdir returns something other than lib, which is true of amd64. Therefore, the file collision still exists on those systems (bug 233025). I was under the (possibly mistaken) impression that /usr/$(get_libdir) (e.g. /usr/lib64) should be used for platform-specific Python modules (e.g. non-pure-Python modules, .so and so on) while /usr/lib should be used as the directory for platform-shared modules (e.g. pure Python stuff). I based this on the Python distutils documentation, for example from distutils.sysconfig.get_python_lib: def get_python_lib(plat_specific=0, standard_lib=0, prefix=None): """Return the directory containing the Python library (standard or site additions). If 'plat_specific' is true, return the directory containing platform-specific modules, i.e. any module from a non-pure-Python module distribution; otherwise, return the platform-shared library directory. If 'standard_lib' is true, return the directory containing standard Python library modules; otherwise, return the directory for site-specific modules. This function, at least in dev-lang/python-2.4.4-r13, contains the following: if os.name == "posix": if plat_specific: lib = "lib64" else: lib = "lib" The whole Python distutils process seems geared towards splitting between "platform-specific" and "platform-shared" packages. The distutils install_lib command, for example, gets the destination of the library files from the install command (in distutils.command.install_lib.finalize_options()) which, for packages which don't contain any non-pure-Python modules, returns the standard /usr/lib directory as the target. If I run the following on my amd64 system: $ python -c 'from distutils.sysconfig import get_python_lib; print get_python_lib()' I get /usr/lib/python2.4/site-packages, which to me would seem to make sense. Likewise, if I run: $ python -c 'from distutils.command.install import install; \ from distutils.dist import Distribution; x = install(Distribution()); \ x.finalize_options(); print "purelib:", x.install_purelib; \ print "platlib:", x.install_platlib; print "install_lib:", \ x.install_lib' On my system, this would print: purelib: /usr/lib/python2.4/site-packages platlib: /usr/lib64/python2.4/site-packages install_lib: /usr/lib/python2.4/site-packages/ where purelib is the install.install_purelib variable ("for pure module distributions", say the comments), platlib is install.install_platlib ("non-pure, dists w/extensions") and install_lib is install.install_lib ("the actual directory to install modules to"). I have posted all of this with greater detail on bug 223025, comment 10. Is Gentoo just following a different logic from the Python distutils, or have I misunderstood something? If someone from Gentoo Python could shed some light here for me, I would appreciate it. Also, if you could please make an informed comment on bug 233025, where me and another user where having precisely the same question (whether the ebuild is wrong for not installing to $(get_libdir) or it's actually right and the collision fix is broken). Thank you kindly.
Forgot to mention, if I'm wrong and all Python modules should indeed be installed to /usr/$(get_libdir) then the aforementioned bug 233025 is also blocking this one. We were uncertain as to whether it should block this or not due to the whole doubt I mentioned above.
Sorry for the triple comment... just wanted to note I said bug 233025 several times above when I really should have said bug 223025. Apologies for the typo.
I've added get_python_sitedir() and get_python_libdir() functions to the python.eclass. Isreal, I'm trying to get access to a lib64 system so I can give you an answer and document this in our project guide[1]. I can't remember where this was discussed when we first started using get_libdir. [1] http://www.gentoo.org/proj/en/Python/developersguide.xml
NOTE: The functions have been named python_get_sitedir and python_get_libdir to follow existing function names. Thanks for pointing that out, Necoro.
Just to list other packages, which need to be addressed and are not already filed: - numeric-23.7 (perhaps this version can be dropped) - pmw-1.2 (filed an ebuild bump under bug#233638) - psycopg-1.1.21 - pychecker-0.8.17 - pyclimate-1.2.1-r1 - pydispatcher (also could need a bump :)) - pygtkglext-1.0.1* (perhaps can be dropped?) - pythong - python-irclib - sancho-0.11-r1 (also: strange slotting: 0.0 and 0) - visual-3.2.9 - wxpython/files/wxpy-config.py - xsv-2.7
Updated: - numeric is gone from the tree - pmw-1.3.2 is in the tree and has been fixed - psycopg-1.1.21 was fixed up - pychecker-0.8.18 doesn't seem to have this problem - >=pyclimate-1.2.2 doesn't seem to have this problem - pydispatcher got bumped and doesn't have the problem anymore - pygtkglext-1.0.1* doesn't seem to have this problem - pythong doesn't seem to have this problem - python-irclib has been fixed - sancho-2.4 doesn't seem to have this problem - visual-3.2.9 has been fixed - xsv-2.7 has been fixed - wxpython/files/wxpy-config.py script doesn't exist anymore So I think we're done here. Just to check: checking if it still has this problem can basically be done by seeing if /usr/lib is in the ebuild, right?
(In reply to comment #10) `grep usr/lib/python ${ebuilds}` reports e.g. mail-filter/tmda.
These seem to be all that's left... dev-lang/python/python-2.4.6.ebuild: python_mod_optimize -f -x "/(site-packages|test|tests)/" /usr/lib/python${SLOT} dev-lang/python/python-2.4.6.ebuild: python_mod_cleanup /usr/lib/python${SLOT} dev-python/numpy/numpy-1.3.0-r1.ebuild: rm -f "${ED}"/usr/lib/python*/site-packages/numpy/*.txt || die dev-python/numpy/numpy-1.3.0-r2.ebuild: rm -f "${ED}"/usr/lib/python*/site-packages/numpy/*.txt || die dev-python/numpy/numpy-1.4.1.ebuild: rm -f "${ED}"/usr/lib/python*/site-packages/numpy/*.txt || die mail-filter/tmda/tmda-1.0.3-r3.ebuild: insinto "/usr/lib/python${pv}/site-packages/TMDA" mail-filter/tmda/tmda-1.0.3-r3.ebuild: insinto "/usr/lib/python${pv}/site-packages/TMDA/pythonlib/email" mail-filter/tmda/tmda-1.1.12.ebuild: insinto "/usr/lib/python${pv}/site-packages/TMDA" mail-filter/tmda/tmda-1.1.12.ebuild: insinto "/usr/lib/python${pv}/site-packages/TMDA/Queue" mail-filter/tmda/tmda-1.1.12.ebuild: insinto "/usr/lib/python${pv}/site-packages/TMDA/pythonlib/email" mail-filter/tmda/tmda-1.1.12.ebuild: insinto "/usr/lib/python${pv}/site-packages/TMDA/pythonlib/email/mime" mail-filter/tmda/tmda-1.1.4.ebuild: insinto "/usr/lib/python${pv}/site-packages/TMDA" mail-filter/tmda/tmda-1.1.4.ebuild: insinto "/usr/lib/python${pv}/site-packages/TMDA/pythonlib/email" Whatever TMDA is doing seems deeply wrong in more than one ways.
(In reply to comment #12) The newer versions are fixed: >=dev-lang/python-2.5 >=dev-python/numpy-1.5.0-r2 >=mail-filter/tmda-1.1.12 This bug can be closed now.
