Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 937642 - distutils-r1.eclass policy on stray top-level files in site-packages blocks JAR library files.
Summary: distutils-r1.eclass policy on stray top-level files in site-packages blocks J...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Eclasses (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Python Gentoo Team
URL:
Whiteboard:
Keywords: PullRequest
: 937761 (view as bug list)
Depends on:
Blocks:
 
Reported: 2024-08-10 05:27 UTC by Ivan Lloro
Modified: 2024-09-12 09:37 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
jpype 1.5.0 ebuild (jpype-1.5.0.ebuild,1.03 KB, text/plain)
2024-08-10 05:27 UTC, Ivan Lloro
Details
Proposed patch (distutils-r1.eclass.patch,350 bytes, patch)
2024-08-10 05:27 UTC, Ivan Lloro
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ivan Lloro 2024-08-10 05:27:04 UTC
Created attachment 899721 [details]
jpype 1.5.0 ebuild

I have created an ebuild for jpype, a Python package that allows a Python-Java cross-language bridge to allow Python programs complete access to Java class libraries).

Jpype installs a JAR library as a top-level file in Python's site-packages. Such action is blocked by _distutils-r1_post_python_install() in distutils-r1.eclass because JAR files are not explicitly considered valid library files (unlike are .so ones). I argue that distutils-r1.eclass should be patched to allow for JAR libs installs.

So far, my workaround is just to overload and nullify _distutils-r1_post_python_install() which works for me but certainly it is not OK for me to push it into GURU as is.
Comment 1 Ivan Lloro 2024-08-10 05:27:55 UTC
Created attachment 899722 [details, diff]
Proposed patch
Comment 2 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2024-08-10 05:32:42 UTC
I suppose that makes sense.  However, it will take a while for this to land since we tend to batch patches.
Comment 3 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2024-08-10 12:26:47 UTC
By the way, did you test stuff using JPype further?  I'm wondering if that's all the fixes we need.
Comment 4 Eli Schwartz gentoo-dev 2024-08-11 05:49:38 UTC
# Given that org.jpype.jar is a library akin to the (accepted) .so files
# the "Stray top-level files in site-packages" in /usr/lib/python3.XX/site-packages/
# maybe should not appply.

This feels somehow weird because a jar file isn't a python module. The reason we allow .so files isn't because they are "libraries", it is because .so is the file extension used by python C-API importable modules. In fact, you cannot use them as ordinary C libraries at all.

I don't think it makes sense to add a special exception for jar files. If we think that packages may be legitimately storing *data* files at the top level, we should either relax the check in general, or allow adding a suppression variable for packages to suppress this check.

Alternatively, jpype could install their *data file* to jpype/org.jpype.jar which would:

- avoid cluttering up the top-level namespace a bit (python ecosystem tooling could be a bit happier about this especially pip for install/removal of jpype)

- sidestep the distutils-r1.eclass check, or depending on how you look at it, "respect the intent".



Another possibility would be to patch it. It looks like the critical bit is here: https://github.com/jpype-project/jpype/blob/653ccffd1df46e4d472217d77f592326ae3d3690/native/common/jp_classloader.cpp#L81

Since an ebuild knows exactly where files will be installed to, we technically don't have to look for data files relative to the imported modules at all -- this data file could be put in /usr/share/jpype/jar-assets/ or wherever you like, as long as the source is patched to respect that. But usually we don't bother with this even though it's basically kind of messy to have data files in site-packages at all.


(The python ecosystem loves putting data files in site-packages because of the word "easy". And because when all you have is a hammer, everything looks like a nail. In this case, the hammer is the wheel file format.)
Comment 5 Ivan Lloro 2024-08-11 12:12:17 UTC
*** Bug 937761 has been marked as a duplicate of this bug. ***
Comment 6 Ivan Lloro 2024-08-11 12:15:11 UTC
@Michał: yes, that is the only change that is required. Beyond that, it works flawlessly in my setup.
Comment 7 Ivan Lloro 2024-08-11 12:46:20 UTC
@Eli: I understand your argument (and, for the record, yes, I abused the term "library" for convenience).

Said so, the "easy wheel" approach is so common in the Python universe that patching all packages that follow it due to a simple philosophical question (no matter how very reasonable) seems like fighting a losing battle.

Restricting stray files that are an obvious clutter (such as test files as described in https://projects.gentoo.org/python/guide/qawarn.html#stray-top-level-files-in-site-packages) makes total sense but for other types of files, it is a less obvious case.

Ultimately, I don't mind patching this particular package myself. My concern now is that this strict Gentoo QA policy may end up creating the need for endless patching from upstream sources that follow an accepted practice in the Python universe.
Comment 8 Larry the Git Cow gentoo-dev 2024-09-08 18:21:44 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=31705da10b8165380b2eb7b21da4cffd7d3a1621

commit 31705da10b8165380b2eb7b21da4cffd7d3a1621
Author:     Michał Górny <mgorny@gentoo.org>
AuthorDate: 2024-08-10 05:33:33 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-09-08 18:21:29 +0000

    distutils-r1.eclass: Allow .jar files in sitedir
    
    Closes: https://bugs.gentoo.org/937642
    Signed-off-by: Michał Górny <mgorny@gentoo.org>
    Signed-off-by: Sam James <sam@gentoo.org>

 eclass/distutils-r1.eclass | 2 ++
 1 file changed, 2 insertions(+)
Comment 9 Larry the Git Cow gentoo-dev 2024-09-12 09:37:25 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/proj/guru.git/commit/?id=90a7c2f4ffef67f4e3bb00d8473557e19cd303bc

commit 90a7c2f4ffef67f4e3bb00d8473557e19cd303bc
Author:     Ivan Lloro <ivan.lloro.boada@gmail.com>
AuthorDate: 2024-09-11 09:40:56 +0000
Commit:     Ivan Lloro <ivan.lloro.boada@gmail.com>
CommitDate: 2024-09-11 13:08:21 +0000

    dev-python/jpype: Remove obsolete workaround to distutils-r1.eclass former blocking QA check
    
    Bug: https://bugs.gentoo.org/937642
    Signed-off-by: Ivan Lloro <ivan.lloro.boada@gmail.com>

 dev-python/jpype/jpype-1.5.0.ebuild | 7 -------
 1 file changed, 7 deletions(-)