Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 531648 - dev-lang/python should not install /usr/lib*/python*/test/
Summary: dev-lang/python should not install /usr/lib*/python*/test/
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Python Gentoo Team
URL:
Whiteboard:
Keywords: PullRequest
: 574322 650336 (view as bug list)
Depends on:
Blocks:
 
Reported: 2014-12-04 10:37 UTC by Patrick Lauer
Modified: 2024-09-22 16:22 UTC (History)
10 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Patrick Lauer gentoo-dev 2014-12-04 10:37:09 UTC
#  du -h /usr/lib64/python2.7/test/
28M     /usr/lib64/python2.7/test/

 * dev-lang/python-2.7.7
         Total files : 4057
         Total size  : 56.68 MiB


So about half the installed size are the python unit tests that no one uses and which are only relevant at install time.

Not installing (unconditionally, or with use=-test) would halve the installed size for great benefit.
Comment 1 Mike Gilbert gentoo-dev 2014-12-04 15:44:29 UTC
Upstream installs them. If you don't want them, use INSTALL_MASK.
Comment 2 Mike Gilbert gentoo-dev 2014-12-04 17:21:03 UTC
Hmm... I see other distros package the test suite separately. So I guess a use flag would not be too abnormal here.
Comment 3 Johan Bergström 2014-12-10 04:37:52 UTC
I'd also like to see test suites go away. This should be documented somewhere since its pretty common for test folders to be installed for setuptools/pip packages as well.
Comment 4 Mike Gilbert gentoo-dev 2014-12-10 16:40:09 UTC
(In reply to Johan Bergström from comment #3)
> I'd also like to see test suites go away. This should be documented
> somewhere since its pretty common for test folders to be installed for
> setuptools/pip packages as well.

Agreed. If we are going to remove tests, we should come up with a consistent policy for it. This would include:

- A consistently named use flag
- A code snippet on the wiki

Personally, I would prefer to keep installing tests unconditionally, but if I'm out-voted it is no great bother to me.
Comment 5 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-12-10 23:00:40 UTC
I'd say this is a case for INSTALL_MASK.
Comment 6 Mike Gilbert gentoo-dev 2015-01-18 02:03:39 UTC
A note on implementing this for dev-lang/python:

The test.support module gets used in the test suites for lib2to3, idlelib, and distutils. These are installed outside of the python/test/ directory.

Also, it looks like some the tests of some third-party packages import test.support. For example:

site-packages/simplejson/tests/test_tool.py:        from test.support import strip_python_stderr
site-packages/bsddb3/tests/test_all.py:    from test import test_support


So, we would probably need to install the test.support module to avoid breaking src_test in a few python-related ebuilds.
Comment 7 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2016-02-10 12:11:22 UTC
*** Bug 574322 has been marked as a duplicate of this bug. ***
Comment 8 . 2016-05-18 16:41:22 UTC
(In reply to Mike Gilbert from comment #6)
> A note on implementing this for dev-lang/python:
> 
> The test.support module gets used in the test suites for lib2to3, idlelib,
> and distutils. These are installed outside of the python/test/ directory.
> 
> Also, it looks like some the tests of some third-party packages import
> test.support. For example:
> 
> site-packages/simplejson/tests/test_tool.py:        from test.support import
> strip_python_stderr
> site-packages/bsddb3/tests/test_all.py:    from test import test_support
> 
> 
> So, we would probably need to install the test.support module to avoid
> breaking src_test in a few python-related ebuilds.

test directory is only for people who do testing. People who do testing, have `test` USE-flag set. So I think

python:
```
use test || rm -rf ....
```

and 

dev-python/simplejson:
```
PYTHON_REQ_USE="test="
```

would solve this case.
Comment 9 Arfrever Frehtes Taifersar Arahesis 2018-03-13 00:15:12 UTC
*** Bug 650336 has been marked as a duplicate of this bug. ***
Comment 11 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-09-01 08:55:28 UTC
Arthur brought this up in #gentoo-python the other day and we started discussing it again.

mgorny dug up the API docs and we found:

* https://docs.python.org/3.12/library/test.html#module-test.support

"""
Note test.support is not a public module. It is documented here to help Python developers write tests. The API of this module is subject to change without backwards compatibility concerns between releases. 
"""

* https://docs.python.org/3.12/library/test.html

"""
Note The test package is meant for internal use by Python only. It is documented for the benefit of the core developers of Python. Any use of this package outside of Python’s standard library is discouraged as code mentioned here can change or be removed without notice between releases of Python. 
"""

i.e. no packages should be relying on it and it's not public API.
Comment 12 Eli Schwartz gentoo-dev 2024-09-01 15:09:42 UTC
I feel like I've mentioned this a few times in IRC, with reference to it being officially an internal "do not use" API.

Note that cpython does *officially* support a mechanism to disable installation of the tests. If you look at the issue in question you will see a comment from me asking about a way to install the tests with a separate `make` target (suitable for implementing split packages on distros where the package manager supports split packages, or implementing a second ebuild to install tests via portage, which doesn't).

I never did get around to implementing that feature. I was asked to submit a separate issue tracking that suggestion, and I did. I also included a comment saying I started working on it but had some design questions -- and the involved individual never responded to the ticket after asking me to open it. :)
Comment 13 Eli Schwartz gentoo-dev 2024-09-01 15:11:09 UTC
The option is available since Python 3.10, implementing it on older versions would require hacky for loops or extglob assuming the effort was found worthwhile.
Comment 14 Larry the Git Cow gentoo-dev 2024-09-05 08:43:31 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=e063af81df5889ab6447d9d15cb614222cfd0ea5

commit e063af81df5889ab6447d9d15cb614222cfd0ea5
Author:     Michał Górny <mgorny@gentoo.org>
AuthorDate: 2024-09-02 13:09:09 +0000
Commit:     Michał Górny <mgorny@gentoo.org>
CommitDate: 2024-09-05 08:42:17 +0000

    dev-lang/python: Do not install tests in this package
    
    Closes: https://bugs.gentoo.org/531648
    Signed-off-by: Michał Górny <mgorny@gentoo.org>

 dev-lang/python/python-3.10.14_p3-r1.ebuild    | 542 ++++++++++++++++++++
 dev-lang/python/python-3.11.9_p2-r1.ebuild     | 573 +++++++++++++++++++++
 dev-lang/python/python-3.12.5_p1-r1.ebuild     | 558 +++++++++++++++++++++
 dev-lang/python/python-3.13.0_rc1_p3-r1.ebuild | 660 +++++++++++++++++++++++++
 4 files changed, 2333 insertions(+)
Comment 15 Larry the Git Cow gentoo-dev 2024-09-11 23:59:11 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ab843bbe1f9c1bcb3431d95b7887c34e97835e1f

commit ab843bbe1f9c1bcb3431d95b7887c34e97835e1f
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2024-09-11 23:57:58 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-09-11 23:58:37 +0000

    dev-debug/systemtap: update USE=install-tests -> USE=test-install in comment
    
    Following discussion with mgorny and us agreeing to use test-install
    for Python.
    
    dev-debug/dtrace was updated in 063c07a77f3d3b04649bf1e00bc90b1fcd316d1a.
    
    Bug: https://bugs.gentoo.org/531648
    Signed-off-by: Sam James <sam@gentoo.org>

 dev-debug/systemtap/systemtap-5.1-r1.ebuild | 2 +-
 dev-debug/systemtap/systemtap-5.1.ebuild    | 2 +-
 dev-debug/systemtap/systemtap-9999.ebuild   | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=063c07a77f3d3b04649bf1e00bc90b1fcd316d1a

commit 063c07a77f3d3b04649bf1e00bc90b1fcd316d1a
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2024-09-11 23:56:28 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-09-11 23:58:31 +0000

    dev-debug/dtrace: rename USE=install-tests -> USE=test-install
    
    Following discussion with mgorny and us agreeing to use test-install
    for Python.
    
    Bug: https://bugs.gentoo.org/531648
    Signed-off-by: Sam James <sam@gentoo.org>

 dev-debug/dtrace/dtrace-2.0.1.1-r2.ebuild | 10 +++++-----
 dev-debug/dtrace/dtrace-9999.ebuild       | 10 +++++-----
 dev-debug/dtrace/metadata.xml             |  2 +-
 3 files changed, 11 insertions(+), 11 deletions(-)