Summary: | dev-python/cython-3.0.0-r1 fails tests | ||
---|---|---|---|
Product: | Gentoo Linux | Reporter: | Paolo Pedroni <paolo.pedroni> |
Component: | Current packages | Assignee: | Python Gentoo Team <python> |
Status: | CONFIRMED --- | ||
Severity: | normal | CC: | bschnzl, DuPol, hatler.jaremy, kingjon3377, sam |
Priority: | Normal | Keywords: | PATCH |
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
See Also: | https://github.com/cython/cython/pull/4630 | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Attachments: |
cython-3.0.0-r1:20230822-091646.log.gz
Portage Tracks Portage Settings Patch to fix test_cython_pyximport test error Patch to fix coverage_cmd_src_pkg_layout test |
Description
Paolo Pedroni
2023-08-22 09:41:48 UTC
Maybe another reason for us to change our mind on the PEP thing... Created attachment 875254 [details]
Portage Tracks
Getting the same on cython-3.0.2-r1.
Also, _test_cython_pyximport.pyx seems to drop the work/cython-3.0.2 directories from the path building pyxbuild.pyx_to_dll
for visibility:
ALL DONE (6/9)
Sharded tests run in 713 seconds (11.9 minutes)
Errors found in shards 7, 5
Errors from shard 7:
======================================================================
ERROR: test_cython_pyximport (Cython.Build.Tests.TestIpythonMagic.TestIPythonMagic.test_cython_pyximport)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/var/tmp/portage/dev-python/cython-3.0.2-r1/work/cython-3.0.2/pyximport/_pyximport3.py", line 314, in create_module
so_path = build_module(spec.name, pyxfilename=spec.origin, pyxbuild_dir=self._pyxbuild_dir,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/tmp/portage/dev-python/cython-3.0.2-r1/work/cython-3.0.2/pyximport/_pyximport3.py", line 197, in build_module
so_path = pyxbuild.pyx_to_dll(pyxfilename, extension_mod,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/tmp/portage/dev-python/cython-3.0.2-r1/work/cython-3.0.2/pyximport/pyxbuild.py", line 27, in pyx_to_dll
assert os.path.exists(filename), "Could not find %s" % os.path.abspath(filename)
AssertionError: Could not find /var/tmp/portage/dev-python/cython-3.0.2-r1/_test_cython_pyximport.pyx
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/tmp/portage/dev-python/cython-3.0.2-r1/work/cython-3.0.2/Cython/Build/Tests/TestIpythonMagic.py", line 122, in test_cython_pyximport
ip.run_cell_magic('cython_pyximport', module_name, code)
File "/usr/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 2493, in run_cell_magic
result = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/var/tmp/portage/dev-python/cython-3.0.2-r1/work/cython-3.0.2/Cython/Build/IpythonMagic.py", line 179, in cython_pyximport
__import__(module_name)
File "/var/tmp/portage/dev-python/cython-3.0.2-r1/work/cython-3.0.2/pyximport/_pyximport3.py", line 332, in create_module
raise exc.with_traceback(tb)
File "/var/tmp/portage/dev-python/cython-3.0.2-r1/work/cython-3.0.2/pyximport/_pyximport3.py", line 314, in create_module
so_path = build_module(spec.name, pyxfilename=spec.origin, pyxbuild_dir=self._pyxbuild_dir,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/tmp/portage/dev-python/cython-3.0.2-r1/work/cython-3.0.2/pyximport/_pyximport3.py", line 197, in build_module
so_path = pyxbuild.pyx_to_dll(pyxfilename, extension_mod,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/tmp/portage/dev-python/cython-3.0.2-r1/work/cython-3.0.2/pyximport/pyxbuild.py", line 27, in pyx_to_dll
assert os.path.exists(filename), "Could not find %s" % os.path.abspath(filename)
ImportError: Building module _test_cython_pyximport failed: ['AssertionError: Could not find /var/tmp/portage/dev-python/cython-3.0.2-r1/_test_cython_pyximport.pyx\n']
Errors from shard 5:
======================================================================
FAIL: runTest (__main__.EndToEndTest.runTest)
[5] End-to-end coverage_cmd_src_pkg_layout
----------------------------------------------------------------------
Traceback (most recent call last):
File "/var/tmp/portage/dev-python/cython-3.0.2-r1/work/cython-3.0.2/runtests.py", line 2047, in runTest
self.assertEqual(0, res, "non-zero exit status, last output was:\n%r\n-- stdout:%s\n-- stderr:%s\n" % (
AssertionError: 0 != 1 : non-zero exit status, last output was:
'/usr/bin/python3.11 -m pip install .'
-- stdout:
-- stderr:error: externally-managed-environment
× This environment is externally managed
╰─>
The system-wide Python installation in Gentoo should be maintained
using the system package manager (e.g. emerge).
If the package in question is not packaged for Gentoo, please
consider installing it inside a virtual environment, e.g.:
python -m venv /path/to/venv
. /path/to/venv/bin/activate
pip install mypackage
To exit the virtual environment, run:
deactivate
The virtual environment is not deleted, and can be re-entered by
re-sourcing the activate file.
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
Created attachment 875255 [details]
Portage Settings
Created attachment 880264 [details, diff] Patch to fix test_cython_pyximport test error I was not able to reproduce the "coverage_cmd_src_pkg_layout" error originally mentioned in the bug. However, I was able to reproduce the "test_cython_pyximport" error mentioned by bschnzl on cython-3.0.6 and cython-3.0.7. Running the tests manually from the "${WORKDIR}/${P}" directory using the same test command failing in the ebuild did not produce any errors. I suspected this to be a path traversal issue instigated by something in the portage build environment which wasn't present in an interactive shell. The errors I received from cython-3.0.6 are below. > ERROR: test_cython_pyximport (Cython.Build.Tests.TestIpythonMagic.TestIPythonMagic.test_cython_pyximport) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/var/tmp/portage/dev-python/cython-3.0.6/work/cython-3.0.6/pyximport/_pyximport3.py", line 314, in create_module > so_path = build_module(spec.name, pyxfilename=spec.origin, pyxbuild_dir=self._pyxbuild_dir, > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File "/var/tmp/portage/dev-python/cython-3.0.6/work/cython-3.0.6/pyximport/_pyximport3.py", line 197, in build_module > so_path = pyxbuild.pyx_to_dll(pyxfilename, extension_mod, > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File "/var/tmp/portage/dev-python/cython-3.0.6/work/cython-3.0.6/pyximport/pyxbuild.py", line 27, in pyx_to_dll > assert os.path.exists(filename), "Could not find %s" % os.path.abspath(filename) > AssertionError: Could not find /var/tmp/portage/dev-python/cython-3.0.6/_test_cython_pyximport.pyx > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): > File "/var/tmp/portage/dev-python/cython-3.0.6/work/cython-3.0.6/Cython/Build/Tests/TestIpythonMagic.py", line 122, in test_cython_pyximport > ip.run_cell_magic('cython_pyximport', module_name, code) > File "/usr/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 2517, in run_cell_magic > result = fn(*args, **kwargs) > ^^^^^^^^^^^^^^^^^^^ > File "/var/tmp/portage/dev-python/cython-3.0.6/work/cython-3.0.6/Cython/Build/IpythonMagic.py", line 179, in cython_pyximport > __import__(module_name) > File "/var/tmp/portage/dev-python/cython-3.0.6/work/cython-3.0.6/pyximport/_pyximport3.py", line 332, in create_module > raise exc.with_traceback(tb) > File "/var/tmp/portage/dev-python/cython-3.0.6/work/cython-3.0.6/pyximport/_pyximport3.py", line 314, in create_module > so_path = build_module(spec.name, pyxfilename=spec.origin, pyxbuild_dir=self._pyxbuild_dir, > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File "/var/tmp/portage/dev-python/cython-3.0.6/work/cython-3.0.6/pyximport/_pyximport3.py", line 197, in build_module > so_path = pyxbuild.pyx_to_dll(pyxfilename, extension_mod, > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File "/var/tmp/portage/dev-python/cython-3.0.6/work/cython-3.0.6/pyximport/pyxbuild.py", line 27, in pyx_to_dll > assert os.path.exists(filename), "Could not find %s" % os.path.abspath(filename) > ImportError: Building module _test_cython_pyximport failed: ['AssertionError: Could not find /var/tmp/portage/dev-python/cython-3.0.6/_test_cython_pyximport.pyx\n'] Investigating the second traceback above, I found that the build_module function in _pyximport3.py has a fair amount of logic around changing the directory. That logic was introduced in the below PR from upstream to ensure Windows paths don't go beyond the supported length. https://github.com/cython/cython/pull/4630 It appears that upstream considered constraining that logic to Windows only, but ultimately decided to "keep it general and see if it actually causes problems that way." See the below comment. https://github.com/cython/cython/pull/4630#issuecomment-1186580262 Leaving that logic general does indeed cause problems. I confirmed this by creating a patch reverting the PR (attached) and attempting to emerge cython again. The patch resolved the "test_cython_pyximport" testing failure on both cython-3.0.6 and cython-3.0.7. I did a cursory review of the open cython issues on GitHub and it doesn't appear that others are running into this. I believe the root cause to be something specific to how portage interacts with the logic from the PR, but I am not sure. More testing and debugging would be needed. Does anyone see any value in me opening an issue upstream for the "test_cython_pyximport" testing failure? Created attachment 881118 [details, diff]
Patch to fix coverage_cmd_src_pkg_layout test
I'm not sure why I didn't run into it sooner, but I hit the coverage_cmd_src_pkg_layout test failure doing a @world rebuild. I applied this patch to work past it. There may be a more elegant solution, but this didn't seem to cause any issues for my system.
|