Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 934922

Summary: dev-python/pip-24.1-r1: --python option is broken by unbundling
Product: Gentoo Linux Reporter: Michał Górny <mgorny>
Component: Current packagesAssignee: Python Gentoo Team <python>
Status: CONFIRMED ---    
Severity: normal    
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---

Description Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2024-06-25 11:05:25 UTC
Apparently pip has `--python` option that reexecutes itself with another Python executable, and uses a magic loader to load its own packages from the original Python's site-packages directory.  Which obviously breaks down when we're unbundling deps.


$ python -m build -s
* Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
  - setuptools>=67.6.1
  - wheel
> /usr/lib/python-exec/python3.12/python -m pip --python /tmp/build-env-v9_gr3tu/bin/python install --use-pep517 --no-warn-script-
  location --no-compile -r /tmp/build-reqs-1l6ykvk9.txt
< Traceback (most recent call last):
<   File "/usr/lib/python3.12/site-packages/pip/__pip-runner__.py", line 50, in <module>
<     runpy.run_module("pip", run_name="__main__", alter_sys=True)
<   File "<frozen runpy>", line 226, in run_module
<   File "<frozen runpy>", line 98, in _run_module_code
<   File "<frozen runpy>", line 88, in _run_code
<   File "/usr/lib/python3.12/site-packages/pip/__main__.py", line 22, in <module>
<     from pip._internal.cli.main import main as _main
<   File "/usr/lib/python3.12/site-packages/pip/_internal/cli/main.py", line 11, in <module>
<     from pip._internal.cli.autocompletion import autocomplete
<   File "/usr/lib/python3.12/site-packages/pip/_internal/cli/autocompletion.py", line 10, in <module>
<     from pip._internal.cli.main_parser import create_main_parser
<   File "/usr/lib/python3.12/site-packages/pip/_internal/cli/main_parser.py", line 9, in <module>
<     from pip._internal.build_env import get_runnable_pip
<   File "/usr/lib/python3.12/site-packages/pip/_internal/build_env.py", line 14, in <module>
<     from certifi import where
< ModuleNotFoundError: No module named 'certifi'

Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/build/__main__.py", line 178, in _handle_build_error
    yield
  File "/usr/lib/python3.12/site-packages/build/__main__.py", line 429, in main
    built = build_call(
            ^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/build/__main__.py", line 238, in build_package
    out = _build(isolation, srcdir, outdir, distribution, config_settings, skip_dependency_check, installer)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/build/__main__.py", line 170, in _build
    return _build_in_isolated_env(srcdir, outdir, distribution, config_settings, installer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/build/__main__.py", line 135, in _build_in_isolated_env
    env.install(builder.build_system_requires)
  File "/usr/lib/python3.12/site-packages/build/env.py", line 136, in install
    self._env_backend.install_requirements(requirements)
  File "/usr/lib/python3.12/site-packages/build/env.py", line 265, in install_requirements
    run_subprocess(cmd)
  File "/usr/lib/python3.12/site-packages/build/_ctx.py", line 71, in run_subprocess
    subprocess.run(cmd, capture_output=True, check=True, env=env)
  File "/usr/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/lib/python-exec/python3.12/python', '-m', 'pip', '--python', '/tmp/build-env-v9_gr3tu/bin/python', 'install', '--use-pep517', '--no-warn-script-location', '--no-compile', '-r', '/tmp/build-reqs-1l6ykvk9.txt']' returned non-zero exit status 1.

ERROR Command '['/usr/lib/python-exec/python3.12/python', '-m', 'pip', '--python', '/tmp/build-env-v9_gr3tu/bin/python', 'install', '--use-pep517', '--no-warn-script-location', '--no-compile', '-r', '/tmp/build-reqs-1l6ykvk9.txt']' returned non-zero exit status 1.
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2024-06-25 11:08:41 UTC
I suppose that our options are:

1. Try to figure out how to get `pip._vendor.*` magic to work with unbundling, and live with the risk of packaging.Version mismatch style conflicts.

2. Modify their loader to cover all of pip's dependencies.  Could get ugly, and may partially break down build isolation.

3. Modify their launcher to run pip with bundled dependencies from the wheel instead.
Comment 2 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2024-06-25 18:48:18 UTC
(In reply to Michał Górny from comment #1)
> I suppose that our options are:
> 
> 1. Try to figure out how to get `pip._vendor.*` magic to work with
> unbundling, and live with the risk of packaging.Version mismatch style
> conflicts.

I've been able to get that to work — but it causes the packaging.Version mismatch problem between system and vendored package.
Comment 3 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2024-07-02 08:51:21 UTC
(In reply to Michał Górny from comment #2)
> (In reply to Michał Górny from comment #1)
> > I suppose that our options are:
> > 
> > 1. Try to figure out how to get `pip._vendor.*` magic to work with
> > unbundling, and live with the risk of packaging.Version mismatch style
> > conflicts.
> 
> I've been able to get that to work — but it causes the packaging.Version
> mismatch problem between system and vendored package.

Actually, this doesn't solve the problem at all.  So either 2. or 3. remains.