Apparently pip has `--python` option that reexecutes itself with another Python executable, and uses a magic loader to load its own packages from the original Python's site-packages directory. Which obviously breaks down when we're unbundling deps. $ python -m build -s * Creating isolated environment: venv+pip... * Installing packages in isolated environment: - setuptools>=67.6.1 - wheel > /usr/lib/python-exec/python3.12/python -m pip --python /tmp/build-env-v9_gr3tu/bin/python install --use-pep517 --no-warn-script- location --no-compile -r /tmp/build-reqs-1l6ykvk9.txt < Traceback (most recent call last): < File "/usr/lib/python3.12/site-packages/pip/__pip-runner__.py", line 50, in <module> < runpy.run_module("pip", run_name="__main__", alter_sys=True) < File "<frozen runpy>", line 226, in run_module < File "<frozen runpy>", line 98, in _run_module_code < File "<frozen runpy>", line 88, in _run_code < File "/usr/lib/python3.12/site-packages/pip/__main__.py", line 22, in <module> < from pip._internal.cli.main import main as _main < File "/usr/lib/python3.12/site-packages/pip/_internal/cli/main.py", line 11, in <module> < from pip._internal.cli.autocompletion import autocomplete < File "/usr/lib/python3.12/site-packages/pip/_internal/cli/autocompletion.py", line 10, in <module> < from pip._internal.cli.main_parser import create_main_parser < File "/usr/lib/python3.12/site-packages/pip/_internal/cli/main_parser.py", line 9, in <module> < from pip._internal.build_env import get_runnable_pip < File "/usr/lib/python3.12/site-packages/pip/_internal/build_env.py", line 14, in <module> < from certifi import where < ModuleNotFoundError: No module named 'certifi' Traceback (most recent call last): File "/usr/lib/python3.12/site-packages/build/__main__.py", line 178, in _handle_build_error yield File "/usr/lib/python3.12/site-packages/build/__main__.py", line 429, in main built = build_call( ^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/build/__main__.py", line 238, in build_package out = _build(isolation, srcdir, outdir, distribution, config_settings, skip_dependency_check, installer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/build/__main__.py", line 170, in _build return _build_in_isolated_env(srcdir, outdir, distribution, config_settings, installer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/build/__main__.py", line 135, in _build_in_isolated_env env.install(builder.build_system_requires) File "/usr/lib/python3.12/site-packages/build/env.py", line 136, in install self._env_backend.install_requirements(requirements) File "/usr/lib/python3.12/site-packages/build/env.py", line 265, in install_requirements run_subprocess(cmd) File "/usr/lib/python3.12/site-packages/build/_ctx.py", line 71, in run_subprocess subprocess.run(cmd, capture_output=True, check=True, env=env) File "/usr/lib/python3.12/subprocess.py", line 571, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/lib/python-exec/python3.12/python', '-m', 'pip', '--python', '/tmp/build-env-v9_gr3tu/bin/python', 'install', '--use-pep517', '--no-warn-script-location', '--no-compile', '-r', '/tmp/build-reqs-1l6ykvk9.txt']' returned non-zero exit status 1. ERROR Command '['/usr/lib/python-exec/python3.12/python', '-m', 'pip', '--python', '/tmp/build-env-v9_gr3tu/bin/python', 'install', '--use-pep517', '--no-warn-script-location', '--no-compile', '-r', '/tmp/build-reqs-1l6ykvk9.txt']' returned non-zero exit status 1.
I suppose that our options are: 1. Try to figure out how to get `pip._vendor.*` magic to work with unbundling, and live with the risk of packaging.Version mismatch style conflicts. 2. Modify their loader to cover all of pip's dependencies. Could get ugly, and may partially break down build isolation. 3. Modify their launcher to run pip with bundled dependencies from the wheel instead.
(In reply to Michał Górny from comment #1) > I suppose that our options are: > > 1. Try to figure out how to get `pip._vendor.*` magic to work with > unbundling, and live with the risk of packaging.Version mismatch style > conflicts. I've been able to get that to work — but it causes the packaging.Version mismatch problem between system and vendored package.
(In reply to Michał Górny from comment #2) > (In reply to Michał Górny from comment #1) > > I suppose that our options are: > > > > 1. Try to figure out how to get `pip._vendor.*` magic to work with > > unbundling, and live with the risk of packaging.Version mismatch style > > conflicts. > > I've been able to get that to work — but it causes the packaging.Version > mismatch problem between system and vendored package. Actually, this doesn't solve the problem at all. So either 2. or 3. remains.
*** Bug 938883 has been marked as a duplicate of this bug. ***
Note that ebuild dev-python/pip-24.0 has recently been faded out of main Gentoo repository. For me at least, reverting to v24 is still is the only way to dodge this (and related 938883) bug. With my limited knowledge of PIP mechanics, diffing v24 with later (e.g. v24.3.1-r2), I can only guess that the cause comes from file _internal/build_env.py at instruction from [pip._vendor.]certifi import where In v24.0 PIP imports `certifi` from `pip._vendor` module; but in later versions it is imported from current scoped Python environment. It seems to persists even with a Python 3.13 main OS distro.
I've just run into this. So, I pulled a pip-24.0 ebuild into my local overlay, then got a different error when running `python -m build` with hatchling. ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv. Is there some other workaround?
Note, this is with me trying to follow this guide: https://packaging.python.org/en/latest/tutorials/packaging-projects/
For the time being, I'm just creating a local venv and installing build there.
(In reply to Michał Górny from comment #8) > For the time being, I'm just creating a local venv and installing build > there. Could you describe the steps, please? Thanks.
If you don't mind dev-python/uv: python -m build --installer=uv ... If you want pip: python -m venv /tmp/venv . /tmp/venv/bin/activate pip install build python -m build ...
(In reply to Michał Górny from comment #10) > If you don't mind dev-python/uv: > > python -m build --installer=uv ... > I don't know enough to mind or not mind it? It did take a surprisingly long time to emerge uv. However, my attempts to use it failed. > * Creating isolated environment: venv+uv... > * Using external uv from /usr/bin/uv > * Installing packages in isolated environment: > - hatchling > * Getting build dependencies for sdist... > > Traceback (most recent call last): > File "/usr/lib/python3.12/site-packages/pyproject_hooks/_impl.py", line 402, in _call_hook > raise BackendUnavailable( > pyproject_hooks._impl.BackendUnavailable: Cannot import 'hatching.build' > > ERROR Backend 'hatching.build' is not available. Does it have something to do with the fact that dev-python/hatchling is installed? I can't exactly uninstall that. > If you want pip: > > python -m venv /tmp/venv > . /tmp/venv/bin/activate > pip install build > python -m build ... I tried these steps. I uninstalled dev-python/build so the third step worked properly. Then I got the same certifi import error on the last step. As far as I can tell, it was still creating a new venv and using that.
(In reply to David Gasaway from comment #11) > > * Creating isolated environment: venv+uv... > > * Using external uv from /usr/bin/uv > > * Installing packages in isolated environment: > > - hatchling > > * Getting build dependencies for sdist... > > > > Traceback (most recent call last): > > File "/usr/lib/python3.12/site-packages/pyproject_hooks/_impl.py", line 402, in _call_hook > > raise BackendUnavailable( > > pyproject_hooks._impl.BackendUnavailable: Cannot import 'hatching.build' > > > > ERROR Backend 'hatching.build' is not available. > > Does it have something to do with the fact that dev-python/hatchling is > installed? I can't exactly uninstall that. That is weird. Could you tell me what package are you trying to build? > > > If you want pip: > > > > python -m venv /tmp/venv > > . /tmp/venv/bin/activate > > pip install build > > python -m build ... > > I tried these steps. I uninstalled dev-python/build so the third step > worked properly. Then I got the same certifi import error on the last step. > As far as I can tell, it was still creating a new venv and using that. You shouldn't need to uninstall it since you are in venv now. Creating a new venv is fine, since that venv will contain a full bundled pip. Alternatively, just unmerge dev-python/pip completely.
(In reply to Michał Górny from comment #12) > That is weird. Could you tell me what package are you trying to build? Sorry, right after posting, I realized my 'hatching' vs. 'hatchling' typo. I'll keep trying this method. What I'm trying to build is the silly little package shown in the guide I linked, as a warmup to converting one of my own from setuptools to hatchling. > You shouldn't need to uninstall it since you are in venv now. Creating a > new venv is fine, since that venv will contain a full bundled pip. > Alternatively, just unmerge dev-python/pip completely. I created a venv per your steps, but the build didn't seem to be using it, it was using '/tmp/build-env-8cznlnk3/bin/python install'. I'll try removing system pip.
uv worked. > $ python -m build --installer=uv > * Creating isolated environment: venv+uv... > * Using external uv from /usr/bin/uv > * Installing packages in isolated environment: > - hatchling > * Getting build dependencies for sdist... > * Building sdist... > * Building wheel from sdist > * Creating isolated environment: venv+uv... > * Using external uv from /usr/bin/uv > * Installing packages in isolated environment: > - hatchling > * Getting build dependencies for wheel... > * Building wheel... > Successfully built example_package_dkg-0.1.0.tar.gz and exa pip still didn't. > $ python -m venv /tmp/venv > $ . /tmp/venv/bin/activate > /home/dave/Projects/packaging_tutorial > $ pip install build > -bash: pip: command not found Should I be doing this instead? > $ /tmp/venv/bin/pip install build > > [notice] A new release of pip is available: 24.3.1 -> 25.0.1 > [notice] To update, run: /tmp/venv/bin/python -m pip install --upgrade pip > ERROR: Can not perform a '--user' install. User site-packages are not visible > in this virtualenv.
(In reply to David Gasaway from comment #14) > Should I be doing this instead? > > > $ /tmp/venv/bin/pip install build Yes, I think I should have been using /tmp/venv/bin/ for both of the last two steps. Turns out `activate` wasn't working properly, but I didn't know enough about venvs to realize it. And, I needed to get rid of a ~/.config/pip/pip.conf file that I apparently created two years ago but have no idea why. I apologize for junking up this bug with my garbage. To sum up, the following steps work for me as a workaround for the original issue. Thanks. python -m venv /tmp/venv source /tmp/venv/bin/activate pip install build python -m build
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=cd58e32401e6e898830b6d051c5e9e7c9acd1831 commit cd58e32401e6e898830b6d051c5e9e7c9acd1831 Author: Michał Górny <mgorny@gentoo.org> AuthorDate: 2025-02-19 13:24:44 +0000 Commit: Michał Górny <mgorny@gentoo.org> CommitDate: 2025-02-19 14:34:18 +0000 dev-python/build: Add a patch to support unbundled pip Bug: https://bugs.gentoo.org/934922 Signed-off-by: Michał Górny <mgorny@gentoo.org> ...ld-1.2.2_p1.ebuild => build-1.2.2_p1-r1.ebuild} | 5 ++++ .../build/files/build-1.2.2_p1-gentoo-pip.patch | 28 ++++++++++++++++++++++ 2 files changed, 33 insertions(+)
I don't see any feasible fix for this, and since build's been patched, I think we can close this.
I confirm that *ebuild* `dev-python/build-1.2.2_p1-r1` combined with `dev-python/pip-25.0.1-r1` are working as expected by now. Thanks.
