Bug 936405 – dev-python/certifi-3021.3.16-r4 is not recognized by dev-python/pip

• Product: Gentoo Linux
• Status: RESOLVED FIXED

[Description Maciej S. Szmigiero 2024-07-20 19:01:52 UTC]

With dev-python/pip-24.1.2 pip can't find the installed dev-python/certifi-3021.3.16-r4:
> $ python -m venv --system-site-packages --without-pip /tmp/venv-test
> $ source /tmp/venv-test/bin/activate
> $ python -m pip install --dry-run --no-index certifi
> ERROR: Could not find a version that satisfies the requirement certifi (from versions: none)
> ERROR: No matching distribution found for certifi

Installing other packages that have certifi as a dependency also fail:
> $ python -m pip install --dry-run --no-index requests
> Requirement already satisfied: requests in /usr/lib/python3.12/site-packages (2.32.3)
> Requirement already satisfied: charset-normalizer<4,>=2 in /usr/lib/python3.12/site-packages (from requests) (3.3.2)
> Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.12/site-packages (from requests) (3.7)
> Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/lib/python3.12/site-packages (from requests) (2.2.2)
> INFO: pip is looking at multiple versions of requests to determine which version is compatible with other requirements. This could take a while.
> ERROR: Could not find a version that satisfies the requirement certifi>=2017.4.17 (from requests) (from versions: none)
> ERROR: No matching distribution found for certifi>=2017.4.17

It looks like the problem is that the "dist name" [1] for certifi is actually "certifi-system-store" as specified in "/usr/lib/python3.12/site-packages/certifi-3021.3.16.dist-info/METADATA" so pip rejects this name as a duplicate for already enumerated "certifi-system-store" package in [2].

However, it looks like upstream (accidentally?) fixed the issue in the commit at $URL.
With this patch applied to dev-python/pip-24.1.2 looking up certifi works as expected:
> $ python -m pip install --dry-run --no-index certifi
> Requirement already satisfied: certifi in /usr/lib/python3.12/site-packages (3021.3.16)

[1]: https://github.com/pypa/pip/blob/9d8e765789883a854326dfad7fe31aeb9e715200/src/pip/_internal/metadata/importlib/_envs.py#L64
[2]: https://github.com/pypa/pip/blob/9d8e765789883a854326dfad7fe31aeb9e715200/src/pip/_internal/metadata/importlib/_envs.py#L69

[Comment 1 Maciej S. Szmigiero 2024-07-20 19:03:35 UTC]

That's on dev-lang/python-3.12.4_p2:3.12 BTW.

[Comment 2 Maciej S. Szmigiero 2024-07-21 10:04:32 UTC]

On some machines the issue only reproduces when dev-python/certifi is reinstalled.

I guess this depends on the order of enumeration of Python distributions - if "certifi" happens to be enumerated before "certifi-system-store" then the issue won't show since then "certifi-system-store" would be eliminated as the duplicate package and not "certifi".

In turn this might also depend on the backing filesystem - it was btrfs in all of my cases.

[Comment 3 Michał Górny 2024-07-22 08:44:52 UTC]

Well, it sounds like it worked for you only by accident.  certifi-system-store is relying on Provides-Dist [1] which simply was never implemented by pip.

Given that certifi-system-store died anyway and we're patching it for a long time already (sigh), I suppose we could just fork it and make it build as "certifi" again to avoid the problem.

[1] https://packaging.python.org/en/latest/specifications/core-metadata/#provides-dist-multiple-use

[Comment 4 Larry the Git Cow 2024-07-22 09:38:56 UTC]

The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=38824b0f84509e4128711d09144726c57cc15969

commit 38824b0f84509e4128711d09144726c57cc15969
Author:     Michał Górny <[e-mail]>
AuthorDate: 2024-07-22 09:32:06 +0000
Commit:     Michał Górny <[e-mail]>
CommitDate: 2024-07-22 09:38:52 +0000

    dev-python/certifi: Bump to 3024.7.22 (our own fork)
    
    Closes: https://bugs.gentoo.org/936405
    Signed-off-by: Michał Górny <[e-mail]>

 dev-python/certifi/Manifest                 |  1 +
 dev-python/certifi/certifi-3024.7.22.ebuild | 43 +++++++++++++++++++++++++++++
 dev-python/certifi/metadata.xml             |  1 +
 3 files changed, 45 insertions(+)