Bug 934549 – app-portage/portage-utils - qwhich returns bogus results when md5-cache missing

• Product: Portage Development
• Status: CONFIRMED ---

[Description James Le Cuirot 2024-06-19 13:29:08 UTC]

When the md5-cache is missing, qwhich from portage-utils 0.97 only returns meaningful results for the first package in each category. Even the meaningful results don't match the output with the md5-cache present.

md5-cache present:

> $ qwhich 7zip
> /var/db/repos/gentoo/app-arch/7zip/7zip-24.06.ebuild
> $ qwhich zip
> /var/db/repos/gentoo/app-arch/zip/zip-3.0-r7.ebuild

md5-cache missing:

> $ qwhich 7zip
> /var/db/repos/gentoo/app-arch/7zip/7zip-24.05.ebuild
> /var/db/repos/gentoo/app-arch/7zip/7zip-24.06.ebuild
> $ qwhich zip
> /var/db/repos/gentoo/app-arch/zip/zip.ebuild

I've tried really hard to get to the bottom of this, but the codebase is a lot more complex than one might imagine. Stepping through with gdb, I can see that it definitely gathers up the other directories into cat_ctx->pkg_ctxs. It seems like atom_explode_cat never sees the version because only the PN gets passed in, but I don't know why.

Regarding the differing results for 7zip, you get the two lines even when specifying --latest, which doesn't seem right. Having said that, you only get a single line without --latest when the cache is present, and that doesn't seem right either.

[Comment 1 Fabian Groffen 2024-06-19 13:32:15 UTC]

thanks, let me check that later

[Comment 2 Fabian Groffen 2024-06-22 13:09:35 UTC]

in the tree with md5-cache missing, do you have a metadata/md5-cache directory?

[Comment 3 Fabian Groffen 2024-06-22 13:11:16 UTC]

oh and q -o might help here too, qwhich loops over all overlays by default, so it will report latest from each.  That said, the difference in behaviour still makes no sense (yet).

[Comment 4 James Le Cuirot 2024-06-22 21:42:46 UTC]

I first hit this on the Gentoo-derivative Flatcar Linux, which hasn't bothered generating a cache in some time because it only has a relatively small set of packages in its repos. However, I can still reproduce it on Gentoo.

I demonstrated this by renaming metadata/md5-cache. If I create an empty md5-cache directory, then I get no results at all.

I do normally have a few overlays on my system, but these packages only exist in the main gentoo repo. If I rename /etc/portage/repos.conf so that it only sees the default main repo, the behaviour remains the same. I still get an additional result, even though "q -o" only lists one repository.

[Comment 5 Fabian Groffen 2024-06-23 07:01:38 UTC]

Ok, that provides some hints.  portage-utils doesn't validate the (md5) cache, it just assumes it is correct.  Perhaps it is time to do the extra work to ensure the cache is sane.

[Comment 6 James Le Cuirot 2024-06-23 09:17:56 UTC]

It's supposed to work when there is no cache. You will find that there this a lot of code for handling ebuilds directly where cachetype is CACHE_EBUILD rather than CACHE_METADATA_MD5. This code probably doesn't get tested very often.