Created attachment 549448 [details] emerge --info The 'emerge --search' function appears to be broken for certain packages. It basically ignores packages even when they exist. For example, media-sound/spek: imbrium /usr/portage # emerge -s spek [ Results for search key : spek ] Searching... * app-accessibility/espeak Latest version available: 1.48.04-r1 Latest version installed: [ Not Installed ] Size of files: 2,949 KiB Homepage: http://espeak.sourceforge.net/ Description: Speech synthesizer for English and other languages License: GPL-3 [ Applications found : 1 ] imbrium /usr/portage # emerge -vp spek These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild N ] media-sound/spek-0.8.3-r1::gentoo USE="-libav" 0 KiB Total: 1 package (1 new), Size of downloads: 0 KiB Yet another package in the same category is fine: imbrium /usr/portage # emerge -s xwax [ Results for search key : xwax ] Searching... * media-sound/xwax Latest version available: 1.5 Latest version installed: [ Not Installed ] Size of files: 76 KiB Homepage: http://xwax.org/ Description: Digital vinyl emulation software License: GPL-2 [ Applications found : 1 ] File permissions seem identical for both: imbrium /usr/portage # ls -la /usr/portage/media-sound/xwax /usr/portage/media-sound/spek /usr/portage/media-sound/spek: total 12 drwxr-xr-x 1 portage portage 90 Jul 18 23:23 . drwxr-xr-x 1 portage portage 5836 Aug 24 23:04 .. drwxr-xr-x 1 portage portage 294 Jul 18 23:23 files -rw-r--r-- 1 portage portage 304 Jul 18 23:23 Manifest -rw-r--r-- 1 portage portage 369 Jul 18 23:23 metadata.xml -rw-r--r-- 1 portage portage 1146 Jul 18 23:23 spek-0.8.3-r1.ebuild /usr/portage/media-sound/xwax: total 16 drwxr-xr-x 1 portage portage 110 Jul 18 23:23 . drwxr-xr-x 1 portage portage 5836 Aug 24 23:04 .. drwxr-xr-x 1 portage portage 58 Jul 18 23:23 files -rw-r--r-- 1 portage portage 600 Jul 18 23:23 Manifest -rw-r--r-- 1 portage portage 803 Jul 18 23:23 metadata.xml -rw-r--r-- 1 portage portage 2174 Jul 18 23:23 xwax-1.5.ebuild -rw-r--r-- 1 portage portage 2178 Jul 18 23:23 xwax-1.6.ebuild I even tried rm -r /usr/portage and a fresh sync, to no avail. Some more example packages with the same issue (VERY non-exhaustive list) media-sound/fmit sys-block/fmdl sys-block/whdd sys-block/devio sys-block/spindown net-irc/miau net-irc/limnoria I feel I must be missing something obvious here, as surely a bug this broad would've been reported. But... even if something is wrong on my system with specific packages, portage is still overlooking in search packages it can clearly install, without reporting any errors. That alone is, I think, a bug.
Try it with --search-index=n in your emerge options. You probably have a stale metadata/pkg_desc_index in the repository. You can update it with this command: egencache --update-pkg-desc-index --repo=gentoo
(In reply to Zac Medico from comment #1) > Try it with --search-index=n in your emerge options. You probably have a > stale metadata/pkg_desc_index in the repository. You can update it with this > command: > > egencache --update-pkg-desc-index --repo=gentoo Thank you, that does seem to be a temporary workaround. However, the file that command updates (/usr/portage/metadata/pkg_desc_index) is one of the files that is pulled from upstream at every sync. Even after I deleted it and pulled a fresh copy, it still missed many packages. Is it really expected that I should have to regen that file locally/manually after every sync? (Oh, and in case it's not clear, I'm using sync-type=git for the gentoo repo.) That said, even if I do update it manually, it leads to a second problem: After running the egencache command provided, I can no longer sync the main repo, as I've now altered a file within it and git throws a fatal error. imbrium /usr/portage # emerge --sync >>> Syncing repository 'gentoo' into '/usr/portage'... /usr/bin/git fetch origin --depth 1 remote: Total 0 (delta 0), reused 0 (delta 0) * Using keys from /usr/share/openpgp-keys/gentoo-release.asc * Refreshing keys from keyserver ... [ ok ] * Trusted signature found on top commit error: Entry 'metadata/pkg_desc_index' not uptodate. Cannot merge. fatal: Could not reset index file to revision 'refs/remotes/origin/master'. !!! git merge error in /usr/portage Action: sync for repo: gentoo, returned code = 128 I know I could add some wrappers and post-hooks to my sync command (to delete the index before sync, and then update it after sync) but that doesn't feel correct. If portage is syncing the search index file, shouldn't that file be correctly indexing the *entire* rest of the same commit?
(In reply to Xepher from comment #2) > (In reply to Zac Medico from comment #1) > > Try it with --search-index=n in your emerge options. You probably have a > > stale metadata/pkg_desc_index in the repository. You can update it with this > > command: > > > > egencache --update-pkg-desc-index --repo=gentoo > > Thank you, that does seem to be a temporary workaround. However, the file > that command updates (/usr/portage/metadata/pkg_desc_index) is one of the > files that is pulled from upstream at every sync. Even after I deleted it > and pulled a fresh copy, it still missed many packages. Is it really > expected that I should have to regen that file locally/manually after every > sync? No, it seems like a problem with the git repo you sync from. Which repo is it?
(In reply to Zac Medico from comment #3) > > No, it seems like a problem with the git repo you sync from. Which repo is > it? Is there more than one mirror for git now? I set things up some time ago after I first noticed git support being added. I only saw one URL suggested for syncing at that time. That said, it's doing signature verification so I figured any/all mirrors would have to be correct or "broken" together, and one couldn't be bad off on its own. Maybe I'm wrong on that. That said, my repo config. imbrium /etc/portage # cat /etc/portage/repos.conf/gentoo.conf [DEFAULT] main-repo = gentoo [gentoo] location = /usr/portage sync-type = git sync-uri = git://anongit.gentoo.org/repo/sync/gentoo.git sync-depth = 1 sync-git-verify-commit-signature = true
(In reply to Xepher from comment #4) > (In reply to Zac Medico from comment #3) > [gentoo] > location = /usr/portage > sync-type = git > sync-uri = git://anongit.gentoo.org/repo/sync/gentoo.git > sync-depth = 1 > sync-git-verify-commit-signature = true Indeed, I can see that media-sound/spek is missing, even though the commit history shows that the file is being updated frequently: https://gitweb.gentoo.org/repo/sync/gentoo.git/log/metadata/pkg_desc_index
There's an ebuild here: https://gitweb.gentoo.org/repo/sync/gentoo.git/tree/media-sound/spek But it's not listed here even though the file is updated frequently: https://gitweb.gentoo.org/repo/sync/gentoo.git/tree/metadata/pkg_desc_index
Hmm, running pmaint locally to update the index shows a big diff as well. Need to check if it's not updating at all on the server, of it's suffering from some pkgcore bug fixed upstream.
I have no clue why it isn't working as expected, and my frustration has exceeded the motivation to debug this. Apparently the same version of snakeoil+pkgcore+pkgcheck on my system finds it, while on jacamar it doesn't. I've set both in virtualenv to avoid any local differences. Apparently, the 'for p in repo' doesn't go over all the packages somehow. On my system it processes 31691 ebuilds; on jacamar only 14719. However, it seems that pkgcore does see those 'extra' packages when generating md5-cache. Also removing md5-cache entries to force it to regenerate doesn't change the result. @radhermit, any clue how to debug this?
Note I'm using older/forked versions of packages from pre-Python3 times: snakeoil: aa9ec925bfd0ad25947e03e81ffdbad4111ff43f pkgcore: repo-mirror-ci @ mgorny/pkgcore
*** Bug 676866 has been marked as a duplicate of this bug. ***
A simple solution would be to remove the pkg_desc_index, since distributing a corrupt pkg_desc_index is worse than nothing.
Good news, everyone. We've finally noticed the silly mistake deep in pkgcore, and fix is on its way.
metadata/pkg_desc_index | 16515 +++++++++++++++++++++++++++++++++++++++++++++++----------------
