portage is broken after a sync. Reproducible: Always Steps to Reproduce: 1. emaint sync -a or emerge <anything> Actual Results: Performing Global Updates (Could take a couple of minutes if you have a lot of binary packages.) .='update pass' *='binary update' #='/var/db update' @='/var/db move' s='/var/db SLOT move' %='binary move' S='binary SLOT move' p='update /etc/portage/package.*' /var/portage/repository/kde/profiles/updates/4Q-2023......... /var/portage/gentoo/profiles/updates/4Q-2023................................... Traceback (most recent call last): File "/usr/lib/python-exec/python3.11/emaint", line 58, in <module> emaint_main(sys.argv[1:]) File "/usr/lib/python3.11/site-packages/portage/emaint/main.py", line 245, in emaint_main returncodes = taskmaster.run_tasks(tasks, func, status, options=task_opts) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/portage/emaint/main.py", line 141, in run_tasks returncode, msgs = getattr(inst, func)(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/portage/emaint/modules/sync/sync.py", line 89, in auto_sync return self._sync(repos, return_messages, emaint_opts=options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/portage/emaint/modules/sync/sync.py", line 261, in _sync self._do_pkg_moves() File "/usr/lib/python3.11/site-packages/portage/emaint/modules/sync/sync.py", line 272, in _do_pkg_moves if self.emerge_config.opts.get("--package-moves") != "n" and _global_updates( ^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/portage/_global_updates.py", line 41, in _global_updates return _do_global_updates( ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/portage/_global_updates.py", line 257, in _do_global_updates bindb.update_ents(repo_map, onUpdate=onUpdate) File "/usr/lib/python3.11/site-packages/portage/dbapi/__init__.py", line 426, in update_ents aux_update(cpv, metadata_updates) File "/usr/lib/python3.11/site-packages/portage/dbapi/bintree.py", line 284, in aux_update raise KeyError(cpv) KeyError: 'kde-plasma/libksysguard-9999'
Is it possible that the package is missing but still listed in $PKGDIR/Packages? Could be related to bug 889300. Does this command solve it? emaint --fix binhost
(In reply to Zac Medico from comment #1) > Is it possible that the package is missing but still listed in > $PKGDIR/Packages? Could be related to bug 889300. Yes, that was the case. > Does this command solve it? emaint --fix binhost Sorry, can't test anymore as I resolved the problem manually. An error message in place of the stacktrace would be helpful.
OK, similar to bug 915474 then.
There are some nasty inconsistencies with respect to binpkg-multi-instance in the global updates code and aux_update that can trigger problems related to pkgdir-index-trusted. For example, bindbapi.aux_update modifies that the file mtime without updating the mtime in the cpv instance that it passes to bintree.inject, and this is inconsistent because the mtime is part of the package key for binpkg-multi-instance. This affects bindbapi.move_slot_ent. Also, bindbbapi.move_ent has a similar problem to bindbapi.aux_update, where it modifies the package without calling aux_update. Since the global update code calls bindbbapi.move_ent and bindbapi.move_slot_ent prior to update_ents, the inconsistency in those calls may have corrupted the information that update_ents is operating on.
That would explain some other issues which keep cropping up here like corrupted half updated binpkgs, I think.
(In reply to Sam James from comment #5) sorry, I didn't finish my thought here: we kept getting weird issues like bug 906675 where it looked like multi-instance got "confused". It might be a different problem though.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=cc372a52ee2aba153c0d7c2290306b7f623e8b7c commit cc372a52ee2aba153c0d7c2290306b7f623e8b7c Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2023-12-09 23:58:07 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2023-12-10 01:27:43 +0000 bindbapi: add SIZE to _pkg_str_aux_keys The _pkg_str_aux_keys are used inside dbapi.update_ents, and need to contain SIZE in order for the fakedbapi _instance_key_multi_instance method to operate correctly. Incorrect operation of _instance_key_multi_instance could prevent binarytree.inject from removing an old instance from its internal state. It could also trigger a KeyError in bindbapi.aux_update as in bug 918597, since it could cause binarytree.getname to return a non-existent path. It could also cause binarytree.getname to return an existing but incorrect path, which might trigger an InvalidBinaryPackageFormat exception as in bug 906675. Bug: https://bugs.gentoo.org/906675 Bug: https://bugs.gentoo.org/918597 Bug: https://bugs.gentoo.org/919668 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/portage/dbapi/bintree.py | 3 +++ 1 file changed, 3 insertions(+)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=69ce8a71cc806b4b333c2e707e7a9291c3d84664 commit 69ce8a71cc806b4b333c2e707e7a9291c3d84664 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-12-10 22:13:09 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-12-10 22:29:08 +0000 _emerge: BinpkgVerifier: give better error message on stale binpkg index portage-3.0.52 defaults to FEATURES="pkgdir-index-trusted" (see NEWS) which has a few benefits, but means that manually editing PKGDIR without regenerating the index with 'emaint binhost -f' will confuse Portage. Give a better error message mentioning that command if we fail to fetch a binpkg but bintree.dbapi.cpv_exists says it should exist. Bug: https://bugs.gentoo.org/915474 Bug: https://bugs.gentoo.org/918597 Signed-off-by: Sam James <sam@gentoo.org> lib/_emerge/BinpkgVerifier.py | 22 +++++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0d365d80099d206e49b592abb30030642f8f09f9 commit 0d365d80099d206e49b592abb30030642f8f09f9 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-12-10 22:34:47 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-12-10 22:34:52 +0000 sys-apps/portage: add 3.0.57 Closes: https://bugs.gentoo.org/918929 Closes: https://bugs.gentoo.org/913628 Closes: https://bugs.gentoo.org/915474 Closes: https://bugs.gentoo.org/918597 Closes: https://bugs.gentoo.org/919072 Closes: https://bugs.gentoo.org/919105 Closes: https://bugs.gentoo.org/919174 Closes: https://bugs.gentoo.org/919311 Closes: https://bugs.gentoo.org/919419 Closes: https://bugs.gentoo.org/919668 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.57.ebuild | 242 +++++++++++++++++++++++++++++++++ 2 files changed, 243 insertions(+)
