After today's package updates, I'm seeing a lot of errors like: !!! Invalid binary package: '/var/cache/binpkg/www-client/chromium/chromium-107.0.5304.18-1.gpkg.tar', gpkg file structure mismatch in /var/cache/binpkg/www-client/chromium/chromium-107.0.5304.18-1.gpkg.tar, ['chromium-107.0.5304.18-1/gpkg-1', 'chromium-107.0.5304.18/m etadata.tar.zst', 'chromium-107.0.5304.18-1/image.tar.zst', 'chromium-107.0.5304.18/Manifest'] A quick look at emerge.log, this package was built on Oct 8th, so I was definitely using Portage 3.0.38.1 at the time.
As observed in #gentoo-portage (or was it -python), it looks like build-id is getting lost somewhere?
['chromium-107.0.5304.18-1/gpkg-1', 'chromium-107.0.5304.18/metadata.tar.zst', 'chromium-107.0.5304.18-1/image.tar.zst', 'chromium-107.0.5304.18/Manifest'] This combination does not make sense. I had no idea how this file created. The gpkg version will always be the new one, so the package last update is "chromium-107.0.5304.18-1". However the metadata and Manifest is old, yet image get update.
If you can't figure out where that could be happening, perhaps we could add asserts around binpkg update code to make it explode when it detects output like this?
(In reply to Michał Górny from comment #3) > If you can't figure out where that could be happening, perhaps we could add > asserts around binpkg update code to make it explode when it detects output > like this? I think it might be the replace failed that the old directory string do not match. As it may looking for "chromium-107.0.5304.18-1" if you enabled multi-binpkg, but only "chromium-107.0.5304.18" exists. Should we unconditional rename the first level of directory to avoid this mismatch?
(In reply to Sheng Yu from comment #4) > (In reply to Michał Górny from comment #3) > > If you can't figure out where that could be happening, perhaps we could add > > asserts around binpkg update code to make it explode when it detects output > > like this? > > I think it might be the replace failed that the old directory string do not > match. As it may looking for "chromium-107.0.5304.18-1" if you enabled > multi-binpkg, but only "chromium-107.0.5304.18" exists. This is a fresh binpkg, and the old directory definitely had -1. Also, as you can see, it's the updated files that lack -1, not the original ones.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=5f256ab11e5510f98d7951ac9770948e00c36ed5 commit 5f256ab11e5510f98d7951ac9770948e00c36ed5 Author: Sheng Yu <syu.os@protonmail.com> AuthorDate: 2022-11-02 19:05:29 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-11-02 22:58:15 +0000 Unconditionally update first level of dir name in binpkgs Enabling / disabling binpkg-multi-instance while doing metadata move / update may cause binary package structure mismatch. This will unconditional replace the first level name (cpv) in the package. Closes: https://bugs.gentoo.org/877271 Signed-off-by: Sheng Yu <syu.os@protonmail.com> Closes: https://github.com/gentoo/portage/pull/929 Signed-off-by: Sam James <sam@gentoo.org> lib/portage/gpkg.py | 36 +++++++++++++++++++++++++++++++++--- 1 file changed, 33 insertions(+), 3 deletions(-)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=203e0d0083b5e7d8098785ec8862c02f6baf5534 commit 203e0d0083b5e7d8098785ec8862c02f6baf5534 Author: Sam James <sam@gentoo.org> AuthorDate: 2022-11-20 03:04:40 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-11-20 03:05:40 +0000 sys-apps/portage: add 3.0.39 Closes: https://bugs.gentoo.org/365655 Closes: https://bugs.gentoo.org/757525 Closes: https://bugs.gentoo.org/871573 Closes: https://bugs.gentoo.org/875812 Closes: https://bugs.gentoo.org/875860 Closes: https://bugs.gentoo.org/877215 Closes: https://bugs.gentoo.org/877271 Closes: https://bugs.gentoo.org/877357 Closes: https://bugs.gentoo.org/877419 Closes: https://bugs.gentoo.org/873757 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.39.ebuild | 273 +++++++++++++++++++++++++++++++++ 2 files changed, 274 insertions(+)
Still happens. But thanks to the verification, now it throws an exception without overwriting the original, so I was able to debug. Traceback: Traceback (most recent call last): File "/usr/lib/python-exec/python3.11/emerge", line 54, in <module> retval = emerge_main() ^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/_emerge/main.py", line 1294, in emerge_main return run_action(emerge_config) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/_emerge/actions.py", line 3456, in run_action and _global_updates( ^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/portage/_global_updates.py", line 41, in _global_updates return _do_global_updates( ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/portage/_global_updates.py", line 257, in _do_global_updates bindb.update_ents(repo_map, onUpdate=onUpdate) File "/usr/lib/python3.11/site-packages/portage/dbapi/__init__.py", line 414, in update_ents aux_update(cpv, metadata_updates) File "/usr/lib/python3.11/site-packages/portage/dbapi/bintree.py", line 294, in aux_update mybinpkg.update_metadata(mydata) File "/usr/lib/python3.11/site-packages/portage/gpkg.py", line 1152, in update_metadata raise InvalidBinaryPackageFormat( portage.exception.InvalidBinaryPackageFormat: gpkg file structure mismatch in /var/cache/binpkg/dev-lang/python/python-2.7.18_p16-1.gpkg.tar The problem is that _create_tarinfo() method is using self.basename. Apparently when performing updates: self.basename = "python-2.7.18_p16" self.prefix = "python-2.7.18_p16-1" This is because portage/dbapi/bintree.py aux_update() does: mybinpkg = portage.gpkg.gpkg(self.settings, cpv, binpkg_path) and str(cpv) doesn't include "-1".
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=ef3ad9396d9fe2ae7d34b640f8318d218be8bae6 commit ef3ad9396d9fe2ae7d34b640f8318d218be8bae6 Author: Michał Górny <mgorny@gentoo.org> AuthorDate: 2022-12-02 03:41:26 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-02 21:07:19 +0000 bintree: Fix breaking GPKG structure on updates Fix passing the wrong basename to the gpkg initializer when performing package updates. This caused newly written files to be missing build_id and therefore causing a structure mismatch. Bug: https://bugs.gentoo.org/877271 Signed-off-by: Michał Górny <mgorny@gentoo.org> Closes: https://github.com/gentoo/portage/pull/955 Signed-off-by: Sam James <sam@gentoo.org> NEWS | 4 +++- lib/portage/dbapi/bintree.py | 7 +++++-- lib/portage/gpkg.py | 3 ++- 3 files changed, 10 insertions(+), 4 deletions(-)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=9cd716999a21554f10d0af33c7b954173290460a commit 9cd716999a21554f10d0af33c7b954173290460a Author: Sam James <sam@gentoo.org> AuthorDate: 2022-12-05 00:47:54 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-12-05 00:47:54 +0000 sys-apps/portage: add 3.0.41 Closes: https://bugs.gentoo.org/883801 Closes: https://bugs.gentoo.org/877271 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.41.ebuild | 283 +++++++++++++++++++++++++++++++++ 2 files changed, 284 insertions(+)
Note that syu mentioned there may be another change needed in update_ent (at least to avoid trying to upgrade signed binpkgs/discard them).
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=2569a1a1d889a76af80ce24a005d54269df7d2e2 commit 2569a1a1d889a76af80ce24a005d54269df7d2e2 Author: Sheng Yu <syu.os@protonmail.com> AuthorDate: 2023-04-07 04:50:49 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-04-07 09:49:11 +0000 dbapi: Catch invalid binpkg error during pkgmove Bug: https://bugs.gentoo.org/877271 Bug: https://bugs.gentoo.org/903917 Bug: https://bugs.gentoo.org/903926 Signed-off-by: Sheng Yu <syu.os@protonmail.com> Closes: https://github.com/gentoo/portage/pull/1022 Signed-off-by: Sam James <sam@gentoo.org> NEWS | 3 +++ lib/portage/dbapi/__init__.py | 8 ++++++-- 2 files changed, 9 insertions(+), 2 deletions(-)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=05f0b3b184011060c86787f93e16a29002bdc54a commit 05f0b3b184011060c86787f93e16a29002bdc54a Author: Sam James <sam@gentoo.org> AuthorDate: 2023-04-07 09:59:46 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-04-07 09:59:55 +0000 sys-apps/portage: add 3.0.46 Closes: https://bugs.gentoo.org/861659 Closes: https://bugs.gentoo.org/877271 Closes: https://bugs.gentoo.org/898224 Closes: https://bugs.gentoo.org/899898 Closes: https://bugs.gentoo.org/902189 Closes: https://bugs.gentoo.org/903917 Closes: https://bugs.gentoo.org/903926 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.46.ebuild | 285 +++++++++++++++++++++++++++++++++ 2 files changed, 286 insertions(+)
