After settings BINPKG_FORMAT='gpkg', older XPAK binary packages and newer gpkg ones seem to be able to co-exist without issues. However, it appears that when a binary package move occurs - such as the recent dev-util -> dev-build move, if BINPKG_FORMAT='gpkg' is set then all moved XPAK files are *copied* to *.gpkg.tar in the new category directory without their contents being changed - which causes portage to error-out with: If the new gpkg files are deleted from the new category directory, then on the next 'emerge' invocation every broken package in that directory will re-appear: it appears that the "%='binary move'" stage is actually copying XPAK files without removing the original, and so the corrupt packages are being created each time the 'binary move' process runs. So it looks as if there are several problems here: * The 'binary move' process renames all (XPAK) packages to match BINPKG_FORMAT (I don't know if the reverse is also true), but doesn't re-pack or convert them to give a valid result; * The 'binary move' process doesn't remove old XPAK files if BINPKG_FORMAT=gpkg, meaning that it will re-create previously deleted corrupt packages every time it runs; * 'emerge -k --buildpkg=y' treats a corrupt package as a fatal error, rather than rebuilding the package in question and self-healing… if not using 'emerge -K' and if explicitly (re)building binary packages, a corrupt package should instead be a warning (or automatically renamed for later inspection)? Reproducible: Always Steps to Reproduce: 1. Have a collection of XPAK & gpkg binary packages; 2. Trigger a category-move affecting a category which still has XPAK packages; 3. Until the XPAK files are manually removed, blocking corruption will be recreated every time the 'binary move' process triggers. Actual Results: 'emerge' aborts with, for example: ``` >>> Extracting info !!! Error Extracting '/var/cache/portage/pkg/amd64/docker/dev-build/automake-wrapper/automake-wrapper-20221207-7.gpkg.tar', Cannot identify tar format: /var/cache/portage/pkg/amd64/docker/dev-build/automake-wrapper/automake-wrapper-20221207-7.gpkg.tar ``` Expected Results: * The 'binary move' process should either not rename packages, or should re-pack them (adding metadata as necessary) - if not possible, perhaps this process should abort the entire operation early? * 'emerge' should treat a corrupt package as a warning and rebuild the package (if --buildpkg=y), rather than aborting and breaking all builds until manual intervention is performed; * 'emaint binhost -f' and/or eclean should find and remove corrupt packages. This was a confusing one to fix, since the problem kept coming back. In the end I had to: * Find all files in PKGDIR named *.gpkg.tar; * Find which of these were corrupt by looking for 'XPAK' after running `file` on each; * Find all other instances of the same filename in other category-directories; * Delete the original and the new gpkg files; * Run 'emaint binhost -f'; * Rebuild the affected packages. … which surely isn't the intent :(
Created attachment 915707 [details] Package move errors Experienced this today, when updating the tree from 2024-12-02 to 2025-01-02 with the llvm category creations. Adding log for completeness.
In the move_ent method we have this code which probably needs adjustment to choose an appropriate file name based on type: allocated_pkg_path = self.getname(mynewcpv, allocate_new=True)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=6417df1f8e35895ba12b71381a57c15406cc5999 commit 6417df1f8e35895ba12b71381a57c15406cc5999 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2025-01-02 03:09:49 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2025-01-02 03:22:55 +0000 binarytree.move_ent: Allocate path with binpkg_format Bug: https://bugs.gentoo.org/923530 Signed-off-by: Zac Medico <zmedico@gentoo.org> NEWS | 2 ++ lib/portage/dbapi/bintree.py | 4 +++- 2 files changed, 5 insertions(+), 1 deletion(-)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=03f41049a0fe0632eabd8cddaaca898e45943201 commit 03f41049a0fe0632eabd8cddaaca898e45943201 Author: Sam James <sam@gentoo.org> AuthorDate: 2025-01-22 00:29:50 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2025-01-22 00:30:02 +0000 sys-apps/portage: add 3.0.67 Closes: https://bugs.gentoo.org/703520 Closes: https://bugs.gentoo.org/707980 Closes: https://bugs.gentoo.org/904702 Closes: https://bugs.gentoo.org/906044 Closes: https://bugs.gentoo.org/923530 Closes: https://bugs.gentoo.org/938164 Closes: https://bugs.gentoo.org/939299 Closes: https://bugs.gentoo.org/940120 Closes: https://bugs.gentoo.org/942512 Closes: https://bugs.gentoo.org/942760 Closes: https://bugs.gentoo.org/945382 Closes: https://bugs.gentoo.org/945861 Closes: https://bugs.gentoo.org/946326 Closes: https://bugs.gentoo.org/947822 Closes: https://bugs.gentoo.org/948067 Closes: https://bugs.gentoo.org/939444 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.67.ebuild | 231 +++++++++++++++++++++++++++++++++ 2 files changed, 232 insertions(+)
