Bug 923530 – Binary package moves rename *.xpak archives to *.gpkg, break future package builds

• Product: Portage Development
• Status: UNCONFIRMED ---

[Description Stuart Shelton 2024-02-02 02:55:40 UTC]

After settings BINPKG_FORMAT='gpkg', older XPAK binary packages and newer gpkg ones seem to be able to co-exist without issues.

However, it appears that when a binary package move occurs - such as the recent dev-util -> dev-build move, if BINPKG_FORMAT='gpkg' is set then all moved XPAK files are *copied* to *.gpkg.tar in the new category directory without their contents being changed - which causes portage to error-out with:

If the new gpkg files are deleted from the new category directory, then on the next 'emerge' invocation every broken package in that directory will re-appear: it appears that the "%='binary move'" stage is actually copying XPAK files without removing the original, and so the corrupt packages are being created each time the 'binary move' process runs.

So it looks as if there are several problems here:

* The 'binary move' process renames all (XPAK) packages to match BINPKG_FORMAT (I don't know if the reverse is also true), but doesn't re-pack or convert them to give a valid result;

* The 'binary move' process doesn't remove old XPAK files if BINPKG_FORMAT=gpkg, meaning that it will re-create previously deleted corrupt packages every time it runs;

* 'emerge -k --buildpkg=y' treats a corrupt package as a fatal error, rather than rebuilding the package in question and self-healing… if not using 'emerge -K' and if explicitly (re)building binary packages, a corrupt package should instead be a warning (or automatically renamed for later inspection)?

Reproducible: Always

Steps to Reproduce:
1. Have a collection of XPAK & gpkg binary packages;
2. Trigger a category-move affecting a category which still has XPAK packages;
3. Until the XPAK files are manually removed, blocking corruption will be recreated every time the 'binary move' process triggers.
Actual Results:  
'emerge' aborts with, for example:

```
>>> Extracting info
!!! Error Extracting '/var/cache/portage/pkg/amd64/docker/dev-build/automake-wrapper/automake-wrapper-20221207-7.gpkg.tar', Cannot identify tar format: /var/cache/portage/pkg/amd64/docker/dev-build/automake-wrapper/automake-wrapper-20221207-7.gpkg.tar
```

Expected Results:  
* The 'binary move' process should either not rename packages, or should re-pack them (adding metadata as necessary) - if not possible, perhaps this process should abort the entire operation early?

* 'emerge' should treat a corrupt package as a warning and rebuild the package (if --buildpkg=y), rather than aborting and breaking all builds until manual intervention is performed;

* 'emaint binhost -f' and/or eclean should find and remove corrupt packages.

This was a confusing one to fix, since the problem kept coming back.  In the end I had to:

* Find all files in PKGDIR named *.gpkg.tar;
* Find which of these were corrupt by looking for 'XPAK' after running `file` on each;
* Find all other instances of the same filename in other category-directories;
* Delete the original and the new gpkg files;
* Run 'emaint binhost -f';
* Rebuild the affected packages.

… which surely isn't the intent :(