Bug 915474 – sys-apps/portage improve error messaging when index contains entries for missing packages

• Product: Portage Development
• Status: RESOLVED FIXED

[Description Agostino Sarubbo 2023-10-09 13:11:21 UTC]

With portage-3.0.52 -k command line option looks broken to me.

How to reproduce:

1) build a package with '-b' option enabled, it will generate a binpkg
2) test if emerge -pvOk will try to emerge the binary created above
3) manually remove the binpkg
4) test again point n°2

With portage-3.0.51:
since there is no more the binpkg on the filesystem emerge will do:
[ebuild   R    ] .... 

With portage-3.0.52:
it goes blindly by emerging the non-existing binary but it always fails in a way like:
!!! Fetching Binary failed for '...'

It works for me only if I manually remove the entry in $PKGDIR/Packages


I saw in the portage NEWS file that there was a little movement related to binpkg, but if you need it I can bisect to find the exactly commit that break it.

[Comment 1 Ionen Wolkens 2023-10-09 13:20:06 UTC]

> I saw in the portage NEWS file that there was a little
> movement related to binpkg
Given "3) manually remove the binpkg", the entry of interest in NEWS is:

Breaking changes:
* FEATURES=pkgdir-index-trusted is now on by default. This improves performance
  with binpkgs, but does require that users run `emaint binhost --fix` if they
  manually add or remove binpkgs from their PKGDIR (bug #889300).

[Comment 2 Agostino Sarubbo 2023-10-09 13:53:05 UTC]

(In reply to Ionen Wolkens from comment #1)
> > I saw in the portage NEWS file that there was a little
> > movement related to binpkg
> Given "3) manually remove the binpkg", the entry of interest in NEWS is:
> 
> Breaking changes:
> * FEATURES=pkgdir-index-trusted is now on by default. This improves
> performance
>   with binpkgs, but does require that users run `emaint binhost --fix` if
> they
>   manually add or remove binpkgs from their PKGDIR (bug #889300).

That works for me.

I'd suggest to add the 'emaint binhost --fix' suggestion into the failure log, so it may help a bit.

[Comment 3 Zac Medico 2023-10-09 22:58:44 UTC]

(In reply to Agostino Sarubbo from comment #0)
> With portage-3.0.52:
> it goes blindly by emerging the non-existing binary but it always fails in a
> way like:
> !!! Fetching Binary failed for '...'
> 
> It works for me only if I manually remove the entry in $PKGDIR/Packages

We can have BinpkgVerifier check for a stale entry in $PKGDIR/Packages when it handles the ENOENT error here. If bintree.dbapi.cpv_exists(self.pkg.cpv) returns True, then that's enough to confirm the stale entry (this method compares various metadata for binpkg-multi-instance support).

[Comment 4 Larry the Git Cow 2023-12-10 22:29:13 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=69ce8a71cc806b4b333c2e707e7a9291c3d84664

commit 69ce8a71cc806b4b333c2e707e7a9291c3d84664
Author:     Sam James <[e-mail]>
AuthorDate: 2023-12-10 22:13:09 +0000
Commit:     Sam James <[e-mail]>
CommitDate: 2023-12-10 22:29:08 +0000

    _emerge: BinpkgVerifier: give better error message on stale binpkg index
    
    portage-3.0.52 defaults to FEATURES="pkgdir-index-trusted" (see NEWS) which
    has a few benefits, but means that manually editing PKGDIR without regenerating
    the index with 'emaint binhost -f' will confuse Portage.
    
    Give a better error message mentioning that command if we fail to fetch a binpkg
    but bintree.dbapi.cpv_exists says it should exist.
    
    Bug: https://bugs.gentoo.org/915474
    Bug: https://bugs.gentoo.org/918597
    Signed-off-by: Sam James <[e-mail]>

 lib/_emerge/BinpkgVerifier.py | 22 +++++++++++++++++-----
 1 file changed, 17 insertions(+), 5 deletions(-)

[Comment 5 Larry the Git Cow 2023-12-10 22:35:36 UTC]

The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0d365d80099d206e49b592abb30030642f8f09f9

commit 0d365d80099d206e49b592abb30030642f8f09f9
Author:     Sam James <[e-mail]>
AuthorDate: 2023-12-10 22:34:47 +0000
Commit:     Sam James <[e-mail]>
CommitDate: 2023-12-10 22:34:52 +0000

    sys-apps/portage: add 3.0.57
    
    Closes: https://bugs.gentoo.org/918929
    Closes: https://bugs.gentoo.org/913628
    Closes: https://bugs.gentoo.org/915474
    Closes: https://bugs.gentoo.org/918597
    Closes: https://bugs.gentoo.org/919072
    Closes: https://bugs.gentoo.org/919105
    Closes: https://bugs.gentoo.org/919174
    Closes: https://bugs.gentoo.org/919311
    Closes: https://bugs.gentoo.org/919419
    Closes: https://bugs.gentoo.org/919668
    Signed-off-by: Sam James <[e-mail]>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-3.0.57.ebuild | 242 +++++++++++++++++++++++++++++++++
 2 files changed, 243 insertions(+)