Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 877357 - sys-apps/portage-3.0.38.1: backtrace when asking to emerge -1av libva-compat
Summary: sys-apps/portage-3.0.38.1: backtrace when asking to emerge -1av libva-compat
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Binary packages support (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on: 873757
Blocks:
  Show dependency tree
 
Reported: 2022-10-16 23:34 UTC by Matthew Thode ( prometheanfire )
Modified: 2022-11-20 03:05 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
backtrace.txt (file_877357.txt,2.42 KB, text/plain)
2022-10-16 23:34 UTC, Matthew Thode ( prometheanfire )
Details
libva-compat binpkgs (libva-binpkgs.tar.xz,892.51 KB, application/x-xz)
2022-10-16 23:37 UTC, Matthew Thode ( prometheanfire )
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Matthew Thode ( prometheanfire ) archtester Gentoo Infrastructure gentoo-dev Security 2022-10-16 23:34:51 UTC
Created attachment 824477 [details]
backtrace.txt

This seems to be triggered by the package move when binpkgs exist
This error occurred before dispatch-conf was run to update for the libva package move.
Comment 1 Matthew Thode ( prometheanfire ) archtester Gentoo Infrastructure gentoo-dev Security 2022-10-16 23:37:21 UTC
Created attachment 824479 [details]
libva-compat binpkgs
Comment 2 John Helmert III archtester Gentoo Infrastructure gentoo-dev Security 2022-10-17 02:45:23 UTC
The relevant snippet from bintree.py:

            try:
                binpkg_format = cpv.binpkg_format
            except AttributeError:
                # In order to force the caller to clarify its intent, do not
                # use default BINPKG_FORMAT unless allocate_new is True.
                # The caller can set cpv.binpkg_format in advance if something
                # other than the default is desired here.
                if allocate_new:
                    binpkg_format = self.settings.get(
                        "BINPKG_FORMAT", SUPPORTED_GENTOO_BINPKG_FORMATS[0]
                    )
                else:
                    binpkg_format = None

            if not binpkg_format:
                # Raise an error if the desired binpkg_format is not clear.
                # The caller should either set allocate_new to True or else
                # ensure that cpv.binpkg_format is set to a particular format.
                raise InvalidBinaryPackageFormat(binpkg_format)

InvalidBinaryPackageFormat is a subclass of PortageException, the contructor of which is:

    def __init__(self, value):
        self.value = value[:]

So the binpkg triggers the else: condition, making binpkg_format=None which falls over when it tries to calculate None[:]. I suppose a naive fix would just be:

diff --git a/lib/portage/dbapi/bintree.py b/lib/portage/dbapi/bintree.py
index c8bb2c88e..fec9d7705 100644
--- a/lib/portage/dbapi/bintree.py
+++ b/lib/portage/dbapi/bintree.py
@@ -2091,7 +2091,7 @@ class binarytree:
                         "BINPKG_FORMAT", SUPPORTED_GENTOO_BINPKG_FORMATS[0]
                     )
                 else:
-                    binpkg_format = None
+                    binpkg_format = ""

             if not binpkg_format:
                 # Raise an error if the desired binpkg_format is not clear.
Comment 3 John Helmert III archtester Gentoo Infrastructure gentoo-dev Security 2022-10-30 03:46:09 UTC
Hm, that patch still makes it backtrace, but this time it successfully throws the portage.exception.InvalidBinaryPackageFormat rather than erroring when trying, which looks like what's supposed to happen.

I guess it's troubling that these invalid binpkgs were generated in the first place?
Comment 4 Sheng Yu 2022-11-01 14:29:39 UTC
Kind weird, you had correct Package Index and correct binpkg, but portage report cannot find format.

This should be fixed in next release as we deprecated BINPKG_FORMAR in metadata.
Comment 5 Sheng Yu 2022-11-01 16:36:43 UTC
Ok, this one is 2 bugs.

The first one is binpkg_format = None that causes raise InvalidBinaryPackageFormat(binpkg_format) failed.

The second one is something not related to binpkg but index:

They moved x11-libs/libva-compat to media-libs/libva-compat but kept package name same. This (rare?) action make the portage package search function confused, as it found 2 packages named libva-compat and try to print the list.

emerge -ak libva-compat

...

[ Results for search key : libva-compat ]
pkg media-libs/libva-compat
*  media-libs/libva-compat
      Latest version available: 1.8.3-r2
      Latest version installed: [ Not Installed ]
      Size of files: 183 KiB
      Homepage:      https://01.org/linuxmedia/vaapi
      Description:   Video Acceleration (VA) API for Linux
      License:       MIT

pkg x11-libs/libva-compat
*  x11-libs/libva-compat
None
... done!

Since x11-libs/libva-compat no longer exists, portage internal dataset is None, and it try to get a file path for None, and you run into this error.

Do a eclean-pkg should able fix this issue.

Fix WIP.
Comment 6 Larry the Git Cow gentoo-dev 2022-11-02 22:58:21 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=445f10f4214c673f8fe0a9cc518c12767be4f159

commit 445f10f4214c673f8fe0a9cc518c12767be4f159
Author:     Sheng Yu <syu.os@protonmail.com>
AuthorDate: 2022-11-02 18:46:03 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-11-02 22:58:15 +0000

    Use binpkg extensions and header to get format
    
    This deprecates the BINPKG_FORMAT in the metadata and Package index.
    
    Also fixed a bug that obsolete packages that have been moved may be
    attempted to be read during package searching.
    
    Closes: https://bugs.gentoo.org/877357
    Closes: https://bugs.gentoo.org/877419
    Signed-off-by: Sheng Yu <syu.os@protonmail.com>
    Closes: https://github.com/gentoo/portage/pull/928
    Signed-off-by: Sam James <sam@gentoo.org>

 lib/_emerge/BinpkgExtractorAsync.py |   4 +-
 lib/_emerge/BinpkgFetcher.py        |  32 ++------
 lib/_emerge/Package.py              |   1 -
 lib/_emerge/actions.py              |  15 +++-
 lib/_emerge/depgraph.py             |  15 +++-
 lib/_emerge/search.py               |   6 +-
 lib/portage/binpkg.py               |  23 ++++--
 lib/portage/dbapi/bintree.py        | 142 +++++++++++++++---------------------
 lib/portage/gpkg.py                 |   4 +
 lib/portage/versions.py             |  16 ----
 lib/portage/xpak.py                 |  18 +++++
 11 files changed, 141 insertions(+), 135 deletions(-)
Comment 7 Larry the Git Cow gentoo-dev 2022-11-08 23:07:51 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=a7dd39c1ae4a5ea4e3252ae8129fbd671c95d5f7

commit a7dd39c1ae4a5ea4e3252ae8129fbd671c95d5f7
Author:     Sheng Yu <syu.os@protonmail.com>
AuthorDate: 2022-11-08 22:52:46 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-11-08 23:07:46 +0000

    Do not drop default PATH in the package index (avoid Packages regeneration)
    
    The new Packages index (introduced in 445f10f4214c673f8fe0a9cc518c12767be4f159)
    needs PATH but it got dropped later on when processing, so we would try
    to rebuild the index on every emerge call.
    
    This fixes regenerating Packages loop.
    
    (Note that this didn't affect a released version.)
    
    Fixes: 445f10f4214c673f8fe0a9cc518c12767be4f159
    Bug: https://bugs.gentoo.org/877357
    Bug: https://bugs.gentoo.org/877419
    Signed-off-by: Sheng Yu <syu.os@protonmail.com>
    Closes: https://github.com/gentoo/portage/pull/934
    Signed-off-by: Sam James <sam@gentoo.org>

 lib/portage/dbapi/bintree.py | 21 +++------------------
 1 file changed, 3 insertions(+), 18 deletions(-)
Comment 8 Larry the Git Cow gentoo-dev 2022-11-20 03:05:54 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=203e0d0083b5e7d8098785ec8862c02f6baf5534

commit 203e0d0083b5e7d8098785ec8862c02f6baf5534
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-11-20 03:04:40 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-11-20 03:05:40 +0000

    sys-apps/portage: add 3.0.39
    
    Closes: https://bugs.gentoo.org/365655
    Closes: https://bugs.gentoo.org/757525
    Closes: https://bugs.gentoo.org/871573
    Closes: https://bugs.gentoo.org/875812
    Closes: https://bugs.gentoo.org/875860
    Closes: https://bugs.gentoo.org/877215
    Closes: https://bugs.gentoo.org/877271
    Closes: https://bugs.gentoo.org/877357
    Closes: https://bugs.gentoo.org/877419
    Closes: https://bugs.gentoo.org/873757
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-3.0.39.ebuild | 273 +++++++++++++++++++++++++++++++++
 2 files changed, 274 insertions(+)