Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 717216 - sys-apps/hwids vs. the proliferation of sys-apps/hwdata*
Summary: sys-apps/hwids vs. the proliferation of sys-apps/hwdata*
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo's Team for Core System packages
URL: https://github.com/vcrhonek/hwdata
Whiteboard:
Keywords: PullRequest
Depends on: 828474
Blocks:
  Show dependency tree
 
Reported: 2020-04-12 13:38 UTC by Andreas Sturmlechner
Modified: 2021-12-18 04:39 UTC (History)
8 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Andreas Sturmlechner gentoo-dev 2020-04-12 13:38:03 UTC
In short, I'm confused.

We have sys-apps/hwids which provides our pci.ids and usb.ids.

sys-apps/hwdata was once (2006) removed and superseded by sys-apps/hwdata-gentoo. Which still contains a blocker against sys-apps/hwdata-redhat, a package dropped only relatively recently in commit 3012cbdf.


Now, Plasma 5.19 is going to require "new" package hwdata (which duplicates pci.ids and usb.ids it seems), moved from former fedora-hosted to github - upstream link: https://github.com/vcrhonek/hwdata

Specifically, we need pnp.ids for KWin. Is it realistic to include that file in hwids?
Comment 1 Mike Gilbert gentoo-dev 2020-12-07 17:39:33 UTC
So it looks like the source of data for that pnp.ids file is an Excel workbook hosted on download.microsoft.com.

https://github.com/vcrhonek/hwdata/blob/master/Makefile#L118

The data seems very static: I doubt any vendors are inventing new ISA PnP ids.

I do wonder why KWin needs such obsolete data. Who is running Plasma on an ISA system?

We can add a static copy to hwids if needed.
Comment 2 Andreas Sturmlechner gentoo-dev 2020-12-07 18:28:16 UTC
The commit introduced monitor vendor decoding from EDID:

https://invent.kde.org/plasma/kwin/-/commit/33a1777a5ab0668da997e18173f4a50643748f1c
https://phabricator.kde.org/D10041
Comment 3 Mike Gilbert gentoo-dev 2020-12-07 19:33:49 UTC
Ah ha!

According to this Wikipedia page, the UEFI forum is the authority on assigning PnP ids.

https://en.wikipedia.org/wiki/Extended_Display_Identification_Data#Structure,_version_1.4

As far as I can tell, the list only seems to be available as an HTML document masquerading as an Excel workbook.

https://www.uefi.org/pnp_id_list

https://www.uefi.org/uefi-pnp-export

I suppose it would be simple enough to write a python script to extract the data.
Comment 4 Pino Toscano 2021-10-23 15:46:04 UTC
Earlier this year, hwdata changed its source for pnp.ids to the UEFI table -- see:
https://github.com/vcrhonek/hwdata/issues/4
https://github.com/vcrhonek/hwdata/commit/4edeb518d114ee364008b1196474d3faddf51451

So let me provide few points more:
- hwdata provides iab.txt, oui.txt, pci.ids, pnp.ids, usb.ids
- hwdata is actively maintained (by a Red Hat employee)
- hwdata is updated and released every month, usually at the beginning of each month
- it is easy to provide feedback to hwdata (see the hwdata issue #4, reported by me), or get fixes there (see the pnp.ids.patch in the repository)
- the installed size of hwdata is smaller than the sys-apps/hwids one

Hence, IMHO would be a good idea for Gentoo to:
a) package hwdata as it is
b) drop from sys-apps/hwids what hwdata provides
c) make sys-apps/hwids depend on hwdata (to avoid breakages)
d) slowly file bugs and transition packages to depend on hwdata directly if they need it

It would be much easier to just update hwdata from upstream every month, rather than maintaining a separate project just for some of these files (especially when hwids is not updated regularly...).
Especially that there are derivative distros of Gentoo: switching to a regularly updated hwdata will benefit them as well.

What do you think?

Pino (in this case, with the hat of the Debian maintainer of hwdata)
Comment 5 Mike Gilbert gentoo-dev 2021-11-24 18:06:50 UTC
(In reply to Pino Toscano from comment #4)

I think this makes sense.

I think the only stuff that hwids provides that hwdata does not is the udev hwdb data. We could probably migrate that back into the udev/systemd packages in Gentoo.
Comment 6 Mike Gilbert gentoo-dev 2021-11-24 19:46:06 UTC
I will plan on working on this soon, once eudev is masked.
Comment 7 Arsen Arsenović gentoo-dev 2021-11-25 21:31:46 UTC
Maybe hwdb can be factored out for reuse in eudev? If it can't right now, I'm pretty sure I'll be able to make that work, since I plan to rebase eudev on top of systemd again. Though, I'd say that's in the middle-distant future, because I'm quite busy right now.

I can look into factoring it out in a few days.
Comment 8 Larry the Git Cow gentoo-dev 2021-11-28 20:08:54 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=8325391d6a8592bf7b3661f8dc47cfc74994c217

commit 8325391d6a8592bf7b3661f8dc47cfc74994c217
Author:     Mike Gilbert <floppym@gentoo.org>
AuthorDate: 2021-11-24 20:32:10 +0000
Commit:     Mike Gilbert <floppym@gentoo.org>
CommitDate: 2021-11-28 19:54:00 +0000

    sys-apps/hwdata: new package
    
    Bug: https://bugs.gentoo.org/717216
    Signed-off-by: Mike Gilbert <floppym@gentoo.org>

 sys-apps/hwdata/Manifest            |  1 +
 sys-apps/hwdata/hwdata-0.353.ebuild | 24 ++++++++++++++++++++++++
 sys-apps/hwdata/metadata.xml        |  7 +++++++
 3 files changed, 32 insertions(+)
Comment 9 Larry the Git Cow gentoo-dev 2021-11-29 13:31:52 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/kde.git/commit/?id=47b618343c70b76a5b62ce4481c17824a72d4fd2

commit 47b618343c70b76a5b62ce4481c17824a72d4fd2
Author:     Andreas Sturmlechner <asturm@gentoo.org>
AuthorDate: 2021-11-29 13:19:23 +0000
Commit:     Andreas Sturmlechner <asturm@gentoo.org>
CommitDate: 2021-11-29 13:20:35 +0000

    kde-plasma/kwin: Add missing RDEPEND on sys-apps/hwdata
    
    Upstream commit 33a1777a5ab0668da997e18173f4a50643748f1c
    
    Bug: https://bugs.gentoo.org/717216
    Package-Manager: Portage-3.0.28, Repoman-3.0.3
    Signed-off-by: Andreas Sturmlechner <asturm@gentoo.org>

 kde-plasma/kwin/kwin-5.23.49.9999.ebuild | 7 ++-----
 kde-plasma/kwin/kwin-9999.ebuild         | 7 ++-----
 2 files changed, 4 insertions(+), 10 deletions(-)
Comment 10 Mike Gilbert gentoo-dev 2021-12-08 21:10:22 UTC
(In reply to Larry the Git Cow from comment #9)

For kwin, I think hwdata should be added to DEPEND as well since the build system looks for the file at build time.
Comment 11 Mike Gilbert gentoo-dev 2021-12-08 21:12:35 UTC
Alternatively, you should pass the path to cmake (hwdata_FOUND, hwdata_DIR, hwdata_PNPIDS_FILE).
Comment 12 Andreas Sturmlechner gentoo-dev 2021-12-09 08:01:58 UTC
(In reply to Mike Gilbert from comment #10)
> (In reply to Larry the Git Cow from comment #9)
> 
> For kwin, I think hwdata should be added to DEPEND as well since the build
> system looks for the file at build time.
It is explicitly marked as runtime dep upstream.

> find_package(hwdata)
> set_package_properties(hwdata PROPERTIES
>     TYPE RUNTIME
>     PURPOSE "Runtime-only dependency needed for mapping monitor hardware vendor IDs to full names"
>     URL "https://github.com/vcrhonek/hwdata"
> )
Comment 13 Andreas Sturmlechner gentoo-dev 2021-12-09 10:02:36 UTC
As a small update on the bug description, sys-apps/hwdata-gentoo was treecleaned in bug 755617.

Shall it be closed or do you want to keep it open for the cleanup of sys-apps/hwids?
Comment 14 Mike Gilbert gentoo-dev 2021-12-09 16:15:17 UTC
(In reply to Andreas Sturmlechner from comment #12)
> It is explicitly marked as runtime dep upstream.

Maybe I misunderstand how cmake works, but doesn't the code in Findhwdata.cmake execute at build time, regardless of how the dependency has been labeled?
Comment 15 Andreas Sturmlechner gentoo-dev 2021-12-09 16:17:05 UTC
That's how it works, but it is of no consequence if missing. Basically a packagers notice, in order to get listed in FeatureSummary when running cmake.
Comment 16 Larry the Git Cow gentoo-dev 2021-12-18 04:39:36 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=a86a1f553906b5caac4ead5e176442bf326d75db

commit a86a1f553906b5caac4ead5e176442bf326d75db
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2021-12-18 03:35:18 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2021-12-18 04:39:25 +0000

    sys-fs/eudev: update live ebuild for hwids/hwdb changes upstream
    
    eudev upstream now includes updated hwdb files and shouldn't need hwids.
    
    Bug: https://bugs.gentoo.org/717216
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-fs/eudev/eudev-9999.ebuild | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)