Bug 935797 – Future of KDE Phonon Backends and HW acceleration on Intel GPU

• Product: Gentoo Linux
• Status: RESOLVED FIXED

[Description Matthias Nagel 2024-07-09 18:16:08 UTC]

This bug report is not a bug report for a specific package, but for a unfortunate combination of packages. Also the bug can probably not be fixed within Gentoo, but needs to be fixed upstream. Moreover, there is not a clear, single solution and I do not feel like even recommending one. This report is supposed to create awareness that "something" needs to be done.

The issue is a result of the following (more or less independent) events

 - transition from Qt5 to Qt6 and Plasma/KDE 6 being at the horizon
 - missing Qt6 support for Gstreamer and its plugins
 - VLC 3.x either depending on (old) FFMPEG 4 or lack of VA API-based HW acceleration with newer FFMPEG versions
 - VLC 4 not being production ready
 - missing VDPAU support on Intel GPUs

Phonon (media-libs/phonon) is the unified KDE multimedia framework. Most KDE apps rely on Phonon for generation of thumbnails for media files or video playback (e.g. thumbnails in Dolphin, video playback with DragonPlayer, etc.) Phonon does not implement playback capabilities itself, but is only a unified wrapper and relies on backends for the actual decoding. For a long time, Gstreamer had been the recommended Phonon backend (media-libs/phonon-gstreamer) and VLC had been an alternative (media-libs/phonon-vlc).

However, Gstreamer has not been ported to Qt6 and hence has become deprecated as a Phonon backend. At the moment, VLC is the only available Phonon backend. This circumstance is also reflected by the latest change in the use flags for media-libs/phonon. The use flags "gstreamer" and "vlc" have been dropped and replaced by "minimal" (see https://bugs.gentoo.org/935033#c9).

On Linux, HW accelerated video decoding and rendering is either provided through VDPAU or VA API. However, the Intel GPU drivers only support VA API.

If one enables support for VA API via enabling the corresponding use flag, VLC (media-video/vlc) requires FFMPEG 4. Between major versions FFMpeg changed its API calls for VA API support, and VLC seem mistakenly to believe that FFMpeg dropped VA API support (see https://bugs.gentoo.org/864721#c11) or have at least no interest in fixing that issue.

VLC 4 seems to be the only version which get fixes for dependencies, but it is on a pre-alpha stage for years (see https://bugs.gentoo.org/864721#c13).

This leaves one with two choices on Intel GPUs: either building  VLC 3 with HW acceleration and being stuck with FFMPEG 4 or using newer version of FFMPEG but disable HW acceleration.

The whole situation is somewhat unfortunate. I see a couple of options here, but most of the options require upstream support and cannot be solved by Gentoo alone:

 - Somehow convince Gstreamer to support Qt6 (very unlikely) and revive Gstreamer as a Phonon backend. Nice side effect: no need for the VLC player on a pure KDE installation
 - Somehow convince VLC to fix the VA API bug in VLC 3 for newer versions of FFMPEG (probably also unlikely)
 - Support MPV (media-video/mpv) as an alternative Phonon backend (see https://github.com/OpenProgger/phonon-mpv). Arch Linux does that inofficially, see https://aur.archlinux.org/packages/phonon-qt6-mpv.
 - Support VDPAU over VA API via libvdpau_va_gl.so. This would allow VLC to use VDPAU calls which then translates into VA API calls for Intel GPU. The library is already available through a Gentoo overlay repository, see http://gpo.zugaina.org/x11-libs/libvdpau-va-gl.
 - Make FFMPEG slotted. This would allow to use FFMPEG 4 with VLC and still update to FFMPEG 6 for all other packages.



Reproducible: Always

[Comment 1 Mart Raudsepp 2024-07-10 19:39:14 UTC]

GStreamer provides Qt6 support fine, and we could package it, but it has nothing to do with phonon backends, as that one is about Qt6 QML plugin, i.e. rendering video with gstreamer into a QML scene graph or something.

For phonon, it'd have to be a phonon gstreamer backend that is functioning for Qt6. Honestly, I don't understand why they wouldn't have that support as primary, but I guess I'm biased. Then again, I wouldn't use phonon but rather gstreamer directly :)

Anyhow, tl;dr: nothing for gstreamer@g.o to do here, as the equivalent of bug 810814 but for the qt6 variant won't help here in any way.

[Comment 2 Matthias Nagel 2024-07-11 16:59:51 UTC]

Sorry, for the technical impreciseness. When I wrote Gstreamer has not been ported to Qt6, I actually meant that the Gstreamer-based Phonon backend has not been ported to Qt6. However, from a user's perspective this is rather a technicality, but the general issue still stands.

We are heading towards the unfortunate situation that KDE users either loose HW-accelerated video decoding and rendering or are stuck with old dependencies which sooner or later will become outdated.

You (@Mart Raudsepp) write that you would rather use Gstreamer directly than via Phonon. You won't hear any objections from my side. You might be right and one layer of abstraction on top of another layer of abstraction has become simply one layer too much to maintain. (If I am not mistaken, Gstreamer is also only an abstraction layer on top of other multimedia libraries such as FFMPEG.) However, that is nothing what users (or package maintainers) can change. The KDE project decided to invent Phonon as another abstraction layer and that is the situation we have to face.

That is why I wrote that I don't have a single, good solution or clear way forward. We either need a new Phonon backend (maybe based on MPV), or get the Gstreamer-based Phonon backend ported to Qt6 or get VLC 3 (or 4) to support VA API with FFMPEG 6.

[Comment 3 Sam James 2024-07-11 17:04:22 UTC]

Yes, it's a horrible mess. I appreciate you filing it and I agree that the situation is pretty poor.

phonon-mpv isn't perfect but forcing people onto VLC when it's incompatible with modern ffpmeg is also asking for trouble.

I don't know what to do yet. There's some progress upstream on a backport to VLC 3.x finally but I don't know if it's going to go anywhere.

[Comment 4 Andreas Sturmlechner 2024-07-11 18:04:17 UTC]

I'm not sure why we need another bug next to bug 864721?

[Comment 5 Matthias Nagel 2024-07-11 18:16:13 UTC]

I posted this bug in order to consolidate the issue. Any of the other, related issues only looks at a single specific package, like the bug you mentioned. While it is problematic for its own sake that VLC does not support VA API for FfMPEG >4, this wouldn't be such a big problem if there were viable alternatives.

Hence, my intention was to provide the bigger picture and also show that there are several solutions out of this unfortunate situation. I listed some at the end of my initial post.

[Comment 6 Gino McCarty 2024-10-06 06:39:16 UTC]

Been using phonon-mpv for some time works great so far. Would make a nice alternative to VLC.

https://github.com/OpenProgger/phonon-mpv

[Comment 7 Sam James 2024-10-06 06:45:31 UTC]

IIRC parona reproduced the crash in https://github.com/OpenProgger/phonon-mpv/issues/20 or something similar which put us off.

[Comment 8 Steven Newbury 2024-10-22 16:00:15 UTC]

phonon-gstreamer was mostly ported to qt6 in 2022, although no 4.11.0 release ever materialised.  It was insufficient by the time was needed, but in the Arch AUR there's a patch which fixes the few build issues:
https://aur.archlinux.org/cgit/aur.git/?h=phonon-qt6-gstreamer-git

It's been reported to work well with Arch, and I've created an ebuild which I've successfully built, and will be testing it.

It seems like there must be some vested interest in using VLC as the phonon (and kalarm) backend, I've no idea what's behind it but there wasn't a lot keeping phonon-qt6-backend-gstreamer from working.

It seems quite problematic to me to force Gentoo Plasma users to use ffmpeg-4, and prevent anything depending on modern ffmpeg from being emerged.

[Comment 9 Andreas Sturmlechner 2024-12-16 18:16:20 UTC]

(In reply to Matthias Nagel from comment #5)
> Hence, my intention was to provide the bigger picture and also show that
> there are several solutions out of this unfortunate situation. I listed some
> at the end of my initial post.
Well, has there anything changed on the upstream sides of these "solutions"?

My point previously was that, as far as things under our downstream influence, the hot-topic-bug 864721 had already been made a blocker against bug 935416, so you know we are aware of the implications for KDE Plasma/Gear users.

(In reply to Matthias Nagel from comment #0)
>  - Support VDPAU over VA API via libvdpau_va_gl.so. This would allow VLC to
> use VDPAU calls which then translates into VA API calls for Intel GPU. The
> library is already available through a Gentoo overlay repository, see
> http://gpo.zugaina.org/x11-libs/libvdpau-va-gl.
Of all the listed options, this is the one that at least sounds (if not too good to be true) the most viable for a downstream to provide, without becoming the new upstream of a dead-end package or forking ffmpeg +all revdep code depending on it into a slottable solution. But even if this is tangible at all, it takes someone to put the effort in it to show that it works and how a downstream can provide it as an alternative to ffmpeg[vaapi].

But also in the meantime, we've just dropped avcodec_vaapi from VLC.

[Comment 10 Andreas Sturmlechner 2025-01-28 21:27:30 UTC]

Let me close this bug with this summary:

- VLC 3.0.x is now fine with >=ffmpeg-5
- phonon is undoubtedly on its way out, but in the long term