Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 698988 - dev-qt/qtwebengine:5 should support Python 3
Summary: dev-qt/qtwebengine:5 should support Python 3
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All All
: Normal normal with 5 votes (vote)
Assignee: Qt Bug Alias
URL:
Whiteboard: fixed in 5.15.3
Keywords: InOverlay
Depends on: qt-5.15.3-stable
Blocks: py3-tracker, python-3-incompatible
  Show dependency tree
 
Reported: 2019-10-31 04:11 UTC by Arfrever Frehtes Taifersar Arahesis
Modified: 2022-04-15 23:20 UTC (History)
45 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
QtWebEngine 5.15.3.9999 ebuild to support python 3 (qtwebengine-5.15.3.9999.ebuild,9.04 KB, text/plain)
2022-04-04 01:31 UTC, Tiernan Hubble
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Arfrever Frehtes Taifersar Arahesis 2019-10-31 04:11:32 UTC
Chromium upstream is working on porting Chromium to Python 3. See bug #698974.
Comment 2 Marc Grondin 2020-11-08 18:23:34 UTC
So i am but a common user but i am trying to purge python 2.7 from my system. QTWebengine is that final thing requiring it.

Using a slightly modified version of the patch linked by Andreas i can get it to patch but the configure step still fails with this

https://paste.marcg.pizza/~marcg/b3713e3ebd50670e784364108ce79ecc087ab554

The modified patch simply changes some of the line offsets
https://git.marcg.pizza/~marcg/thebeach/blob/master/dev-qt/qtwebengine/files/qtwebengine-5.15.1-python3-build.patch

I've seen reports that doing
sed -i 's|python_major_version, 2|python_major_version, 3|' configure.pri
was all that was required but that does not help in my case.

What am i missing here? Is it caused by dependencies in other parts of the QT framework/build system.

This is the ebuild i am using
https://git.marcg.pizza/~marcg/thebeach/blob/master/dev-qt/qtwebengine/qtwebengine-5.15.1-r1.ebuild
Comment 3 Chiitoo gentoo-dev 2020-11-08 19:41:43 UTC
The main issue is Chromium, which as far as I understand still has quite a bit of work to be done towards making this possible.
Comment 4 perestoronin 2020-11-30 04:29:45 UTC
qtwebengine updated to 5.15.2 and patches obsolete from this bug.

Please update patches to qtwebengine 5.15.2 (for chromium + python 3)
Comment 5 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2020-11-30 04:31:46 UTC
(In reply to perestoronin from comment #4)
> qtwebengine updated to 5.15.2 and patches obsolete from this bug.
> 
> Please update patches to qtwebengine 5.15.2 (for chromium + python 3)

The patches are from FreeBSD.
Comment 6 Manfred Knick 2021-01-31 11:07:07 UTC
(In reply to Chiitoo from comment #3)
> The main issue is Chromium, ...
Well, I have no chromium installed on my system,
and can't detect any dependency upon it.

Hard-coded in    qtwebengine-5.15.2.ebuild :

. . . PYTHON_COMPAT=( python2_7 )

Can anybody confirm off by heart
if this is still necessary for latest version -5.15.2 ?
(Test-compile takes hours, unfortunately ...)



(In reply to Marc Grondin from comment #2)
> ... i am trying to purge python 2.7 from my
> system. QTWebengine is that final thing requiring it.

+ 1
Comment 7 Manfred Knick 2021-01-31 11:14:40 UTC
(In reply to Manfred Knick from comment #6)

> if this is still necessary for latest version -5.15.2 ?

Sorry for the noise!
Just found it, finally:


All Platforms

On all platforms, the following tools are required at build time:

    Python 2.7.5 or later. Python 3 is not supported.               <--- !
    ...

[ https://doc.qt.io/qt-5/qtwebengine-platform-notes.html#linux ]
Comment 8 Chiitoo gentoo-dev 2021-01-31 11:40:20 UTC
(In reply to Manfred Knick from comment #6)
> (In reply to Chiitoo from comment #3)
> > The main issue is Chromium, ...
> Well, I have no chromium installed on my system,
> and can't detect any dependency upon it.

Right, but Qt WebEngine uses Chromium "inside", and as long as Chromium requires Python 2 to build, Qt WebEngine will require Python 2 to build.
Comment 9 Julien Delquié 2021-02-03 17:43:28 UTC
According to the Implementation support timeline [1], what will happen after 2021-02-15 (Package removal)? No more build of dev-qt/qtwebengine?

Today, here is the list of installed packages that depend on it, on my system:
dev-qt/qtwebengine-5.15.2 pulled in by:
    app-text/calibre-5.6.0 requires >=dev-qt/qtwebengine-5.12
    dev-python/PyQtWebEngine-5.15.2 requires dev-qt/qtwebengine:5[widgets]
    kde-apps/kaccounts-providers-20.12.1 requires >=dev-qt/qtwebengine-5.15.1:5
    kde-apps/kalgebra-20.12.1 requires >=dev-qt/qtwebengine-5.15.1:5[widgets]
    kde-apps/kimagemapeditor-20.12.1 requires >=dev-qt/qtwebengine-5.15.1:5[widgets]
    kde-apps/ktp-text-ui-20.12.1 requires >=dev-qt/qtwebengine-5.15.1:5[widgets]
    kde-apps/marble-20.12.1 requires >=dev-qt/qtwebengine-5.15.1:5[widgets]
    kde-apps/parley-20.12.1-r1 requires >=dev-qt/qtwebengine-5.15.1:5[widgets]
    media-gfx/digikam-7.1.0-r1 requires >=dev-qt/qtwebengine-5.12.3:5[widgets]
    net-libs/signon-ui-0.15_p20171022 requires dev-qt/qtwebengine:5
    net-p2p/ktorrent-20.12.1 requires >=dev-qt/qtwebengine-5.15.1:5
    www-client/falkon-3.1.0-r1 requires >=dev-qt/qtwebengine-5.12.3:5/5.15=[widgets], >=dev-qt/qtwebengine-5.12.3:5=[widgets]

I won't be able to build these or something after this date?


[1] https://wiki.gentoo.org/wiki/Project:Python/Implementations
Comment 10 Manfred Knick 2021-02-03 18:15:30 UTC
(In reply to Julien Delquié from comment #9)
> ...
> [1] https://wiki.gentoo.org/wiki/Project:Python/Implementations

. . . "Dates in bold indicate when events took place, 
. . .  dates in parentheses are planned dates."              <-----
Comment 11 Julien Delquié 2021-02-04 07:36:11 UTC
(In reply to Manfred Knick from comment #10)
> (In reply to Julien Delquié from comment #9)
> > ...
> > [1] https://wiki.gentoo.org/wiki/Project:Python/Implementations
> 
> . . . "Dates in bold indicate when events took place, 
> . . .  dates in parentheses are planned dates."              <-----

So, date will be changed when reached if nothing has been done upstream?
Comment 12 Andreas Sturmlechner gentoo-dev 2021-02-09 21:51:19 UTC
Voting here is cute, but we depend on upstream.
Comment 13 Julien Delquié 2021-02-10 07:26:04 UTC
(In reply to Andreas Sturmlechner from comment #12)
> Voting here is cute, but we depend on upstream.

I know about dependency on upstream.

But I don't know what is the strategy in this case specific case:
- try to implement an equivalent of the FreeBSD's KDE maintainer patch -> awesome but really risky imho, and I think this is clearly not the way Gentoo devs will choose,
- waiting for upstream to move to python 3 and have a new dev-qt/qtwebengine one day (+ keep python 2.x and current dev-qt/qtwebengine during this period) -> cool thing, and what I hope will happen, despite dates in parentheses (I know, it's just « planned »),
- just trash everything because this takes too long, and is irritating for everyone here -> the thing I really fear, here, but I could understand that this situation is boring, tiring, etc. for Gentoo devs.
Comment 14 Attila Tóth 2021-02-10 22:14:08 UTC
(In reply to Julien Delquié from comment #13)
> (In reply to Andreas Sturmlechner from comment #12)
> > Voting here is cute, but we depend on upstream.
> 
> I know about dependency on upstream.
> 
> But I don't know what is the strategy in this case specific case:
> - try to implement an equivalent of the FreeBSD's KDE maintainer patch ->
> awesome but really risky imho, and I think this is clearly not the way
> Gentoo devs will choose,
> - waiting for upstream to move to python 3 and have a new dev-qt/qtwebengine
> one day (+ keep python 2.x and current dev-qt/qtwebengine during this
> period) -> cool thing, and what I hope will happen, despite dates in
> parentheses (I know, it's just « planned »),
> - just trash everything because this takes too long, and is irritating for
> everyone here -> the thing I really fear, here, but I could understand that
> this situation is boring, tiring, etc. for Gentoo devs.

Quite a lot of change has been included in 5.15.2_p20210205 , which was addressed in the FreeBSD patch. Maybe there is a progress. But the build system still expects python2...
Comment 15 Seong-ho Cho 2021-05-09 05:49:41 UTC
two thumbs up. 

Python 2 is zombie that should be buried. :P
Comment 16 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2021-05-09 05:55:41 UTC
(In reply to Seong-ho Cho from comment #15)
> two thumbs up. 
> 
> Python 2 is zombie that should be buried. :P

We know, but this isn't really something we can do ourselves.
Comment 17 Dmitry Derevyanko 2021-05-09 07:05:11 UTC
Is it possible to add binary version of qtwebengine? dev-qt/qtwebengine-bin
Comment 18 water.devil 2021-06-18 06:37:59 UTC
binary version of qtwebengine would help great, because I'm no longer able to compile it. PC freezes.

In my case calibre depends on qtwebengine.

Or upgrade it. 1 year and 8 months seems long enough!
Many thanks
Comment 19 Manfred Knick 2021-06-18 08:02:49 UTC
(In reply to water.devil from comment #18)

> Or upgrade it. 1 year and 8 months seems long enough!
Let me kindly suppose to re-direct your anger and this complaint @ Qt,
because comments 7 and 8 above still prove valid.

Just check yourself:

. . . . . Python 2.7.5 or later. Python 3 is not supported.               <--- !

[ https://doc.qt.io/qt-5/qtwebengine-platform-notes.html#all-platforms ]
Comment 20 Thomas Bettler 2021-06-18 08:07:05 UTC
Things might not change until Qt includes chromium >= 93 for qtwebengine, wich might not happen within the next few months.

On the plus side, lets mention, latest chromium ebuilds work now without py2 :)
Comment 21 Thomas Bettler 2021-06-18 08:11:18 UTC
watch out for https://github.com/qt/qtwebengine-chromium/commits/upstream-master moving to chromium >= 93
Comment 22 Andreas Sturmlechner gentoo-dev 2021-09-29 17:46:54 UTC
We know that chromium has py3 support upstream.

That's irrelevant for qtwebengine:5 which remains stuch at chromium-87 with only security bugs being backported. Unless someone does the work and backports py3 support.
Comment 23 Thomas Bettler 2021-12-13 12:35:16 UTC
@asturm: I totally agree.

For the curious:
https://github.com/qt/qtwebengine/tree/6.3 includes support for chromium-94 thus builds without python-2. Although we will still have to wait until this version will be released and eventually lands in gentoo... let's be patient.
Comment 24 Michael 2021-12-13 12:57:58 UTC
(In reply to Thomas Bettler from comment #23)
> Although we will still have to wait until this
> version will be released and eventually lands in gentoo... 

And all BDEPS ported to it. It'll be years.
Comment 25 Manfred Knick 2021-12-13 13:06:00 UTC
Yes, indeed - Thomas is right, twice:

"This version of Qt WebEngine is based on Chromium version 90.0.4430, with additional security fixes from newer versions."
[ https://doc-snapshots.qt.io/qt6-dev/qtwebengine-overview.html#qt-webengine-core-module ]

As Michael just pointed out ("Mid-Air Collision"):
The vast majority of applications (including KDE, e.g.) still depends upon QT 5.
Comment 26 Michael 2021-12-13 13:20:48 UTC
So is the 6.3 branch going to be chromium 90 or 94?
Comment 27 Andreas Sturmlechner gentoo-dev 2021-12-13 13:31:25 UTC
No offtopic chatter please, this bug is about qtwebengine:5 which can't go away until revdeps are ported.
Comment 28 Andreas Sturmlechner gentoo-dev 2021-12-13 14:03:25 UTC
(In reply to Michael from comment #26)
> So is the 6.3 branch going to be chromium 90 or 94?

To answer your question anyway:
https://gitweb.gentoo.org/proj/qt.git/tree/dev-qt/qtwebengine/qtwebengine-6.9999.ebuild
Comment 29 Thomas Bettler 2021-12-13 21:09:24 UTC
@asturm: you'd like to look at qtwebengine:5 only ? - would you like to clarify the title?
Comment 30 Tiernan Hubble 2022-04-03 16:15:05 UTC
Looks like Arch found a way to get python 3 working with the latest qtwebengine 5.15.3: https://github.com/archlinux/svntogit-packages/tree/packages/qt5-webengine/trunk

The PKGBUILD does the following:
1. Applies "qt5-webengine-python3.patch" in the main source directory.
2. Applies "qt5-webengine-chromium-python3.patch" in the "src/3rdparty" subdirectory.
3. Completely removes the "src/3rdparty/chromium/third_party/catapult" directory and replaces it with a more recent git commit from the chromium repo: "https://chromium.googlesource.com/catapult#commit=5eedfe23148a234211ba477f76fc2ea2e8529189"

I hacked together an ebuild based on the "qtwebengine-5.15.3.9999.ebuild" Live ebuild from the "qt" repo. It worked perfectly, and I've now completely removed python2 from my system and rebuilt qtwebengine successfully twice. Unfortunately there's an unrelated issue where QtWebEngineProcess crashes immediately on loading a URL. It's definitely not related to python3 because it was happening before my change, I actually stumbled upon the Arch python3 fix when I was searching for a solution to that problem. I'm not sure if it's related to the Qt 5.15.3 update or using a newer git version of qtwebengine/chromium, still trying to track it down. Anyway...

The patches (and completely replacing a directory) are obviously huge, so I'm not sure if this is feasible for main Gentoo repos, but I'm posting this information in case anyone's interested. I have a local qtwebengine ebuild I've been using for a while, it supports building the whole thing with thinlto, and was working fine up until this latest update. Once I've tracked it down I might look at pushing it up somewhere.
Comment 31 Michael 2022-04-03 20:13:20 UTC
(In reply to Tiernan Hubble from comment #30)
> Looks like Arch found a way to get python 3 working with the latest
.............
> Once I've tracked
> it down I might look at pushing it up somewhere.

You can upload the ebuild here to bugzilla, I'd be happy to test it.
Comment 32 Tiernan Hubble 2022-04-04 01:31:57 UTC
Created attachment 768647 [details]
QtWebEngine 5.15.3.9999 ebuild to support python 3

Attached my modified version of qtwebengine-5.15.3.9999.ebuild (original here: https://github.com/gentoo/qt/blob/master/dev-qt/qtwebengine/qtwebengine-5.15.3.9999.ebuild ).

You can copy the entire "qtwebengine" directory from the Gentoo Qt repo ( https://github.com/gentoo/qt/tree/master/dev-qt/qtwebengine ) and then overwrite "qtwebengine-5.15.3.9999.ebuild" with this file.

You'll also need to put the following 2 patches from Arch in the "files" directory:
https://raw.githubusercontent.com/archlinux/svntogit-packages/packages/qt5-webengine/trunk/qt5-webengine-python3.patch
https://raw.githubusercontent.com/archlinux/svntogit-packages/packages/qt5-webengine/trunk/qt5-webengine-chromium-python3.patch

I haven't actually tested this particular version of the ebuild because I stripped out all the thinlto stuff I have in my local copy, that I'm still trying to get working. Apparently Clang 14 causes Javascript crashes in qtwebengine 5.x, even with conservative CFLAGS. It does build fine with GCC though. Doesn't have anything to do with python3, in any case.

This is the live (git) version that has all the latest security fixes. It should probably work with any reasonable recent qtwebengine-5.15.x version though. (Arch seems to use 5.15.8).
Comment 33 Michael 2022-04-04 03:35:45 UTC
I build-tested the ebuild and it certainly works. Clang-13, libstd++, generally uneventful. The only app that uses qtwebengine on my system is a chm viewer, so the runtime testing is limited to that, but it shows the CHMs no problem.
Comment 34 Andreas Sturmlechner gentoo-dev 2022-04-05 11:13:03 UTC
(In reply to Tiernan Hubble from comment #32)
> Created attachment 768647 [details]
> QtWebEngine 5.15.3.9999 ebuild to support python 3
You've made unrelated changes to that ebuild so I can't really pick it, but I am going to look at the modifications Arch have made.
Comment 35 Tiernan Hubble 2022-04-05 13:54:08 UTC
(In reply to Andreas Sturmlechner from comment #34)
> You've made unrelated changes to that ebuild so I can't really pick it, but
> I am going to look at the modifications Arch have made.

The only unrelated changes I made were to put some source file names into variables, because I needed to completely override src_unpack in order to control how and where the catapult archive is unpacked (it needs to be in a specific directory in the existing qtwebengine/chromium source, and it needs to remove the contents of the existing directory first). So things started to get unmaintainable with having to copy/paste these filenames in multiple methods.

Ideally it would be better to find a way to make it work without having to override src_unpack, but I couldn't find a way to do that.
Comment 36 Michael 2022-04-05 14:12:00 UTC
(In reply to Tiernan Hubble from comment #35)
> Ideally it would be better to find a way to make it work without having to
> override src_unpack, but I couldn't find a way to do that.

Someone told me on IRC that qtwebengine doesn't really use catapult, so, possibly it could be wiped out and the package built without it. Just an idea, I don't really know how difficult it would be to disable it in the build system.
Comment 37 Tiernan Hubble 2022-04-05 17:51:59 UTC
(In reply to Michael from comment #36)
> Someone told me on IRC that qtwebengine doesn't really use catapult, so,
> possibly it could be wiped out and the package built without it. Just an
> idea, I don't really know how difficult it would be to disable it in the
> build system.

I tried with just the 2 Arch patches and not the Catapult replacement, and I got Catapult errors. I'm not sure how tighly integrated Catapult is with the qtwebengine/chromium builds, but it definitely needs either Catapult replaced, or another patch to disable the parts of the build that call it.
Comment 38 Andreas Sturmlechner gentoo-dev 2022-04-05 18:00:14 UTC
https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=a5fb9285a1dd3bc61a9ab25b488806b869d2cb81

profiles: Mask dev-qt/qtwebengine-5.15.3_p20220330 for now

--- a/profiles/package.mask
+++ b/profiles/package.mask
 
+# Andreas Sturmlechner <asturm@gentoo.org> (2022-04-05)
+# First version with py3 buildsystem support.
+# May be succeeded quickly by another snapshot.
+~dev-qt/qtwebengine-5.15.3_p20220330


https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=040d7675bae6b7af62f251b8fbab4dde4df81a19

dev-qt/qtwebengine: 5.15.3_p20220330 version bump, py3

Patched with security patches up to Chromium version:
98.0.4758.102 plus fixes for CVE-2022-0971, CVE-2022-1096

Patches sourced from ArchLinux:
- Enable build with >=python-3
- Fix build with >=ffmpeg-5
- Enable screencast support using pipewire-3

Snapshotted at:
Branch: 5.15
Commit: dcdf9656f794e1903163a5533d0a325eb3dce423

Submodule qtwebengine-chromium.git:
Branch: 87-based
Commit: d13d0924c4e18ecc4b79adf0fec142ee9a9eaa14

Bug: https://bugs.gentoo.org/835761
Closes: https://bugs.gentoo.org/831487
Comment 39 Michael 2022-04-06 02:08:28 UTC
(In reply to Tiernan Hubble from comment #37)
> I tried with just the 2 Arch patches and not the Catapult replacement, and I
> got Catapult errors. I'm not sure how tighly integrated Catapult is with the
> qtwebengine/chromium builds, but it definitely needs either Catapult
> replaced, or another patch to disable the parts of the build that call it.

Just for reference:
https://sources.debian.org/patches/chromium/89.0.4389.114-1~deb10u1/disable/catapult.patch/
Comment 40 Andreas Sturmlechner gentoo-dev 2022-04-06 07:58:49 UTC
None of these patches apply to 5.15 which is chromium-87-based. I'm not going to look into it myself, but would try if someone produces a rebased patch.
Comment 41 Andreas Sturmlechner gentoo-dev 2022-04-06 08:01:43 UTC
(In reply to Andreas Sturmlechner from comment #40)
> None of these patches apply to 5.15 which is chromium-87-based. I'm not
> going to look into it myself, but would try if someone produces a rebased
> patch.
(that's regarding catapult removal)
Comment 42 Michael 2022-04-06 09:26:01 UTC
(In reply to Andreas Sturmlechner from comment #41)
> (In reply to Andreas Sturmlechner from comment #40)
> > None of these patches apply to 5.15 which is chromium-87-based. I'm not
> > going to look into it myself, but would try if someone produces a rebased
> > patch.
> (that's regarding catapult removal)

Is it needed for something, though?
It seems you've added newer catapult into the newest snapshot. So, what's the advantage of removing it now?
Comment 43 Andreas Sturmlechner gentoo-dev 2022-04-06 11:35:17 UTC
(In reply to Michael from comment #42)
> It seems you've added newer catapult into the newest snapshot. So, what's
> the advantage of removing it now?
Not having to build it, obviously.
Comment 44 Chiitoo gentoo-dev 2022-04-06 12:05:53 UTC
I couldn't resist, and went ahead and created a patch based on the Debian patch to remove the need to mess with catapult.
Comment 45 Larry the Git Cow gentoo-dev 2022-04-09 15:05:13 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/qt.git/commit/?id=b70d10d2259fa7c4a100ebb90f47719c40eb2cb8

commit b70d10d2259fa7c4a100ebb90f47719c40eb2cb8
Author:     Andreas Sturmlechner <asturm@gentoo.org>
AuthorDate: 2022-04-09 13:40:33 +0000
Commit:     Andreas Sturmlechner <asturm@gentoo.org>
CommitDate: 2022-04-09 13:40:51 +0000

    dev-qt/qtwebengine: Support build w/ py3, IUSE=screencast, fix ffmpeg5
    
    ${PN}-5.15.3_p20220406-patchset contains patches to drop catapult as well.
    
    Bug: https://bugs.gentoo.org/698988
    Thanks-to: Jimi Huotari <chiitoo@gentoo.org>
    Package-Manager: Portage-3.0.30, Repoman-3.0.3
    Signed-off-by: Andreas Sturmlechner <asturm@gentoo.org>

 dev-qt/qtwebengine/Manifest                        |  1 +
 ... => qtwebengine-5.15.3_p20220406-ffmpeg5.patch} | 32 +++++++++++-----------
 dev-qt/qtwebengine/metadata.xml                    |  1 +
 dev-qt/qtwebengine/qtwebengine-5.15.3.9999.ebuild  | 12 +++++---
 4 files changed, 26 insertions(+), 20 deletions(-)
Comment 46 Andreas Sturmlechner gentoo-dev 2022-04-15 22:13:41 UTC
I think we can consider this fixed.
Comment 47 Chiitoo gentoo-dev 2022-04-15 23:20:35 UTC
\ö/

Thank you, everyone!