Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 669082 - dev-qt/qtwebengine-bin package
Summary: dev-qt/qtwebengine-bin package
Status: UNCONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All Linux
: Normal normal with 28 votes (vote)
Assignee: Default Assignee for New Packages
URL:
Whiteboard:
Keywords:
: 758341 (view as bug list)
Depends on:
Blocks:
 
Reported: 2018-10-20 16:19 UTC by Pavel Kozlov
Modified: 2021-04-12 14:43 UTC (History)
40 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Pavel Kozlov 2018-10-20 16:19:07 UTC
Hello!

dev-qt/qtwebengine takes really huge time to compile - and it became worse after every next version. Last time it took more than 8 hours to build on my laptop! Of course it is not new, but still core-i5 with 6 gigabytes of memory...

     Sat Oct 20 15:05:28 2018 >>> dev-qt/qtwebengine-5.11.1
       merge time: 8 hours, 9 minutes and 46 seconds.

I know that gentoo has some bin versions of packages which were created in intention to reduce compile-time which doesn't make really much sense - like firefox-bin or libreoffice-bin. In fact, firefox takes just about 1.5 hour to compile in my laptop, which is acceptable, I don't use firefox-bin, I use firefox package - it is OK for me.

But why qtwebengine takes so long? Can it be improved? I am OK to use -bin version for it, is this possible?

emerge --info (first part with processor and memory)

Portage 2.3.49 (python 3.6.5-final-0, default/linux/amd64/17.1/desktop/plasma, gcc-7.3.0, glibc-2.26-r7, 4.18.9-gentoo x86_64)
=================================================================
System uname: Linux-4.18.9-gentoo-x86_64-Intel-R-_Core-TM-_i5-3230M_CPU_@_2.60GHz-with-gentoo-2.4.1
KiB Mem:     5909444 total,   1578476 free
KiB Swap:    1048572 total,    851256 free

...
Comment 1 jms 2019-04-23 18:19:50 UTC
oh yes please a bin package.
this is becoming a pain .
It will rise earth temperature by another 1°c by itself.
Comment 2 Till Schäfer 2019-10-30 01:53:52 UTC
+1 

compilation times are skyrocketing here on my laptop with core 2 duo / 4GiB RAM:

2019-01-07T21:53:17 >>> dev-qt/qtwebengine-5.11.1: 7 hours, 29 minutes, 17 seconds
2019-06-05T07:06:21 >>> dev-qt/qtwebengine-5.12.3: 16 hours, 34 minutes, 2 seconds


current merge (5.12.5) is running for 18 hours now and is not finished yet.
Comment 3 Bodo Graumann 2019-11-03 10:11:01 UTC
Yes please provide a binary package! I have masked this package on my system because of the ridiculous long build time. This project includes a large part of the chromium code base.

I need to install nextcloud-client, but am not able because of this package. :-(
Maybe as a quick fix somebody can point me to a public binhost?
Comment 4 Andreas Sturmlechner gentoo-dev 2019-11-12 17:40:18 UTC
For everyone with such long build times, please look at USE=jumbo-build, build will be much, much faster. Make sure your MAKEOPTS value is not unreasonable though taking into account available RAM. Demand can go up to 2GB per process.
Comment 5 Chiitoo gentoo-dev 2019-11-12 18:05:48 UTC
We'd already have a binary package if it was a simple thing to provide while also doing it right.

With regards to the build time, I'll add that it might be worth checking if swapping and/or thermal throttling is happening.  Either will very likely slow down the build considerably.

I'll also add that it looks like jumbo-build will be going away at around Chromium 80 or some time after, but only time will really tell.
Comment 6 jeff.lemos.a 2019-12-07 02:51:49 UTC
I have found this flag: --no-keep-memory; in case the system runs out of memory when processing ld at the end, when using jumbo-build. Putting it on /portage/env could be a good idea in such case
Comment 7 Erik Solomon 2020-01-20 08:21:39 UTC
Till Schäfer

It's still over 4:30 hours with my nice 8 core and 16GB.  Never touches swap, though.
Comment 8 Simon 2020-05-22 19:41:15 UTC
+1 for this request, this package is a pain.

(In reply to Chiitoo from comment #5)
> We'd already have a binary package if it was a simple thing to provide while
> also doing it right.
> 
> With regards to the build time, I'll add that it might be worth checking if
> swapping and/or thermal throttling is happening.  Either will very likely
> slow down the build considerably.
> 
> I'll also add that it looks like jumbo-build will be going away at around
> Chromium 80 or some time after, but only time will really tell.

What are the challenges/what needs to be done to make a binary package for this possible?
Comment 9 Paul Gover 2020-05-23 11:37:42 UTC
My guess is the choice of USE flags.  I'd go for:
alsa bindist -designer geolocation -kerberos pulseaudio +system-ffmpeg +system-icu widgets
which is pretty much the defaults apart from removing designer.
Comment 10 jms 2020-05-23 11:45:16 UTC
what I ve got
(alsa bindist jumbo-build pulseaudio system-ffmpeg system-icu widgets -debug -designer -geolocation -kerberos -test)
Comment 11 Vasilis Lourdas 2020-05-23 11:51:10 UTC
Having +system-icu and +system-ffmpeg would cause a rebuild in case either of these libraries changed their ABI. In my opinion, it might be best if these flags were off for the *binary* package of qtwebengine to reduce any external dependencies. Just like when both icu and libreoffice-bin get bumped together, because the former depends on the latter.
Comment 12 jms 2020-05-23 12:21:27 UTC
(In reply to Vasilis Lourdas from comment #11)
> Having +system-icu and +system-ffmpeg would cause a rebuild in case either
> of these libraries changed their ABI. In my opinion, it might be best if
> these flags were off for the *binary* package of qtwebengine to reduce any
> external dependencies. Just like when both icu and libreoffice-bin get
> bumped together, because the former depends on the latter.

Agreed.
-system-icu -system-ffmpeg
This makes sense to me for a binary package
and
-debug  -test
+alsa +bindist 
what about theses?
+pulseaudio (I am for it I use it)
-designer (don't think many people use it)
-geolocation  
-kerberos
Comment 13 Vasilis Lourdas 2020-05-23 12:25:06 UTC
(In reply to jms from comment #12)
> Agreed.
> -system-icu -system-ffmpeg
> This makes sense to me for a binary package
> and
> -debug  -test
> +alsa +bindist 
> what about theses?
> +pulseaudio (I am for it I use it)
> -designer (don't think many people use it)
> -geolocation  
> -kerberos

Yes, these make sense, but I also think +widgets is needed too.
Comment 14 jms 2020-05-23 12:38:11 UTC
(In reply to Vasilis Lourdas from comment #13)
> (In reply to jms from comment #12)
> > Agreed.
> > -system-icu -system-ffmpeg
> > This makes sense to me for a binary package
> > and
> > -debug  -test
> > +alsa +bindist 
> > what about theses?
> > +pulseaudio (I am for it I use it)
> > -designer (don't think many people use it)
> > -geolocation  
> > -kerberos
> 
> Yes, these make sense, but I also think +widgets is needed too.
yes I forgot
Comment 15 Chiitoo gentoo-dev 2020-05-24 18:30:26 UTC
(In reply to Simon from comment #8)
> What are the challenges/what needs to be done to make a binary package for
> this possible?

Mainly probably reverse dependencies, and bundling the most annoying ones while also keeping them up-to-date especially with regards to security updates and such.
Comment 16 Simon 2020-06-03 17:58:18 UTC
(In reply to Chiitoo from comment #15)
> (In reply to Simon from comment #8)
> > What are the challenges/what needs to be done to make a binary package for
> > this possible?
> 
> Mainly probably reverse dependencies, and bundling the most annoying ones
> while also keeping them up-to-date especially with regards to security
> updates and such.

OK, thanks for the clarification. That seems doable.
Is there already something in place within gentoo's infrastructure to build and distribute binary packages? Or would this need to be done by a dev on his/her own account? I assume we could use a binpkg for this?
Comment 17 manu 2020-06-06 20:20:45 UTC
definitely +1...this package has become a huge deal when updating my laptop. Though it's really old (11-years old, dual core, 4GB RAM), updating everything else every now and then is fine...but this one: > 14 hours
Comment 18 nme 2020-09-23 16:36:46 UTC
+1 for the bin package. This would relieve my aging laptops a bit.
Comment 19 Vasilis Lourdas 2020-10-17 11:50:50 UTC
Newer qtwebengine package. Still hours to build, especially when you maintain 4 Gentoo installations. Is there hope to finally see a binary version of this package?
Comment 20 Paul Gover 2020-10-17 12:20:35 UTC
Vasilis, if your 4 installations share a common CPU architecture, you should be able to build a binary package on just one (the most powerful), and install that on the other 3.  For added value, you might be able to use DISTCC to spread the compilation across all the machines.  See the wiki for details of both.

That said, I'd still like to see a qtwebengine-bin package!
Comment 21 Vasilis Lourdas 2020-10-20 17:55:42 UTC
Ok, this is ridiculous. Today I got a libjpeg-turbo update which causes a qtwebengine rebuild. In my (relatively strong) PC, it takes an hour to compile. In my 2-core laptop, it took more than 3 hours to compile (even with the jumbo-build USE flag). And just because libjpeg-turbo was updated, another rebuild is needed.

Today marks exactly 2 years after this bug was opened. Is someone willing to provide a binary package for this time-consuming package?
Comment 22 urcindalo 2020-11-20 10:51:17 UTC
+1 for a much needed qtwebengine-bin

There are some packages I don't emerge and some USE flags I don't use so that they don't trigger the qtwebengine monster.
Comment 23 Vasilis Lourdas 2020-11-25 15:53:59 UTC
Another day, another qtwebengine rebuild. And as I thought Zoom needed it (and in fact it did at first, but not now), the latest digikam-7.0 requires it. Today dev-libs/re2 causes a rebuild of this freaking package...
Comment 24 email200202 2020-11-26 12:23:15 UTC
+1 for the bin package.

It takes 3 hours on my 8cores/16G computer.
Comment 25 Vasilis Lourdas 2020-11-26 13:18:56 UTC
(In reply to email200202 from comment #24)
> +1 for the bin package.
> 
> It takes 3 hours on my 8cores/16G computer.

You should enable the jumbo-build USE flag for this package, for your system, it shouldn't take more than an hour.
Comment 26 email200202 2020-11-26 14:49:08 UTC
(In reply to Vasilis Lourdas from comment #25)
> (In reply to email200202 from comment #24)
> > +1 for the bin package.
> > 
> > It takes 3 hours on my 8cores/16G computer.
> 
> You should enable the jumbo-build USE flag for this package, for your
> system, it shouldn't take more than an hour.

It takes 3 hours with jumbo-build USE flag enabled.

In the latest version dev-qt/qtwebengine-5.15.2, jumbo-build USE flag was removed!

# cat /usr/portage/dev-qt/qtwebengine/qtwebengine-5.15.2.ebuild
....
IUSE="alsa bindist designer geolocation kerberos pulseaudio +system-ffmpeg +system-icu widgets"
....
src_prepare() {
    if use ppc64; then
        eapply "${WORKDIR}/${PN}-ppc64"
    fi

    # QTBUG-88657 - jumbo-build is broken
    #if ! use jumbo-build; then
        sed -i -e 's|use_jumbo_build=true|use_jumbo_build=false|' \
            src/buildtools/config/common.pri || die
    #fi
Comment 27 email200202 2020-11-27 06:28:09 UTC
dev-qt/qtwebengine-5.15.2 merge took 6 hours 43 minutes!



# genlop -t dev-qt/qtwebengine

     Wed May 20 20:30:20 2020 >>> dev-qt/qtwebengine-5.14.2
       merge time: 2 hours, 50 minutes and 28 seconds.

     Wed May 27 18:48:10 2020 >>> dev-qt/qtwebengine-5.15.0
       merge time: 3 hours, 9 minutes and 3 seconds.

     Wed Jul 15 17:01:58 2020 >>> dev-qt/qtwebengine-5.15.0
       merge time: 3 hours, 1 minute and 21 seconds.

     Fri Sep 11 18:40:38 2020 >>> dev-qt/qtwebengine-5.15.1
       merge time: 3 hours, 9 minutes.

     Mon Oct 19 15:41:27 2020 >>> dev-qt/qtwebengine-5.15.1
       merge time: 2 hours, 59 minutes and 23 seconds.

     Thu Nov  5 03:37:44 2020 >>> dev-qt/qtwebengine-5.15.1
       merge time: 2 hours, 59 minutes and 29 seconds.

     Fri Nov 27 05:03:23 2020 >>> dev-qt/qtwebengine-5.15.2
       merge time: 6 hours, 43 minutes and 28 seconds.
Comment 28 Andreas Sturmlechner gentoo-dev 2020-12-04 01:03:51 UTC
*** Bug 758341 has been marked as a duplicate of this bug. ***
Comment 29 FlyingWaffle 2020-12-20 01:50:11 UTC
This really needs to be addressed; I'm on a decently new/good specced laptop and even after I started using the -jumbo-build flag in October or so you can see from genlop output how much that has NOT helped:

Sat Jan 11 21:32:07 2020 >>> dev-qt/qtwebengine-5.14.0
merge time: 5 hours, 14 minutes and 48 seconds.

Mon Feb  3 01:53:42 2020 >>> dev-qt/qtwebengine-5.14.1
merge time: 9 hours, 9 minutes and 53 seconds.

Sun Feb 16 05:14:32 2020 >>> dev-qt/qtwebengine-5.14.1
merge time: 9 hours, 46 minutes and 27 seconds.

Sun Mar 22 01:07:15 2020 >>> dev-qt/qtwebengine-5.14.1
merge time: 7 hours, 36 minutes and 29 seconds.

Sun Apr  5 08:53:14 2020 >>> dev-qt/qtwebengine-5.14.1
merge time: 8 hours, 41 minutes and 20 seconds.

Sun Apr 12 02:44:46 2020 >>> dev-qt/qtwebengine-5.14.2
merge time: 8 hours, 34 minutes and 8 seconds.

Fri May  1 19:35:51 2020 >>> dev-qt/qtwebengine-5.14.2
merge time: 6 hours, 6 minutes and 27 seconds.

Sun May 10 09:02:10 2020 >>> dev-qt/qtwebengine-5.14.2
merge time: 8 hours and 18 seconds.

Sun May 31 02:55:19 2020 >>> dev-qt/qtwebengine-5.15.0
merge time: 9 hours, 57 minutes and 12 seconds.

Wed Jul 15 15:20:16 2020 >>> dev-qt/qtwebengine-5.15.0
merge time: 6 hours, 59 minutes and 19 seconds.

Fri Sep 11 16:15:06 2020 >>> dev-qt/qtwebengine-5.15.1
merge time: 5 hours, 31 minutes and 28 seconds.

Sat Oct 31 16:44:17 2020 >>> dev-qt/qtwebengine-5.15.1
merge time: 4 hours, 12 minutes and 21 seconds.

Wed Nov  4 16:13:45 2020 >>> dev-qt/qtwebengine-5.15.1
merge time: 4 hours, 26 minutes and 38 seconds.

Fri Nov 27 23:15:26 2020 >>> dev-qt/qtwebengine-5.15.2
merge time: 9 hours, 33 minutes and 18 seconds.

Tue Dec  8 22:32:36 2020 >>> dev-qt/qtwebengine-5.15.2
merge time: 11 hours, 45 minutes and 28 seconds.
Comment 30 Tiger 2020-12-30 12:51:31 UTC
CPU : Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz
RAM : 4Gb

dev-qt/qtwebengine-5.15.2: 19 hours, 58 minutes, 22 seconds average for 1 merge
dev-qt/qtwebengine-5.14.1: 15 hours, 20 minutes, 50 seconds average for 1 merge

USE flags :
dev-qt/qtwebengine-5.15.2 -alsa -bindist -debug -designer -geolocation -kerberos -pulseaudio -system-ffmpeg -system-icu -test widgets

Shrug...

Worst of this : it is required only for KmyMoney (which I will simply download the AppImage in the future) ...

Previous "jumbo-build" USE flag looks to have saved like 4 hours...
Comment 31 Vasilis Lourdas 2021-01-10 12:58:39 UTC
(In reply to Tiger from comment #30)
> CPU : Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz
> RAM : 4Gb
> 
> dev-qt/qtwebengine-5.15.2: 19 hours, 58 minutes, 22 seconds average for 1
> merge
> dev-qt/qtwebengine-5.14.1: 15 hours, 20 minutes, 50 seconds average for 1
> merge
> 
> USE flags :
> dev-qt/qtwebengine-5.15.2 -alsa -bindist -debug -designer -geolocation
> -kerberos -pulseaudio -system-ffmpeg -system-icu -test widgets
> 
> Shrug...
> 
> Worst of this : it is required only for KmyMoney (which I will simply
> download the AppImage in the future) ...
> 
> Previous "jumbo-build" USE flag looks to have saved like 4 hours...

Today I synced and I also got the stable 5.15.2 ebuild without the jumbo-build flag. I copied the ebuild to my local overlay, re-enabled the jumbo-build flag since I don't use GCC 10 and I will compile it.

Let's hope that in 2021 the developers will decide to provide a binary package.
Comment 32 Till Schäfer 2021-01-11 08:34:58 UTC
another +1 

this gets troubling to build on my haswell i7-4810MQ system 
2020-11-20T02:53:17 >>> dev-qt/qtwebengine-5.15.1: 2 hours, 18 minutes, 14 seconds
2020-11-26T15:58:09 >>> dev-qt/qtwebengine-5.15.2: 5 hours, 6 minutes, 46 seconds
Comment 33 Ulrich Müller gentoo-dev 2021-01-11 09:24:58 UTC
There won't be any progress here if all that people do is posting additional +1 comments. Someone has to step up and write an ebuild, and be prepared to maintain the package (e.g., rebuild it against new versions of dev-libs/icu).
Comment 34 Paul Osmialowski 2021-01-13 10:32:07 UTC
I'm sorry for yet another +1, but this package is a potential death sentence on Gentoo on many of my desktop machines.
Comment 35 Paul Gover 2021-01-13 12:57:37 UTC
Ulrich, is it possible to release a binpkg binary as an ebuild?  The discussions earlier seem to imply a binary package along the lines of say firefox, which can optionally contain specific versions of dependencies; I understand why that can make sense for such packages, but the need here is simply (for certain values of "simple") avoiding the mega-compilation.

IIUC, two things need to be developed: something in catalyst to build and package qtwebengine for various platforms, and an ebuild that installs the result.  I do something different on my home setup - I compile it on my ryzen-9 desktop machine (in less than an hour), and install the binpkg on my laptop (where it would probably take a day).  But AFAIK ebuilds don't do that.  Or can they?