Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 669082 - dev-qt/qtwebengine-bin package
Summary: dev-qt/qtwebengine-bin package
Status: UNCONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All Linux
: Normal normal with 35 votes (vote)
Assignee: Default Assignee for New Packages
URL:
Whiteboard:
Keywords:
: 758341 (view as bug list)
Depends on:
Blocks:
 
Reported: 2018-10-20 16:19 UTC by Pavel Kozlov
Modified: 2022-06-22 13:32 UTC (History)
50 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Pavel Kozlov 2018-10-20 16:19:07 UTC
Hello!

dev-qt/qtwebengine takes really huge time to compile - and it became worse after every next version. Last time it took more than 8 hours to build on my laptop! Of course it is not new, but still core-i5 with 6 gigabytes of memory...

     Sat Oct 20 15:05:28 2018 >>> dev-qt/qtwebengine-5.11.1
       merge time: 8 hours, 9 minutes and 46 seconds.

I know that gentoo has some bin versions of packages which were created in intention to reduce compile-time which doesn't make really much sense - like firefox-bin or libreoffice-bin. In fact, firefox takes just about 1.5 hour to compile in my laptop, which is acceptable, I don't use firefox-bin, I use firefox package - it is OK for me.

But why qtwebengine takes so long? Can it be improved? I am OK to use -bin version for it, is this possible?

emerge --info (first part with processor and memory)

Portage 2.3.49 (python 3.6.5-final-0, default/linux/amd64/17.1/desktop/plasma, gcc-7.3.0, glibc-2.26-r7, 4.18.9-gentoo x86_64)
=================================================================
System uname: Linux-4.18.9-gentoo-x86_64-Intel-R-_Core-TM-_i5-3230M_CPU_@_2.60GHz-with-gentoo-2.4.1
KiB Mem:     5909444 total,   1578476 free
KiB Swap:    1048572 total,    851256 free

...
Comment 1 jms 2019-04-23 18:19:50 UTC
oh yes please a bin package.
this is becoming a pain .
It will rise earth temperature by another 1°c by itself.
Comment 2 Till Schäfer 2019-10-30 01:53:52 UTC
+1 

compilation times are skyrocketing here on my laptop with core 2 duo / 4GiB RAM:

2019-01-07T21:53:17 >>> dev-qt/qtwebengine-5.11.1: 7 hours, 29 minutes, 17 seconds
2019-06-05T07:06:21 >>> dev-qt/qtwebengine-5.12.3: 16 hours, 34 minutes, 2 seconds


current merge (5.12.5) is running for 18 hours now and is not finished yet.
Comment 3 Bodo Graumann 2019-11-03 10:11:01 UTC
Yes please provide a binary package! I have masked this package on my system because of the ridiculous long build time. This project includes a large part of the chromium code base.

I need to install nextcloud-client, but am not able because of this package. :-(
Maybe as a quick fix somebody can point me to a public binhost?
Comment 4 Andreas Sturmlechner gentoo-dev 2019-11-12 17:40:18 UTC
For everyone with such long build times, please look at USE=jumbo-build, build will be much, much faster. Make sure your MAKEOPTS value is not unreasonable though taking into account available RAM. Demand can go up to 2GB per process.
Comment 5 Chiitoo gentoo-dev 2019-11-12 18:05:48 UTC
We'd already have a binary package if it was a simple thing to provide while also doing it right.

With regards to the build time, I'll add that it might be worth checking if swapping and/or thermal throttling is happening.  Either will very likely slow down the build considerably.

I'll also add that it looks like jumbo-build will be going away at around Chromium 80 or some time after, but only time will really tell.
Comment 6 jeff.lemos.a 2019-12-07 02:51:49 UTC
I have found this flag: --no-keep-memory; in case the system runs out of memory when processing ld at the end, when using jumbo-build. Putting it on /portage/env could be a good idea in such case
Comment 7 Erik Solomon 2020-01-20 08:21:39 UTC
Till Schäfer

It's still over 4:30 hours with my nice 8 core and 16GB.  Never touches swap, though.
Comment 8 Simon 2020-05-22 19:41:15 UTC
+1 for this request, this package is a pain.

(In reply to Chiitoo from comment #5)
> We'd already have a binary package if it was a simple thing to provide while
> also doing it right.
> 
> With regards to the build time, I'll add that it might be worth checking if
> swapping and/or thermal throttling is happening.  Either will very likely
> slow down the build considerably.
> 
> I'll also add that it looks like jumbo-build will be going away at around
> Chromium 80 or some time after, but only time will really tell.

What are the challenges/what needs to be done to make a binary package for this possible?
Comment 9 Paul Gover 2020-05-23 11:37:42 UTC
My guess is the choice of USE flags.  I'd go for:
alsa bindist -designer geolocation -kerberos pulseaudio +system-ffmpeg +system-icu widgets
which is pretty much the defaults apart from removing designer.
Comment 10 jms 2020-05-23 11:45:16 UTC
what I ve got
(alsa bindist jumbo-build pulseaudio system-ffmpeg system-icu widgets -debug -designer -geolocation -kerberos -test)
Comment 11 Vasilis Lourdas 2020-05-23 11:51:10 UTC
Having +system-icu and +system-ffmpeg would cause a rebuild in case either of these libraries changed their ABI. In my opinion, it might be best if these flags were off for the *binary* package of qtwebengine to reduce any external dependencies. Just like when both icu and libreoffice-bin get bumped together, because the former depends on the latter.
Comment 12 jms 2020-05-23 12:21:27 UTC
(In reply to Vasilis Lourdas from comment #11)
> Having +system-icu and +system-ffmpeg would cause a rebuild in case either
> of these libraries changed their ABI. In my opinion, it might be best if
> these flags were off for the *binary* package of qtwebengine to reduce any
> external dependencies. Just like when both icu and libreoffice-bin get
> bumped together, because the former depends on the latter.

Agreed.
-system-icu -system-ffmpeg
This makes sense to me for a binary package
and
-debug  -test
+alsa +bindist 
what about theses?
+pulseaudio (I am for it I use it)
-designer (don't think many people use it)
-geolocation  
-kerberos
Comment 13 Vasilis Lourdas 2020-05-23 12:25:06 UTC
(In reply to jms from comment #12)
> Agreed.
> -system-icu -system-ffmpeg
> This makes sense to me for a binary package
> and
> -debug  -test
> +alsa +bindist 
> what about theses?
> +pulseaudio (I am for it I use it)
> -designer (don't think many people use it)
> -geolocation  
> -kerberos

Yes, these make sense, but I also think +widgets is needed too.
Comment 14 jms 2020-05-23 12:38:11 UTC
(In reply to Vasilis Lourdas from comment #13)
> (In reply to jms from comment #12)
> > Agreed.
> > -system-icu -system-ffmpeg
> > This makes sense to me for a binary package
> > and
> > -debug  -test
> > +alsa +bindist 
> > what about theses?
> > +pulseaudio (I am for it I use it)
> > -designer (don't think many people use it)
> > -geolocation  
> > -kerberos
> 
> Yes, these make sense, but I also think +widgets is needed too.
yes I forgot
Comment 15 Chiitoo gentoo-dev 2020-05-24 18:30:26 UTC
(In reply to Simon from comment #8)
> What are the challenges/what needs to be done to make a binary package for
> this possible?

Mainly probably reverse dependencies, and bundling the most annoying ones while also keeping them up-to-date especially with regards to security updates and such.
Comment 16 Simon 2020-06-03 17:58:18 UTC
(In reply to Chiitoo from comment #15)
> (In reply to Simon from comment #8)
> > What are the challenges/what needs to be done to make a binary package for
> > this possible?
> 
> Mainly probably reverse dependencies, and bundling the most annoying ones
> while also keeping them up-to-date especially with regards to security
> updates and such.

OK, thanks for the clarification. That seems doable.
Is there already something in place within gentoo's infrastructure to build and distribute binary packages? Or would this need to be done by a dev on his/her own account? I assume we could use a binpkg for this?
Comment 17 manu 2020-06-06 20:20:45 UTC
definitely +1...this package has become a huge deal when updating my laptop. Though it's really old (11-years old, dual core, 4GB RAM), updating everything else every now and then is fine...but this one: > 14 hours
Comment 18 nme 2020-09-23 16:36:46 UTC
+1 for the bin package. This would relieve my aging laptops a bit.
Comment 19 Vasilis Lourdas 2020-10-17 11:50:50 UTC
Newer qtwebengine package. Still hours to build, especially when you maintain 4 Gentoo installations. Is there hope to finally see a binary version of this package?
Comment 20 Paul Gover 2020-10-17 12:20:35 UTC
Vasilis, if your 4 installations share a common CPU architecture, you should be able to build a binary package on just one (the most powerful), and install that on the other 3.  For added value, you might be able to use DISTCC to spread the compilation across all the machines.  See the wiki for details of both.

That said, I'd still like to see a qtwebengine-bin package!
Comment 21 Vasilis Lourdas 2020-10-20 17:55:42 UTC
Ok, this is ridiculous. Today I got a libjpeg-turbo update which causes a qtwebengine rebuild. In my (relatively strong) PC, it takes an hour to compile. In my 2-core laptop, it took more than 3 hours to compile (even with the jumbo-build USE flag). And just because libjpeg-turbo was updated, another rebuild is needed.

Today marks exactly 2 years after this bug was opened. Is someone willing to provide a binary package for this time-consuming package?
Comment 22 urcindalo 2020-11-20 10:51:17 UTC
+1 for a much needed qtwebengine-bin

There are some packages I don't emerge and some USE flags I don't use so that they don't trigger the qtwebengine monster.
Comment 23 Vasilis Lourdas 2020-11-25 15:53:59 UTC
Another day, another qtwebengine rebuild. And as I thought Zoom needed it (and in fact it did at first, but not now), the latest digikam-7.0 requires it. Today dev-libs/re2 causes a rebuild of this freaking package...
Comment 24 email200202 2020-11-26 12:23:15 UTC
+1 for the bin package.

It takes 3 hours on my 8cores/16G computer.
Comment 25 Vasilis Lourdas 2020-11-26 13:18:56 UTC
(In reply to email200202 from comment #24)
> +1 for the bin package.
> 
> It takes 3 hours on my 8cores/16G computer.

You should enable the jumbo-build USE flag for this package, for your system, it shouldn't take more than an hour.
Comment 26 email200202 2020-11-26 14:49:08 UTC
(In reply to Vasilis Lourdas from comment #25)
> (In reply to email200202 from comment #24)
> > +1 for the bin package.
> > 
> > It takes 3 hours on my 8cores/16G computer.
> 
> You should enable the jumbo-build USE flag for this package, for your
> system, it shouldn't take more than an hour.

It takes 3 hours with jumbo-build USE flag enabled.

In the latest version dev-qt/qtwebengine-5.15.2, jumbo-build USE flag was removed!

# cat /usr/portage/dev-qt/qtwebengine/qtwebengine-5.15.2.ebuild
....
IUSE="alsa bindist designer geolocation kerberos pulseaudio +system-ffmpeg +system-icu widgets"
....
src_prepare() {
    if use ppc64; then
        eapply "${WORKDIR}/${PN}-ppc64"
    fi

    # QTBUG-88657 - jumbo-build is broken
    #if ! use jumbo-build; then
        sed -i -e 's|use_jumbo_build=true|use_jumbo_build=false|' \
            src/buildtools/config/common.pri || die
    #fi
Comment 27 email200202 2020-11-27 06:28:09 UTC Comment hidden (obsolete)
Comment 28 Andreas Sturmlechner gentoo-dev 2020-12-04 01:03:51 UTC
*** Bug 758341 has been marked as a duplicate of this bug. ***
Comment 29 FlyingWaffle 2020-12-20 01:50:11 UTC Comment hidden (obsolete)
Comment 30 Tiger 2020-12-30 12:51:31 UTC
CPU : Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz
RAM : 4Gb

dev-qt/qtwebengine-5.15.2: 19 hours, 58 minutes, 22 seconds average for 1 merge
dev-qt/qtwebengine-5.14.1: 15 hours, 20 minutes, 50 seconds average for 1 merge

USE flags :
dev-qt/qtwebengine-5.15.2 -alsa -bindist -debug -designer -geolocation -kerberos -pulseaudio -system-ffmpeg -system-icu -test widgets

Shrug...

Worst of this : it is required only for KmyMoney (which I will simply download the AppImage in the future) ...

Previous "jumbo-build" USE flag looks to have saved like 4 hours...
Comment 31 Vasilis Lourdas 2021-01-10 12:58:39 UTC
(In reply to Tiger from comment #30)
> CPU : Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz
> RAM : 4Gb
> 
> dev-qt/qtwebengine-5.15.2: 19 hours, 58 minutes, 22 seconds average for 1
> merge
> dev-qt/qtwebengine-5.14.1: 15 hours, 20 minutes, 50 seconds average for 1
> merge
> 
> USE flags :
> dev-qt/qtwebengine-5.15.2 -alsa -bindist -debug -designer -geolocation
> -kerberos -pulseaudio -system-ffmpeg -system-icu -test widgets
> 
> Shrug...
> 
> Worst of this : it is required only for KmyMoney (which I will simply
> download the AppImage in the future) ...
> 
> Previous "jumbo-build" USE flag looks to have saved like 4 hours...

Today I synced and I also got the stable 5.15.2 ebuild without the jumbo-build flag. I copied the ebuild to my local overlay, re-enabled the jumbo-build flag since I don't use GCC 10 and I will compile it.

Let's hope that in 2021 the developers will decide to provide a binary package.
Comment 32 Till Schäfer 2021-01-11 08:34:58 UTC
another +1 

this gets troubling to build on my haswell i7-4810MQ system 
2020-11-20T02:53:17 >>> dev-qt/qtwebengine-5.15.1: 2 hours, 18 minutes, 14 seconds
2020-11-26T15:58:09 >>> dev-qt/qtwebengine-5.15.2: 5 hours, 6 minutes, 46 seconds
Comment 33 Ulrich Müller gentoo-dev 2021-01-11 09:24:58 UTC
There won't be any progress here if all that people do is posting additional +1 comments. Someone has to step up and write an ebuild, and be prepared to maintain the package (e.g., rebuild it against new versions of dev-libs/icu).
Comment 34 Paul Osmialowski 2021-01-13 10:32:07 UTC
I'm sorry for yet another +1, but this package is a potential death sentence on Gentoo on many of my desktop machines.
Comment 35 Paul Gover 2021-01-13 12:57:37 UTC
Ulrich, is it possible to release a binpkg binary as an ebuild?  The discussions earlier seem to imply a binary package along the lines of say firefox, which can optionally contain specific versions of dependencies; I understand why that can make sense for such packages, but the need here is simply (for certain values of "simple") avoiding the mega-compilation.

IIUC, two things need to be developed: something in catalyst to build and package qtwebengine for various platforms, and an ebuild that installs the result.  I do something different on my home setup - I compile it on my ryzen-9 desktop machine (in less than an hour), and install the binpkg on my laptop (where it would probably take a day).  But AFAIK ebuilds don't do that.  Or can they?
Comment 36 Marek Kozlowski 2021-05-07 06:06:20 UTC Comment hidden (obsolete)
Comment 37 Marek Kozlowski 2021-05-07 12:36:47 UTC Comment hidden (obsolete)
Comment 38 Marek Kozlowski 2021-05-07 14:25:33 UTC
(In reply to Marek Kozlowski from comment #37)
> (In reply to Marek Kozlowski from comment #36)
> > It is also worth mentioning that 4GB of RAM with all non-critical processes
> > turned off is insufficient for compiling the newest qtwebengine. I confirm
> > that the lack of it forces me to switching to a different distro till the
> > *-bin package appears.
> 
> I've just checked with 6GB - is not enough either!

The same with 8GB! Do I really need a supercomputer to just install calibre (a required dependency)?
Comment 39 Andreas Sturmlechner gentoo-dev 2021-05-07 16:36:40 UTC
Please don't spam this bug. Tune your -j value to a realistic number for your machinery, and definitely *less* than total GB of RAM/2.
Comment 40 Marek Kozlowski 2021-05-08 05:25:56 UTC
(In reply to Andreas Sturmlechner from comment #39)
> Please don't spam this bug. Tune your -j value to a realistic number for
> your machinery, and definitely *less* than total GB of RAM/2.

-j is jest to 3 ( https://wiki.gentoo.org/wiki/Lenovo_ThinkPad_T400 ). 8GB of RAM. 3 < 8/2. It still fails.
Comment 41 oulx 2021-05-08 06:04:56 UTC
I'm running Intel i9 @3.60GHz, 10 cores, 20 threads; 64 GB RAM. dev-qt/qtwebengine is failing for me too, even with -j1 and jumbo-build. (Tried emerging today, several times). qtwebengine's emerge is awful. Count me as a +1-er.
Comment 42 Vasco Gervasi 2021-05-09 17:22:30 UTC
While building on a dual socket AMD Epyc 64 cores I noticed a peak RAM usage of 150 GB. I think the problem should be addressed upstream :(.
Comment 43 Matthew Sorensen 2021-05-09 17:28:47 UTC
(In reply to Vasco Gervasi from comment #42)
> While building on a dual socket AMD Epyc 64 cores I noticed a peak RAM usage
> of 150 GB. I think the problem should be addressed upstream :(.

If you have a lot of RAM it will try to utilize most of it, which is why you might see a couple hundred MB being used while idle on a system with 2 GB of RAM, but on a system with 8 GB it'll be several times more (take that with a grain of salt though, I'm not an expert).
Comment 44 Paul Osmialowski 2021-06-02 12:51:19 UTC Comment hidden (obsolete)
Comment 45 Vasilis Lourdas 2021-06-02 18:24:25 UTC
Check also bug #793935.
Comment 46 Vasilis Lourdas 2021-06-02 19:32:12 UTC
Ok, so I am willing to donate a small amount to any developer that spends time to create a ebuild for a binary package. I know I can't do it myself, if I did, I would have already tried it.

If anyone here is willing to do the same, would this help?
Comment 47 Simon 2021-06-02 20:29:40 UTC
Does anyone know if there's a standardized way within the gentoo org of building and providing binary packages? I'd assume something like that would already exist. Then it would just be as simple as adding qtwebengine to the list of packages to build and add a -bin ebuild to the tree to use the built artifact.
Comment 48 Michael 'veremitz' Everitt 2021-06-03 02:40:22 UTC
Because everybody's dependencies are frequently different, its difficult/impossible to build a 'generic' binary. This is why compiling from source works, because the build system can detect and adapt to what's present/etc. 

Most Gentoo binary packages come from upstream, where they can take care of compatibility testing, which Gentoo doesn't really have the capacity to.
Comment 49 Vasilis Lourdas 2021-06-03 14:13:49 UTC
(In reply to Michael 'veremitz' Everitt from comment #48)
> Because everybody's dependencies are frequently different, its
> difficult/impossible to build a 'generic' binary. This is why compiling from
> source works, because the build system can detect and adapt to what's
> present/etc. 
> 
> Most Gentoo binary packages come from upstream, where they can take care of
> compatibility testing, which Gentoo doesn't really have the capacity to.

Allow me to disagree. The binary version of Libreoffice is built by Gentoo and is not the upstream binary version. For such a complex package (consider that it is a user application with UI etc.), I tend to think it's "relatively" easy to provide a binary package for a Qt library.
Comment 50 Michael 'veremitz' Everitt 2021-06-03 14:46:28 UTC
well, that is the first I've seen of that, indeed tamiko is building and hosting that himself (evident from SRC_URI).
Comment 51 email200202 2021-06-03 16:21:50 UTC
I merge dev-qt/qtwebengine and other large packages with --buildpkg flag on one computer and then I use the binary packages on other computers.

The binary package is not large:
# ls -lh /usr/portage/packages/dev-qt/qtwebengine/qtwebengine-5.15.2_p20210521-2.xpak 
-rw-r--r-- 1 root portage 56M May 31 04:10 /usr/portage/packages/dev-qt/qtwebengine/qtwebengine-5.15.2_p20210521-2.xpak

The 'emerge' tool checks for dependencies and use flags before accepting the binary package. Any conflict will cause 'emerge' to compile from the sources instead.

If we define a new use flag say "binary" (or other mechanism) to direct 'emerge'  to download the binary package from a gentoo server and to invoke the binary install functions, we will have a binary install without the need for a different ebuild.

dev-qt/qtwebengine is not the only package which takes unreasonable time to merge. So we need a solution which does not need different *-bin package and a maintainer for it. 

Just my two cents.
Comment 52 Matthew Sorensen 2021-06-03 20:22:05 UTC
(In reply to Michael 'veremitz' Everitt from comment #48)
> Because everybody's dependencies are frequently different, its
> difficult/impossible to build a 'generic' binary. This is why compiling from
> source works, because the build system can detect and adapt to what's
> present/etc. 
> 
> Most Gentoo binary packages come from upstream, where they can take care of
> compatibility testing, which Gentoo doesn't really have the capacity to.

No? All a binary is, is code that's compiled to machine code. Compiling from source allows you to select the features you want from a program and the ability to customize optimization options. Unless you're using --native or something, it is very easy to create a generic binary. 

Compatibility testing should not be a problem as the vast majority of distributed binaries are compiled with -O2, so any "incompatibility" that happens is on the fault of the developer.
Comment 53 Adam 2021-06-14 22:44:24 UTC
Perhaps the solution isn't to distribute *-bin packages at all. Instead gentoo.org or some other trusted authority could host binpkgs for the larger packages on architectures in which they are stable.

I would host it myself (as I've already built many of these packages), but it's probably not a good idea to have people installing binaries from some random guy's website.

Another alternative would be to distribute completely pre-configured images like sakaki's rasperry pi images.
Comment 54 gerion 2021-06-14 23:01:34 UTC
Just as a bit meta discussion:
NixOS has exactly that. The package manager tries to install an (officially) prebuilded package and if that fails it compiles it by hand.

The whole point why this works is because in Nix the inputs and outputs are determined. With this, one can check if there is a package for exactly "this particular input configuration".

Gentoo's approach with bin-packages currently is to take a source package, apply a very specific use flag combination/compiler version/compiler flags, and use that to provide an extra binary package.

In my understanding, the Nix approach should be possible for Gentoo, too, if the inputs can be specified exactly:
1. dependencies are already specified in the ebuild (and part of the input set)
2. use flags are already specified in the ebuild (and part of the input set)
3. compiler version and flags are not part of the ebuild (are there a necessary input? Compiler are normally interoperable, just like CFLAGS except march=)

I'm not exactly sure but quite a time ago there was a discussion about a binary package projekt on the Gentoo mailing list: https://archives.gentoo.org/gentoo-dev/message/e2eaa90b45a8a342d58231a06cf18137
Comment 55 Reva Denis 2021-07-28 09:45:49 UTC Comment hidden (obsolete)
Comment 56 Vasco Gervasi 2021-09-03 14:59:56 UTC
Hi,
a quick update, I was able to compile dev-qt/qtwebengine 5.15.2_p20210824 on my i7-4810QM with 16 GB of RAM in 3 hours and 23 mins.
A great improvement from not being able to do it and the 5 hours of 5.15.2.
Comment 57 Andreas Sturmlechner gentoo-dev 2021-09-03 17:41:32 UTC
<5.15.2 had IUSE=jumbo-build (so default disabled)
5.15.2 did not contain IUSE=jumbo-build because it was broken.
5.15.2_p* contains IUSE=+jumbo-build (so default enabled)
Comment 58 Vasilis Lourdas 2021-12-02 19:11:31 UTC
Someone pointed me to this link: https://gentoo.osuosl.org/experimental/amd64/binpkg/default/linux/17.1/x86-64/dev-qt/qtwebengine/

Contains xpak binaries (?) for the latest version. I synced today and it built 5.15.2_p20211019 in 1 hour and 40' in my PC, while the previous builds were finished in a hour.

I will certainly try these binaries out.

On a sidenote, this freaking package was *one* of the reasons I switched from Gentoo to OpenSUSE Tumbleweed in my 7th Gen i5 laptop...
Comment 59 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2021-12-02 19:38:41 UTC
(In reply to Vasilis Lourdas from comment #58)
> Someone pointed me to this link:
> https://gentoo.osuosl.org/experimental/amd64/binpkg/default/linux/17.1/x86-
> 64/dev-qt/qtwebengine/
> 
> Contains xpak binaries (?) for the latest version. I synced today and it
> built 5.15.2_p20211019 in 1 hour and 40' in my PC, while the previous builds
> were finished in a hour.
> 
> I will certainly try these binaries out.

More info here: https://dilfridge.blogspot.com/2021/09/experimental-binary-gentoo-package.html.

> 
> On a sidenote, this freaking package was *one* of the reasons I switched
> from Gentoo to OpenSUSE Tumbleweed in my 7th Gen i5 laptop...

Maybe we can get you back then ;)
Comment 60 Andreas Sturmlechner gentoo-dev 2021-12-03 10:36:40 UTC
(In reply to Vasilis Lourdas from comment #58)
> On a sidenote, this freaking package was *one* of the reasons I switched
> from Gentoo to OpenSUSE Tumbleweed in my 7th Gen i5 laptop...
These sidenotes are irrelevant in bugs.
Comment 61 MeatShooter 2021-12-03 20:21:48 UTC Comment hidden (obsolete)
Comment 62 Michael 2021-12-09 06:24:52 UTC
For those who are struggling with compiling qtwebengine, you can try building it with clang. It uses 30-50% the RAM comparing to gcc, when compiling chromium/qtwebengine, which might allow you to bump up MAKEOPTS, which in turn might make it less painful. 

Hang in there, guys. :-)
Comment 63 Andreas Sturmlechner gentoo-dev 2021-12-09 08:05:01 UTC
There are plenty of bugs filed by people trying to build qtwebengine with clang, not because of resource exhaustion, but regular build errors.
Comment 64 Michael 2021-12-09 12:32:52 UTC
(In reply to Andreas Sturmlechner from comment #63)
> There are plenty of bugs filed by people trying to build qtwebengine with
> clang, not because of resource exhaustion, but regular build errors.

Yeah, I know. Nothing comes easy with qtwebengine, and it takes some fiddling. But it's entirely possible to do:

     Wed Dec  8 21:22:31 2021 >>> dev-qt/qtwebengine-5.15.2_p20211207
       merge time: 1 hour, 29 minutes and 16 seconds.

And the ram usage is so much lower that it might be worth it in certain situations. I remember someone told me on #gentoo-chat that they were able to lower the build time from 8 to 2 hours. They had a decent CPU, but only 8 gigs of ram, so that's a bit specific, but still.
Comment 65 Andreas Sturmlechner gentoo-dev 2021-12-09 14:53:18 UTC
(In reply to Michael from comment #64)
> I remember someone told me on #gentoo-chat that they were able
> to lower the build time from 8 to 2 hours.
That's just misconfiguration. GCC may as well take just 2 hours in that scenario without overly optimistic MAKEOPTS settings leading to swapping.

See also comment #4:
> Make sure your MAKEOPTS value is not unreasonable though taking into account
> available RAM. Demand can go up to 2GB per process.
Comment 66 Michael 2021-12-09 15:17:20 UTC
If I remember correctly, he had a ryzen 3600, 6 cores, 12 threads, and 8GB is of course grossly underspecced, it needs at very least 3 times as much. But the point is GCC was chugging along with -j3 or -j4, because when building qtwebengine it can easily gobble up to 2.5-3GB per thread. And with clang he was able to bump it up to -j10 or something like that, which of course sped it up significantly. 

And of course it is misconfigured, in the sense that ryzen 5 needs more ram, otherwise it'll not be fully utilized. 

That particular anecdote is of course just that. An anecdote. It was a low hanging fruit for speeding things up. For me switching to clang didn't change the compile time at all, because my system is not quite as grossly underspecced in terms of RAM. 

I'm just mentioning all that, because maybe there's someone else out there bottlenecking on RAM.
Comment 67 Andreas Sturmlechner gentoo-dev 2021-12-10 08:37:56 UTC
(In reply to Michael from comment #66)
> And of course it is misconfigured, in the sense that ryzen 5 needs more ram,
> otherwise it'll not be fully utilized. 
It will never take a Ryzen 5 3600 eight hours to build qtwebengine even with only 8 GB RAM, with aptly adjusted MAKEOPTS, without swapping. Even an old i7-4790K will do it in 2 hours, with 8 GB RAM. So first and foremost we will always tell people to set their MAKEOPTS to realistic values.

The reason I am objecting are all the other issues that switching to clang systemwide brings. I would never recommend it as a quick fix; it takes some commitment.
Comment 68 Michael 2021-12-10 08:46:56 UTC
(In reply to Andreas Sturmlechner from comment #67)
> The reason I am objecting are all the other issues that switching to clang
> systemwide brings. I would never recommend it as a quick fix; it takes some
> commitment.

Just for the record, I was not at all suggesting systemwide clang. Only for anything with chromium in it. Which is basically chromium and qtwebengine.
Comment 69 Roberto Castagnola 2021-12-10 11:00:01 UTC
(In reply to Andreas Sturmlechner from comment #67)
> It will never take a Ryzen 5 3600 eight hours to build qtwebengine even with
> only 8 GB RAM, with aptly adjusted MAKEOPTS, without swapping. Even an old
> i7-4790K will do it in 2 hours, with 8 GB RAM. So first and foremost we will
> always tell people to set their MAKEOPTS to realistic values.

I have an old i7-2600K with 8GB RAM, and I have used MAKEOPTS="-j4" in the past.
These are the last build times for this package:

     Sat Nov  2 00:49:44 2019 >>> dev-qt/qtwebengine-5.12.5
       merge time: 3 hours, 35 minutes and 18 seconds.

     Sat Apr  4 12:20:30 2020 >>> dev-qt/qtwebengine-5.14.1
       merge time: 4 hours, 6 minutes and 8 seconds.

     Thu May 21 19:34:34 2020 >>> dev-qt/qtwebengine-5.14.2
       merge time: 4 hours, 33 minutes and 49 seconds.

     Thu Oct 29 15:14:33 2020 >>> dev-qt/qtwebengine-5.15.1
       merge time: 17 hours, 45 minutes and 20 seconds.

     Thu Dec  3 03:36:11 2020 >>> dev-qt/qtwebengine-5.15.2
       merge time: 16 hours, 10 minutes and 7 seconds.

     Tue Feb  2 16:18:39 2021 >>> dev-qt/qtwebengine-5.15.2
       merge time: 16 hours, 12 minutes and 5 seconds.

The big difference was due to the use-flag jumbo-build that was broken for 5.15 at the beginning; when it was available again, re-enabling it the build failed with both -j4 and -j3.
I was able to build it a bit faster only enabling jumbo-build and using -j4 at the beginning; every time the build failed, I have changed the option reducing the amount of thread directly in Makefile and restarted the build with command 'ebuild', ending with -j2 to be able to complete the build.
PS: the build is started with nothing else running and swap disabled; I run 'openrc boot' to keep the max resource available only for the build

I will try to use clang only for this package (I have to figure out how to do it); the only other solution will be the binary package.

Further, is there a way to define a different PORTAGE_TMPDIR for a single package?
I tryed to add it into /etc/portage/env/dev-qt/qtwebengine but it didn't work.
Comment 70 Michael 2021-12-10 11:09:40 UTC
(In reply to Roberto Castagnola from comment #69)
> Further, is there a way to define a different PORTAGE_TMPDIR for a single
> package?
> I tryed to add it into /etc/portage/env/dev-qt/qtwebengine but it didn't
> work.

Yes, there is a way to do that, and you are on the right track. But in addition you need a package.env file with something like `dev-qt/qtwebengine tmpdir.conf`
and a /etc/portage/env/tmpdir.conf with `PORTAGE_TMPDIR=....` in it. 

You'll have to figure out how to use package.env if you want to set the compiler on a per package basis.
Comment 71 Michael 2021-12-10 12:53:10 UTC
I'm fully expecting you to be able to go to at very least -j4 with clang.
Comment 72 Roberto Castagnola 2021-12-13 18:48:39 UTC
(In reply to Michael from comment #71)
> I'm fully expecting you to be able to go to at very least -j4 with clang.

with MAKEOPTS="-j5"
     Mon Dec 13 05:03:35 2021 >>> dev-qt/qtwebengine-5.15.2_p20211207
       merge time: 7 hours, 37 minutes and 1 second.

still a lot of time but better than before.

I needed few workaround to compile it with clang:
bug #813249 : I have defined EXTRA_GN as suggested
bug #828099 : I have used the patch www-client/chromium/files/chromium-glibc-2.34.patch (adapted for qtwebengine code)

Further, almost at the end, the build failed because it tried to run clang++ without the full path. Not sure if it is an issue of qtwebengine build or of clang.
I have tried to make a link for clang++ in /usr/bin, but in this way the build fails at the beginning.
My workauround was to define CC and CXX with full path.
Comment 73 Michael 2021-12-13 22:07:47 UTC
(In reply to Roberto Castagnola from comment #72)
> I needed few workaround to compile it with clang

Build system in qtwebengine is notoriously flaky, so not unexpected.

> I have tried to make a link for clang++ in /usr/bin, but in this way the
> build fails at the beginning.
> My workauround was to define CC and CXX with full path.

I haven't hit that one yet, but thanks for the heads-up. :-)
Comment 74 gangstervano@yandex.ru 2021-12-17 23:19:27 UTC Comment hidden (obsolete)
Comment 75 Manuel Garcia Wolff 2022-02-04 19:49:43 UTC
On my i7 (7th gen) laptop with 32 GB of RAM it takes around 3 hours to merge qtwebengine, which is quite long for a package that got merged, on average, twice a month in the last year. My MAKEOPTS is set to "-j6".

I wonder if disabling the system-ffmpeg and system-icu USE flags could reduce the amount of rebuilds that are needed?
Comment 76 Tobias Leupold 2022-02-04 22:23:15 UTC
I build qtwebengine like twice a week on my Ryzen 5 with 12 threads and 32 GB RAM. It still utterly sucks. So I think cheaping out building tiny packages (compared with qtwebengine) like ffmpeg and icu won't help much here.
Comment 77 Krzysiek 2022-02-05 22:33:35 UTC
On my system (Quad Core, 8 GB), switching to clang reduced the compilation time from ~16h to ~5h (with jumbo-build).
Comment 78 Pulkit Sukhija 2022-03-03 10:34:23 UTC
(In reply to Krzysiek from comment #77)
> On my system (Quad Core, 8 GB), switching to clang reduced the compilation
> time from ~16h to ~5h (with jumbo-build).

Any special changes you did for building with clang?
Comment 79 Krzysiek 2022-03-03 22:01:09 UTC
(In reply to Pulkit Sukhija from comment #78)
> (In reply to Krzysiek from comment #77)
> > On my system (Quad Core, 8 GB), switching to clang reduced the compilation
> > time from ~16h to ~5h (with jumbo-build).
> 
> Any special changes you did for building with clang?

No, I just edited files:

/etc/portage/env/compiler-clang
/etc/portage/package.env

according to this manual:
https://wiki.gentoo.org/wiki/Clang#Clang_environments
Comment 80 Richard H. 2022-03-03 22:10:12 UTC
(In reply to Krzysiek from comment #79)
> (In reply to Pulkit Sukhija from comment #78)
> > (In reply to Krzysiek from comment #77)
> > > On my system (Quad Core, 8 GB), switching to clang reduced the compilation
> > > time from ~16h to ~5h (with jumbo-build).
> > 
> > Any special changes you did for building with clang?
> 
> No, I just edited files:
> 
> /etc/portage/env/compiler-clang
> /etc/portage/package.env
> 
> according to this manual:
> https://wiki.gentoo.org/wiki/Clang#Clang_environments

Yeah the same for me. qtwebengine builds without any problems using CLang nowadays.
Comment 81 Pulkit Sukhija 2022-03-04 05:07:37 UTC
(In reply to Krzysiek from comment #79)
> (In reply to Pulkit Sukhija from comment #78)
> > (In reply to Krzysiek from comment #77)
> > > On my system (Quad Core, 8 GB), switching to clang reduced the compilation
> > > time from ~16h to ~5h (with jumbo-build).
> > 
> > Any special changes you did for building with clang?
> 
> No, I just edited files:
> 
> /etc/portage/env/compiler-clang
> /etc/portage/package.env
> 
> according to this manual:
> https://wiki.gentoo.org/wiki/Clang#Clang_environments

Oh, alright, 'cause for me it failed at somewhat 2800 tars, maybe i didn't edit the confs the correct way
Comment 82 Konstantinos Tsardounis 2022-06-05 18:42:03 UTC
sys-kernel/gentoo-kernel-bin is good, but a qtwebengine-bin will be a real redemption for gentoo. I had it disabled for many years, but last seasons I started using www-client/falkon and it hurts so much to wait for hours and hours for it to compile...
Comment 83 sebaro 2022-06-22 13:32:04 UTC
Even with 16GB of RAM the build may fail so a binary package will be great (I see chromium-bin was added recently).

---
/usr/lib/gcc/x86_64-pc-linux-gnu/11.3.0/include/g++-v11/bits/unique_ptr.h:155:63: internal compiler error: Segmentation fault
---