Summary: | dev-qt/qtwebengine-6.7.3 fails to build (ozone_platform_qt.cpp:9:10: fatal error: media/gpu/buildflags.h: No such file or directory) | ||
---|---|---|---|
Product: | Gentoo Linux | Reporter: | jlm <WoLibertygentoo> |
Component: | Current packages | Assignee: | Qt Bug Alias <qt> |
Status: | RESOLVED FIXED | ||
Severity: | normal | CC: | gentoo, ionen, WoLibertygentoo |
Priority: | Normal | Keywords: | PATCH |
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Bug Depends on: | |||
Bug Blocks: | 351559 | ||
Attachments: |
qtwebengine-6.7.3-missing-gn-deps.patch
build log showing failure mode of comment #5 qtwebengine-6.7.3-missing-gn-deps.patch qtwebengine-6.7.3-missing-gn-deps.patch qtwebengine-6.7.3-missing-gn-deps.patch qtwebengine-6.7.3-missing-gn-deps.patch qtwebengine-6.7.3-missing-gn-deps.patch more-fixes.patch more-fixes.patch |
Description
jlm
2024-10-25 16:44:32 UTC
Created attachment 906685 [details, diff]
qtwebengine-6.7.3-missing-gn-deps.patch
I can't reproduce but these kind of rare race conditions with qtwebengine have happened before and I can take a guess at at a likely fix.
Could you try the attached patch? Since can't reproduce, can't confirm if it works myself before pushing.
On a side-note, please always attach the full build.log (compress it if too big), it makes it difficult to confirm certain things otherwise (not *only* interested in the error itself from that).
will try it as soon as the current build ends, I'm trying to see if this is a side effect of ccache and jumbo-build USE flag, so I desactivated jumbo-build and trying to rebuild (In reply to jlm from comment #2) > will try it as soon as the current build ends, I'm trying to see if this is > a side effect of ccache and jumbo-build USE flag, so I desactivated > jumbo-build and trying to rebuild Being a race condition, changing things around is likely to workaround it. It may also randomly work again even if you change nothing, so hard to say if a patch really helps but a successful build with same conditions would give hope that might have. There are such issues that indeed happened only with ccache (because it builds faster, and a thing that need some generated header started building before it was generated), but that doesn't mean the fix was to disable ccache. (on a side-note, disabling jumbo-build can increase build time by roughly 50%) test disabling jumbobuild but keeping ccache : passed test re-enabling jumbobuild, ccache on, provided patch not applied : same error test jumbobuild on, ccache on, provided patch applied : error changed (seems to pass previous error) ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/components/signin/internal/identity_manager/profile_oauth2_token_service.h:19:10: fatal error: components/signin/public/base/signin_buildflags.h: No such file or directory seems that signin_buildflags.h has also to be added to generation list? best resgards Interesting, wonder what is causing these to surface? Last time it was a ninja update which affected most users (me included), but unsure here (either way is still a bug given the dependencies do seem to be missing). I don't have enough information to update the patch though, need to know which file that is including profile_oauth2_token_service.h that it is failing on (ideally the full build.log would make things simpler). Given chromium "seems" to have the right dependencies, I do think it's coming from Qt's side as well though, probably src/core/pref_service_adapter.cpp? and if so think fix will belong in src/core/configure/BUILD.root.gn.in seems to have compiled fine with last sync (2024-11-04) (In reply to jlm from comment #0) > /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6. > 7.3/src/core/ozone/ozone_platform_qt.cpp:9:10: fatal error: > media/gpu/buildflags.h: No such file or directory > 9 | #include "media/gpu/buildflags.h" I hit this exact issue just now. Without changing anything but simply resuming the build with `ebuild /var/db/repos/gentoo/dev-qt/qtwebengine/qtwebengine-6.7.3.ebuild merge`, I was apparently able to proceed past it but then got the same error as reported in comment #5: > ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/ > components/signin/internal/identity_manager/profile_oauth2_token_service.h: > 19:10: fatal error: components/signin/public/base/signin_buildflags.h: No > such file or directory I tried resuming the build yet again but could not proceed past this latter error. Even clearing MAKEOPTS (to disable parallel building) did not resolve the issue. # emerge -pqv '=dev-qt/qtwebengine-6.7.3::gentoo' [ebuild R ] dev-qt/qtwebengine-6.7.3 USE="alsa jumbo-build opengl qml system-icu vaapi vulkan widgets -accessibility -bindist -custom-cflags -designer -geolocation -kerberos (-pdfium) -pulseaudio -screencast -test -webdriver" You'll note that I have previously built this version successfully, so I guess the failure is arising due to a change in some build-time dependency. (In reply to Matt Whitlock from comment #8) > > ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/ > > components/signin/internal/identity_manager/profile_oauth2_token_service.h: > > 19:10: fatal error: components/signin/public/base/signin_buildflags.h: No > > such file or directory As I stated in comment #6, I still need a build.log or at least know which file that is including profile_oauth2_token_service.h it is failing on to be able to patch further something that I cannot reproduce myself without making guesses. With FEATURES="-ccache distcc" MAKEOPTS="", it built to completion, even without the patch from comment #1, so I can't provide a failing build.log from that. Now I'm testing FEATURES="ccache -distcc" MAKEOPTS="". I see, if someone else runs into this, please make sure to keep the build.log of the 2nd failure if run into it. Could patch the first one either way, but seems like they go hand-in-hand. Created attachment 908706 [details] build log showing failure mode of comment #5 FEATURES="ccache -distcc" MAKEOPTS="" also built to completion. However, FEATURES="ccache -distcc" MAKEOPTS="-j5" (with a populated cache from the previous build) failed with the error from comment #5. Build log is attached. If there are any other files from my build tree that you need, I can provide them. Created attachment 908717 [details, diff] qtwebengine-6.7.3-missing-gn-deps.patch Attaching new patch for both fixes, would appreciate if someone affected could try it in case it either doesn't help or there's a 3rd issue, albeit understand it may or may not be hard to tell whether it really fixed it given it's somewhat random (just knowing it built fine once is good enough for me). (In reply to Matt Whitlock from comment #12) > Created attachment 908706 [details] > build log showing failure mode of comment #5 And interesting, actually didn't fail on the file I thought it was and the fix I had in mind (without seeing the log) wasn't in the right place. Was scratching my head for a bit given all the deps are there for web_history_service.cc, but then noticed that a bit (added by Qt) *removes* a dependency in src/3rdparty/chromium/components/history/core/browser/BUILD.gn if (is_qtwebengine) { ... deps -= [ ... "//components/signin/public/identity_manager", If I understand this right, removing identity_manager despite the header being used leads to losing the public dep from src/3rdparty/chromium/components/signin/public/identity_manager/BUILD.gn with "//components/signin/public/base:signin_buildflags" Given Qt seems to be trying to avoid parts of identity_manager (only selects a few files from it), I "think" the maybe not best but safest fix is to keep it removed and add base:signin_buildflags which is what I did in the above patch. (In reply to Ionen Wolkens from comment #13) > Created attachment 908717 [details, diff] [details, diff] > qtwebengine-6.7.3-missing-gn-deps.patch > > Attaching new patch for both fixes, would appreciate if someone affected > could try it in case it either doesn't help or there's a 3rd issue Now attempting to build with this patch and FEATURES="ccache -distcc" MAKEOPTS="-j5". Will post back with results. (In reply to Matt Whitlock from comment #14) > (In reply to Ionen Wolkens from comment #13) > > Created attachment 908717 [details, diff] [details, diff] [details, diff] > > qtwebengine-6.7.3-missing-gn-deps.patch > > > > Attaching new patch for both fixes, would appreciate if someone affected > > could try it in case it either doesn't help or there's a 3rd issue > > Now attempting to build with this patch and FEATURES="ccache -distcc" > MAKEOPTS="-j5". Will post back with results. Thanks. With the patch from comment #13, it gets further along but still fails: In file included from ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/third_party/blink/renderer/platform/heap/cross_thread_persistent.h:8, from ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/third_party/blink/public/platform/web_private_ptr.h:36, from ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/third_party/blink/public/platform/web_crypto_algorithm.h:35, from ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/third_party/blink/public/platform/web_crypto.h:36, from ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/components/webcrypto/algorithm_dispatch.h:14, from ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/components/webcrypto/algorithm_dispatch.cc:5: ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/third_party/blink/renderer/platform/heap/persistent.h:8:10: fatal error: third_party/blink/renderer/platform/heap/heap_buildflags.h: No such file or directory 8 | #include "third_party/blink/renderer/platform/heap/heap_buildflags.h" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. Let me know if you need the build.log or if the above suffices. Created attachment 908721 [details, diff] qtwebengine-6.7.3-missing-gn-deps.patch (In reply to Matt Whitlock from comment #16) > With the patch from comment #13, it gets further along but still fails: Still more huh... I *think* this one is fixed in Qt 6.8.0 by [1], see no harm in backporting to 6.7.3 in the interim though. I've attached a new patch that combines everything, hopefully it's the right fix and also doesn't hit a 4th error (many thanks for testing). [1] https://crrev.com/ac2ad9c9f01bf255fd123785bd37d3b0ec44e410 With the patch from comment #17 and FEATURES="ccache -distcc" MAKEOPTS="-j5", the build fails with: ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/qtwebengine/browser/extensions/api/webrtc_desktop_capture_private/webrtc_desktop_capture_private_api.cc:56:10: fatal error: qtwebengine/common/extensions/api/webrtc_desktop_capture_private.h: No such file or directory 56 | #include "qtwebengine/common/extensions/api/webrtc_desktop_capture_private.h" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. Created attachment 908722 [details, diff]
qtwebengine-6.7.3-missing-gn-deps.patch
Well another try, bit different but "think" should be right for that one.
The patch from comment #19 leads to the same error: ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/qtwebengine/browser/extensions/api/webrtc_desktop_capture_private/webrtc_desktop_capture_private_api.cc:56:10: fatal error: qtwebengine/common/extensions/api/webrtc_desktop_capture_private.h: No such file or directory 56 | #include "qtwebengine/common/extensions/api/webrtc_desktop_capture_private.h" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. I guess that wasn't the fix this time. Created attachment 908726 [details, diff]
qtwebengine-6.7.3-missing-gn-deps.patch
Take two, think this one is more straight forward than thought (assuming it works).
Sorry for the wait, got distracted by another issue.
(In reply to Ionen Wolkens from comment #21) > Created attachment 908726 [details, diff] [details, diff] > qtwebengine-6.7.3-missing-gn-deps.patch No wait, don't bother trying that one, I tried the wrong patch and that one is flat out broken. Looking again for the right fix. Created attachment 908728 [details, diff]
qtwebengine-6.7.3-missing-gn-deps.patch
Take three, looked deeper and looks like //qtwebengine/common/extensions/api:api is the right one (didn't really see it given the way it generates it).
Interesting considering Qt has this dependency in its top level, but weird ordering may still be possible to fail? I think anyway... will see if it helps.
With the patch from comment #23 and FEATURES="ccache -distcc" MAKEOPTS="-j5", the build ran to completion. I'll repeat it several times with differing degrees of parallelism to confirm. The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=956d6a85d63efd962759c9388ddcb86bb0282dcb commit 956d6a85d63efd962759c9388ddcb86bb0282dcb Author: Ionen Wolkens <ionen@gentoo.org> AuthorDate: 2024-10-26 02:03:15 +0000 Commit: Ionen Wolkens <ionen@gentoo.org> CommitDate: 2024-11-18 09:02:34 +0000 dev-qt/qtwebengine: quick fix for build race conditions (qt6) Done as two separate patches given one is a chromium backport (fixed in 6.8), and the others are qtwebengine-side issues. Thanks to Matt Whitlock for testing. Quite possible there are more issues, but there is no harm in fixing the ones we know about so far. Closes: https://bugs.gentoo.org/942142 Signed-off-by: Ionen Wolkens <ionen@gentoo.org> .../qtwebengine-6.7.3-missing-gn-deps-blink.patch | 38 ++++++++++++++++++++++ .../files/qtwebengine-6.7.3-missing-gn-deps.patch | 33 +++++++++++++++++++ dev-qt/qtwebengine/qtwebengine-6.7.3.ebuild | 2 ++ dev-qt/qtwebengine/qtwebengine-6.8.0.ebuild | 1 + dev-qt/qtwebengine/qtwebengine-6.8.9999.ebuild | 1 + 5 files changed, 75 insertions(+) I got another one, this time with MAKEOPTS="-j6": In file included from ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/chrome/common/url_constants.h:24, from ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/qtwebengine/browser/extensions/api/tabs/tabs_api.cc:50: ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/chrome/common/webui_url_constants.h:17:10: fatal error: build/config/chromebox_for_meetings/buildflags.h: No such file or directory 17 | #include "build/config/chromebox_for_meetings/buildflags.h" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. Created attachment 909264 [details, diff] more-fixes.patch Here's another patch. Given the other fixes are merged, new patch only has the latest fix and meant to be used on the top of the rest. Will merge with the in-tree one later if seems to be working. (In reply to Matt Whitlock from comment #26) > tabs_api.cc Looks like another qtwebengine-specific issue. src/3rdparty/chromium/chrome/browser/extensions/BUILD.gn: "api/tabs/tabs_api.cc", src/3rdparty/chromium/qtwebengine/browser/extensions/api/BUILD.gn: "tabs/tabs_api.cc", Seeing how the first BUILD.gn has "//build/config/chromebox_for_meetings:buildflags" in deps, I assume the one failing is the added-by-Qt qtwebengine one below which does not have it. Qt did add it in their top level BUILD.root.gn.in making it "unlikely" to fail, but unlucky ordering can still happen. (In reply to Ionen Wolkens from comment #27) > assume well, no need to assume /qtwebengine/ is right there in the error output's path, but it was to compare with what chromium is doing anyway :) (In reply to Ionen Wolkens from comment #27) > Created attachment 909264 [details, diff] [details, diff] > more-fixes.patch > > Here's another patch. Thank you for continuing to crank out the patches. It's rather unbelievable just how bad Qt has been about this. Where is their quality control? It seems like they're completely lacking any rigorous process for declaring intra-build dependencies and are just tossing s#!† over the wall whenever it happens to pass a cursory trial run. Anyway, I know this must be so tiresome for you, so really, thank you for dealing with this mess. I'm building again now with your latest patch. Will report back if I find any more underspecified deps. (In reply to Matt Whitlock from comment #29) > Anyway, I know this must be so tiresome for you Not really, given I'm not trying to reproduce (my last build went fine, albeit it was the 6.8.1 branch that I'm preparing for the next bump), I don't bother trying to build it (which is would be the tiresome part :) and just looking at files is pretty quick bare a few more obscure dependency chains. Using the patch from comment #27, got another one: In file included from ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/components/autofill/core/browser/autofill_field.h:18, from ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/components/autofill/core/browser/form_structure.h:19, from ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/components/autofill/core/browser/autofill_download_manager.h:25, from ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/components/autofill/core/browser/autofill_manager.h:26, from ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/components/autofill/core/browser/browser_autofill_manager.h:31, from ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/components/autofill/core/browser/autofill_external_delegate.cc:29: ../../../../../qtwebengine-everywhere-src-6.7.3/src/3rdparty/chromium/components/autofill/core/browser/form_parsing/regex_patterns.h:12:10: fatal error: components/autofill/core/browser/form_parsing/buildflags.h: No such file or directory 12 | #include "components/autofill/core/browser/form_parsing/buildflags.h" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. Just a question: why aren't all these generated header files generated right at the start of the build before doing anything else? Created attachment 909554 [details, diff] more-fixes.patch And another. (In reply to Matt Whitlock from comment #31) > autofill_external_delegate.cc This seems introduced by Qt as well. With "is_qtwebengine" it resets sources and deps, and then adds its own more limited set which misses the dependency. That header is generated by the same BUILD.gn which is normally well aware that it needs its own header. buildflag_header("buildflags") { header = "form_parsing/buildflags.h" ... deps = [ ":buildflags", ... if (is_qtwebengine) { sources = [] sources = [ ... "autofill_external_delegate.cc", ... deps = [] deps = [ ... and then this does not have ":buildflags". This new patch adds it which hopefully is right. > Just a question: why aren't all these generated header files generated right > at the start of the build before doing anything else? Well, not all of them are needed or even possible to generate based on other settings / platform and that'd need its own custom set of conditions. At a glance some of these generated headers even have their own dependency strings and could build (possibly unneeded) large components before they have the right information/tools to be generated. So instead it just generate them when something requests it (assuming the dependency is there). Not to say that it couldn't work with the right conditions, but whole thing has gotten pretty complex. fwiw most of these issues seem at least unlikely to be hit without ccache, so they probably don't hit that many people that just build it without and move on (like Qt's test builds, or even Qt's developers that just rebuild specific objects after making changes rather than the whole thing). (In reply to Ionen Wolkens from comment #33) > fwiw most of these issues seem at least unlikely to be hit without ccache, > so they probably don't hit that many people that just build it without and > move on (like Qt's test builds, or even Qt's developers that just rebuild > specific objects after making changes rather than the whole thing). .. not to mention that it also won't hit everyone, I do use ccache a lot for qtwebengine testing myself, but I haven't hit any of these.. go figure. Using the patch from comment #32, it's looking like I am no longer seeing build failures at any degree of build parallelism. (In reply to Matt Whitlock from comment #35) > Using the patch from comment #32, it's looking like I am no longer seeing > build failures at any degree of build parallelism. Nice, thanks. The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=fd94ed9d933d7b27ddc4a50e76397b136468630c commit fd94ed9d933d7b27ddc4a50e76397b136468630c Author: Ionen Wolkens <ionen@gentoo.org> AuthorDate: 2024-11-18 19:00:06 +0000 Commit: Ionen Wolkens <ionen@gentoo.org> CommitDate: 2024-11-19 08:59:02 +0000 dev-qt/qtwebengine: more race condition fixes (qt6) Bug: https://bugs.gentoo.org/942142 Signed-off-by: Ionen Wolkens <ionen@gentoo.org> .../files/qtwebengine-6.7.3-missing-gn-deps.patch | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) |