Summary: | dev-qt/qtwebengine: No progress information from parallel build with ninja | ||
---|---|---|---|
Product: | Gentoo Linux | Reporter: | Jonathan Marten <jjm> |
Component: | Current packages | Assignee: | Qt Bug Alias <qt> |
Status: | CONFIRMED --- | ||
Severity: | normal | CC: | gyakovlev, ionen, pacho, sam, xxc3ncoredxx |
Priority: | Normal | ||
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
See Also: | https://bugs.gentoo.org/show_bug.cgi?id=917245 | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Bug Depends on: | 911938 | ||
Bug Blocks: |
Description
Jonathan Marten
2023-06-22 18:29:18 UTC
Hm, is this because of the default in newer Portage? But you have MAKEOPTS="-j6"... From the --info output: MAKEFLAGS="-j6" Unset: ... MAKEOPTS, ... Portage doesn't take into account MAKEFLAGS when deciding on whether to use default values or not[1]. The question is, should it do that? 1: https://github.com/gentoo/portage/blob/f537f520422af0900049c760b435ab3e0938240d/lib/portage/package/ebuild/doebuild.py#L588 Ignore the MAKEFLAGS="-j6" in the emerge --info output, this is set in my make.conf but does not seem to be used anywhere - it may be a historical artifact. MAKEOPTS is not set in make.conf which means that it is calculated automatically by doebuild.py (for 4 CPUs). The option for Ninja is manually set in the package environment file, any value will work here. No, make will respect MAKEFLAGS. It's relevant to whether the default for output synchronisation we added in Portage should be used or not. Confirmed that the same happens (make -j4, ninja -j4, no progress output) if neither MAKEFLAGS nor MAKEOPTS is set in make.conf or environment. Yes, that is expected behavior. Portage is "blind" to MAKEFLAGS in make.conf, and makes it's decisions purely on whether MAKEOPTS is set or not. In the case out output synchronization, it adds an extra check if GNUMAKEFLAGS is set to avoid clobbering it on the off chance someone has GNU Make-specific options set (so setting either one will prevent the default output synchronization from being set as well). Instead of setting MAKEOPTS="-j1" in the qtwebengine env file, you should be able to set MAKEOPTS="-j$(nproc)" and see the same progress output because now MAKEOPTS is set which prevents the output synchronization default from taking effect. Confirmed that with no MAKE* settings in make.conf but setting NINJAFLAGS="-j7 -l8 -v" MAKEOPTS="-j5" in package env file (random numbers just to make it clear where they come from) results in parallel build of both gn and engine but with logging as expected: 9739 \_ make -j5 10450 \_ ninja -j7 -l8 -v -C /var/tmp/portage/dev-qt/qtwebengine- with the environment of process 9739 containing MAKEOPTS=-j5 Removing MAKEOPTS from the package anv file and allowing it to be automatically calculated gives (no logging in this case): 12244 \_ make -j4 12954 \_ ninja -j7 -l8 -v -C /var/tmp/portage/dev-qt/qtwebengine- with the environment of process 12244 containing MAKEOPTS=-j4 GNUMAKEFLAGS=--output-sync=line Same problem (no logging until build complete) also seen with net-libs/nodejs, same workaround. (In reply to Oskari Pirhonen from comment #6) > Yes, that is expected behavior. Portage is "blind" to MAKEFLAGS in > make.conf, and makes it's decisions purely on whether MAKEOPTS is set or > not. In the case out output synchronization, it adds an extra check if > GNUMAKEFLAGS is set to avoid clobbering it on the off chance someone has GNU > Make-specific options set (so setting either one will prevent the default > output synchronization from being set as well). Let's go off MAKEFLAGS as well, even though it's odd for people to set it. The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=87d0a7447f1464157f18a8ba0991719777003325 commit 87d0a7447f1464157f18a8ba0991719777003325 Author: Oskari Pirhonen <xxc3ncoredxx@gmail.com> AuthorDate: 2023-08-13 03:11:15 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-08-19 15:05:19 +0000 doebuild: gate -Oline behind MAKEFLAGS check Add a check to see if MAKEFLAGS is set (alongside the existing MAKEOPTS and GNUMAKEFLAGS checks) before enabling output synchronization. This is only tangentially related to the bug below via being a (likely rare) case of MAKEFLAGS being set in the wild. Bug: https://bugs.gentoo.org/909009 Signed-off-by: Oskari Pirhonen <xxc3ncoredxx@gmail.com> Signed-off-by: Sam James <sam@gentoo.org> lib/portage/package/ebuild/doebuild.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) Note that this bug shouldn't be automatically closed with the portage-3.0.51 release IMO since it alone doesn't really address the underlying issue. An idea I have for actually fixing it is to remove any output synchronization in the relevant ebuilds themselves. Maybe by either checking for the option in the vars or just unconditionally appending `--output-sync=none`. The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=a6a68c0255b6d3132dd555b1519a035204bffde2 commit a6a68c0255b6d3132dd555b1519a035204bffde2 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-08-20 07:57:47 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-08-20 07:58:22 +0000 sys-apps/portage: add 3.0.51 Bug: https://bugs.gentoo.org/587484 Closes: https://bugs.gentoo.org/673624 Closes: https://bugs.gentoo.org/691434 Closes: https://bugs.gentoo.org/835927 Closes: https://bugs.gentoo.org/909009 Closes: https://bugs.gentoo.org/911335 Closes: https://bugs.gentoo.org/911629 Closes: https://bugs.gentoo.org/911938 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.51.ebuild | 234 +++++++++++++++++++++++++++++++++ 2 files changed, 235 insertions(+) We should pass -Onone like we do in nodejs: commit 223940a28118fc34c3cc2c7b630b83ddc6d9de99 Author: Sam James <sam@gentoo.org> Date: Wed Nov 29 05:12:21 2023 +0000 net-libs/nodejs: adapt to newer Portage's make output synchronisation Newer Portage defaults to GNU Make's output synchronisation if no MAKEOPTS are set by the user. nodejs's Makefile fires up a ninja process which then does all the work, so everything gets buffered and shown at once (once all of nodejs is built). Pass -Onone to override the Portage default as it's not helpful here. Closes: https://bugs.gentoo.org/917245 Signed-off-by: Sam James <sam@gentoo.org> I assume qt6 doesn't need to do anything here? I don't think it still use make for anything (unless I'm missing something that gn does) and output looks normal to me. |