Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 592660

Summary: dev-qt/qtwebengine does not respect MAKEOPTS="-jN"
Product: Gentoo Linux Reporter: Chí-Thanh Christopher Nguyễn <chithanh>
Component: Current packagesAssignee: Qt Bug Alias <qt>
Status: RESOLVED FIXED    
Severity: normal CC: aliens.satin0y, bob.mt.wya, Dan.Johansson, eugene.shalygin, gentoo-bugs-augustin, nrndda
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://bugs.gentoo.org/show_bug.cgi?id=590920
https://bugs.gentoo.org/show_bug.cgi?id=559064
Whiteboard:
Package list:
Runtime testing required: ---

Description Chí-Thanh Christopher Nguyễn gentoo-dev 2016-09-01 13:03:32 UTC
(Split off from bug 559064 comment 13)

dev-qt/qtwebengine will always call ninja without any parallel build options, which makes ninja default to the number of CPUs in a system.
This is however too much for my Raspberry Pi 3, where this will fill up swap and cause the system to grind to a halt.

Setting MAKEOPTS or NINJAOPTS via make.conf or /etc/portage/env has no effect whatsoever.
Setting NINJAFLAGS per kensington on #gentoo-kde appears to be a possible workaround.

# emerge --info
Portage 2.2.26 (python 3.4.3-final-0, default/linux/arm64/13.0/desktop, gcc-4.9.3, glibc-2.22-r4, 4.6.3-raspberrypi3-arm64 aarch64)
=================================================================
System uname: Linux-4.6.3-raspberrypi3-arm64-aarch64-with-gentoo-2.2
KiB Mem:      871684 total,    101892 free
KiB Swap:    1084408 total,   1061608 free
Timestamp of repository gentoo: Thu, 01 Sep 2016 00:45:01 +0000
sh bash 4.3_p42-r1
ld GNU ld (Gentoo 2.25.1 p1.1) 2.25.1
app-shells/bash:          4.3_p42-r1::gentoo
dev-lang/perl:            5.20.2::gentoo
dev-lang/python:          2.7.10-r1::gentoo, 3.4.3-r1::gentoo
dev-util/cmake:           3.3.1-r1::gentoo
dev-util/pkgconfig:       0.28-r2::gentoo
sys-apps/baselayout:      2.2::gentoo
sys-apps/openrc:          0.19.1::gentoo
sys-apps/sandbox:         2.10-r1::gentoo
sys-devel/autoconf:       2.13::gentoo, 2.69::gentoo
sys-devel/automake:       1.14.1::gentoo, 1.15::gentoo
sys-devel/binutils:       2.25.1-r1::gentoo
sys-devel/gcc:            4.9.3::gentoo
sys-devel/gcc-config:     1.7.3::gentoo
sys-devel/libtool:        2.4.6::gentoo
sys-devel/make:           4.1-r1::gentoo
sys-kernel/linux-headers: 4.3::gentoo (virtual/os-headers)
sys-libs/glibc:           2.22-r4::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000

ACCEPT_KEYWORDS="arm64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="aarch64-unknown-linux-gnu"
CFLAGS="-O2 -pipe -march=armv8-a+crc -mtune=cortex-a53"
CHOST="aarch64-unknown-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe -march=armv8-a+crc -mtune=cortex-a53"
DISTDIR="/var/cache/portage/distfiles"
FCFLAGS="-O2"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2"
GENTOO_MIRRORS="http://ftp.spline.de/mirrors/gentoo/"
LANG="de_DE.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j4"
PKGDIR="/var/cache/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
USE="X a52 aac acl acpi alsa arm64 berkdb bluetooth branding bzip2 cairo cdda cdr cli consolekit cracklib crypt cups cxx dbus dri dts dvdr emboss encode exif fam firefox flac fortran gdbm gif glamor gpm gtk iconv ipv6 jpeg lcms libnotify mad mng modules mp3 mp4 mpeg multilib ncurses nls nptl ogg opengl openmp pam pango pcre pdf png policykit ppds qt3support qt5 readline seccomp session spell ssl startup-notification svg tcpd tiff truetype udev udisks unicode upower usb vorbis wxwidgets xattr xcb xml xv xvid zlib" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" L10N="de en" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="de_DE" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_4" RUBY_TARGETS="ruby20 ruby21" USERLAND="GNU" VIDEO_CARDS="fbdev" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
Comment 1 Michael Palimaka (kensington) gentoo-dev 2016-09-01 16:25:37 UTC
MAKEOPTS needs munging before being passed to NINJAOPTS - we cannot assign it verbatim.

Bug #590920 comment #2 suggests introducing a new eclass, instead of duplicating the login in cmake-utils. Thoughts?
Comment 2 Davide Pesavento (RETIRED) gentoo-dev 2016-09-02 06:14:21 UTC
(In reply to Michael Palimaka (kensington) from comment #1)
> Bug #590920 comment #2 suggests introducing a new eclass, instead of
> duplicating the login in cmake-utils. Thoughts?

Sounds sensible to me. Note that multiprocessing.eclass also has makeopts_jobs() and makeopts_loadavg() functions to extract those values from MAKEOPTS.
Comment 3 Toralf Förster gentoo-dev 2017-02-12 20:58:19 UTC
NINJAFLAGS="-j1"
?
Comment 4 Thomas Juerges 2017-05-24 08:40:04 UTC
Has there been any progress?  This bug is affecting all systems with low memory.
Comment 5 Chí-Thanh Christopher Nguyễn gentoo-dev 2017-05-26 09:11:42 UTC
(In reply to Thomas Jürges from comment #4)
There is a new ninja-utils eclass which has been committed last week. Porting qtwebengine ebuild to it would presumably solve the problem.

Until then, set NINJAFLAGS as a workaround per comment 0 and comment 3.
Comment 6 Thomas Juerges 2017-05-26 13:40:28 UTC
I can confirm that setting the environment variable NINJAFLAGS alleviates the issue.

Cheers
    T.
Comment 7 Davide Pesavento (RETIRED) gentoo-dev 2017-05-26 23:38:33 UTC
(In reply to Chí-Thanh Christopher Nguyễn from comment #5)
> (In reply to Thomas Jürges from comment #4)
> There is a new ninja-utils eclass which has been committed last week.
> Porting qtwebengine ebuild to it would presumably solve the problem.

The ebuild doesn't call ninja directly, it's wrapped by a qmake-based build system. So we can't use the eninja eclass function.
Comment 8 Michael Palimaka (kensington) gentoo-dev 2017-06-03 16:03:11 UTC
I pushed a fix to the overlay for testing:

https://gitweb.gentoo.org/proj/qt.git/commit/?id=3210bde4818f62d3acef982a1311f1ba2be8a020

If someone is able to confirm that this approach works, I'll land it in main tree too.
Comment 9 Michael Palimaka (kensington) gentoo-dev 2017-06-04 11:42:19 UTC
I used the wrong variable name. Fixed commit:
https://gitweb.gentoo.org/proj/qt.git/commit/?id=522bef331aed4670a4c97d635d6d9fe8df0bb31e
Comment 10 Michael Palimaka (kensington) gentoo-dev 2017-06-04 13:21:02 UTC
Thanks, fixed in git.

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=90f45d0172ad22d6b8a6e77025499156b704dcc9
Comment 11 . 2017-06-04 18:36:24 UTC
(In reply to Michael Palimaka (kensington) from comment #8)
> I pushed a fix to the overlay for testing:
> 
> https://gitweb.gentoo.org/proj/qt.git/commit/
> ?id=3210bde4818f62d3acef982a1311f1ba2be8a020
> 
> If someone is able to confirm that this approach works, I'll land it in main
> tree too.

what if someone wants to use custom NINJAFLAGS?
Comment 12 Davide Pesavento (RETIRED) gentoo-dev 2017-06-04 18:51:39 UTC
(In reply to Jan Chren (rindeal) from comment #11)
> what if someone wants to use custom NINJAFLAGS?

Good point.
Comment 13 Arfrever Frehtes Taifersar Arahesis 2017-06-04 21:31:27 UTC
(In reply to Jan Chren (rindeal) from comment #11)

I suggest:

export NINJAFLAGS="${NINJAFLAGS:--j$(makeopts_jobs) -l$(makeopts_loadavg "${MAKEOPTS}" 0) -v}"