Bug 703256 – www-client/firefox-71.0 build with PGO stuck with profileserver.py sleeping background

• Product: Gentoo Linux
• Status: RESOLVED OBSOLETE

[Description lfs9azh 2019-12-18 04:35:00 UTC]

emerge firefox-71 with PGO stuck with profileserver.py sleeping background.
CPU:x9100 
RAM:6G
SWAP:4G



 ps aux|grep firefox


portage   8320  0.8  0.4 400896 27964 pts/4    Sl+  02:58   4:34 /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/ff/_virtualenvs/init/bin/python /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/build/pgo/profileserver.py
portage   8498  0.0  0.3 327164 22928 pts/4    S    02:58   0:00 /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/ff/_virtualenvs/init/bin/python /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/build/pgo/profileserver.py
portage   8499  0.0  5.0 2763988 303336 pts/4  Sl   02:58   0:08 /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/ff/dist/firefox/firefox http://localhost:8888/index.html -profile /var/tmp/portage/www-client/firefox-71.0-r1/temp/tmpTaPvss
portage   8564  0.0  3.6 2580988 216360 pts/4  Sl   02:58   0:03 /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/ff/dist/firefox/firefox -contentproc -childID 1 -isForBrowser -prefsLen 1 -prefMapSize 206912 -parentBuildID 20191218004207 -greomni /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/ff/dist/firefox/omni.ja -appomni /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/ff/dist/firefox/browser/omni.ja -appdir /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/ff/dist/firefox/browser 12317 tab
portage   8602  0.0  2.7 2521348 163892 pts/4  Sl   02:58   0:07 /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/ff/dist/firefox/firefox -contentproc -childID 2 -isForBrowser -prefsLen 7001 -prefMapSize 206912 -parentBuildID 20191218004207 -greomni /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/ff/dist/firefox/omni.ja -appomni /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/ff/dist/firefox/browser/omni.ja -appdir /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/ff/dist/firefox/browser 12317 tab
portage   8649  0.0  2.1 2469048 128740 pts/4  Sl   02:58   0:00 /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/ff/dist/firefox/firefox -contentproc -childID 3 -isForBrowser -prefsLen 7981 -prefMapSize 206912 -parentBuildID 20191218004207 -greomni /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/ff/dist/firefox/omni.ja -appomni /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/ff/dist/firefox/browser/omni.ja -appdir /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/ff/dist/firefox/browser 12317 tab
root     12589  0.0  0.0  10640   816 pts/2    S+   11:59   0:00 grep --colour=auto firefox
root     21092  0.0  2.4 156668 146808 pts/0   S+   00:39   0:18 /usr/bin/python3.6 -b /usr/lib/python-exec/python3.6/emerge -a firefox
root     27420  0.0  0.1  18740  9980 pts/4    Ss+  00:42   0:00 /usr/bin/python3.6m /usr/lib/portage/python3.6/pid-ns-init 250 250 250 18 0,1,2 /usr/bin/sandbox [www-client/firefox-71.0-r1] sandbox /usr/lib/portage/python3.6/ebuild.sh compile
portage  27423  0.0  0.0   2384  1716 pts/4    S+   00:42   0:00 [www-client/firefox-71.0-r1] sandbox /usr/lib/portage/python3.6/ebuild.sh compile
portage  27506  0.0  0.0   4060  3172 pts/4    S+   00:42   0:00 /usr/bin/gmake -j2 -O -C /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/ff






If I kill one of above process with "profileserver.py",I got this:







702:40.09 rm -f jarlog/en-US.log
702:40.09 JARLOG_FILE=jarlog/en-US.log /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/ff/_virtualenvs/init/bin/python /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/build/pgo/profileserver.py
702:40.09 (firefox:12149): GLib-GIO-CRITICAL **: 02:58:43.429: g_dbus_proxy_new: assertion 'G_IS_DBUS_CONNECTION (connection)' failed
702:40.09 1576609123583 [e-mail]     WARN    Loading extension '[e-mail]': Reading manifest: Invalid extension permission: mozillaAddons
702:40.09 1576609123583 [e-mail]     WARN    Loading extension '[e-mail]': Reading manifest: Invalid extension permission: telemetry
702:40.09 1576609123583 [e-mail]     WARN    Loading extension '[e-mail]': Reading manifest: Invalid extension permission: resource://pdf.js/
702:40.09 1576609123583 [e-mail]     WARN    Loading extension '[e-mail]': Reading manifest: Invalid extension permission: about:reader*
702:40.10 ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
702:40.10 ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
702:40.10 ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
702:40.10 jarlog: /var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/ff/jarlog/en-US.log
702:40.10 (firefox:12317): GLib-GIO-CRITICAL **: 02:58:49.614: g_dbus_proxy_new: assertion 'G_IS_DBUS_CONNECTION (connection)' failed
702:40.10 (firefox:12317): GLib-GIO-CRITICAL **: 02:58:49.615: g_dbus_proxy_new: assertion 'G_IS_DBUS_CONNECTION (connection)' failed
702:40.10 (firefox:12317): GLib-GIO-CRITICAL **: 02:58:49.615: g_dbus_proxy_new: assertion 'G_IS_DBUS_CONNECTION (connection)' failed
702:40.10 JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
702:40.10 ###!!! [Parent][MessageChannel] Error: (msgtype=0x370120,name=PContent::Msg_AsyncMessage) Channel error: cannot send/recv
702:40.10 gmake[1]: *** [Makefile:207: profiledbuild] Terminated
702:40.10 gmake[1]: Leaving directory '/var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0/ff'
702:40.10 gmake: *** [client.mk:125: build] Error 2
702:40.11 116 compiler warnings present.
 * ERROR: www-client/firefox-71.0-r1::gentoo failed (compile phase):
 *   Failed to run './mach build --verbose'
 * 
 * Call stack:
 *     ebuild.sh, line  125:  Called src_compile
 *   environment, line 5268:  Called virtx './mach' 'build' '--verbose'
 *   environment, line 6646:  Called die
 * The specific snippet of code:
 *       [[ ${retval} -ne 0 ]] && die "Failed to run '$@'";
 * 
 * If you need support, post the output of `emerge --info '=www-client/firefox-71.0-r1::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=www-client/firefox-71.0-r1::gentoo'`.
 * The complete build log is located at '/var/tmp/portage/www-client/firefox-71.0-r1/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/www-client/firefox-71.0-r1/temp/environment'.
 * Working directory: '/var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0'
 * S: '/var/tmp/portage/www-client/firefox-71.0-r1/work/firefox-71.0'

>>> Failed to emerge www-client/firefox-71.0-r1, Log file:





emerge --info "=www-client/firefox-71.0-r1::gentoo"
Portage 2.3.79 (python 3.6.9-final-0, default/linux/amd64/17.1/no-multilib, gcc-9.2.0, glibc-2.29-r7, 4.19.86-gentoo x86_64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-4.19.86-gentoo-x86_64-Intel-R-_Core-TM-2_Extreme_CPU_X9100_@_3.06GHz-with-gentoo-2.6
KiB Mem:     5987632 total,    580392 free
KiB Swap:    4194300 total,   4193788 free
Timestamp of repository gentoo: Wed, 18 Dec 2019 00:45:01 +0000
sh bash 4.4_p23-r1
ld GNU ld (Gentoo 2.32 p2) 2.32.0
app-shells/bash:          4.4_p23-r1::gentoo
dev-java/java-config:     2.2.0-r4::gentoo
dev-lang/perl:            5.30.1::gentoo
dev-lang/python:          2.7.17::gentoo, 3.5.7::gentoo, 3.6.9::gentoo
dev-util/cmake:           3.14.6::gentoo
dev-util/pkgconfig:       0.29.2::gentoo
sys-apps/baselayout:      2.6-r1::gentoo
sys-apps/openrc:          0.41.2::gentoo
sys-apps/sandbox:         2.13::gentoo
sys-devel/autoconf:       2.13-r1::gentoo, 2.69-r4::gentoo
sys-devel/automake:       1.16.1-r1::gentoo
sys-devel/binutils:       2.32-r1::gentoo
sys-devel/gcc:            9.2.0-r2::gentoo
sys-devel/gcc-config:     2.1::gentoo
sys-devel/libtool:        2.4.6-r3::gentoo
sys-devel/make:           4.2.1-r4::gentoo
sys-kernel/linux-headers: 4.19::gentoo (virtual/os-headers)
sys-libs/glibc:           2.29-r7::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: webrsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000
    sync-webrsync-verify-signature: true


ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="@FREE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /var/lib/i2pd/certificates"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=native -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="X acl acpi amd64 apm berkdb bzip2 cli consolekit crypt cxx dri encode fontconfig gdbm gif iconv infinality jpeg libtirpc ncurses networkmanager nls nptl openmp pam pcre plasma png policykit pulseaudio readline seccomp split-usr ssl svg tcpd tiff truetype udev unicode xattr zlib" ABI_X86="64" ADA_TARGET="gnat_2018" ALSA_CARDS="hda-intel" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmxext mmx sse sse2 sse3 sse4_1 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" NETBEANS_MODULES="apisupport cnd groovy gsf harness ide identity j2ee java mobility nb php profiler soa visualweb webcommon websvccommon xml" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-2" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" RUBY_TARGETS="ruby24 ruby25" USERLAND="GNU" VIDEO_CARDS="i965" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

Reproducible: Always

Steps to Reproduce:
1.emerge -a firefox
2.
3.
Actual Results:  
unable to build firefox with PGO opened.

[Comment 1 Ionen Wolkens 2019-12-18 07:07:55 UTC]

Is it really reproducible for you? I've had this randomly happen once a few days ago with clang+lto+pgo but could never reproduce it, just worked fine the next time (I did reboot before trying again though, so it may have cleared/fixed something).

[Comment 2 lfs9azh 2019-12-18 09:01:41 UTC]

(In reply to Ionen Wolkens from comment #1)
> Is it really reproducible for you? I've had this randomly happen once a few
> days ago with clang+lto+pgo but could never reproduce it, just worked fine
> the next time (I did reboot before trying again though, so it may have
> cleared/fixed something).

Yes,everythime!
And when the build process stucked, the loadaverage was 2.x 2.x 2.x
But cpu idle was 9x.x%

[Comment 3 lfs9azh 2019-12-18 09:02:41 UTC]

(In reply to Ionen Wolkens from comment #1)
> Is it really reproducible for you? I've had this randomly happen once a few
> days ago with clang+lto+pgo but could never reproduce it, just worked fine
> the next time (I did reboot before trying again though, so it may have
> cleared/fixed something).

My cpu is 2c2t

[Comment 4 lfs9azh 2020-02-16 03:15:22 UTC]

Thanks, Jonas Stein
but no luck.

[Comment 5 Thomas Deutschmann (RETIRED) 2020-10-10 20:33:02 UTC]

The version this bug was reported against was removed. If you are still experiencing the issue with >=www-client/firefox-78.3.1 or newer please file a new bug or re-open.