Summary: | www-client/firefox-62.0.2: build error if distcc used: UnicodeEncodeError: 'ascii' codec can't encode characters in position 166-168: ordinal not in range(128) | ||
---|---|---|---|
Product: | Gentoo Linux | Reporter: | Piotr Karbowski (RETIRED) <slashbeast> |
Component: | Current packages | Assignee: | Mozilla Gentoo Team <mozilla> |
Status: | RESOLVED FIXED | ||
Severity: | normal | CC: | jasmin+gentoo |
Priority: | Normal | Keywords: | PATCH |
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
URL: | https://bugzilla.mozilla.org/show_bug.cgi?id=1532169 | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Attachments: |
upstream fix from firefox-63.0_beta
Build log, with distcc enabled. build.log of firefox-63.0 fix-logger.patch |
Description
Piotr Karbowski (RETIRED)
![]() 0:58.47 /usr/lib64/ccache/bin/x86_64-pc-linux-gnu-g++ -o Unified_cpp_xpcom_components0.o -c -I/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/ff/dist/stl_wrappers -I/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/ff/dist/system_wrappers -include /var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/config/gcc_hidden.h -DNDEBUG=1 -DTRIMMED=1 -DSTATIC_EXPORTABLE_JS_API -DMOZ_HAS_MOZGLUE -DMOZILLA_INTERNAL_API -DIMPL_LIBXUL -I/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/xpcom/components -I/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/ff/xpcom/components -I/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/ff/xpcom -I/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/xpcom/base -I/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/xpcom/build -I/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/xpcom/ds -I/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/chrome -I/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/modules/libjar -I/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/ff/dist/include -I/usr/include/nspr -I/usr/include/nss -I/usr/include/pixman-1 -fPIC -DMOZILLA_CLIENT -include /var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/ff/mozilla-config.h -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wall -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wsign-compare -Wtype-limits -Wunreachable-code -Wwrite-strings -Wno-invalid-offsetof -Wc++1z-compat -Wduplicated-cond -Wimplicit-fallthrough -Wno-error=maybe-uninitialized -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=free-nonheap-object -Wno-error=multistatement-macros -Wno-error=class-memaccess -Wformat -Wformat-security -Wformat-overflow=2 -fno-sized-deallocation -pipe -march=skylake -fno-delete-null-pointer-checks -fno-lifetime-dse -fno-schedule-insns -fno-schedule-insns2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread -pipe -freorder-blocks -O2 -fomit-frame-pointer -I/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/widget/gtk/compat-gtk3 -pthread -I/usr/include/gtk-3.0/unix-print -I/usr/include/gtk-3.0 -I/usr/include/at-spi2-atk/2.0 -I/usr/include/at-spi-2.0 -I/usr/include/dbus-1.0 -I/usr/lib64/dbus-1.0/include -I/usr/include/gtk-3.0 -I/usr/include/cairo -I/usr/include/pango-1.0 -I/usr/include/harfbuzz -I/usr/include/pango-1.0 -I/usr/include/fribidi -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/uuid -I/usr/include/freetype2 -I/usr/include/libdrm -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/libpng16 -I/usr/include/gio-unix-2.0/ -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -MD -MP -MF .deps/Unified_cpp_xpcom_components0.o.pp -fdiagnostics-color /var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/ff/xpcom/components/Unified_cpp_xpcom_components0.cpp
0:58.47 In file included from /var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/ff/dist/include/js/TracingAPI.h:11,
0:58.47 from /var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/ff/dist/include/js/GCPolicyAPI.h:44,
0:58.47 from /var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/ff/dist/include/js/RootingAPI.h:22,
0:58.47 from /var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/ff/dist/include/js/CallArgs.h:73,
0:58.48 from /var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/ff/dist/include/js/CallNonGenericMethod.h:12,
0:58.48 from /var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/js/src/NamespaceImports.h:15,
0:58.48 from /var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/js/src/gc/Barrier.h:10,
0:58.48 from /var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/js/src/gc/Policy.h:13,
0:58.48 from /var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/js/src/gc/Rooting.h:10,
0:58.48 from /var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/js/src/vm/JSAtom.h:12,
0:58.48 from /var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/js/src/vm/JSAtom-inl.h:10,
0:58.48 from /var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/js/src/vm/JSAtom.cpp:11:
Exception in thread ProcessReader:
Traceback (most recent call last):
File "/usr/lib64/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib64/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/testing/mozbase/mozprocess/mozprocess/processhandler.py", line 1028, in _read
callback(line.rstrip())
File "/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/testing/mozbase/mozprocess/mozprocess/processhandler.py", line 944, in __call__
e(*args, **kwargs)
File "/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/python/mach/mach/mixin/process.py", line 86, in handleLine
line_handler(line)
File "/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/python/mozbuild/mozbuild/controller/building.py", line 681, in on_line
self.log(logging.INFO, 'build_output', {'line': message}, '{line}')
File "/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/python/mach/mach/mixin/logging.py", line 54, in log
extra={'action': action, 'params': params})
File "/usr/lib64/python2.7/logging/__init__.py", line 1238, in log
self._log(level, msg, args, **kwargs)
File "/usr/lib64/python2.7/logging/__init__.py", line 1293, in _log
self.handle(record)
File "/usr/lib64/python2.7/logging/__init__.py", line 1303, in handle
self.callHandlers(record)
File "/usr/lib64/python2.7/logging/__init__.py", line 1343, in callHandlers
hdlr.handle(record)
File "/usr/lib64/python2.7/logging/__init__.py", line 766, in handle
self.emit(record)
File "/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/python/mozbuild/mozbuild/controller/building.py", line 564, in emit
self.fh.write(msg)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 147-149: ordinal not in range(128)
21:41.27 0 compiler warnings present.
21:41.34 Exception when writing resource usage file: u'compile'
21:41.34 We know it took a while, but your build finally finished successfully!
To view resource usage of the build, run |mach resource-usage|.
Error running mach:
['build', '--verbose']
The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You should consider filing a bug for this issue.
If filing a bug, please include the full output of mach, including this error
message.
The details of the failure are as follows:
KeyError: u'compile'
File "/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/python/mozbuild/mozbuild/mach_commands.py", line 173, in build
mach_context=self._mach_context)
File "/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/python/mozbuild/mozbuild/controller/building.py", line 1275, in build
telemetry_data = monitor.get_resource_usage()
File "/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/python/mozbuild/mozbuild/controller/building.py", line 449, in get_resource_usage
o['tiers'] = self.tiers.tiered_resource_usage()
File "/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/python/mozbuild/mozbuild/controller/building.py", line 157, in tiered_resource_usage
self.add_resources_to_dict(t_entry, phase=tier)
File "/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/python/mozbuild/mozbuild/controller/building.py", line 166, in add_resources_to_dict
end=end, phase=phase, per_cpu=False)
File "/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/testing/mozbase/mozsystemmonitor/mozsystemmonitor/resourcemonitor.py", line 471, in aggregate_cpu_percent
data = self.phase_usage(phase)
File "/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2/testing/mozbase/mozsystemmonitor/mozsystemmonitor/resourcemonitor.py", line 430, in phase_usage
time_start, time_end = self.phases[phase]
* ERROR: www-client/firefox-62.0.2::gentoo failed (compile phase):
* (no error message)
*
* Call stack:
* ebuild.sh, line 124: Called src_compile
* environment, line 5135: Called die
* The specific snippet of code:
* MOZ_MAKE_FLAGS="${MAKEOPTS}" SHELL="${SHELL:-${EPREFIX}/bin/bash}" MOZ_NOSPAM=1 ./mach build --verbose || die
*
* If you need support, post the output of `emerge --info '=www-client/firefox-62.0.2::gentoo'`,
* the complete build log and the output of `emerge -pqv '=www-client/firefox-62.0.2::gentoo'`.
* The complete build log is located at '/var/portage/tmp/portage/www-client/firefox-62.0.2/temp/build.log'.
* The ebuild environment file is located at '/var/portage/tmp/portage/www-client/firefox-62.0.2/temp/environment'.
* Working directory: '/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2'
* S: '/var/portage/tmp/portage/www-client/firefox-62.0.2/work/firefox-62.0.2'
>>> Failed to emerge www-client/firefox-62.0.2, Log file:
I had the same problem. For some reason in my case the error occurs, when I'm compiling with distcc (on several machines) but succeeds, when I'm compiling local. Try disabling distcc and/or ccache as a workaround. Hi, can you please provide the `emerge --info` output? It's the same issue as Sven's. I disabled distcc and it built okay. In my case I was using a single remote host with no local compilation, unless distcc was retrying failed remote build. emerge --info output fwiw Portage 2.3.49 (python 3.6.6-final-0, default/linux/amd64/17.0/no-multilib, gcc-8.2.0, glibc-2.27-r6, 4.18.5 x86_64) ================================================================= System uname: Linux-4.18.5-x86_64-Intel-R-_Core-TM-_i5-6300U_CPU_@_2.40GHz-with-gentoo-2.6 KiB Mem: 16249600 total, 7957540 free KiB Swap: 16777212 total, 16764112 free Timestamp of repository gentoo: Sun, 23 Sep 2018 07:00:01 +0000 Head commit of repository gentoo: c9f394543b1fd098b046f497978ff5d551474f20 sh bash 4.4_p23 ld GNU ld (Gentoo 2.31.1 p3) 2.31.1 distcc 3.2rc1 x86_64-pc-linux-gnu [disabled] ccache version 3.4.3 [enabled] app-shells/bash: 4.4_p23::gentoo dev-java/java-config: 2.2.0-r4::gentoo dev-lang/perl: 5.26.2::gentoo dev-lang/python: 2.7.15::gentoo, 3.5.5-r1::gentoo, 3.6.6::gentoo dev-util/ccache: 3.4.3::gentoo dev-util/cmake: 3.12.2::gentoo dev-util/pkgconfig: 0.29.2::gentoo sys-apps/baselayout: 2.6-r1::gentoo sys-apps/openrc: 0.38.2::gentoo sys-apps/sandbox: 2.13::gentoo sys-devel/autoconf: 2.13::gentoo, 2.69-r4::gentoo sys-devel/automake: 1.16.1-r1::gentoo sys-devel/binutils: 2.31.1-r1::gentoo sys-devel/gcc: 8.2.0-r2::gentoo sys-devel/gcc-config: 2.0::gentoo sys-devel/libtool: 2.4.6-r5::gentoo sys-devel/make: 4.2.1-r4::gentoo sys-kernel/linux-headers: 4.17::gentoo (virtual/os-headers) sys-libs/glibc: 2.27-r6::gentoo Repositories: gentoo location: /var/portage/tree sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 sync-rsync-verify-metamanifest: yes sync-rsync-extra-opts: sync-rsync-verify-max-age: 24 sync-rsync-verify-jobs: 1 deadbeef-overlay location: /var/lib/layman/deadbeef-overlay masters: gentoo priority: 50 foo-overlay location: /var/lib/layman/foo-overlay masters: gentoo priority: 50 seden location: /var/lib/layman/seden masters: gentoo priority: 50 ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O3 -pipe -march=skylake" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /etc/stunnel/stunnel.conf /usr/lib64/libreoffice/program/sofficerc /usr/share/gnupg/qualified.txt /usr/share/themes/oxygen-gtk/gtk-2.0 /usr/share/themes/oxygen-gtk/gtk-3.0" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php7.2/ext-active/ /etc/php/cgi-php7.2/ext-active/ /etc/php/cli-php7.2/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O3 -pipe -march=skylake" DISTDIR="/var/portage/distfiles" EMERGE_DEFAULT_OPTS=" --with-bdeps y --binpkg-respect-use y --buildpkg-exclude 'virtual/* sys-kernel/*-sources games-*/* app-emulation/emul-linux-*' --backtrack 300 " ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs buildpkg ccache distlocks ebuild-locks fixlafiles merge-sync multilib-strict news noinfo parallel-fetch preserve-libs protect-owned sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://ftp.heanet.ie/pub/gentoo/ http://distfiles.gentoo.org/" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="--quiet -j4" PKGDIR="/var/portage/packages" PORTAGE_COMPRESS="pigz" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/portage/tmp" USE="acl amd64 berkdb bzip2 cli crypt cups cxx dri fontconfig fortran gdbm glamor iconv jpeg libtirpc logrotate modules ncurses nls nptl opengl openmp pam pcre pic png readline seccomp smp ssl tcpd threads unicode vim-syntax xattr xft zlib zsh-completion" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon plan sheets stage words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 fma3 mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse synaptics" KERNEL="linux" L10N="en" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LLVM_TARGETS="BPF" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6 php7-0" POSTGRES_TARGETS="postgres9_5 postgres10" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" RUBY_TARGETS="ruby21 ruby23 ruby24 ruby25" USERLAND="GNU" VIDEO_CARDS="intel i965" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS Please provide complete build.log. It would be interesting to know if this is a regression, i.e. if you were able to compile previous firefox v62.0 using distcc. Created attachment 548022 [details, diff] upstream fix from firefox-63.0_beta I got a similar warning when compiling firefox-60.2.1 with rust-bin-1.28.0-r1, and decided to fix it for myself via epatch_user. Source: https://bugzilla.mozilla.org/show_bug.cgi?id=1481016#c6 (My warning was never turned into a compile failure, but you may want to try anyway?) Created attachment 549664 [details]
Build log, with distcc enabled.
Sadly, the rust fix does not change anything here, still the same issue with building firefox with FEATURES=distcc. Uh, how could I missed the attached patch. This patch is part of firefox-63.0 and we already have reports of successful build attempts of firefox-63.0 using distcc. So please retry and see if this is also fixed for you, now. Created attachment 552804 [details]
build.log of firefox-63.0
I cannot confirm that firefox-63.0 works with distcc. I have the same type of crash as I had before. Attached new log. From reading your log I don't think you run into a distcc/compilation problem. I think your TERM setting is causing the problem. Please retry with TERM=dumb. I just did the build without distcc and it went fine, firefox-63.0 built successfully. Since I have no distcc in make.conf, I enable it on ad-hoc basis by doing export MAKEOPTS='-j32 -l6'; export FEATURES='distcc' I will try tomorrow ith distcc and the TERM=dumb Created attachment 567680 [details, diff]
fix-logger.patch
Confirmed.
The attached patch is a poor man's workaround.
(In reply to Thomas Deutschmann from comment #15) > Created attachment 567680 [details, diff] [details, diff] > fix-logger.patch > > Confirmed. > > The attached patch is a poor man's workaround. Is this patch safe for all cases or will we need some sort of detection for utf8 vs iso8859-1 / ascii and make it conditional on that? It should be safe for everyone. However, when the error happens, output will now contain unreadable characters instead, something like:
> AudioEventTimeline.h:27:8: note: ���struct mozilla::dom::AudioTimelineEvent��� declared here
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=5f1462820b95e68cd9fa572f5183fc85587b424b commit 5f1462820b95e68cd9fa572f5183fc85587b424b Author: Thomas Deutschmann <whissi@gentoo.org> AuthorDate: 2019-03-27 23:09:28 +0000 Commit: Thomas Deutschmann <whissi@gentoo.org> CommitDate: 2019-03-27 23:17:45 +0000 www-client/firefox: bump to v66.0.2 - drop 6004_revert-bug1497976_bug1507655.patch which causes build problems for some users. [Bug 681548] - TerminalLoggingHandler patched to allow non-ascii characters. [Bug 666948] - Add rust-1.33+ support. [Bug 679656] Bug: https://bugs.gentoo.org/679656 Closes: https://bugs.gentoo.org/666948 Closes: https://bugs.gentoo.org/681548 Package-Manager: Portage-2.3.62, Repoman-2.3.12 Signed-off-by: Thomas Deutschmann <whissi@gentoo.org> www-client/firefox/Manifest | 185 +++++++++++---------- ...firefox-66.0.1.ebuild => firefox-66.0.2.ebuild} | 2 +- 2 files changed, 94 insertions(+), 93 deletions(-) |