| Summary: | sys-auth/ykpers-1.19.3-r1 - a2x: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1885: ordinal not in range(128) | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Martin Dummer <martin.dummer> |
| Component: | Current packages | Assignee: | Marek Szuba <marecki> |
| Status: | RESOLVED DUPLICATE | ||
| Severity: | normal | CC: | martin.dummer |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: | full build log | ||
# emerge --info Portage 2.3.67 (python 3.6.8-final-0, default/linux/amd64/17.0/desktop/plasma, gcc-8.3.0, glibc-2.29-r2, 5.0.20-gentoo-int6 x86_64) ================================================================= System uname: Linux-5.0.20-gentoo-int6-x86_64-Intel-R-_Core-TM-_i5-2520M_CPU_@_2.50GHz-with-gentoo-2.6 KiB Mem: 7881916 total, 2560264 free KiB Swap: 8388604 total, 7844360 free Timestamp of repository gentoo: Tue, 18 Jun 2019 09:00:01 +0000 Head commit of repository gentoo: ae98975cbc8d7c75ac789ca508ba6441341f674f sh bash 5.0_p7 ld GNU ld (Gentoo 2.31.1 p7) 2.31.1 distcc 3.3.2 x86_64-pc-linux-gnu [disabled] app-shells/bash: 5.0_p7::gentoo dev-java/java-config: 2.2.0-r4::gentoo dev-lang/perl: 5.30.0::gentoo dev-lang/python: 2.7.16::gentoo, 3.6.8::gentoo dev-util/cmake: 3.14.5::gentoo dev-util/pkgconfig: 0.29.2::gentoo sys-apps/baselayout: 2.6-r1::gentoo sys-apps/openrc: 0.41.2::gentoo sys-apps/sandbox: 2.17::gentoo sys-devel/autoconf: 2.13-r1::gentoo, 2.69-r4::gentoo sys-devel/automake: 1.11.6-r3::gentoo, 1.16.1-r1::gentoo sys-devel/binutils: 2.31.1-r6::gentoo sys-devel/gcc: 8.3.0-r1::gentoo sys-devel/gcc-config: 2.0::gentoo sys-devel/libtool: 2.4.6-r5::gentoo sys-devel/make: 4.2.1-r4::gentoo sys-kernel/linux-headers: 5.0-r1::gentoo (virtual/os-headers) sys-libs/glibc: 2.29-r2::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 sync-rsync-extra-opts: sync-rsync-verify-jobs: 1 sync-rsync-verify-max-age: 24 sync-rsync-verify-metamanifest: yes md-private location: /usr/local/portage masters: gentoo priority: 0 atom location: /var/lib/layman/atom masters: gentoo priority: 50 das-labor location: /var/lib/layman/das-labor masters: gentoo priority: 50 vdr-devel location: /var/lib/layman/vdr-devel masters: gentoo priority: 50 vmware location: /var/lib/layman/vmware masters: gentoo priority: 50 ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="@FREE" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -march=core2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt /var/bind" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php7.3/ext-active/ /etc/php/cgi-php7.3/ext-active/ /etc/php/cli-php7.3/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -march=core2 -pipe" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--keep-going --ask-enter-invalid --quiet-build=y --verbose-conflicts" ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs clean-logs compress-build-logs compressdebug config-protect-if-modified distlocks ebuild-locks fail-clean fakeroot fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="C" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LINGUAS="en de" MAKEOPTS="-j4" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="X a52 aac aalib acl acpi activities alsa amd64 ao apache2 apng arping audiofile berkdb bluetooth branding bzip2 cairo cdda cdr cli clucene consolekit cron crypt css cups curl cxx dbus declarative device-mapper dhcpcd dri dts dvb dvd dvdr ebook embedded emboss enca encode exif extras fam ffmpeg flac fontconfig fortran gd gdbm geolocation gif git glade glamor gnutls gphoto2 gpm gstreamer gtk httpd hybrid-auth iconv icu id3tag ieee1394 ipv6 ithreads java javafx jpeg kde kerberos kipi kvm kwallet lame lcms ldap lensfun libass libkms libnotify libproxy libtirpc live lm_sensors logrotate lzo macvtap mad mikmod minizip mjpeg mmap mng mod modemmanager mp3 mp4 mpeg mtp multilib musicbrainz mysql ncurses networkmanager nfsv3 nfsv4 nls nptl nptlonly nsplugin ntlm ntp ogg old-daemons openconnect openexr opengl openipmi openmp pam pango pcre pdf phonon plasma pm-utils png policykit ppds pulseaudio python qemu qml qt5 rdp readline resolvconf rss samba sasl sdl seccomp semantic-desktop smartcard smi smp sndfile snmp spell spice sql sqlite ssl startup-notification stl svg symlink tcpd threads threadsafe thumbnail tiff truetype udev udisks unicode upower usb usbredir utils v4l v4l2 vaapi vcd vdpau vim-syntax vnc vorbis webkit widgets wifi wxwidgets x264 xattr xcb xcomposite xinerama xml xrandr xscreensaver xv xvid xvmc zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" CAMERAS="canon ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse synaptics evdev" KERNEL="linux" L10N="en de" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LIRC_DEVICES="userspace" NETBEANS_MODULES="apisupport cnd groovy gsf harness ide identity j2ee java mobility nb php profiler soa visualweb webcommon websvccommon xml" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-3" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" QEMU_SOFTMMU_TARGETS="i386 x86_64 arm" QEMU_USER_TARGETS="i386 x86_64 arm" RUBY_TARGETS="ruby24" SANE_BACKENDS="epson epson2" USERLAND="GNU" VIDEO_CARDS="intel i965" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS The failing command is /usr/bin/a2x which belongs to app-text/asciidoc. The failing emerge of sys-auth/ykpers-1.19.3-r1 happened with app-text/asciidoc-8.6.10_p20181016 It is successful after I downgraded to app-text/asciidoc-8.6.10 So I assume the problem is in app-text/asciidoc-8.6.10_p20181016 Please attach the entire build log to this bug report. Created attachment 580150 [details]
full build log
(In reply to Martin Dummer from comment #2) > The failing command is > > /usr/bin/a2x > > which belongs to app-text/asciidoc. > > The failing emerge of sys-auth/ykpers-1.19.3-r1 happened with > app-text/asciidoc-8.6.10_p20181016 > > It is successful after I downgraded to app-text/asciidoc-8.6.10 > > So I assume the problem is in app-text/asciidoc-8.6.10_p20181016 How reliably can you reproduce this? I'm using python-3.6.8 with asciidoc-8.6.10_p20181016 and it compiles just fine. It looks like some sort of corruption to me. I looked at the byte frequency distribution of ykinfo.1.adoc. There are zero "0xe2" symbols in the file. In fact, ykinfo.1.adoc is only 1703 bytes long, so I'm not sure why the error says position 1885. Can you reinstall 'app-text/asciidoc-8.6.10_p20181016' and see if the build failure is reproducible? I ran into this issue while building ykpers in a catalyst stage4, where the locale is set to the non-unicode "POSIX". Setting LC_CTYPE to some UTF-8 locale worked around it. (In reply to Daniel Cordero from comment #6) > locale is set to the non-unicode "POSIX". Setting LC_CTYPE to some UTF-8 > locale worked around it. This workaround helped me in Bug 702338 Same in bug 707026 I had the same issue, I can confirm setting/changing to a UTF8 locale fixed the issue.
For reference:
undertaker ~ # equery list asciidoc
* Searching for asciidoc ...
[IP-] [ ] app-text/asciidoc-9.0.0_rc1:0
undertaker ~ # emerge --info
Portage 2.3.89 (python 3.6.10-final-0, default/linux/amd64/17.1/desktop/plasma/systemd, gcc-9.2.0, glibc-2.30-r4, 5.4.20-gentoo x86_64)
=================================================================
System uname: Linux-5.4.20-gentoo-x86_64-AMD_Ryzen_5_3600_6-Core_Processor-with-gentoo-2.7
KiB Mem: 32908368 total, 21813644 free
KiB Swap: 34052416 total, 34052416 free
Timestamp of repository gentoo: Thu, 27 Feb 2020 15:00:01 +0000
Head commit of repository gentoo: 5396d2d6f7ac6aa024f6f5d68b9c32913c13f3bc
sh bash 5.0_p16
ld GNU ld (Gentoo 2.34 p1) 2.34.0
app-shells/bash: 5.0_p16::gentoo
dev-lang/perl: 5.30.1::gentoo
dev-lang/python: 2.7.17-r1::gentoo, 3.6.10::gentoo, 3.7.6::gentoo, 3.8.2::gentoo
dev-util/cmake: 3.16.4::gentoo
sys-apps/baselayout: 2.7::gentoo
sys-apps/sandbox: 2.18::gentoo
sys-devel/autoconf: 2.13-r1::gentoo, 2.69-r5::gentoo
sys-devel/automake: 1.16.1-r2::gentoo
sys-devel/binutils: 2.34::gentoo
sys-devel/gcc: 9.2.0-r2::gentoo
sys-devel/gcc-config: 2.2.1::gentoo
sys-devel/libtool: 2.4.6-r6::gentoo
sys-devel/make: 4.3::gentoo
sys-kernel/linux-headers: 5.5::gentoo (virtual/os-headers)
sys-libs/glibc: 2.30-r4::gentoo
Repositories:
gentoo
location: /var/db/repos/gentoo
sync-type: rsync
sync-uri: rsync://rsync.gentoo.org/gentoo-portage
priority: -1000
sync-rsync-verify-max-age: 24
sync-rsync-verify-metamanifest: yes
sync-rsync-extra-opts:
sync-rsync-verify-jobs: 1
brave-overlay
location: /var/lib/layman/brave-overlay
masters: gentoo
priority: 50
gentoo-playground
location: /var/lib/layman/gentoo-playground
masters: gentoo
priority: 50
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=znver1"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe -march=znver1"
DISTDIR="/var/cache/distfiles"
ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2 -pipe -march=znver1"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe -march=znver1"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="en"
MAKEOPTS="-j8"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="X a52 aac acl acpi activities alsa amd64 archive berkdb bluetooth bluray bmp branding bzip2 cairo cdda cddb cdr cli corefonts crypt cups cxx dbus declarative dga dhclient discover dri dts dvd dvdr emboss encode exif faac faad fat ffmpeg flac fortran freerdp gdbm geolocation gif gnome-keyring gpm gtk gtk3 h iconv icu imlib ios ipod jpeg jpeg2k jpg kate kde kipi kwallet lastfm lcms ldap libnotify libtirpc lzma lzo mad mkv mng modplug moodbar mp3 mp4 mpeg mtp multilib musicbrainz ncurses networkmanager nls nptl ntfs offensive ogg opengl openmp pam pango pcre pdf phonon plasma png policykit ppds pulseaudio qml qt5 quicktime rar rdp readline samba sdl seccomp semantic-desktop share spell split-usr sqlite sqlite3 ssl startup-notification svg symlink systemd taglib tcpd theora tiff toolbar truetype udev udisks uefi unicode upnp upower usb vaapi vdpau vim-syntax vorbis webengine webp widgets wifi win32codecs wxwidgets x264 xattr xcb xcomposite xfs xml xv xvid y zeroconf zlib" ABI_X86="64" ADA_TARGET="gnat_2018" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" CAMERAS="canon" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-2" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" RUBY_TARGETS="ruby24 ruby25" USERLAND="GNU" VIDEO_CARDS="nv nouveau amdgpu fbdev vesa" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset: CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
I was looking at the devmanual for unrelated reasons and I saw that the python-utils-r1.eclass contains a function to export a utf8 ctype https://devmanual.gentoo.org/eclass-reference/python-utils-r1.eclass/index.html#lbAE > python_export_utf8_locale > Attempts to export a usable UTF-8 locale in the LC_CTYPE variable. > Does nothing if LC_ALL is defined, or if the current locale uses a UTF-8 charmap. > This may be used to work around the quirky open() behavior of python3. > Return value: 0 on success, 1 on failure. Could this be added to the affected ebuilds? (In reply to Daniel Cordero from comment #10) > > python_export_utf8_locale > > Attempts to export a usable UTF-8 locale in the LC_CTYPE variable. Good idea, I will test it. To get rid of consolekit as per one of the recent news items, I was rebuilding ykpers on several machines. One had no problems, while the other showed this exact same error. For me it helped to build with MAKEOPTS="-j1". The machine where it failed usually runs with -j13 the other with -j5. Is this still a problem with 1.20.0-r1? Actually, scratch that - this has already been figured out in another ticket. *** This bug has been marked as a duplicate of bug 730444 *** |
ebuild sys-auth/ykpers-1.19.3-r1 fails in compile stage when generating manpages. Relevant output is: /bin/sh /var/tmp/portage/sys-auth/ykpers-1.19.3-r1/work/yubikey-personalization-1.19.3/build-aux/missing a2x -L --format=manpage -a revdate="Version 1.19.3" --xsltproc-opts="--nonet" ykchalresp.1.adoc /bin/sh /var/tmp/portage/sys-auth/ykpers-1.19.3-r1/work/yubikey-personalization-1.19.3/build-aux/missing a2x -L --format=manpage -a revdate="Version 1.19.3" --xsltproc-opts="--nonet" ykinfo.1.adoc Traceback (most recent call last): File "/usr/bin/a2x", line 931, in <module> source_options = get_source_options(sys.argv[-1]) File "/usr/bin/a2x", line 337, in get_source_options for line in f: File "/usr/lib64/python3.6/encodings/ascii.py", line 26, in decode return codecs.ascii_decode(input, self.errors)[0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1885: ordinal not in range(128) make[1]: *** [Makefile:1417: ykpersonalize.1] Error 1