Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 524398 - sys-apps/portage-2.2.14_rc1 - 'emerge -v' crashes with UnicodeDecodeError exception
Summary: sys-apps/portage-2.2.14_rc1 - 'emerge -v' crashes with UnicodeDecodeError exc...
Status: RESOLVED WORKSFORME
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Unclassified (show other bugs)
Hardware: All Linux
: Normal minor (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-10-03 17:11 UTC by simon
Modified: 2014-10-03 18:49 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description simon 2014-10-03 17:11:08 UTC
I'm getting a crash of emerge when using verbose output.

emerge -av1 portage

These are the packages that would be merged, in order:

Calculating dependencies... done!
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.3/emerge", line 50, in <module>
    retval = emerge_main()
  File "/usr/lib64/python3.3/site-packages/_emerge/main.py", line 1057, in emerge_main
    return run_action(emerge_config)
  File "/usr/lib64/python3.3/site-packages/_emerge/actions.py", line 4082, in run_action
    emerge_config.args, spinner)
  File "/usr/lib64/python3.3/site-packages/_emerge/actions.py", line 352, in action_build
    favorites=favorites)
  File "/usr/lib64/python3.3/site-packages/_emerge/depgraph.py", line 7512, in display
    return display(self, mylist, favorites, verbosity)
  File "/usr/lib64/python3.3/site-packages/_emerge/resolver/output.py", line 853, in __call__
    self.verbose_size(pkg, None, pkg_info)
  File "/usr/lib64/python3.3/site-packages/_emerge/resolver/output.py", line 334, in verbose_size
    self.verboseadd += localized_size(mysize)
  File "/usr/lib64/python3.3/site-packages/portage/localization.py", line 41, in localized_size
    formatted_num = locale.format('%d', num_kib, grouping=True)
  File "/usr/lib64/python3.3/locale.py", line 195, in format
    return _format(percent, value, grouping, monetary, *additional)
  File "/usr/lib64/python3.3/locale.py", line 216, in _format
    formatted, seps = _group(formatted, monetary=monetary)
  File "/usr/lib64/python3.3/locale.py", line 140, in _group
    conv = localeconv()
  File "/usr/lib64/python3.3/locale.py", line 111, in localeconv
    d = _localeconv()
UnicodeDecodeError: 'locale' codec can't decode byte 0xa4 in position 0: Invalid or incomplete multibyte or wide character

running with env LC_ALL=C works as expected.
similar to https://bugs.gentoo.org/show_bug.cgi?id=519124

Reproducible: Always

Steps to Reproduce:
1. call emerge  with parameter "-v" 

Actual Results:  
abort with UnicodeDecodeError: 'locale' codec can't decode byte 0xa4 in position 0: Invalid or incomplete multibyte or wide character

Expected Results:  
regular output and start of emerge.

locale
LANG=en_US.utf8
LC_CTYPE=en_US.utf8
LC_NUMERIC=de_DE@euro
LC_TIME=C
LC_COLLATE=de_DE@euro
LC_MONETARY=de_DE@euro
LC_MESSAGES=C
LC_PAPER="en_US.utf8"
LC_NAME=de_DE@euro
LC_ADDRESS=de_DE@euro
LC_TELEPHONE=de_DE@euro
LC_MEASUREMENT=de_DE@euro
LC_IDENTIFICATION=en_US
LC_ALL=

remerged portage and python!
no change.


emerge --info
Portage 2.2.14_rc1 (python 3.3.5-final-0, default/linux/amd64/13.0/desktop, gcc-4.7.3, glibc-2.19-r1, 3.16.0-gentoo x86_64)
=================================================================
System uname: Linux-3.16.0-gentoo-x86_64-Intel-R-_Core-TM-2_Quad_CPU_Q9550_@_2.83GHz-with-gentoo-2.2
KiB Mem:     8175300 total,     67972 free
KiB Swap:    9437180 total,   9436956 free
Timestamp of tree: Fri, 03 Oct 2014 14:15:01 +0000
ld GNU ld (Gentoo 2.23.2 p1.0) 2.23.2
distcc 3.1 x86_64-pc-linux-gnu [disabled]
ccache version 3.1.9 [disabled]
app-shells/bash:          4.2_p50
dev-java/java-config:     2.2.0
dev-lang/python:          2.7.7, 3.3.5-r1
dev-util/ccache:          3.1.9-r3
dev-util/cmake:           2.8.12.2-r1
dev-util/pkgconfig:       0.28-r1
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.12.4
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.11.6, 1.12.6, 1.13.4
sys-devel/binutils:       2.23.2
sys-devel/gcc:            4.6.4, 4.7.3-r1
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4.2-r1
sys-devel/make:           3.82-r4
sys-kernel/linux-headers: 3.13 (virtual/os-headers)
sys-libs/glibc:           2.19-r1
Repositories: gentoo LocalOverlay kde
Installed sets: @system
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA googleearth AdobeFlash-10.1 DOOM3"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O2 -pipe -w"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=native -O2 -pipe -w"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--jobs=4 --load-average=7.0 --keep-going"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch protect-owned sandbox sfperms splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="ftp://ftp.tu-clausthal.de/pub/linux/gentoo/ ftp://sunsite.informatik.rwth-aachen.de/pub/Linux/gentoo ftp://linux.rz.ruhr-uni-bochum.de/gentoo-mirror/ ftp://ftp.join.uni-muenster.de/pub/linux/distributions/gentoo http://mirrors.sec.informatik.tu-darmstadt.de/gentoo/ ftp://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/ "
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j5"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/overlays/local /var/lib/layman/kde"
SYNC="rsync://nikki/gentoo-portage"
USE="X a52 aac acl acpi alsa amd64 audiofile bash-completion branding bzip2 cairo cdaudio cdda cddb cdparanioa cdr cli consolekit cracklib crypt cups cxx dbus dri dts dvd dvdr emboss encode exif fam ffmpeg firefox flac fortran ftp gdbm gif glamor gnome gnutls gtk hal hddtemp iconv imap ipv6 jpeg jpeg2k kde kdeenablefinal kdehiddenvisibility lame lcms ldap libnotify lm_sensors mad matroska mmx mng modules mp2 mp3 mp4 mpeg mplayer mtp multilib musepack ncurses nls nptl offensive ogg opengl openmp opus pam pango pcre pdf png policykit ppds qt3 qt3support qt4 rar readline real samba sdl semantic-desktop session spell sse sse2 ssl startup-notification svg taglib tcpd theora threads tiff truetype udev udisks unicode upower usb vcd vdpau vim-syntax vorbis win32codecs wma wxwidgets x264 xcb xcomposite xine xml xv xvid zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" RUBY_TARGETS="ruby19 ruby20" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
Comment 1 Jouni Kosonen 2014-10-03 18:15:30 UTC
That pretty much looks like it's caused by mixing UTF-8 and ISO-8859-15 locale settings. Python3 assumes unicode and 'a4' is ISO-8859 for 'ยค'.

I don't think there's any software that works *better* in a mixed-encoding locale that it would in a pure UTF-8 environment, but I could be wrong.

Unless there's a very good reason to stick with the mixed-encoding locale I'd suggest replacing all instances of de_DE@euro in /etc/env.d/02locales with de_DE.UTF-8 (or de_DE.utf8, if that's how it's named in your /etc/locale.gen) and trying again in a new session.
Comment 2 simon 2014-10-03 18:42:57 UTC
Changing as suggested worked and resolved the issue. 
Thanks for the hint. 

Now i just have to check why LC_NUMERIC can't be updated  (env-update && source /etc/profile) but that's off topic - marking as resolved.
 bash: warning: setlocale: LC_NUMERIC: cannot change locale (de_DE.utf8)


Thanks again.