Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 519124 - sys-apps/portage-2.2.11 - emerge crashes with UnicodeDecodeError exception under et_EE
Summary: sys-apps/portage-2.2.11 - emerge crashes with UnicodeDecodeError exception un...
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Unclassified (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
: 519230 (view as bug list)
Depends on:
Blocks: 484436
  Show dependency tree
 
Reported: 2014-08-05 11:08 UTC by Priit Laes (IRC: plaes)
Modified: 2014-10-03 16:58 UTC (History)
7 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Suggested patch for portage (portage_localization.patch,452 bytes, patch)
2014-08-06 19:45 UTC, Jouni Kosonen
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Priit Laes (IRC: plaes) 2014-08-05 11:08:38 UTC
After upgrading portage to sys-apps/portage-2.2.11, following exception happens when running emerge with -pv arguments (like `emerge -1pv wireshark`):

These are the packages that would be merged, in order:

Calculating dependencies... done!
Traceback (most recent call last):
  File "/usr/bin/emerge", line 51, in <module>
    retval = emerge_main()
  File "/usr/lib64/portage/pym/_emerge/main.py", line 1054, in emerge_main
    return run_action(emerge_config)
  File "/usr/lib64/portage/pym/_emerge/actions.py", line 4066, in run_action
    emerge_config.args, spinner)
  File "/usr/lib64/portage/pym/_emerge/actions.py", line 418, in action_build
    favorites=favorites)
  File "/usr/lib64/portage/pym/_emerge/depgraph.py", line 7328, in display
    return display(self, mylist, favorites, verbosity)
  File "/usr/lib64/portage/pym/_emerge/resolver/output.py", line 853, in __call__
    self.verbose_size(pkg, None, pkg_info)
  File "/usr/lib64/portage/pym/_emerge/resolver/output.py", line 334, in verbose_size
    self.verboseadd += localized_size(mysize)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 2: ordinal not in range(128)

Command succeeds when running it as LC_ALL=C emerge -1pv wireshark
Comment 1 Dmitry Suloev 2014-08-06 09:45:32 UTC
Same problem with RU locale.
Comment 2 Ivan Budiakov 2014-08-06 10:57:28 UTC
The same problem. Portage-2.2.11 2.2.11-r1. Portage-2.2.10 no problem.

werebear ~ # emerge --version
Portage 2.2.10 (default/linux/amd64/13.0/no-multilib, gcc-4.8.3, glibc-2.19-r1, 3.16.0-gentoo x86_64)

werebear ~ # emerge -uavDN  world

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild     U  ] sys-devel/gnuconfig-20140728 [20140212] 45 kB
[ebuild     U  ] sys-apps/portage-2.2.11-r1 [2.2.10] USE="(ipc) -build -doc -epydoc (-pypy) -python2 -python3 (-selinux) -xattr" LINGUAS="ru" PYTHON_TARGETS="python2_7 python3_3 -pypy -python3_2 -python3_4 (-pypy2_0%) (-python2_6%)" 0 kB
[ebuild     U  ] sys-libs/e2fsprogs-libs-1.42.11 [1.42.10] USE="nls static-libs" 591 kB
[ebuild     U  ] media-libs/libbluray-0.6.1 [0.5.0-r1] USE="truetype xml -aacs -java -static-libs -utils" 573 kB
[ebuild     U  ] sys-fs/e2fsprogs-1.42.11 [1.42.10] USE="nls -static-libs" 6,205 kB
[ebuild     U  ] mail-mta/exim-4.84_rc1 [4.83] USE="dcc dkim dnsdb dovecot-sasl dsn exiscan-acl gnutls ipv6 lmtp maildir mysql pam postgres prdr spf sqlite srs ssl syslog tcpd -X -dlfunc -dmarc -doc -ldap -mbx -nis -perl -pkcs11 -proxy -radius -redis -sasl (-selinux) -tpda" 0 kB

Total: 6 packages (6 upgrades), Size of downloads: 7,412 kB

Would you like to merge these packages? [Yes/No] n

Quitting.

werebear ~ # emerge --version
Portage 2.2.11-r1 (python 2.7.8-final-0, default/linux/amd64/13.0/no-multilib, gcc-4.8.3, glibc-2.19-r1, 3.16.0-gentoo x86_64)

werebear ~ # emerge -uavDN  world

These are the packages that would be merged, in order:

Calculating dependencies... done!
Traceback (most recent call last):
  File "/usr/bin/emerge", line 51, in <module>
    retval = emerge_main()
  File "/usr/lib64/portage/pym/_emerge/main.py", line 1054, in emerge_main
    return run_action(emerge_config)
  File "/usr/lib64/portage/pym/_emerge/actions.py", line 4066, in run_action
    emerge_config.args, spinner)
  File "/usr/lib64/portage/pym/_emerge/actions.py", line 352, in action_build
    favorites=favorites)
  File "/usr/lib64/portage/pym/_emerge/depgraph.py", line 7328, in display
    return display(self, mylist, favorites, verbosity)
  File "/usr/lib64/portage/pym/_emerge/resolver/output.py", line 853, in __call__
    self.verbose_size(pkg, None, pkg_info)
  File "/usr/lib64/portage/pym/_emerge/resolver/output.py", line 334, in verbose_size
    self.verboseadd += localized_size(mysize)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 1: ordinal not in range(128)


werebear ~ # locale
LANG=ru_RU.UTF-8
LC_CTYPE="ru_RU.UTF-8"
LC_NUMERIC="ru_RU.UTF-8"
LC_TIME="ru_RU.UTF-8"
LC_COLLATE="ru_RU.UTF-8"
LC_MONETARY="ru_RU.UTF-8"
LC_MESSAGES="ru_RU.UTF-8"
LC_PAPER="ru_RU.UTF-8"
LC_NAME="ru_RU.UTF-8"
LC_ADDRESS="ru_RU.UTF-8"
LC_TELEPHONE="ru_RU.UTF-8"
LC_MEASUREMENT="ru_RU.UTF-8"
LC_IDENTIFICATION="ru_RU.UTF-8"
LC_ALL=


werebear ~ # eselect python list
Available Python interpreters:
  [1]   python2.7 *
  [2]   python3.3
  [3]   python3.4

werebear ~ # emerge --info
Portage 2.2.11-r1 (python 2.7.8-final-0, default/linux/amd64/13.0/no-multilib, gcc-4.8.3, glibc-2.19-r1, 3.16.0-gentoo x86_64)
=================================================================
System uname: Linux-3.16.0-gentoo-x86_64-AMD_A8-3870_APU_with_Radeon-tm-_HD_Graphics-with-gentoo-2.2
KiB Mem:     7636764 total,     69692 free
KiB Swap:    2008120 total,   2008120 free
Timestamp of tree: Wed, 06 Aug 2014 10:30:01 +0000
ld GNU ld (Gentoo 2.24 p1.4) 2.24
app-shells/bash:          4.2_p47
dev-lang/python:          2.7.8, 3.3.5-r1, 3.4.1
dev-util/cmake:           2.8.12.2-r1
dev-util/pkgconfig:       0.28-r2
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.12.4
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.69
sys-devel/automake:       1.14.1
sys-devel/binutils:       2.24-r3
sys-devel/gcc:            4.8.3
sys-devel/gcc-config:     1.8
sys-devel/libtool:        2.4.2-r1
sys-devel/make:           4.0-r1
sys-kernel/linux-headers: 3.16 (virtual/os-headers)
sys-libs/glibc:           2.19-r1
Repositories: gentoo sunrise nektoo werebear77-overlay
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -march=amdfam10 -mcx16 -mpopcnt -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /var/bind"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.5/ext-active/ /etc/php/cgi-php5.5/ext-active/ /etc/php/cli-php5.5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--quiet-build=n"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://mirror.yandex.ru/gentoo-distfiles/ http://mirror.corbina.net/gentoo/  http://trumpetti.atm.tut.fi/gentoo/  http://distfiles.gentoo.org http://www.ibiblio.org/pub/Linux/distributions/gentoo"
LANG="ru_RU.UTF-8"
LC_ALL=""
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j5"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/var/lib/layman/sunrise /var/lib/layman/nektoo /usr/local/portage/overlay"
USE="3dnow acl acpi amd64 bash-completion berkdb bzip2 clamav cli cracklib crypt curl curlwrappers cxx dbus djvu dri fastcgi fbcon fortran ftp gdbm geoip gif gnutls hddtemp iconv idn imap ipv6 jbig jpeg jpeg2k libffi lzo maildir mime mmx mmxext modules mpi mysql ncurses nls nptl openmp pam pch pcre pdf png postgres private-headers python readline samba session slang sockets sqlite sqlite3 sse sse2 sse3 sse4a ssl szip tcpd threads tiff unicode usb xml zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias asis auth_digest authn_dbd cern_meta charset_lite dbd dumpio ident imagemap log_forensic proxy proxy_ajp proxy_balancer proxy_connect proxy_ftp proxy_http substitute version access_compat authn_core authz_core cache_disk cgid lbmethod_bybusyness lbmethod_byrequests lbmethod_bytraffic lbmethod_heartbeat proxy_fcgi proxy_scgi ratelimit remoteip reqtimeout slotmem_shm socache_shmcb unixd" APACHE2_MPMS="event" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64 pc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="ru en" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-4" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" RUBY_TARGETS="ruby19 ruby20" USERLAND="GNU" VIDEO_CARDS="radeon" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, SYNC, USE_PYTHON
Comment 3 Jouni Kosonen 2014-08-06 19:45:08 UTC
Created attachment 382406 [details, diff]
Suggested patch for portage

It looks like the new localized_size function returns unusable content for locales where the grouping character is non-breaking space (U+00A0, bytestream 0xC2 0xA0) and portage is using python-2.

Using the _() -wrapper should be enough to fix this.
Comment 4 Jeroen Roovers gentoo-dev 2014-08-06 20:50:52 UTC
*** Bug 519230 has been marked as a duplicate of this bug. ***
Comment 5 Dmitry Suloev 2014-08-07 07:47:53 UTC
(In reply to Jouni Kosonen from comment #3)
> Created attachment 382406 [details, diff] [details, diff]
> Suggested patch for portage
> 
> It looks like the new localized_size function returns unusable content for
> locales where the grouping character is non-breaking space (U+00A0,
> bytestream 0xC2 0xA0) and portage is using python-2.
> 
> Using the _() -wrapper should be enough to fix this.

Patch helps, thx.
Comment 6 Alexander Berntsen (RETIRED) gentoo-dev 2014-08-11 11:24:44 UTC
This is fixed in git.
Comment 7 Brian Dolbec gentoo-dev 2014-08-22 16:55:27 UTC
Released in portage-2.2.12
Comment 8 simon 2014-10-03 16:00:11 UTC
Getting similar issue with 
Portage 2.2.14_rc1 (python 3.3.5-final-0, default/linux/amd64/13.0/desktop, gcc-4.7.3, glibc-2.19-r1, 3.16.0-gentoo x86_64)

--
emerge -av1 portage

These are the packages that would be merged, in order:

Calculating dependencies... done!
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.3/emerge", line 50, in <module>
    retval = emerge_main()
  File "/usr/lib64/python3.3/site-packages/_emerge/main.py", line 1057, in emerge_main
    return run_action(emerge_config)
  File "/usr/lib64/python3.3/site-packages/_emerge/actions.py", line 4082, in run_action
    emerge_config.args, spinner)
  File "/usr/lib64/python3.3/site-packages/_emerge/actions.py", line 352, in action_build
    favorites=favorites)
  File "/usr/lib64/python3.3/site-packages/_emerge/depgraph.py", line 7512, in display
    return display(self, mylist, favorites, verbosity)
  File "/usr/lib64/python3.3/site-packages/_emerge/resolver/output.py", line 853, in __call__
    self.verbose_size(pkg, None, pkg_info)
  File "/usr/lib64/python3.3/site-packages/_emerge/resolver/output.py", line 334, in verbose_size
    self.verboseadd += localized_size(mysize)
  File "/usr/lib64/python3.3/site-packages/portage/localization.py", line 41, in localized_size
    formatted_num = locale.format('%d', num_kib, grouping=True)
  File "/usr/lib64/python3.3/locale.py", line 195, in format
    return _format(percent, value, grouping, monetary, *additional)
  File "/usr/lib64/python3.3/locale.py", line 216, in _format
    formatted, seps = _group(formatted, monetary=monetary)
  File "/usr/lib64/python3.3/locale.py", line 140, in _group
    conv = localeconv()
  File "/usr/lib64/python3.3/locale.py", line 111, in localeconv
    d = _localeconv()
UnicodeDecodeError: 'locale' codec can't decode byte 0xa4 in position 0: Invalid or incomplete multibyte or wide character

--

locale
LANG=en_US.utf8
LC_CTYPE=en_US.utf8
LC_NUMERIC=de_DE@euro
LC_TIME=C
LC_COLLATE=de_DE@euro
LC_MONETARY=de_DE@euro
LC_MESSAGES=C
LC_PAPER="en_US.utf8"
LC_NAME=de_DE@euro
LC_ADDRESS=de_DE@euro
LC_TELEPHONE=de_DE@euro
LC_MEASUREMENT=de_DE@euro
LC_IDENTIFICATION=en_US
LC_ALL=

remerged portage and python!
Is this a new one or maybe some local issue on my system?
Comment 9 Arfrever Frehtes Taifersar Arahesis 2014-10-03 16:58:19 UTC
(In reply to simon from comment #8)

File a new bug.