Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 577862 - sys-apps/portage: emerge UnicodeDecodeError raised from locale.format() via portage.localization.localized_size
Summary: sys-apps/portage: emerge UnicodeDecodeError raised from locale.format() via p...
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core - Interface (emerge) (show other bugs)
Hardware: All All
: Normal normal (vote)
Assignee: Portage team
Keywords: InVCS
Depends on:
Blocks: portage-2.3.0
  Show dependency tree
Reported: 2016-03-20 19:53 UTC by Zac Medico
Modified: 2017-08-11 20:04 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Note You need to log in before you can comment on or make changes to this bug.
Description Zac Medico gentoo-dev 2016-03-20 19:53:55 UTC
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.4/emerge", line 50, in <module>
    retval = emerge_main()
  File "/usr/lib/python3.4/site-packages/_emerge/", line 1174, in emerge_main
    return run_action(emerge_config)
  File "/usr/lib/python3.4/site-packages/_emerge/", line 3231, in run_action
    emerge_config.args, spinner)
  File "/usr/lib/python3.4/site-packages/_emerge/", line 356, in action_build
  File "/usr/lib/python3.4/site-packages/_emerge/", line 7967, in display
    return display(self, mylist, favorites, verbosity)
  File "/usr/lib/python3.4/site-packages/_emerge/resolver/", line 861, in __call__
    self.verbose_size(pkg, None, pkg_info)
  File "/usr/lib/python3.4/site-packages/_emerge/resolver/", line 334, in verbose_size
    self.verboseadd += localized_size(mysize)
  File "/usr/lib/python3.4/site-packages/portage/", line 41, in localized_size
    formatted_num = locale.format('%d', num_kib, grouping=True)
  File "/usr/lib/python3.4/", line 195, in format
    return _format(percent, value, grouping, monetary, *additional)
  File "/usr/lib/python3.4/", line 216, in _format
    formatted, seps = _group(formatted, monetary=monetary)
  File "/usr/lib/python3.4/", line 140, in _group
    conv = localeconv()
  File "/usr/lib/python3.4/", line 111, in localeconv
    d = _localeconv()
UnicodeDecodeError: 'locale' codec can't decode byte 0xc2 in position 0: Invalid or incomplete multibyte or wide character
Comment 1 Zac Medico gentoo-dev 2016-03-20 19:56:57 UTC
It must be a python bug or something wrong with the locale.

We'll just have to handle the UnicodeDecodeError inside that portage.localization.localized_size function, and just fallback to some kind of dumb formatting.
Comment 2 Zac Medico gentoo-dev 2016-03-21 04:00:23 UTC
Looking at Modules/_localemodule.c in the python source tree, it must be failing to decode the locale data in the PyLocale_localeconv function where it calls this code:

#define RESULT_STRING(s)\
    do { \
        x = PyUnicode_DecodeLocale(l->s, NULL); \
        RESULT(#s, x); \
    } while (0)

The PyUnicode_DecodeLocale function is defined in Objects/unicodeobject.c, and it is capable of raising UnicodeDecodeError.

So, I think our best course of action is to handle the UnicodeDecodeError as suggested in comment #1.
Comment 3 Zac Medico gentoo-dev 2016-03-21 04:17:12 UTC
There's a patch in the following branch:

It can be applied as follows:

mkdir -p /etc/portage/patches/sys-apps/portage
wget -O /etc/portage/patches/sys-apps/portage/bug_577862.patch \
emerge -1 portage

I've posted the patch for review here:
Comment 5 Zac Medico gentoo-dev 2017-08-11 20:04:59 UTC
Fixed in 2.3.0.