Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 498748 - app-portage/gentoolkit-0.3.0.8-r2 - equery u uwsgi: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 10: ordinal not in range(128)
Summary: app-portage/gentoolkit-0.3.0.8-r2 - equery u uwsgi: UnicodeDecodeError: 'asci...
Status: CONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Tools (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage Tools Team
URL:
Whiteboard:
Keywords:
: 501348 (view as bug list)
Depends on:
Blocks:
 
Reported: 2014-01-21 06:03 UTC by Benjamin Peterson
Modified: 2021-12-04 20:58 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Benjamin Peterson 2014-01-21 06:03:33 UTC
Running the command below gives the following error. It works when I run equery with Python 3.

% equery u uwsgi     
[ Legend : U - final flag setting for installation]
[        : I - package is installed with flag     ]
[ Colors : set, unset                             ]
 * Found these USE flags for www-servers/uwsgi-2.0:
 U I
 - - apache2                              : Add Apache2 support
 + - caps                                 : Use Linux capabilities library to control privilege
 - - cgi                                  : Add CGI script support
 - - debug                                : Enable extra debug codepaths, like asserts and extra output. If you want to get meaningful backtraces see http://www.gentoo.org/proj/en/qa/backtraces.xml
 + - embedded                             : Embed plugins instead of building them as real plugins. When disabling this USE flag make sure you specify all used plugins in your uWSGI configuration. Note: Language plugins will always be
                                            built as real plugins.
 - - expat                                : Enable the use of dev-libs/expat for XML parsing
 - - jemalloc                             : Use dev-libs/jemalloc instead of libc's malloc
 - - json                                 : Support json as a configuration file format.
 - - lua                                  : Enable Lua scripting support
 + + pcre                                 : Add support for Perl Compatible Regular Expressions
 - - perl                                 : Add optional support/bindings for the Perl language
 - - php                                  : Include support for the PHP language
 - - php_targets_php5-3                   : Build against PHP 5.3
 - - php_targets_php5-4                   : Build against PHP 5.4
 + + php_targets_php5-5                   : Build against PHP 5.5
 + + python                               : Add optional support/bindings for the Python language
 - - python_gevent                        : Use dev-python/gevent for async request dispatching and as LoopEngine (see http://projects.unbit.it/uwsgi/wiki/Gevent).
 - - python_targets_python2_6             : Build with Python 2.6
 + + python_targets_python2_7             : Build with Python 2.7
 - - python_targets_python3_2             : Build with Python 3.2
 + + python_targets_python3_3             : Build with Python 3.3
 + - routing                              : Enable routing support
 - - ruby                                 : Add support/bindings for the Ruby language
 + + ruby_targets_ruby19                  : Build with MRI Ruby 1.9.x
 - - ruby_targets_ruby20                  : Build with MRI Ruby 2.0.x
 - - ruby_targets_ruby21                  : Build with MRI Ruby 2.1.x
 + + ssl                                  : Add support for Secure Socket Layer connections
 - - uwsgi_plugins_alarm_curl             : Plugin to send a logline to a curl url.
 - - uwsgi_plugins_alarm_xmpp             : Plugin to send a logline via XMPP/Jabber (see http://uwsgi-docs.readthedocs.org/en/latest/AlarmSubsystem.html).
 + - uwsgi_plugins_cache                  : Enable the caching framework
Traceback (most recent call last):
  File "/usr/lib/python-exec/python2.7/equery", line 38, in <module>
    equery.main(sys.argv)
  File "/usr/lib64/python2.7/site-packages/gentoolkit/equery/__init__.py", line 357, in main
    loaded_module.main(module_args)
  File "/usr/lib64/python2.7/site-packages/gentoolkit/equery/uses.py", line 301, in main
    display_useflags(output)
  File "/usr/lib64/python2.7/site-packages/gentoolkit/equery/uses.py", line 116, in display_useflags
    desc = twrap.fill(desc)
  File "/usr/lib64/python2.7/textwrap.py", line 338, in fill
    return "\n".join(self.wrap(text))
  File "/usr/lib64/python2.7/textwrap.py", line 329, in wrap
    return self._wrap_chunks(chunks)
  File "/usr/lib64/python2.7/site-packages/gentoolkit/textwrap_.py", line 95, in _wrap_chunks
    lines.append(indent + ''.join(cur_line))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 10: ordinal not in range(128)
Comment 1 Benjamin Peterson 2014-01-21 06:06:45 UTC
I guess the problem is the "’" in the description of uwsgi_plugins_carbon:

Python 3.3.3 (default, Jan 19 2014, 01:10:27) 
[GCC 4.7.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> ord("’")
8217

So maybe this is a bug in uwsgi, too?
Comment 2 Claudio Calvelli 2014-01-24 08:56:08 UTC
I get the same when running "equery uses sys-devel/gcc" and I tried on different systems (with different CPU architectures) as I thought it may be a problem with one of the installs.

# equery uses sys-devel/gcc
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.3/equery", line 38, in <module>
    equery.main(sys.argv)
  File "/usr/lib/python3.3/site-packages/gentoolkit/equery/__init__.py", line 357, in main
    loaded_module.main(module_args)
  File "/usr/lib/python3.3/site-packages/gentoolkit/equery/uses.py", line 289, in main
    global_usedesc = get_global_useflags()
  File "/usr/lib/python3.3/site-packages/gentoolkit/equery/uses.py", line 158, in get_global_useflags
    for line in open_file:
  File "/usr/lib/python3.3/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 540: ordinal not in range(128)
Comment 3 piotr5 2014-02-11 16:35:04 UTC
I haven't updated yet, but for quite some time already: for me this error in the previous comment happens everytime I execute "equery u" no matter the package I always get:

$ equery u dev-libs/botan
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.3/equery", line 38, in <module>
    equery.main(sys.argv)
  File "/usr/lib64/python3.3/site-packages/gentoolkit/equery/__init__.py", line 357, in main
    loaded_module.main(module_args)
  File "/usr/lib64/python3.3/site-packages/gentoolkit/equery/uses.py", line 289, in main
    global_usedesc = get_global_useflags()
  File "/usr/lib64/python3.3/site-packages/gentoolkit/equery/uses.py", line 158, in get_global_useflags
    for line in open_file:
  File "/usr/lib64/python3.3/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 540: ordinal not in range(128)
Comment 4 piotr5 2014-02-11 16:44:21 UTC
needless to say, the problem is immediately solved by:

$ sudo eselect python set 1
$ sudo eselect python list
Available Python interpreters:
  [1]   python2.7 *
  [2]   python3.3
Comment 5 piotr5 2014-02-11 16:51:06 UTC
doesn't solve the original problem though, the package uwsgi crashes "equery u" on python-2.7.5-r3 and any package crashes this command on python-3.3.3
Comment 6 Paul Varner (RETIRED) gentoo-dev 2014-02-11 17:42:28 UTC
The underlying issue is Python changed how it detects and handles locales in Python 3 and not all of the "bugs" have been found yet in the gentoolkit code.

There are two workarounds to this problem:

1. Switch to Python 2.7 when using gentoolkit
2. Set LC_CTYPE to a UTF8 locale on your system.

Here is output on my dev system using python 3.3

# locale
LANG=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

# equery uses gcc
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.3/equery", line 38, in <module>
    equery.main(sys.argv)
  File "/usr/lib64/python3.3/site-packages/gentoolkit/equery/__init__.py", line 357, in main
    loaded_module.main(module_args)
  File "/usr/lib64/python3.3/site-packages/gentoolkit/equery/uses.py", line 297, in main
    global_usedesc = get_global_useflags()
  File "/usr/lib64/python3.3/site-packages/gentoolkit/equery/uses.py", line 159, in get_global_useflags
    for line in open_file:
  File "/usr/lib64/python3.3/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 540: ordinal not in range(128)

# env LC_CTYPE="en_US.utf8" equery uses gcc
[ Legend : U - final flag setting for installation]
[        : I - package is installed with flag     ]
[ Colors : set, unset                             ]
 * Found these USE flags for sys-devel/gcc-4.7.3-r1:
 U I
 - - awt             : Useful only when building GCJ, this enables Abstract Window Toolkit (AWT) peer support on top of GTK+
 + + cxx             : Build support for C++ (bindings, extra libraries, code generation, ...)
 - - doc             : Add extra documentation (API, Javadoc, etc). It is recommended to enable per package instead of globally
 + + fortran         : Add support for fortran
 - - gcj             : Enable building with gcj (The GNU Compiler for the Javatm Programming Language)
 - - go              : Build the GCC Go language frontend.
 - - graphite        : Add support for the framework for loop optimizations based on a polyhedral intermediate representation
 - - mudflap         : Add support for mudflap, a pointer use checking library
 - - multislot       : Allow for SLOTs to include minor version (3.3.4 instead of just 3.3)
 + + nls             : Add Native Language Support (using gettext - GNU locale utilities)
 - - nopie           : Disable PIE support (NOT FOR GENERAL USE)
 - - nossp           : Disable SSP support (NOT FOR GENERAL USE)
 + + nptl            : Enable support for Native POSIX Threads Library, the new threading module (requires linux-2.6 or better
                       usually)
 - - objc            : Build support for the Objective C code language
 - - objc++          : Build support for the Objective C++ language
 - - objc-gc         : Build support for the Objective C code language Garbage Collector
 + + openmp          : Build support for the OpenMP (support parallel computing), requires >=sys-devel/gcc-4.2 built with
                       USE="openmp"
 - - regression-test : Run the testsuite and install the results (requires FEATURES=test)
 - - vanilla         : Do not add extra patches which change default behaviour; DO NOT USE THIS ON A GLOBAL SCALE as the severity
                       of the meaning changes drastically
Comment 7 Paul Varner (RETIRED) gentoo-dev 2014-02-11 17:55:29 UTC
I missed "equery uses uwsgi" being broken on python 2.7 as well.  The workaround for that situation is to use Python 3.3 and set LC_CTYPE to a UTF8 locale.
Comment 8 Paul Varner (RETIRED) gentoo-dev 2014-02-11 18:50:05 UTC
I've fixed the file in the tree that was causing the issue, so you should not see the problem anymore.
Comment 9 Jeroen Roovers (RETIRED) gentoo-dev 2014-02-15 11:45:18 UTC
*** Bug 501348 has been marked as a duplicate of this bug. ***
Comment 10 Jeroen Roovers (RETIRED) gentoo-dev 2014-02-15 11:45:56 UTC
I just fixed it again.

--- uwsgi_plugins.desc  11 Feb 2014 18:33:45 -0000      1.2
+++ uwsgi_plugins.desc  15 Feb 2014 11:43:52 -0000
@@ -22,7 +22,7 @@
 fastrouter - Proxy/load-balancer/router speaking the uwsgi protocol
 forkptyrouter - Pseudoterminal server to connect to instances running inside a jail/separate namespace
 geoip - Export GeoIP information as routing variables (requires dev-libs/geoip)
-graylog2 - Send logs to a Graylog2 server in Graylog2’s native GELF format
+graylog2 - Send logs to a Graylog2 server in Graylog2's native GELF format
 http - make uWSGI natively speak HTTP, using this plugin you do not need a front-end HTTP server
 ldap - Lets you configure the uWSGI server via LDAP
 legion_cache_fetch - caching for the legion subsystem
Comment 11 Paul Varner (RETIRED) gentoo-dev 2016-06-30 21:41:04 UTC
This is now partially fixed with the fix for bug 587606. Python 2.7 should always work now even if there are Unicode characters in the files containing USE descriptions. 

Python 3 still requires that at a minimum LC_CTYPE be set to a UTF-8 locale to avoid getting a traceback.