If network (or host) is unreachable, urlopen raise urlopen error with message "Errno 101] Network is unreachable". I have bug with ru_RU.UTF-8 locale. Bug appear due localization of glibc message, activated by "locale.setlocale(locale.LC_ALL, '')" in _emerge/main.py module. I have python2.7. !!! Error fetching binhost package info from 'http://5.255.226.146/calculate/CLDX/grp/x86_64' Traceback (most recent call last): File "/usr/lib/python-exec/python2.7/emerge", line 50, in <module> retval = emerge_main() File "/usr/lib64/python2.7/site-packages/_emerge/main.py", line 1070, in emerge_main return run_action(emerge_config) File "/usr/lib64/python2.7/site-packages/_emerge/actions.py", line 3713, in run_action getbinpkgs="--getbinpkg" in emerge_config.opts) File "/usr/lib64/python2.7/site-packages/portage/dbapi/bintree.py", line 627, in populate self._populate(getbinpkgs) File "/usr/lib64/python2.7/site-packages/portage/dbapi/bintree.py", line 1010, in _populate writemsg("!!! %s\n\n" % str(e)) UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 27: ordinal not in range(128)
There's a patch in this branch: https://github.com/zmedico/portage/tree/bug_532784 I've posted it for review here: http://thread.gmane.org/gmane.linux.gentoo.portage.devel/5012
This is in the master branch now: https://github.com/gentoo/portage/commit/4496ee37d6fa327ada635c67500e82f830141a9e
compiled from 9999 today. !!! Error fetching binhost package info from 'http://5.255.226.146/calculate/CLDX/grp/x86_64' Traceback (most recent call last): File "/usr/lib/python-exec/python2.7/emerge", line 50, in <module> retval = emerge_main() File "/usr/lib64/python2.7/site-packages/_emerge/main.py", line 1087, in emerge_main return run_action(emerge_config) File "/usr/lib64/python2.7/site-packages/_emerge/actions.py", line 2811, in run_action getbinpkgs="--getbinpkg" in emerge_config.opts) File "/usr/lib64/python2.7/site-packages/portage/dbapi/bintree.py", line 630, in populate self._populate(getbinpkgs) File "/usr/lib64/python2.7/site-packages/portage/dbapi/bintree.py", line 1031, in _populate _encodings["stdio"], errors="replace")) TypeError: coercing to Unicode: need string or buffer, URLError found
(In reply to Mike Hiretsky from comment #3) > File "/usr/lib64/python2.7/site-packages/portage/dbapi/bintree.py", line > 1031, in _populate > _encodings["stdio"], errors="replace")) > TypeError: coercing to Unicode: need string or buffer, URLError found Hmm, this one is tricky. It seems impossible to reproduce with my locale: Python 2.7.7 (default, Sep 9 2014, 07:37:40) [GCC 4.7.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import locale >>> locale.setlocale(locale.LC_ALL, '') 'en_US.utf8' >>> import urllib2 >>> e = urllib2.URLError('foo') >>> unicode(e) u'<urlopen error foo>' >>> e = urllib2.URLError(u'foo') >>> unicode(e) u'<urlopen error foo>' I guess I'll have to generate a Russian locale, and do some experimenting...
(In reply to Zac Medico from comment #4) You probably need to add "ru" to LINGUAS in make.conf and rebuild glibc.
!!! Error fetching binhost package info from 'http://mirror.cnet.kz/calculate/CLDX/grp/x86_64' Traceback (most recent call last): File "/usr/lib/python-exec/python2.7/emerge", line 50, in <module> retval = emerge_main() File "/usr/lib64/python2.7/site-packages/_emerge/main.py", line 1154, in emerge_main return run_action(emerge_config) File "/usr/lib64/python2.7/site-packages/_emerge/actions.py", line 2818, in run_action getbinpkgs="--getbinpkg" in emerge_config.opts) File "/usr/lib64/python2.7/site-packages/portage/dbapi/bintree.py", line 633, in populate self._populate(getbinpkgs) File "/usr/lib64/python2.7/site-packages/portage/dbapi/bintree.py", line 1034, in _populate _encodings["stdio"], errors="replace")) TypeError: coercing to Unicode: need string or buffer, URLError found
I get the following error when the binhost url reports a 404. LANG=en_US.UTF-8 > !!! Error fetching binhost package info from 'http://builds.developer.core-os.net/sdk/amd64/668.0.0/pkgs/' > Traceback (most recent call last): > File "/usr/lib/python-exec/python2.7/emerge", line 50, in <module> > retval = emerge_main() > File "/usr/lib64/python2.7/site-packages/_emerge/main.py", line 1154, in emerge_main > return run_action(emerge_config) > File "/usr/lib64/python2.7/site-packages/_emerge/actions.py", line 2818, in run_action > getbinpkgs="--getbinpkg" in emerge_config.opts) > File "/usr/lib64/python2.7/site-packages/portage/dbapi/bintree.py", line 633, in populate > self._populate(getbinpkgs) > File "/usr/lib64/python2.7/site-packages/portage/dbapi/bintree.py", line 1034, in _populate > _encodings["stdio"], errors="replace")) > TypeError: coercing to Unicode: need string or buffer, HTTPError found Previously this gracefully logged an error and continued on.
(In reply to Zac Medico from comment #4) > (In reply to Mike Hiretsky from comment #3) > > File "/usr/lib64/python2.7/site-packages/portage/dbapi/bintree.py", line > > 1031, in _populate > > _encodings["stdio"], errors="replace")) > > TypeError: coercing to Unicode: need string or buffer, URLError found > > Hmm, this one is tricky. It seems impossible to reproduce with my locale: > > Python 2.7.7 (default, Sep 9 2014, 07:37:40) > [GCC 4.7.3] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import locale > >>> locale.setlocale(locale.LC_ALL, '') > 'en_US.utf8' > >>> import urllib2 > >>> e = urllib2.URLError('foo') > >>> unicode(e) > u'<urlopen error foo>' > >>> e = urllib2.URLError(u'foo') > >>> unicode(e) > u'<urlopen error foo>' > > I guess I'll have to generate a Russian locale, and do some experimenting... The above did not cause an error because you did not specify an encoding like the code in question does. unicode() without the encoding argument works but with one it does not: >>> e = Exception("failure") >>> unicode(e) u'failure' >>> unicode(e, 'utf_8') Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: coercing to Unicode: need string or buffer, exceptions.Exception found The documentations reports: "If encoding and/or errors are given, unicode() will decode the object which can either be an 8-bit string or a character buffer using the codec for encoding... If no optional parameters are given, unicode() will mimic the behaviour of str()" So using unicode() in this way is invalid, just drop the extra args. :)
Created attachment 402260 [details, diff] [PATCH] bintree.py: do not pass unicode encoding with non-string type
(In reply to mike@marineau.org from comment #9) > Created attachment 402260 [details, diff] [details, diff] > [PATCH] bintree.py: do not pass unicode encoding with non-string type Maybe we should also add errors="replace" to the _unicode arguments.
(In reply to Zac Medico from comment #10) > Maybe we should also add errors="replace" to the _unicode arguments. Actually, that probably raises TypeError. So, lets apply the patch as is.
(In reply to Zac Medico from comment #11) > (In reply to Zac Medico from comment #10) > > Maybe we should also add errors="replace" to the _unicode arguments. > > Actually, that probably raises TypeError. So, lets apply the patch as is. I'm going modify it to handle UnicodeDecodeError (as in comment #0).
This is in the master branch: https://gitweb.gentoo.org/proj/portage.git/commit/?id=1e43b60d734f601cde727d690c7ae4d576146659
Released in portage-2.2.19