Summary: | sys-apps/portage-2.2.27: emerge -a crashes: UnicodeDecodeError: 'utf-8' codec can't decode byte 0x84 in position 0: invalid start byte | ||
---|---|---|---|
Product: | Portage Development | Reporter: | Andrew Savchenko <bircoph> |
Component: | Core - Interface (emerge) | Assignee: | Portage team <dev-portage> |
Status: | RESOLVED FIXED | ||
Severity: | normal | Keywords: | InVCS |
Priority: | Normal | ||
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Bug Depends on: | |||
Bug Blocks: | 573774 | ||
Attachments: | emerge --info |
Description
Andrew Savchenko
2016-01-30 11:31:11 UTC
$ eix -e portage sys-apps/portage Available versions: 2.2.8-r2 2.2.20.1{tbz2} 2.2.24 2.2.26{tbz2} (~)2.2.27{tbz2} **9999{tbz2} {build doc epydoc +ipc pypy2_0 python2 python3 selinux xattr LINGUAS="ru" PYTHON_TARGETS="pypy pypy2_0 python2_6 python2_7 python3_2 python3_3 python3_4 python3_5"} Installed versions: 2.2.27{tbz2}(08:32:35 PM 01/22/2016)(doc ipc -build -epydoc -selinux -xattr LINGUAS="ru" PYTHON_TARGETS="python2_7 python3_4 -pypy -python3_3 -python3_5") Created attachment 424240 [details]
emerge --info
I can't recreate this in HEAD with Norwegian letters, Hiragana, Katakana, or Kanji. 1. Could you try this in HEAD and see if the issue persists? 2. If so, could you try to recreate it with e.g. a Kanji? 3. If you're unable to recreate it with Kanji, could you provide more info? Like how you switch layouts, which specific Cryllic letter you entered, and other useful info? (In reply to Alexander Berntsen from comment #3) > I can't recreate this in HEAD with Norwegian letters, Hiragana, Katakana, or > Kanji. In any case, nothing prevents the user from entering characters that will fail to decode. Under python3, the input function (which returns a unicode string) will raise UnicodeDecodeError. Under python2, raw_input returns bytes, so it won't raise UnicodeDecodeError, but comparison with values from the responses variable can raise UnicodeDecodeError if that variable contains unicode strings (it might because of unicode_literals being enabled in the calling module). There's a patch in the following branch: https://github.com/zmedico/portage/tree/bug_573386 I've posted it for review here: https://archives.gentoo.org/gentoo-portage-dev/message/fcb0fb00154ce53f67c7bafb482130b7 Hi, (In reply to Alexander Berntsen from comment #3) > I can't recreate this in HEAD with Norwegian letters, Hiragana, Katakana, or > Kanji. > > 1. Could you try this in HEAD and see if the issue persists? Yes, it does. > 2. If so, could you try to recreate it with e.g. a Kanji? I can recreate this with kanji (e.g. "幸"). > 3. If you're unable to recreate it with Kanji, could you provide more info? > Like how you switch layouts, which specific Cryllic letter you entered, and > other useful info? I use xterm, probably this matters. Any russian letter is sufficient to reproduce (e.g. "н"). I switch layouts using Xorg's grp:lwin_toggle XkbOptions switch. (In reply to Zac Medico from comment #5) > There's a patch in the following branch: > > https://github.com/zmedico/portage/tree/bug_573386 Thanks, I'll try it in a while. (In reply to Zac Medico from comment #5) > There's a patch in the following branch: > > https://github.com/zmedico/portage/tree/bug_573386 This patch fixes my issue, thanks. One little suggestion: replace "None" string by something more informational for an end-user like "Wrong unicode string". (In reply to Andrew Savchenko from comment #8) > (In reply to Zac Medico from comment #5) > > There's a patch in the following branch: > > > > https://github.com/zmedico/portage/tree/bug_573386 > > This patch fixes my issue, thanks. > > One little suggestion: replace "None" string by something more informational > for an end-user like "Wrong unicode string". I've updated it to display the string returned by str(errors='replace'). This is in the master branch: https://gitweb.gentoo.org/proj/portage.git/commit/?id=1929e2bdab686004697c3d0fd999a563db43349c (In reply to Zac Medico from comment #9) > (In reply to Andrew Savchenko from comment #8) > This is in the master branch: > > https://gitweb.gentoo.org/proj/portage.git/commit/ > ?id=1929e2bdab686004697c3d0fd999a563db43349c I messed up making a last-moment change to that commit, and this fixes it: https://gitweb.gentoo.org/proj/portage.git/commit/?id=f292c3519082ad0932ad3a83a19e547c53b80824 Fixed in 2.2.28. |