Summary: | app-emulation/qemu-2.12.0-r1(python3.5) - "UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 54: ordinal not in range(128)" if en_US.UTF-8 locale is not available | ||
---|---|---|---|
Product: | Gentoo Linux | Reporter: | PrSo <soprwa> |
Component: | Current packages | Assignee: | Gentoo QEMU Project <qemu+disabled> |
Status: | RESOLVED FIXED | ||
Severity: | normal | CC: | aidanamarks, atoth, mike, neil, zsojka |
Priority: | Normal | ||
Version: | unspecified | ||
Hardware: | All | ||
OS: | All | ||
URL: | https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg01123.html | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Attachments: |
build log
output of emerge --info Patch |
Created attachment 535606 [details]
output of emerge --info
Same here. Nothing to do with locale / Polish language. Same problem with python 3.4 and 3.6 by the way, not jot 3.5 *not just Above error message occurs if you try to convert a string to utf-8 while being on a non-utf8 locale. According to [1] the qemu build system has a workaround for exactly that problem. It sets (in Makefile) PYTHON_UTF8 = LC_ALL= LANG=C LC_CTYPE=en_US.UTF-8 $(PYTHON) But that only works if the en_US.UTF-8 locale is available... Can you please try again but generate the en_US.UTF-8 locale first: - add "en_US.UTF-8 UTF-8" to /etc/locale.gen - run # locale-gen [1] https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg01123.html can confirm the workaround succeeded for me with all python3 targets (In reply to Matthias Maier from comment #5) > Above error message occurs if you try to convert a string to utf-8 while > being on a non-utf8 locale. According to [1] the qemu build system has a > workaround for exactly that problem. It sets (in Makefile) > > PYTHON_UTF8 = LC_ALL= LANG=C LC_CTYPE=en_US.UTF-8 $(PYTHON) > > But that only works if the en_US.UTF-8 locale is available... > > Can you please try again but generate the en_US.UTF-8 locale first: > - add "en_US.UTF-8 UTF-8" to /etc/locale.gen > - run # locale-gen > > [1] https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg01123.html Setting en_US.UTF-8 in /etc/locale.gen indeed works, but that's little odd. If I understood qemu-devel mailing list post correctly the patch sets LANG to "C" overriding en_US.UTF-8, and since mine was set to pl_PL.UTF-8 it seems that in qemu case setting en_US.UTF-8 is a MUST (beside others UTF-8 locales). Maybe we should add pre-emerge checks to the qemu ebuild, similar to those related to kernel .config file, or try to modify makefile to override other locales too. Thanks, Przemek. Created attachment 535682 [details, diff]
Patch
QEMU upstream do not know how to use Python properly...
It is easy to use explicit decoding/encoding, which is then independent of locale.
I attach untested patch.
Please test this patch with LC_ALL etc. (In reply to Arfrever Frehtes Taifersar Arahesis from comment #9) > Please test this patch with LC_ALL etc. I meant LC_ALL=C (In reply to Arfrever Frehtes Taifersar Arahesis from comment #8) > Created attachment 535682 [details, diff] [details, diff] > Patch > > QEMU upstream do not know how to use Python properly... > It is easy to use explicit decoding/encoding, which is then independent of > locale. > > I attach untested patch. It seems that the patch works without any drawbacks with python 3.5. I have set back the locale to pl_PL.UTF-8 only in locale.gen, and quemu compiles fine. Thanks, Przemek. The patch works for me too, thanks! % grep -v '^#' /etc/locale.gen en_GB ISO-8859-1 en_GB.UTF-8 UTF-8 The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=d5141d7cae6dfa31b8b7023b6f984ea4725289ae commit d5141d7cae6dfa31b8b7023b6f984ea4725289ae Author: Matthias Maier <tamiko@gentoo.org> AuthorDate: 2018-06-12 15:33:08 +0000 Commit: Matthias Maier <tamiko@gentoo.org> CommitDate: 2018-06-12 15:40:57 +0000 app-emulation/qemu: fix compilation if en_US.UTF-8 is unavailable Thanks to Arfrever for the proper patch. Closes: https://bugs.gentoo.org/657766 Package-Manager: Portage-2.3.40, Repoman-2.3.9 app-emulation/qemu/Manifest | 1 + app-emulation/qemu/qemu-2.12.0-r1.ebuild | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) |
Created attachment 535604 [details] build log Hi all. After recent update there is a problem during compilation of qemu-2.12.0-r1 with python 3.5. It seems that there is something wrong with 'ascii' codec on python side. I have no idea if it is related but I am using Polish-utf8 language set in my environment. Build log and output of "emerge info" attached.