Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 477964 - =app-misc/media-player-info-19: chokes on utf-8 character during install
Summary: =app-misc/media-player-info-19: chokes on utf-8 character during install
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Freedesktop bugs
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-07-24 03:33 UTC by Taahir Ahmed
Modified: 2013-07-24 17:46 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
cave installation log (contains python exception) (1374632299-install-app-misc_media-player-info-19:0::gentoo.out,24.00 KB, text/plain)
2013-07-24 03:36 UTC, Taahir Ahmed
Details
output of emerge --info (emerge-info.txt,5.79 KB, text/plain)
2013-07-24 04:00 UTC, Taahir Ahmed
Details
Output of "emerge app-misc/media-player-info" (file_477964.txt,23.17 KB, text/plain)
2013-07-24 06:39 UTC, Taahir Ahmed
Details
proposed patch (media-player-info-19-utf8.patch,2.30 KB, patch)
2013-07-24 12:45 UTC, Alexandre Rostovtsev (RETIRED)
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Taahir Ahmed 2013-07-24 03:33:00 UTC
app-misc/media-player-info-19 throws a python exception during install using cave as a package manager, but not when using emerge.

According to cave's maintainer, this is because cave sets a C locale when running ebuilds, and relies on the ebuild to ensure that it sets a different locale if needed.

The exception occurs when the ebuild executes a helper python script ("mpi2udev.py"); there is a single utf-8 character in the media-players directory, and the helper script throws an encoding error on reaching it:

basis env.d # cd /var/tmp/paludis/app-misc-media-player-info-19/work/media-player-info-19/media-players/

basis media-players # grep --color='auto' -P -n "[\x80-\xFF]" -r
./

./palm_pre.mpi:2:Product=PrÄ

However, when I use emerge to install the package, it succeeds, because emerge lets my configured en_US.UTF-8 locale through to the ebuild.  On a system that didn't run a UTF-8 locale, I can only assume that emerge would also fail to install the package.

I have attached the logs of attempting to install the package under both cave and emerge, as well as emerge --info output.

Reproducible: Always

Steps to Reproduce:
1.cave resolve --complete '=app-misc/media-player-info-19' -x
2.
3.
Actual Results:  
Python exception as shown in the cave installation log.

Expected Results:  
The package should either not attempt to process utf-8 characters during install (by removing that single character from the directory of .mpi files), or require a UTF-8 locale.
Comment 1 Taahir Ahmed 2013-07-24 03:36:32 UTC
Created attachment 354060 [details]
cave installation log (contains python exception)
Comment 2 Taahir Ahmed 2013-07-24 04:00:45 UTC
Created attachment 354062 [details]
output of emerge --info

Note that the USE flags in particular are not the authoritative ones on my system -- those are in the paludis configuration directory.
Comment 3 Samuli Suominen (RETIRED) gentoo-dev 2013-07-24 06:24:43 UTC
Upstream bug report:

http://bugs.freedesktop.org/show_bug.cgi?id=67242
Comment 4 Taahir Ahmed 2013-07-24 06:36:24 UTC
Comment on attachment 354060 [details]
cave installation log (contains python exception)

Apologies for the shell-init errors.  My bash-completion was broken at the time.  I re-ran the install with it fixed, and the python errors are the same.
Comment 5 Taahir Ahmed 2013-07-24 06:39:39 UTC
Created attachment 354064 [details]
Output of "emerge app-misc/media-player-info"
Comment 6 Pacho Ramos gentoo-dev 2013-07-24 06:51:46 UTC
I am not sure if this is valid: every time someone has suggested to force LC_ALL=C in portage, issues like this (usually with python) rised, even a C.utf8 from debian was suggested
Comment 7 Zac Medico gentoo-dev 2013-07-24 07:00:32 UTC
The python script that throws the exception should force utf-8 codec when it opens the file. Then it would work regardless of locale.
Comment 8 Zac Medico gentoo-dev 2013-07-24 07:07:41 UTC
In portage we use io.open(file, encoding='utf_8') for things like this. It works with python-2.6 and up.
Comment 9 Samuli Suominen (RETIRED) gentoo-dev 2013-07-24 09:33:09 UTC
Anyone good enough with python to provide a patch based on Comment #8? ;-)
Comment 10 Alexandre Rostovtsev (RETIRED) gentoo-dev 2013-07-24 12:45:03 UTC
Created attachment 354090 [details, diff]
proposed patch

See if this patch works. Note: it requires >=python-3.2 (configparser in python-3.0 and 3.1 apparently does not allow setting input encodings).
Comment 11 Alexandre Rostovtsev (RETIRED) gentoo-dev 2013-07-24 14:13:38 UTC
Fixed upstream in version 21.

+*media-player-info-21 (24 Jul 2013)
+
+  24 Jul 2013; Alexandre Rostovtsev <tetromino@gentoo.org>
+  +media-player-info-21.ebuild:
+  Version bump, fixes build failure in non-utf8 locales (bug #477964, thanks to
+  Taahir Ahmed).
Comment 12 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-07-24 17:46:25 UTC
(In reply to Alexandre Rostovtsev from comment #10)
> Created attachment 354090 [details, diff] [details, diff]
> proposed patch
> 
> See if this patch works. Note: it requires >=python-3.2 (configparser in
> python-3.0 and 3.1 apparently does not allow setting input encodings).

You have to use ConfigParser.readfp() on a file opened with particular encoding.