Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 49223 - easytag 0.31_pre2-r3 encoding (utf8, iso8859-15) problems
Summary: easytag 0.31_pre2-r3 encoding (utf8, iso8859-15) problems
Status: RESOLVED UPSTREAM
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: x86 Linux
: High normal (vote)
Assignee: Daniel Drake (RETIRED)
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-04-27 17:21 UTC by Alexander Jenisch
Modified: 2004-06-02 12:40 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander Jenisch 2004-04-27 17:21:49 UTC
starting easytag with my standard encondig - everything "POSIX", but LC_CTYPE="de_AT@euro"
--------------------------------------------------------------------------
nt@eNTi $ easytag 
Starting EasyTAG 0.30.2 ...
Currently using id3lib version 3.8.3 ...
You are running a GTK2 port of EasyTAG. This port is currently unofficial.
Please report all bugs, that you cannot reproduce in the GTK1 version,
to Daniel Drake <dsd@gentoo.org> or the easytag-mailing mailing list (subscription required)

Loading default 'Fill Tag' masks...
text is Fill tag preview...
Loading default 'Rename File' masks...
Tree_Select_Dir; path:/home/nt/media/audio/music/

** (easytag:31650): WARNING **: convert_string(): Failed conversion. Charsets: UTF-8 -> ISO-8859-15. String Mein Baby War Beim Fris\x72r. Errcode 1 (Invalid byte sequence in conversion input)

** (easytag:31650): WARNING **: convert_string(): Failed conversion. Charsets: UTF-8 -> ISO-8859-15. String Die \x72rzte. Errcode 1 (Invalid byte sequence in conversion input)

** (easytag:31841): WARNING **: convert_string(): Failed conversion. Charsets: UTF-8 -> utf-8. String Mein Baby War Beim Fris\x72r. Errcode 1 (Invalid byte sequence in conversion input)

** (easytag:31841): WARNING **: convert_string(): Failed conversion. Charsets: UTF-8 -> utf-8. String Die \x72rzte. Errcode 1 (Invalid byte sequence in conversion input)

** (easytag:31841): WARNING **: Invalid UTF8 string passed to pango_layout_set_text()

** (easytag:31841): WARNING **: Invalid UTF8 string passed to pango_layout_set_text()

--------------------------------------------------------------------------
first two WARINGS will come after selecting the directory containg the two files, the others after klicking on each of the two once. the special character, as well as the rest of the tag will be "cut out" text area of both files (one being an ogg, the other an mp3)
--------------------------------------------------------------------------
--------------------------------------------------------------------------
starting easytag with LC_ALL="en_US.utf8"
--------------------------------------------------------------------------
nt@eNTi $ LC_ALL="en_US.utf8" easytag
Starting EasyTAG 0.30.2 ...
Currently using id3lib version 3.8.3 ...
You are running a GTK2 port of EasyTAG. This port is currently unofficial.
Please report all bugs, that you cannot reproduce in the GTK1 version,
to Daniel Drake <dsd@gentoo.org> or the easytag-mailing mailing list (subscription required)

Loading default 'Fill Tag' masks...
text is Fill tag preview...
Loading default 'Rename File' masks...
Tree_Select_Dir; path:/home/nt/media/audio/music/

** (easytag:32056): WARNING **: convert_string(): Failed conversion. Charsets: UTF-8 -> ISO-8859-15. String Mein Baby War Beim Fris\x72r. Errcode 1 (Invalid byte sequence in conversion input)

** (easytag:32056): WARNING **: convert_string(): Failed conversion. Charsets: UTF-8 -> ISO-8859-15. String Die \x72rzte. Errcode 1 (Invalid byte sequence in conversion input)

** (easytag:32056): WARNING **: convert_string(): Failed conversion. Charsets: UTF-8 -> utf-8. String Mein Baby War Beim Fris\x72r. Errcode 1 (Invalid byte sequence in conversion input)

** (easytag:32056): WARNING **: convert_string(): Failed conversion. Charsets: UTF-8 -> utf-8. String Die \x72rzte. Errcode 1 (Invalid byte sequence in conversion input)

** (easytag:32056): WARNING **: Invalid UTF8 string passed to pango_layout_set_text()

** (easytag:32056): WARNING **: Invalid UTF8 string passed to pango_layout_set_text()
----------------------------------------------------------------
first two errors the same as above. the other warnings are produced by the mp3 only. ogg behaves ok. not chars cut out in text areas, no warnings. mp3 has same behavoir as above. tell me if you want me to test some things for you.
Comment 1 Daniel Drake (RETIRED) gentoo-dev 2004-04-28 05:55:26 UTC
Thanks for reporting the issue
It's definately a bug, which I will look into. If you are after a temporary workaround, you can probably do this:

LANG=de_AT easytag

That should work without errors, at least, that is what I had tested :)

thanks.
Comment 2 Alexander Jenisch 2004-04-28 06:09:26 UTC
i don't know what encondings you are using, but "LANG=de_AT easytag" produces the same warnings and errors, as the first part and "LC_ALL=POSIX LANG=de_AT easytag" does the same as the second part. so it's LC_CTYPE which you should look after. this one is set to "de_AT@euro" on my system. the rest is "POSIX". so that i have an english system, with german characters.
Comment 3 Daniel Drake (RETIRED) gentoo-dev 2004-05-04 14:59:18 UTC
Ok, please try 0.31_pre2-r4 which is on its way into portage right now. I'm not 100% confident that I've fixed the issue, but we'll see. Please reopen bug if the issue still exists.

Thanks again for the bug reports :)
Comment 4 Alexander Jenisch 2004-05-05 05:08:45 UTC
nothing changed here. same errors. what did you do? i'm still on "POSIX" with all but LC_TYPE, which is on "de_AT@euro". tell me what i should try and i'll do it :).
Comment 5 Daniel Drake (RETIRED) gentoo-dev 2004-05-05 05:35:44 UTC
Well I have a slight problem reproducing the problem, e.g. if I run:
 # LC_ALL="en_US.utf8" easytag

Then I get:
 (process:8214): Gtk-WARNING **: Locale not supported by C library.
 Using the fallback 'C' locale.

(same for other LC_ settings)

I remerged glibc with USE=nls and then remerged gtk+ - still no luck. Don't suppose you have any ideas?

However, I was able to get a few of those messages even though it had fallen back to the "C" locale. So I went through and fixed all them, but thats why I wasn't sure if it would have fixed the problem for your locale..
Comment 6 Daniel Drake (RETIRED) gentoo-dev 2004-05-05 07:43:07 UTC
Oh, forgot to mention.
Could you please repost the warnings using the newer version, I added some extra warnings to give a little more info. TY.
Comment 7 Alexander Jenisch 2004-05-05 08:04:37 UTC
Loading default 'Fill Tag' masks...
text is Fill tag preview...
Loading default 'Rename File' masks...
Tree_Select_Dir; path:/home/nt/media/audio/music/

** (easytag:32446): WARNING **: convert_string(): Failed conversion. Charsets: UTF-8 -> ISO-8859-15. String Mein Baby War Beim Fris\xf6r. Errcode 1 (Invalid byte sequence in conversion input)

** (easytag:32446): WARNING **: convert_string(): Failed conversion. Charsets: UTF-8 -> ISO-8859-15. String Die \xc4rzte. Errcode 1 (Invalid byte sequence in conversion input)

------------------------------------------------------------------------------
when selecting directory, that contains files with special characters, in this case it would be "Die 
Comment 8 Alexander Jenisch 2004-05-05 08:04:37 UTC
Loading default 'Fill Tag' masks...
text is Fill tag preview...
Loading default 'Rename File' masks...
Tree_Select_Dir; path:/home/nt/media/audio/music/

** (easytag:32446): WARNING **: convert_string(): Failed conversion. Charsets: UTF-8 -> ISO-8859-15. String Mein Baby War Beim Fris\xf6r. Errcode 1 (Invalid byte sequence in conversion input)

** (easytag:32446): WARNING **: convert_string(): Failed conversion. Charsets: UTF-8 -> ISO-8859-15. String Die \xc4rzte. Errcode 1 (Invalid byte sequence in conversion input)

------------------------------------------------------------------------------
when selecting directory, that contains files with special characters, in this case it would be "Die Ärzte" and not "Die \xc4rzte".
------------------------------------------------------------------------------

** (easytag:32446): WARNING **: convert_string(): Failed conversion. Charsets: U      
TF-8 -> utf-8. String Mein Baby War Beim Fris\xf6r. Errcode 1 (Invalid byte sequ      
ence in conversion input)

** (easytag:32446): WARNING **: convert_string(): Failed conversion. Charsets: U      
TF-8 -> utf-8. String Die \xc4rzte. Errcode 1 (Invalid byte sequence in conversi      
on input)

** (easytag:32446): WARNING **: Invalid UTF8 string passed to pango_layout_set_t      
ext()

** (easytag:32446): WARNING **: Invalid UTF8 string passed to pango_layout_set_t      
ext()

** (easytag:32446): WARNING **: convert_from_utf8(): Failed conversion to ISO-88      
59-15. String Mein Baby War Beim Fris\xf6r. Errcode 1 (Invalid byte sequence in       
conversion input)

** (easytag:32446): WARNING **: convert_from_utf8(): String was invalid UTF8.


** (easytag:32446): WARNING **: convert_from_utf8(): Failed conversion to ISO-88      
59-15. String Die \xc4rzte. Errcode 1 (Invalid byte sequence in conversion input)

------------------------------------------------------------------------------
when selecting ogg or mp3
------------------------------------------------------------------------------
Comment 9 Daniel Drake (RETIRED) gentoo-dev 2004-05-15 01:08:16 UTC
-r5 is in portage, please try it.
I haven't specifically gone about fixing your problems, but the character set handling/conversion is _much_ more sensible now. Let me know what happens.
Comment 10 Alexander Jenisch 2004-05-26 08:36:07 UTC
sorry, for not responding so long...

well mp3 is still broken as it used to, ogg seems to be displayed ok. BUT: another strange behavior has sneaked into easytag now. just look at that screenshot to see what's happening to mp3 as well as to ogg files now:

http://members.inode.at/a.jenisch/easytag_repeat.jpg
Comment 11 Daniel Drake (RETIRED) gentoo-dev 2004-05-26 09:40:43 UTC
eek.

That screenshot you showed is a screenshot of the tag from another application, right? Do the title and artist tracks appear as corrupted in the easytag display?

Thanks for persisting with this.
Comment 12 Daniel Drake (RETIRED) gentoo-dev 2004-05-30 12:37:14 UTC
Ok. I've got rid of the "locale not supported error", and when starting with:
LC_COLLATE="POSIX" LC_MESSAGES="POSIX" LC_MONETARY="POSIX" LC_CTYPE="de_AT@euro" LC_NUMERIC="POSIX" LC_TIME="POSIX" ./easytag
everything works fine. 

I tagged a MP3 file and an ogg file with 
Comment 13 Daniel Drake (RETIRED) gentoo-dev 2004-05-30 12:37:14 UTC
Ok. I've got rid of the "locale not supported error", and when starting with:
LC_COLLATE="POSIX" LC_MESSAGES="POSIX" LC_MONETARY="POSIX" LC_CTYPE="de_AT@euro" LC_NUMERIC="POSIX" LC_TIME="POSIX" ./easytag
everything works fine. 

I tagged a MP3 file and an ogg file with äääää, checked the MP3 in xmms and the ogg with rhythmbox (yes, xmms's ogg tag unicode parsing is broken) and all was fine. There is a setting in easytag prefernces ("Tag settings") where you can choose between using ISO-8859-1 for tags, or specifying encodings. For now, you should be using the first option (always use ISO-8859-1). 
Easytag 0.31 is about to go into portage, but I don't think it will affect the problems you are having. Please check the above setting and try 0.31 anyway and let me know what happens.
Comment 14 Daniel Drake (RETIRED) gentoo-dev 2004-06-02 12:40:32 UTC
I mentioned this bug to the Easytag author, as this is the only remaining issue I know of with my GTK2 work. He says:
------
I had a look to the related bug:
   The screenshot sended by "/Alexander Jenisch <mailto:nt-@gmx.de> 2004-05-26 08:36 PST" shows UTF-8 strings displayed to ISO-8859-1 (or assimilates). In fact, this problem could be due to the previous version of EasyTAG.
Indeed, if you have a locale set to UTF-8, it write logically UTF-8 strings in the tag. But EasyTAG let the encoding flag (in tag) to ISO-8859-1. And this was very bad! So other applications, according to the flag, read ISO-8859-1 strings.
Since the version (0.30.1), an option (activated by default) force the conversion to ISO-8859-1 to be correct. Because UTF-8 was supported only into ID3v2.4 (id3lib 3.8.13 supports only ID3v2.3!)

In the preferences window, the two fields to convert the character set were initially implemented for russian people. I'll change them to allow to the user to fix easily this kind of problem.
But it can already fix his tag by using "UTF-8 to <system_locale>" for reading tag and then "ISO-8859-1 to //<system_locale>"  to write tags. 
------

So basically, I think this is a core easytag issue - a "by-default" incompatibility with certain locales, where the options need to be tweaked.
It has not been fixed in 0.31 (as far as I can see) but some commented code has been added, which looks like much improved charset handling. So, this will hopefully be fixed for the next version. Stay tuned :)