Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 144523 - Broken non-UTF-8 locale filenames in gtk+-2.8.19 and newer
Summary: Broken non-UTF-8 locale filenames in gtk+-2.8.19 and newer
Status: RESOLVED WONTFIX
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Library (show other bugs)
Hardware: All Linux
: Normal major (vote)
Assignee: Gentoo Linux Gnome Desktop Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-08-20 06:54 UTC by Daniel Zavodsky
Modified: 2006-09-19 14:41 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Zavodsky 2006-08-20 06:54:55 UTC
With gtk+-2.18.19 and newer, gtk+-2 based applications no longer interpret non-UTF-8 encoded filenames properly. For characters outside the standard ASCII range, a question mark is printed. GQview prints out the following more specific information box:

<--cut here-->
Filename encoding locale mismatch
One or more filenames are not encoded with the preferred locale character set. Operations on, and display of these files with gqview may not succeed.

If your filenames are not encoded in utf-8, try setting the environment variable G_BROKEN_FILENAMES=1
It appears G_BROKEN_FILENAMES is set to 1
The locale appears to be set to "cs_CZ"
(set by LANG environment variable)
<--cut here-->

All gtk+2 applications misinterpret these characters (making filesystem operations unreliable), and seemingly ignore the G_BROKEN_FILENAMES environment variable - the behavior is the same whether it is set to 0, 1 or unset. I have reproduced this bug on both amd64 and x86 architectures. With gtk+-2.8.19 (stable) and gtk+-2.8.20-r1 (not yet marked stable). Recompiling applications linked to it after the upgrade does not help either.


emerge --info:

Gentoo Base System version 1.12.4
Portage 2.1-r2 (default-linux/amd64/2005.0, gcc-3.4.6, glibc-2.3.6-r4, 2.6.14-gr7 x86_64)
=================================================================
System uname: 2.6.14-gr7 x86_64 AMD Athlon(tm) 64 Processor 3000+
distcc 2.18.3 x86_64-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
app-admin/eselect-compiler: [Not Present]
dev-lang/python:     2.4.3-r1
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     [Not Present]
dev-util/confcache:  [Not Present]
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.59-r7
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2
sys-devel/binutils:  2.16.1-r3
sys-devel/gcc-config: 1.3.13-r3
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.11-r2
ACCEPT_KEYWORDS="amd64"
AUTOCLEAN="yes"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O3 -march=athlon64 -fomit-frame-pointer -pipe -ftracer -funit-at-a-time -fforce-addr -fpeel-loops -funswitch-loops"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/lib64/mozilla/defaults/pref /usr/share/X11/xkb /usr/share/config"
CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/revdep-rebuild /etc/terminfo"
CXXFLAGS="-O3 -march=athlon64 -fomit-frame-pointer -pipe -ftracer -funit-at-a-time -fforce-addr -fpeel-loops -funswitch-loops"
DISTDIR="/mnt/home/sharn-lugonn/Downloads/Gentoo/Distfiles"
FEATURES="autoconfig candy distlocks metadata-transfer sandbox sfperms strict"
GENTOO_MIRRORS="ftp://ftp.sh.cvut.cz/MIRRORS/gentoo"
LANG="cs_CZ"
LDFLAGS="-Wl,-O1 -Wl,--enable-new-dtags -Wl,--sort-common"
LINGUAS="cs"
MAKEOPTS="-j2"
PKGDIR="/home/sharn-lugonn/Downloads/Gentoo/Packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/home/sharn-lugonn/Downloads/Gentoo/Overlay"
SYNC="rsync://ftp.sh.cvut.cz/gentoo-portage"
USE="amd64 X a52 aalib acpi alsa arts avi berkdb bitmap-fonts bzip2 cdparanoia cdr cjk cli crypt dga dlloader dri dvd dvdread emboss encode esd ffmpeg firefox fortran gif gnome gstreamer gtk gtk2 gtkhtml imagemagick imlib immqt-bc isdnlog javascript jpeg jpeg2k kdeenablefinal lzw lzw-tiff mad mp3 mpeg musepack ncurses nls ogg oggvorbis openal opengl pam pda pdflib perl pic png pppd python qt3 qt4 quicktime readline reflection sdl session slang sndfile spell spl ssl tcltk threads tiff truetype truetype-fonts type1-fonts usb vorbis xmms xorg xpm xrandr xv xvid zlib elibc_glibc input_devices_keyboard input_devices_mouse input_devices_evdev kernel_linux linguas_cs userland_GNU video_cards_radeon video_cards_vesa video_cards_fglrx"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_RSYNC_EXTRA_OPTS
Comment 1 Christian Ihle 2006-08-21 10:26:38 UTC
I have the same problem on 2 machines. It makes it hard to use gtk+ apps, as they can't see files correctly.

machine1 ~ $ emerge --info
Portage 2.1-r2 (default-linux/x86/2006.0, gcc-3.4.6, glibc-2.3.6-r4, 2.6.17.1 i686)
=================================================================
System uname: 2.6.17.1 i686 AMD Athlon(tm) XP 2100+
Gentoo Base System version 1.6.15
app-admin/eselect-compiler: [Not Present]
dev-lang/python:     2.4.3-r1
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     [Not Present]
dev-util/confcache:  [Not Present]
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.59-r7
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2
sys-devel/binutils:  2.16.1-r3
sys-devel/gcc-config: 1.3.13-r2
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.11-r2
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=athlon-xp -O2 -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/gconf /etc/java-config/vms/ /etc/revdep-rebuild /etc/terminfo"
CXXFLAGS="-march=athlon-xp -O2 -pipe -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks fixpackages metadata-transfer sandbox sfperms strict userpriv usersandbox"
GENTOO_MIRRORS="http://mirror.gentoo.no/ ftp://ftp.du.se/pub/os/gentoo ftp://ftp.tu-clausthal.de/pub/linux/gentoo/"
MAKEOPTS="-j2"                  
PKGDIR="/usr/portage/packages"  
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_TMPDIR="/var/tmp"       
PORTDIR="/usr/portage"          
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.no.gentoo.org/gentoo-portage"
USE="x86 3dnow 3dnowext X aac acpi alsa apache2 apm arts asf audiofile avi berkdb bitmap-fonts bzip2 cairo cdparanoia cli crypt cups curl cvs dlloader dri dvd dvdr dvdread emboss encode exif expat ffmpeg flac foomaticdb gdbm gif gtk2 hal idn imagemagick imlib ipv6 isdnlog java jpeg kde kdeenablefinal lcms libg++ libwww lirc live mad mikmod mmx mmxext motif mp3 mpeg musepack musicbrainz mysql nas ncurses network nls nptl nptlonly nvidia ogg openal opengl oss pam pcre pdflib perl php pic png ppds pppd python qt3 qt4 quicktime rdesktop readline real reflection ruby samba sdl session slang spl sql sqlite sse ssl subversion svg tcltk tcpd theora tiff truetype truetype-fonts type1-fonts udev usb vcd vorbis win32codecs x264 xcomposite xine xinerama xml xorg xv xvid xvmc zeroconf zlib elibc_glibc input_devices_keyboard input_devices_mouse kernel_linux lirc_devices_imon_pad userland_GNU video_cards_nvidia video_cards_nv video_cards_vga video_cards_vesa"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS, PORTAGE_RSYNC_EXTRA_OPTS
                                
machine2 ~ $ emerge --info
Portage 2.1-r2 (default-linux/x86/2005.1, gcc-3.4.6, glibc-2.3.6-r4, 2.6.17.4 i686)
=================================================================
System uname: 2.6.17.4 i686 Intel(R) Pentium(R) M processor 1.70GHz
Gentoo Base System version 1.12.4
app-admin/eselect-compiler: [Not Present]
dev-lang/python:     2.4.3-r1
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     [Not Present]
dev-util/confcache:  [Not Present]
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.59-r7
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2
sys-devel/binutils:  2.16.1-r3
sys-devel/gcc-config: 1.3.13-r3
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.11-r2
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=pentium-m -O2 -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/gconf /etc/java-config/vms/ /etc/revdep-rebuild /etc/terminfo"
CXXFLAGS="-march=pentium-m -O2 -pipe -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks fixpackages metadata-transfer sandbox sfperms strict userpriv usersandbox"
GENTOO_MIRRORS="http://mirror.gentoo.no/ ftp://ftp.du.se/pub/os/gentoo"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="x86 X aac acpi alsa apache2 arts asf avi berkdb bitmap-fonts bzip2 cairo cli crypt cups curl cvs dbus dlloader dri dvd dvdr dvdread emboss encode exif expat ffmpeg flac gdbm gif gimp glut gphoto2 gtk2 hal idn ieee1394 imagemagick imlib ipv6 isdnlog java jpeg kde kdeenablefinal lcms libg++ libwww live logitech-mouse mad mikmod mmx mmxext motif mp3 mpeg musepack musicbrainz mysql nas ncurses network nls nptl nptlonly ogg oggvorbis opengl oss pam pcmcia pcre pdflib perl php pic png ppds pppd python qt3 qt4 quicktime rdesktop readline real reflection ruby samba sdl session spl sql sqlite sse sse2 ssl subversion svg tcltk tcpd theora tidy tiff truetype truetype-fonts type1-fonts usb vcd vorbis wifi win32codecs x264 xcomposite xml xml2 xorg xv xvid xvmc zeroconf zlib elibc_glibc input_devices_evdev input_devices_keyboard input_devices_mouse input_devices_synaptics kernel_linux userland_GNU video_cards_vga video_cards_vesa video_cards_radeon"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 2 Daniel Zavodsky 2006-09-19 09:10:39 UTC
Setting the following variable in /etc/env.d/50glib2 fixes the problem:

G_FILENAME_ENCODING="@locale"

This should be made default in the ebuild.
Comment 3 Mart Raudsepp gentoo-dev 2006-09-19 12:30:41 UTC
(In reply to comment #2)
> Setting the following variable in /etc/env.d/50glib2 fixes the problem:
> 
> G_FILENAME_ENCODING="@locale"
> 
> This should be made default in the ebuild.

I disagree.
A filename should be independent of the locale used. A locale can change, but the filenames don't change together with the locale. Once you change the locale, your whole systems filenames become a mess if you had saved filenames in the locale encoding, instead of UTF-8.

UTF-8 on the other hand has at least these pros (and locale specific encoding has the opposite as a con):
a) always determinable what should be shown to the user in UI, if the basic assumption is kept that filenames should be expressed in UTF-8 - it is a set "encoding" that everyone can agree on without leaving any other language out, based on the b) below
b) is able to express ANY languages alphabet or scripts, as it supports all of unicode, as well, it is an unicode encoding.
b2) your filenames aren't constrained to the 200ish glyphs that your non-UTF-8 locale describes - you can have any glyph that is possible.

The comments in the various ebuild versions seem to indicate that UTF-8 is meant to be the default, and the user can set G_FILENAME_ENCODING to what he wants.
@locale is a special value for that which means to use whatever the current locale happens to me.
Because G_BROKEN_FILENAMES was set in earlier versions, then I guess now G_FILENAME_ENCODING=UTF-8 is set to be a bit backwards compatible if G_BROKEN_FILENAMES is set (the combination of these means to use UTF-8, which would be done also in the lack of both variables as well), and to give an example for G_FILENAME_ENCODING so that the user can change UTF-8 to something else.
While I personally disagree with changing away from UTF-8, due to the reasons given above (b), I even more so disagree with using @locale because that means that the encoding is quite indeterminable, as per a) above.

Long ago I had non UTF-8 filenames on my systems, but I took the time to convert the ones that I care about to UTF-8. Now I don't have any problems with exchanging files named in my own languages alphabet with others, and all works great. If there are a considerable amount of 8-bit encoding filenames, and less UTF-8 filenames, then one might consider changing G_FILENAME_ENCODING to his encoding that is used as a majority.

http://developer.gnome.org/doc/API/2.0/glib/glib-Character-Set-Conversion.html discusses this topic, but from a more technical point of view.

For new systems, having everything as UTF-8 is what should be done, hence the default certainly shouldn't be changed, imho. We live in a world of unicode.
Comment 4 Daniel Zavodsky 2006-09-19 14:41:08 UTC
Thank you for the explanation... though converting the filenames into UTF-8 is not an option for me (I would have to convert not only my whole system to UTF-8, but all computers on the network), I understand that for new systems such a behavior is desirable.