Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 71747 - Xorg takes ages to load on a utf8 locale
Summary: Xorg takes ages to load on a utf8 locale
Status: VERIFIED INVALID
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Gentoo X packagers
URL:
Whiteboard:
Keywords:
: 101046 (view as bug list)
Depends on: 65186
Blocks:
  Show dependency tree
 
Reported: 2004-11-19 05:09 UTC by Björn Michaelsen
Modified: 2011-01-05 00:31 UTC (History)
8 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
strace of twm startup with en_US.UTF-8 locale (twm_en_US.log,242.94 KB, text/plain)
2004-12-11 03:21 UTC, Maurice van der Pot (RETIRED)
Details
strace of twm startup with POSIX locale (twm_posix.log,94.37 KB, text/plain)
2004-12-11 03:22 UTC, Maurice van der Pot (RETIRED)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Björn Michaelsen 2004-11-19 05:09:44 UTC
On a utf8 locale (de_DE.utf8) xorg takes at least 10 min. to start and X is eating 100% CPU the whole time. The weird thing is fluxbox/rox is coming up after ca. 5 min. but isnt responding for another 5 min. Tested with twm too takes a long time there to. (I see this on two systems.)


Reproducible: Always
Steps to Reproduce:
1. export LC_ALL="de_DE.utf8"
2. startx
3. wait

Actual Results:  
X comes up, but doesnt take input (apart from mouse moves)

Expected Results:  
a pretty fast start in a few minutes

x11-base/xorg-x11-6.8.0-r3
media-libs/freetype-2.1.5-r1

Portage 2.0.51-r3 (default-linux/x86/2004.3, gcc-3.3.4, glibc-2.3.4.20040808-r1,
2.6.8.1 i686)
=================================================================
System uname: 2.6.8.1 i686 AMD Athlon(tm) XP 2400+
Gentoo Base System version 1.4.16
distcc 2.16 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
Autoconf: sys-devel/autoconf-2.59-r5
Automake: sys-devel/automake-1.8.5-r1
Binutils: sys-devel/binutils-2.14.90.0.8-r1
Headers:  sys-kernel/linux26-headers-2.6.8.1
Libtools: sys-devel/libtool-1.5.2-r5
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CFLAGS="-O2 -march=athlon-xp -mmmx -m3dnow -fomit-frame-pointer -fforce-addr -pipe"
CHOST="i686-pc-linux-gnu"
COMPILER=""
CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config
/usr/kde/3/share/config /usr/share/config /usr/share/texmf/dvipdfm/config/
/usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/
/usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-O2 -march=athlon-xp -mmmx -m3dnow -fomit-frame-pointer -fforce-addr
-pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs autoconfig candy ccache distlocks sandbox sfperms"
GENTOO_MIRRORS="ftp://ftp.gentoo.skynet.be/pub/gentoo/"
MAKEOPTS="-j1"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.de.gentoo.org/gentoo-portage"
USE="3dnow 3ds X Xaw3d aalib acpi acpi4linux alsa avi berkdb bitmap-fonts bzlib
cdr crypt cscope cups curl dga dvd encode esd f2c fam fbcon ffmpeg foomaticdb
fortran gdbm gif glut gstreamer gtk gtk2 guile imagemagick imlib java jikes
joystick jpeg junit ldap libg++ libwww ltsp mad maildir mcal mikmod mmx mng
motif moznocompose moznoirc moznomail mozsvg mpeg mpeg4 ncurses nls nptl nvidia
offensive oggvorbis openal opengl oscar oss pam pcre pdflib perl physfs pic png
ppds python readline sasl sdl slang spell sqlite sse ssl tcpd tetex tiff
transcode truetype unicode usb v4l v4l2 vim-with-x x86 xinerama xml xml2 xmms
xsl xv xvid zlib"
Comment 1 Björn Michaelsen 2004-11-19 05:16:13 UTC
Arrgh! Just after filing this bug I found that LC_ALL should be "de_DE.utf-8", not "de_DE.utf-8". Sorry. Still, Xorg should complain about the locale since it is otherwise hard to find out what is wrong.
Comment 2 Björn Michaelsen 2004-11-19 07:00:43 UTC
Sorry to change this bug again. LC_ALL should be *.utf8 according to locales -a. This causes the long load times.
Comment 3 Donnie Berkholz (RETIRED) gentoo-dev 2004-11-19 22:23:12 UTC
Is there anything useful in /var/log/Xorg.0.log (attach it please) or on the console you start X from?

When looking in xorg's locale files, I don't even see one for de_DE.utf-8. That may be because I have nls off, but not sure.

Could you run `qpkg -l xorg-x11 | grep locale`?
Comment 4 Björn Michaelsen 2004-11-20 05:39:01 UTC
There is nothing different in Xorg.0.log or on the console. I started twm once with a utf8 locale and it complained about some missing fonts for this locale.
locales -a says the locales name is de_DE.utf8, not de_DE.utf-8 ...
"qpkg -l xorg-x11 | grep locale" doesnt return any de_DE locale

A "locate de_DE" yields among others /etc/X11/xserver/de_DE.utf8 and /etc/X11/xserver/de_DE.UTF-8. However, they dont seems to belong to *any* package (qpkg -f /etc/X11/xserver/de_DE.utf8 yields nothing). Strange.
Comment 5 Maurice van der Pot (RETIRED) gentoo-dev 2004-12-11 03:21:06 UTC
Created attachment 45726 [details]
strace of twm startup with en_US.UTF-8 locale

I have some more info about this.

`qpkg -l xorg-x11 | grep locale` does show my locale (en_US.UTF-8)

I noticed that it doesn't matter what the locale is when X is started, the
delay
is in starting the window manager. If I start X with en_US.UTF-8 locale and twm

with POSIX locale, everything is fine for instance.

I have attached a couple of straces, one for starting twm in en_US.UTF-8 and
one
in POSIX. There is *a lot* of stuff in the former about fonts that isn't in 
the latter, so I think it can help identify the cause of this problem.
Comment 6 Maurice van der Pot (RETIRED) gentoo-dev 2004-12-11 03:22:06 UTC
Created attachment 45727 [details]
strace of twm startup with POSIX locale
Comment 7 Donnie Berkholz (RETIRED) gentoo-dev 2004-12-11 12:43:11 UTC
So what you're telling me is that this is a problem between window managers and locales and has nothing to do with X proper.
Comment 8 Maurice van der Pot (RETIRED) gentoo-dev 2004-12-17 10:05:46 UTC
A workaround for flux has been provided as fix for bug #65186: USE="disablexmb" emerge fluxbox
Comment 9 fctk86 2005-01-05 00:27:42 UTC
Maurice van der Pot's workaround used to work for me but now (after having switched to ~x86) it works no more...
Comment 10 Maurice van der Pot (RETIRED) gentoo-dev 2005-01-05 09:33:45 UTC
I have always been using ~x86.

Which versions of what software are you talking about now?
(xorg-x11, fluxbox, twm, ...)
Comment 11 Radoslaw Szkodzinski 2005-01-26 15:36:42 UTC
I have a similar problem with pl_PL.UTF-8 and ion3.  I get long startup times (~1minute) with all non-TTF apps (like xfontsel, xmessage) and this error: Warning: missing charsets in String to FontSet conversion  x11-base/xorg-x11-6.8.1.902  zen encodings # emerge info Portage 2.0.51-r15 (default-linux/x86/2004.3, gcc-3.4.3, glibc-2.3.4.20041102-r0, 2.6.11-rc2-mm1 i686) ================================================================= System uname: 2.6.11-rc2-mm1 i686 AMD Athlon(tm) XP 2400+ Gentoo Base System version 1.6.8 Python:              dev-lang/python-2.3.4 [2.3.4 (#1, Jan  7 2005, 00:34:28)] dev-lang/python:     2.3.4 sys-devel/autoconf:  2.59-r6, 2.13 sys-devel/automake:  1.4_p6, 1.5, 1.9.4, 1.8.5-r2, 1.6.3, 1.7.9 sys-devel/binutils:  2.15.92.0.2-r2 sys-devel/libtool:   1.5.10-r3 virtual/os-headers:  2.6.8.1-r2 ACCEPT_KEYWORDS="x86 ~x86" AUTOCLEAN="yes" CFLAGS="-O2 -march=athlon-xp -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.3/env /usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/qmail/alias /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-O2 -march=athlon-xp -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs autoconfig ccache distlocks sandbox sfperms" GENTOO_MIRRORS="http://gentoo.prz.rzeszow.pl" MAKEOPTS="-j3" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage /usr/local/bmg-main.alternative" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="x86 3dnow X Xaw3d aalib alsa apache2 apm arts artswrappersuid avi berkdb  bitmap-fonts bzip2 bzlib caps cdr chroot cjk crypt cups curl dba emacs encode extensions f77 fam fbcon flac font-server foomaticdb fortran fpx gd gd-external gdbm gif gimpprint glut gpm gsm gtk gtk2 idea imagemagick imlib ipv6 ithreads jabber jack jack-tmpfs java javascript jbig jpeg jpeg2k latex lcms lesstif libclamav libg++ libwww mad memlimit mew mikmod mimencode mmx mng motif mozilla mpeg mpi mysql ncurses nls nogg normalizemime nptl nptlonly oggvorbis opengl oss pam pcre pdflib perl php pic png posix ppds pwdb python qt quicktime readline samba sdl session slang sndfile spell sse ssl svg tcltk tcpd tetex threads tidy tiff truetype truetype-fonts type1-fonts ucs2 unicode wmf xml xml2 xmms xprint xv zlib linguas_en" Unset:  ASFLAGS, CBUILD, CTARGET, LDFLAGS
Comment 12 fctk86 2005-01-28 06:29:12 UTC
@Maurice van der Pot
sorry for the *huge* late but i didn't read your answer... :(
anyway, now i have:
-fluxbox 0.9.12-r1
-xorg 6.8.1.902
Comment 13 Zsolt Barat 2005-02-05 10:42:55 UTC
long loading of xapps seems to be related to XLC_LOCALE in en_US.UTF-8. as other utf encodings are based on this, native xapps tries to load different asian fontsets defined here like: JISX0208.1983-0 and KSC5601.1987-0.
from strace xcalc:
select(4, [3], NULL, NULL, NULL)        = 1 (in [3])
read(3, "\1\3175\0\0\0\0\0\0\0\0\0\4\0\0\0xH]\t\1\0\0\0\0\6e\t{"..., 32) = 32
writev(3, [{"1\0\n\0\1\0\36\0", 8}, {"*-*-*-*-*-*-*-*-KSC5601.1987-0", 30}, {"\0\0", 2}], 3) = 40
read(3, 0xbfffd380, 32)                 = -1 EAGAIN (Resource temporarily unavailable)
select(4, [3], NULL, NULL, NULL)        = 1 (in [3])
read(3, "\1\3176\0\0\0\0\0\0\0\0\0\4\0\0\0xH]\t\1\0\0\0\0\6e\t{"..., 32) = 32
writev(3, [{"1\0\n\0\1\0 \0", 8}, {"*-*-*-*-*-*-*-*-*-KSC5601.1987-0", 32}], 2) = 40
read(3, 0xbfffd380, 32)                 = -1 EAGAIN (Resource temporarily unavailable)

i think installing these fonts (probably with USE=cjk) would solve this.
or modifying XLC_LOCALE?
Comment 14 Stephan 2005-06-11 02:39:15 UTC
Hello.
Got the same Problem with various tools.
Just add +cjk to your package.use for xorg-x11.
This solves the problem.
Maybe +cjk to global use flags would be an idea.

cheers.
Comment 15 Zsolt Barat 2005-09-18 14:46:04 UTC
what happend with this? was it sent upstream? don't want to depend on cjk-fonts
since i don't need them.
Comment 16 Maurice van der Pot (RETIRED) gentoo-dev 2005-09-19 08:11:22 UTC
re comment #7 (Donnie Berkholz):

I also noticed the problem with xcalc, so I started X without a window manager 
with only an xterm from .xinitrc, then launched xcalc and the problem was still 
occurred.
Comment 17 Joshua Baergen (RETIRED) gentoo-dev 2005-09-19 17:43:16 UTC
I haven't noticed this at all with x-modular.  Could someone check 6.8.99.15 to
see if this is still a problem for them in the newer versions?
Comment 18 Rohen Peterson 2005-12-16 23:03:51 UTC
ion3 also takes ages to load when compiled with LC_CTYPE="en_US.UTF-8", with xorg 6.8.2 and xorg 6.8.99.15.
Comment 19 Rohen Peterson 2005-12-16 23:19:11 UTC
Stupid for not checking this earlier, but the unstable version of ion3 loads quickly.
Comment 20 Chris Bainbridge (RETIRED) gentoo-dev 2006-03-19 05:58:59 UTC
I still see these utf-8 delays with the modular xorg-7. This is probably the same bug as http://bugs.gentoo.org/show_bug.cgi?id=101046 https://bugs.freedesktop.org/show_bug.cgi?id=4939 http://bugs.gentoo.org/show_bug.cgi?id=83769  and https://bugs.freedesktop.org/show_bug.cgi?id=2475 and related to http://bugs.gentoo.org/show_bug.cgi?id=93443
Comment 21 Donnie Berkholz (RETIRED) gentoo-dev 2006-04-04 15:22:06 UTC
Does it speed up again if you install font-daewoo-misc, font-isas-misc and font-jis-misc?
Comment 22 Donnie Berkholz (RETIRED) gentoo-dev 2006-04-19 21:24:14 UTC
Please reopen when you respond.
Comment 23 Maurice van der Pot (RETIRED) gentoo-dev 2006-04-21 10:23:32 UTC
(In reply to comment #21)
> Does it speed up again if you install font-daewoo-misc, font-isas-misc and
> font-jis-misc?

Yes. I can start xcalc without delay with these installed. 
But after unmerging them, the behaviour did not change back.
Does that make sense?

This is with x11-base/xorg-x11-7.0-r1
Comment 24 SpanKY gentoo-dev 2006-07-15 20:38:09 UTC
*** Bug 101046 has been marked as a duplicate of this bug. ***
Comment 25 Chris Bainbridge (RETIRED) gentoo-dev 2007-02-15 14:33:30 UTC
Yes, install those fonts, restart xorg, and eterm starts without any delay.
Comment 26 Pawel Madej aka Nysander 2008-11-24 23:10:12 UTC
shouldn't this bug be closed as we have much newer xorg and reported does not take part in resolving it?
Comment 27 Björn Michaelsen 2008-11-25 00:49:44 UTC
@Pawel: I wont be able to help testing as I have stopped using fluxbox years ago. So from my point of view this is RESOLVED/WORKSFORME. However, there might be others who are still hit by this (the duplicate is a hint), and I cannot speak for them.
Comment 28 Pawel Madej aka Nysander 2008-11-25 21:57:02 UTC
well last duplicate was 2 years ago and no new comments here after that.
Comment 29 SpanKY gentoo-dev 2009-01-07 14:01:02 UTC
no, the bug cannot be closed until it's fixed.  it's trivial to reproduce with Eterm, so spend all of three seconds doing so.
Comment 30 DEMAINE Benoît-Pierre, aka DoubleHP 2009-01-07 14:52:29 UTC
Comment 7 is right.

In reply to c29 : no, developper of Eterm admitted I am right: the problem is not X, but, that various applications (including window managers) do not handle UTF at all, what slow them down ... and *THIS* can be fixed by respective upstream. Eterm still dont support UTF (in the way that, it still can not print it properly), but, once patched, it is not slow anymore. For details, see the difference of the source of Eterm, SVN revision 38477 (which fixes the pb) and the previous one (which did not).

Wont be possible to repro when SVN-r.38477 is released in stable. But, until all apps are fixed, << it can still happen again>>.

In reply to c28: fact there is no activity does not people dont meet it again. Several times I have been reproached to confirm bugs, and Gentoo maints often tell: "YES WE KNOW WE DONT NEED ALL PEOPLE TO CONFIRM <so do I>". And now, when we dont, you assume bug faded away :/

What may have happen is that applications (Eterm now, and maybe Fluxbox ?) have been fixed ?

For details, see bug 253947 comments 0 and 4. Good luck.

I also think that the title of the bug is wrong: X is the graphical layer. X is quick to launch. What is *really* slow to start in what people describe is the window manager in X. Who ever tried to start "just X" and then the WM later on ? I did. And I say! X is quick.

in reply to c12: this is not enough: just giving your profile, USE flags and versions is way not enough. If i take the correct version of your ebuilds, and your USE flags, I am still missing the default USE flags of your profile by that time: that's why emerge --info is compleetely useless; what we need to be able to debug problems "years later" is "emerge -vp".
Comment 31 Rémi Cardona (RETIRED) gentoo-dev 2009-02-09 09:04:20 UTC
Reading this up again, I feel this has (almost) nothing to do with xorg-server, but rather individual apps who have trouble with UTF-8 locales.

I'll be closing this bug. I however strongly urge everyone to open bugs for each individual application to get them fixed, since this bug has become too much of a mess to be of any real use.

Thanks
Comment 32 Raphaël Droz 2011-01-01 17:38:52 UTC
This bug is still present at this time (x11-base/xorg-server 1.9.2, libXt 1.0.9 and xmessage 1.0.3 with -cjk)
The behaviour shown in https://qa.mandriva.com/show_bug.cgi?id=60967 is reproducible without difficulties:

time -p LC_CTYPE=C xmessage hello -timeout 1
real 1.01

time -p xmessage hello -timeout 1
Warning: Missing charsets in String to FontSet conversion
real 4.11

I now realize that here is the main performance bottleneck of my fluxbox desktop experience.

Does someone knows about upstream thoughts on this, or dug xmessage code ?
(I'm not willing to open hundreds of bugs against every and each X apps)
Comment 33 DEMAINE Benoît-Pierre, aka DoubleHP 2011-01-02 14:33:55 UTC
Raphael ... read this https://bugs.freedesktop.org/show_bug.cgi?id=30806 and you will see they just don't mind. One tuto gave once the bad example, every-one followed the bad example, and now, we are stuck.
Comment 34 Raphaël Droz 2011-01-02 15:48:35 UTC
I found this report to be the most representative of the situation:
https://bugs.freedesktop.org/show_bug.cgi?id=4939

I tested the source code provided there:
time ./a.out fr_FR.iso8859-15
0m0.041s
time ./a.out fr_FR.utf-8
0m2.980s
Comment 35 Rémi Cardona (RETIRED) gentoo-dev 2011-01-02 18:55:48 UTC
(In reply to comment #33)
> Raphael ... read this https://bugs.freedesktop.org/show_bug.cgi?id=30806 and
> you will see they just don't mind. One tuto gave once the bad example,
> every-one followed the bad example, and now, we are stuck.

Benoît-Pierre, stop trolling. You've been mumbling about X and just about everything else for _years_ now. Please stop.

(In reply to comment #34)
> I found this report to be the most representative of the situation:
> https://bugs.freedesktop.org/show_bug.cgi?id=4939
> 
> I tested the source code provided there:
> time ./a.out fr_FR.iso8859-15
> 0m0.041s
> time ./a.out fr_FR.utf-8
> 0m2.980s

And let me point you to comment #7 where the real culprit was found : broken apps. If you have issues with _specific_ apps, please open a new bug report and we'll try to sort it out.

Closing this bug for good.

Thanks
Comment 36 Raphaël Droz 2011-01-02 21:06:40 UTC
I may be wrong by posting here, but the huge slowdown I notice is not related to the WM used (but indeed I use fluxbox)
rm .xinitrc
export LANG=C; export LC_ALL=C; 
xinit xterm --

and the same behaviour happens with the POF code given in the freedesktop.org bug report 4939.

If the code given there is said "broken", fine, please explain how and I will then use xcalc ou xmessage as my test case.
But I fear I do not think that an average 3 seconds for this C line is "normal".
XCreateFontSet(d, "-*-*-*-R-*-*-*-120-*-*-*-*,*")

According to freedesktop bug 7832 (which is about a similar problem around the libXfont), it seems that there is still room for improvement for such routines
Comment 37 Raphaël Droz 2011-01-05 00:31:28 UTC
If the bug on freedesktop is not fixed there, I would vote for an libX11 ebuild flag (say "+cjk" or "nocjk").
If cjk locale should not be considered, then an epatch for :
libX11/nls/en_US.UTF-8/XLC_LOCALE.pre
(/usr/share/X11/locale/en_US.UTF-8/XLC_LOCALE)
would do the trick (but I do not know about side effects)

It would comment this part
   fs17 class (Korean Character)
fs13   { ... }
  fs18 class (Chinese Han Character)
fs14	{ ... }

KSC5601.1987-0 and GB2312.1980-0 are two problematic locales.
Kanji/Kana locale (JISX0201.1976-0) doesn't cause slow downs to me.

Otherwise, if cjk is selected, libX11 would pull the 2 essential font ebuilds.