Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 21420 - The paradox of Freetype/Xft/fontconfig - CJK glyphs do not hint/antialias under some circumstances
Summary: The paradox of Freetype/Xft/fontconfig - CJK glyphs do not hint/antialias und...
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Library (show other bugs)
Hardware: All Linux
: High trivial (vote)
Assignee: CJK Team
Depends on:
Reported: 2003-05-21 09:35 UTC by Zhen Lin
Modified: 2003-10-12 09:45 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---

LANG="" Big5 document (CJK-with-antialias-2.png,197.11 KB, image/png)
2003-05-21 09:40 UTC, Zhen Lin
LANG="zh_CN" UTF-8 page that demonstrates the problems. (CJK-with-antialias.png,134.53 KB, image/png)
2003-05-21 09:42 UTC, Zhen Lin
LANG="" Same UTF-8 page as before. (CJK-without-antialias.png,130.23 KB, image/png)
2003-05-21 09:45 UTC, Zhen Lin
The raw HTML of the problem page. (log.html,11.46 KB, text/html)
2003-05-21 09:47 UTC, Zhen Lin
CJK glyph table (CJK.png,177.18 KB, image/png)
2003-06-12 02:48 UTC, Zhen Lin
CJK glyph table, size 20 (CJK2.png,297.40 KB, image/png)
2003-06-13 03:58 UTC, Zhen Lin

Note You need to log in before you can comment on or make changes to this bug.
Description Zhen Lin 2003-05-21 09:35:54 UTC
See title.
Will attach screenshots.

More details as attachments come.

Reproducible: Always
Steps to Reproduce:
1. Visit a UTF-8 site, with CJK glyphs mixed with Latin glyphs, in the same line
and in close proximity; using Mozilla 1.2
2. Note the ugliness.
3. Revisit, with LANG=zh_CN this time.
4. Note the prettiness.

Actual Results:  
Exactly as described above.

Expected Results:  
Antialias and hint, always.

[ebuild   R  ] media-libs/freetype-2.1.4  -doc +zlib -prebuilt
[ebuild   R  ] media-libs/fontconfig-2.2.0-r2  -doc
[ebuild   R  ] x11-base/xfree-4.3.0-r2  -3dfx +sse +mmx -3dnow -xml +truetype
+nls +cjk -doc
[ebuild   R  ] net-www/mozilla-1.2.1-r5  +java +crypt -ipv6 -gtk2 +ssl +ldap
+gnome -mozsvg -mozcalendar -mozaccess -mozinterfaceinfo -mozp3p -mozxmlterm
-moznoirc -moznomail -moznocompose -moznoxft
Comment 1 Zhen Lin 2003-05-21 09:40:29 UTC
Created attachment 12248 [details]
LANG="" Big5 document


Note that Mozilla never has antialiased or hinted UI text. This was a bug
attributed to fontconfig...
Comment 2 Zhen Lin 2003-05-21 09:42:21 UTC
Created attachment 12249 [details]
LANG="zh_CN" UTF-8 page that demonstrates the problems.


Note how the Mozilla UI text is not only not smoothed, it is also oversized!
Comment 3 Zhen Lin 2003-05-21 09:45:24 UTC
Created attachment 12250 [details]
LANG="" Same UTF-8 page as before.


No smoothing, variable sizes, unreadable Latin text/scrawl.
Comment 4 Zhen Lin 2003-05-21 09:47:04 UTC
Created attachment 12251 [details]
The raw HTML of the problem page.

UTF-8 binary encoded, not sure if there will be corruption...
Comment 5 Zhen Lin 2003-05-26 04:46:07 UTC
More paradoxes!

A page, that used to contain only UTF-8 Japanese, was appended with some Shift JIS, which Mozilla attempted to render as a Chinese encoding, causing the entire page's CJK glyphs to become antialiased.

I suspect that my configuration antialiases Chinese natively-encoded text.

Or fonts. Perhaps there are outline-fonts (Type 1, TrueType etc.) that are designed for use with Chinese encodings but supports the other CJK glyphs.

Is there any way for me to discover what fonts are really being used to render the page?
Comment 6 Zhen Lin 2003-06-12 02:48:09 UTC
Created attachment 13144 [details]
CJK glyph table

Now that I have installed some Japanese fonts, the problems have been reduced;

Now, when I look at the attached page, all the Japanese glyphs are smooth and
evenly sized, but the Chinese ones are not. 

Also, this leads to this problem: Mozilla is currently set to render Japanese
using Fixed (JIS) but is obviously using one of the MS {Mincho|Gothic} fonts.
Comment 7 Alastair Tse (RETIRED) gentoo-dev 2003-06-12 13:51:52 UTC
hmm interesting .. 

do you have chinese truetype fonts installed? like media-fonts/arphicfonts ?

i'd like to help you fix this problem, but the screenshots you attached have been scaled down and i can't see any detail of the fonts.

which version of xfree and fontconfig are you using now? i believe the current version for stable enables anti-aliasing for cjk fonts.
Comment 8 Zhen Lin 2003-06-13 03:58:52 UTC
Created attachment 13197 [details]
CJK glyph table, size 20

The image has not been scaled down... Has it?

I was using a 16pixel fontsize. 

Also, note that when I use the Kai (Chinese) font, all the previously aliased
glyphs disappear. Is there a way for fontconfig/freetype to recognise that
those glyphs are a) outline/vector and b) exist? I thought there was a
preference given to outline fonts.

I've just copied & installed all the Windows CJK fonts (batang.ttc, gulim.ttc,
mingliu.ttc, msgothic.ttc, msmincho.ttc, simsun.ttc)
 ... Now the aliased glyphs are consistent with each other... Not exactly what
I wanted. (MingLiU CJK characters are still aliased)

If also have installed all Chinese fonts in portage, as well as Kai from Mac OS

[ebuild   R  ] media-libs/freetype-2.1.4  -doc +zlib -prebuilt
[ebuild   R  ] media-libs/fontconfig-2.2.0-r2  -doc
[ebuild   R  ] x11-base/xfree-4.3.0-r2	-3dfx +sse +mmx -3dnow -xml +truetype
+nls +cjk -doc

It would be nice to know exactly what was edited out so that I could institute
a reverse rule to that... I believe there used to be a CJK no antialias rule
but I can't find it anymore.
Comment 9 Zhen Lin 2003-06-13 23:56:12 UTC
I have finally tracked down the problem... Despite the fact that I specify TrueType outline fonts, FreeType still renders a TrueType _bitmap_ font instead.

After editing the truetype font to dike out the embedded bitmaps, the font smoothens.

Is there a way to specify to FontConfig/FreeType that outline, antialiased fonts are preferred?
Comment 10 Alastair Tse (RETIRED) gentoo-dev 2003-06-14 02:46:56 UTC
indeed. sorry about that, epiphany automatically scales down these images. i don't know if that is a bug of feature! grrr .. i'll take a close look at it now
Comment 11 Alastair Tse (RETIRED) gentoo-dev 2003-06-14 03:21:51 UTC
now that i can see the images properly, the first one with mozilla's UI, did you compile with gtk2 support? because if you didn't, the UI doesn't use xft support. this is what it looks like if you compile with gtk2.

unfortunately, moz-1.3 is still in ~x86 for some reason. also, i believe mozilla uses X's dpi settings to figure out the size of rendered fonts. if you are using gnome, you should goto applications->desktop settings->fonts and change the dpi settings to match your screen more correctly.

i'm not sure how you find out what gtk2 is using for your fonts. mozilla automatically detects the local of a page and translates the fonts to utf-8, and same goes for gtk2 with pango. i noticed on redhat they have a custom extension that makes fontconfig NOT choose bitmap fonts over truetype fonts. 

but the problem is if the characters do not exist for that font, then it will eventually choose them. so it really depends on what fonts you have installed and what you use in your mozilla settings. you can choose which fonts mozilla uses when rendering chinese language pages in the preferences.

the no anti-alias rule has been removed for some reason. that was possibly a workaround for older fontconfigs. if you're interested, the rule looks something like this:

        <match target="font">
                <test name="lang" compare="contains">
                <edit name="hinting" mode="assign">

i hope this helps you. i'll try and find out more about the fontconfig settings when i have time. but it all comes down to the fact we don't have good complete free chinese fonts available. i'll try and find more freely distributable fonts to add to gentoo when i have some time to see if there is an improvement over the arphicfonts
Comment 12 Zhen Lin 2003-06-14 05:09:13 UTC
I have also managed to fix the Mozilla aliased fonts problem... Just make Helvetica inaccessible and Mozilla will use your specified font, antialiased; without GTK2.

I have managed to fix the problem, partially. Perhaps I should file a bugreport saying that bitmaps are preferred over outline instead of vice-versa.

In the meantime, why not set the resolution to WORKSFORME? I haven't been able to consistently reproduce results.
Comment 13 Alastair Tse (RETIRED) gentoo-dev 2003-06-14 09:01:55 UTC
ok at least in my fonts.conf, it actually aliases helvetica to mean arial to get around that specific problem. it says:

 Check user preference to avoid bitmap fonts and replace
 bitmap face names with equivalent scalable fonts
   <match target="pattern">
           <test name="prefer_outline">
           <test name="family">
           <edit name="family" mode="prepend" binding="same">

if you don't have that in your /etc/fonts/fonts.conf, you should add it to /etc/fonts/local.conf.

i was just trying to get to the bottom of all this before i close the bug. it seems the problem with the bad characters comes down to the current chinese fonts available with gentoo (arphic/twmoe) being incomplete for unicode. i'm testing the freely available bitstream cyberbit font which is much better and will put it in portage soon. i'll close this bug when i have them in portage.
Comment 14 Zhen Lin 2003-07-12 07:46:24 UTC
Finally fixed the problem - installed Arial Unicode MS, set as a fallback font when glyphs are missing, and CJK gets rendered nicely.

However, Arial Unicode MS is unsatisfactory for the character 0x96F6 '&#38646;', which renders as 0xF9B2 '&#63922;' instead. Perhaps this is due to ignorance on the artist's part. 

Additionally, at 16 pixels, it seems that CJK hinting/autohinting is bad - the baseline is not aligned. How do I disable this one case in fontconfig and Xft?
Comment 15 Zhen Lin 2003-07-12 08:09:27 UTC
Looks like Bitstream Cyberbit is no longer open. However, using a copy I found, it seems that the zero character in both fonts are designed alike... 
Comment 16 Alastair Tse (RETIRED) gentoo-dev 2003-07-12 08:35:48 UTC
yeah, it isn't free anymore, thats why this bug still hasn't been closed. i've tried contacting them, but i haven't got any response. however, i've found that the Hong Kong government makes available a ISO10646/GB18030 compliant version of Ming that seems alright to include, since we're not redistributing the fonts themselves.

Comment 17 Alastair Tse (RETIRED) gentoo-dev 2003-10-12 09:44:58 UTC
i've done quite a bit of research on free fonts for chinese character sets
and there is pretty much no realy high-quality font we can directly distribute
except for one provided by the Hong Kong Government.

i've mailed them but they seem to have no real position on whether an ebuild
satisfies their terms of use. i'll try and bring it up with other devs, but
for the moment, we can't really fix the situation. so i'm closing this now
because it is unlikely we'll come to a real resolution on this bug.
Comment 18 Alastair Tse (RETIRED) gentoo-dev 2003-10-12 09:45:43 UTC
i might as well add, here is a summary of all the chinese fonts i could find.