Bug 13416

Summary: Font subfamilies merged
Product: fontconfig Reporter: Sam Liddicott <sam>
Component: fc-cacheAssignee: fontconfig-bugs
Status: RESOLVED MOVED QA Contact: Behdad Esfahbod <freedesktop>
Severity: normal    
Priority: medium CC: akira, belegdol, fonts-bugs
Version: 2.4   
Hardware: x86 (IA32)   
OS: All   
Whiteboard:
i915 platform: i915 features:
Attachments: output of fc-query for problem font familes

Description Sam Liddicott 2007-11-28 02:37:26 UTC
I have the full set of adobe helvetica including Helvetica Inserat and Helvetica Neue.

fontconfig seems to be merging these into one happy family called "Helvetica" regardless of variant (Neue or Inserat) or style.

As I'm using fontconfig 2.4.2 on Ubuntu I have also reported this on Launchpad:
https://bugs.launchpad.net/ubuntu/+source/fontconfig/+bug/24764
where there seems to be a similar problem that was bug #4924 here, apparently fixed. (So maybe it's a different problem).

fc-list | grep -i helvetica
seems to be missing out most of the Inserat / Neue / fractions variants and just using the name "Helvetica"

Helvetica:style=Fractions Bold
Helvetica:style=Ultra Compressed
Helvetica:style=.Black Oblique
Helvetica:style=Rounded Bold Condensed Oblique
Helvetica:style=Narrow Bold
Helvetica:style=Black
Helvetica:style=Narrow Bold Italic
Helvetica:style=Condensed Light
Helvetica:style=Oblique
Helvetica:style=Fractions
Helvetica:style=Condensed Medium
Helvetica:style=Narrow
Helvetica:style=Compressed
Helvetica:style=Rounded Black
Helvetica:style=Rounded Bold Condensed
Helvetica:style=Condensed Bold Oblique
Helvetica:style=Light
Helvetica Inserat:style=Roman
Helvetica:style=Light Oblique
Helvetica:style=Narrow Italic
Helvetica:style=Rounded Black Oblique
Helvetica:style=Bold
Helvetica:style=Condensed Bold
Helvetica Neue:style=Regular
Helvetica:style=Regular
Helvetica:style=Condensed Black Oblique
Helvetica:style=Condensed Light Oblique
Helvetica:style=Condensed Oblique
Helvetica:style=Condensed Black
Helvetica:style=Bold Oblique
Helvetica:style=Extra Compressed
Helvetica:style=Rounded Bold Oblique
Helvetica:style=Rounded Bold

I also notice that Gnome Font Viewer gets it wrong.

HLB__*.PFB is called "Helvetica Neue, Regular" by Gnome Font Viewer which is wrong, the B after HL in HLB means BOLD.
Kfontview correctly calls it "Helvetica Neue, Bold"
fontforge also knows that the font is bold.
firefox chooses the wrong Helvetica
openoffice can't find any Helvetica at all

Note that fc-list did not show any bold versions of Helvetica Neue



# FC_DEBUG=384 fc-cache -f
shows:
        Scanning file /usr/share/fonts/type1/dbam/hlb_____.pfb...
using FreeType family "Helvetica Neue"
using FreeType style "Regular"
        Type1 weight Bold maps to 200
        Style Regular maps to width -1
        Style Regular maps to slant -1
        Style Regular maps to decorative 0
...

So it knows after a fashion that this is Bold, but still calls it style "Regular"

and then:
        Scanning file /usr/share/fonts/type1/dbam/hlbc____.pfb...
using FreeType family "Helvetica Neue"
using FreeType style "Regular"
        Type1 weight Bold maps to 200
        Style Regular maps to width -1
        Style Regular maps to slant -1
        Style Regular maps to decorative 0

the hlbc_*.pfb is bold condensed, but even kfontview did not "know" this.

Inkscape shows the fonts available as:

Hevetica: Regular, Black, Compressed, Condensed Black, Condensed Light, Extra Compressed, Fractions, Light, Narrow, Rounded Black, Ultra Compressed, Condensed Medium, .Black Oblique, Condensed Black Oblique, Condensed Oblique, Light Oblique, Narrow Italic, Oblique, Rounded Black Oblique, Bold, Condensed Bold, Fractions Bold, Narrow Bold, Rounded Bold, Rounded Bold Condensed, Bold Oblique, Condensed Bold Oblique, Bold Oblique, Narrow Bold Italic, Rounded Bold Condensed Oblique, Rounded Bold Italic

(and shows the wrong face, e.g. i see a fractions face when Regular is selected)

Helvetica Inserat: Roman, Italic, Bold, Bold Italic

Helvetica Neue: Regular, Regular, Regular, Regular, Regular, Regular, Regular, Regular, Regular, Regular, Regular, Regular, Regular, Regular

Which clearly isn't right.
Comment 1 Sam Liddicott 2007-11-28 02:39:21 UTC
The font file list (in case it helps) is:
hebco___.pfb
hebco___.pfm
hebc____.pfb
hebc____.pfm
heblo___.pfb
heblo___.pfm
hebl____.pfb
hebl____.pfm
hebo____.pfb
hebo____.pfm
heb_____.pfb
heb_____.pfm
hir_____.pfb
hir_____.pfm
hlao____.pfb
hlao____.pfm
hla_____.pfb
hla_____.pfm
hlavo___.pfb
hlavo___.pfm
hlav____.pfb
hlav____.pfm
hlbco___.pfb
hlbco___.pfm
hlbc____.pfb
hlbc____.pfm
hlbi____.pfb
hlbi____.pfm
hlbli___.pfb
hlbli___.pfm
hlbl____.pfb
hlbl____.pfm
hlbou___.pfb
hlbou___.pfm
hlb_____.pfb
hlb_____.pfm
hlbvo___.pfb
hlbvo___.pfm
hlbv____.pfb
hlbv____.pfm
hlco____.pfb
hlco____.pfm
hlc_____.pfb
hlc_____.pfm
hlhco___.pfb
hlhco___.pfm
hlhc____.pfb
hlhc____.pfm
hlhi____.pfb
hlhi____.pfm
hlh_____.pfb
hlh_____.pfm
hlhvo___.pfb
hlhvo___.pfm
hlhv____.pfb
hlhv____.pfm
hli_____.pfb
hli_____.pfm
hljo____.pfb
hljo____.pfm
hlj_____.pfb
hlj_____.pfm
hllco___.pfb
hllco___.pfm
hllc____.pfb
hllc____.pfm
hlli____.pfb
hlli____.pfm
hll_____.pfb
hll_____.pfm
hllvo___.pfb
hllvo___.pfm
hllv____.pfb
hllv____.pfm
hlmco___.pfb
hlmco___.pfm
hlmc____.pfb
hlmc____.pfm
hlmi____.pfb
hlmi____.pfm
hlm_____.pfb
hlm_____.pfm
hlmvo___.pfb
hlmvo___.pfm
hlmv____.pfb
hlmv____.pfm
hlr_____.pfb
hlr_____.pfm
hltco___.pfb
hltco___.pfm
hltc____.pfb
hltc____.pfm
hlti____.pfb
hlti____.pfm
hlt_____.pfb
hlt_____.pfm
hltvo___.pfb
hltvo___.pfm
hltv____.pfb
hltv____.pfm
hluli___.pfb
hluli___.pfm
hlul____.pfb
hlul____.pfm
hlvo____.pfb
hlvo____.pfm
hlv_____.pfb
hlv_____.pfm
hlzco___.pfb
hlzco___.pfm
hlzc____.pfb
hlzc____.pfm
hlzvo___.pfb
hlzvo___.pfm
hlzv____.pfb
hlzv____.pfm
hvblo___.pfb
hvblo___.pfm
hvbl____.pfb
hvbl____.pfm
hvbo____.pfb
hvbo____.pfm
hvb_____.pfb
hvb_____.pfm
hvcbl___.pfb
hvcbl___.pfm
hvcbo___.pfb
hvcbo___.pfm
hvcb____.pfb
hvcb____.pfm
hvcdo___.pfb
hvcdo___.pfm
hvclo___.pfb
hvclo___.pfm
hvcl____.pfb
hvcl____.pfm
hvco____.pfb
hvco____.pfm
hvc_____.pfb
hvc_____.pfm
hvek____.pfb
hvek____.pfm
hvfrb___.pfb
hvfrb___.pfm
hvfr____.pfb
hvfr____.pfm
hvk_____.pfb
hvk_____.pfm
hvlo____.pfb
hvlo____.pfm
hvl_____.pfb
hvl_____.pfm
hvnbi___.pfb
hvnbi___.pfm
hvnb____.pfb
hvnb____.pfm
hvni____.pfb
hvni____.pfm
hvn_____.pfb
hvn_____.pfm
hvo_____.pfb
hvo_____.pfm
hv______.pfb
hv______.pfm
hvuk____.pfb
hvuk____.pfm
Comment 2 Keith Packard 2008-01-13 17:00:09 UTC
As you can see from the debug output, the Type1 versions of these fonts have some ambiguous data (the style name 'Regular' shouldn't occur on a Bold font). I've found the .otf versions include far better information.

Probably the only reasonable fix here is to provide configuration files that correctly map these fonts to the desired patterns using the new 'scan' version of the match/edit rules.
Comment 3 Julian Sikorski 2008-06-22 11:04:09 UTC
Not sure if this is the same bug, but what I am seeing on my system (Fedora 9) is quite similar. I have both Arial (from msttcorefonts rpm) and Arial Narrow (put in ~/.fonts) installed. The issue is that the latter gets confused with the former. There is no such font as Arial Narrow listed in the applications, and the only way I can make the text in openoffice use the narrow font is to erase the msttcorefonts package. It is still listed as Arial in the apps, but it has the narrow glyphs. For reference, Fedora 9 uses fontconfig-2.5.0. Relevant fc-list output:

[jsikorski@snowball arialn]$ fc-list | grep Arial
Arial Black:style=Normalny,Normal,obyčejné,Standard,Κανονικά,Regular,Normaali,Normál,Normale,Standaard,Обычный,Normálne,Navadno,Arrunta
Arial,Arial Narrow:style=Normalny,Narrow,Normal,obyčejné,Standard,Κανονικά,Regular,Normaali,Normál,Normale,Standaard,Обычный,Normálne,Navadno,Arrunta
Arial:style=Pogrubiona kursywa,Negreta cursiva,tučné kurzíva,fed kursiv,Fett Kursiv,Έντονα Πλάγια,Bold Italic,Negrita Cursiva,Lihavoitu Kursivoi,Gras Italique,Félkövér dőlt,Grassetto Corsivo,Vet Cursief,Halvfet Kursiv,Negrito Itálico,Полужирный Курсив,Tučná kurzíva,Fet Kursiv,Kalın İtalik,Krepko poševno,nghiêng đậm,Lodi etzana
Arial,Arial Narrow:style=Pogrubiona kursywa,Narrow,Negreta cursiva,tučné kurzíva,fed kursiv,Fett Kursiv,Έντονα Πλάγια,Bold Italic,Negrita Cursiva,Lihavoitu Kursivoi,Gras Italique,Félkövér dőlt,Grassetto Corsivo,Vet Cursief,Halvfet Kursiv,Negrito Itálico,Полужирный Курсив,Tučná kurzíva,Fet Kursiv,Kalın İtalik,Krepko poševno,Lodi etzana
Arial:style=Kursywa,Cursiva,kurzíva,kursiv,Πλάγια,Italic,Kursivoitu,Italique,Dőlt,Corsivo,Cursief,Itálico,Курсив,İtalik,Poševno,nghiêng,Etzana
Arial:style=Normalny,Normal,obyčejné,Standard,Κανονικά,Regular,Normaali,Normál,Normale,Standaard,Обычный,Normálne,Navadno,thường,Arrunta
Arial,Arial Narrow:style=Pogrubiony,Narrow,Negreta,tučné,fed,Fett,Έντονα,Bold,Negrita,Lihavoitu,Gras,Félkövér,Grassetto,Vet,Halvfet,Negrito,Полужирный,Fet,Kalın,Krepko,Lodia
Arial:style=Pogrubiony,Negreta,tučné,fed,Fett,Έντονα,Bold,Negrita,Lihavoitu,Gras,Félkövér,Grassetto,Vet,Halvfet,Negrito,Полужирный,Fet,Kalın,Krepko,đậm,Lodia
Arial,Arial Narrow:style=Kursywa,Narrow,Cursiva,kurzíva,kursiv,Πλάγια,Italic,Kursivoitu,Italique,Dőlt,Corsivo,Cursief,Itálico,Курсив,İtalik,Poševno,Etzana
[jsikorski@snowball arialn]$ 
Comment 4 Nicolas Mailhot 2008-06-22 11:24:27 UTC
Using a single family for all its different faces is fine.
Not being able to distinguish between them afterwards is not
Comment 5 Julian Sikorski 2008-06-22 13:25:26 UTC
I was trying various applications and so far only Firefox (3.0) was able to display Arial Narrow properly:
http://mrmazda.no-ip.com/auth/Font/fonts-comps-narrow.html
Not sure if this information brings any value, though. OpenOffice.org, Abiword, Gnome and KDE font selectors are broken. Maybe Firefox uses some tricks to solve this issue? This is likely, since the site mentioned above defaults to monospace for Arial Narrow in Konqueror.
Comment 6 Julian Sikorski 2008-10-12 11:33:27 UTC
Any updates on this? I have filed a separate bug in Red Hat bugzilla, which might contain some more useful information:
https://bugzilla.redhat.com/show_bug.cgi?id=466678
Comment 7 Julian Sikorski 2008-12-10 05:26:39 UTC
Please have a look into this issue, no document using Arial Narrow font can be displayed properly at this point. It's likely that in order to fix this properly bug #18725 will have to be fixed first.
Comment 8 dmotd 2010-03-23 22:32:31 UTC
Created attachment 34392 [details]
output of fc-query for problem font familes
Comment 9 dmotd 2010-03-23 22:35:44 UTC
Comment on attachment 34392 [details]
output of fc-query for problem font familes

Same issue is appearing here, in this case 'Helvetica Condensed' and 'Helvetica
Black' are both being registered as their own family and also as 'Helvetica',
resulting in duplicate style entries in the Helvetica family, and the rendering
of Helvetica as 'Helvetica Condensed'.

I am using truetype versions of these fonts, I've attatched output of fc-query
for each font.

fontconfig version 2.8.0 on archlinux.
Comment 10 Nicolas Mailhot 2010-03-24 02:34:26 UTC
It is normal that font sub-families are not limited to n,r,b,bi. The Opentype spec allowed more subfamilies a long time ago. So fonts with the same family name are going to be merged with lots of different sub-families.

Now the people that write the Opentype spec did it in two times :
1. at first they did not put any constraint on font subfamily names; "red beetle" was a valid style name
2. then when microsoft tried to make use of the new fonts in vista it realised apps needed a fixed list of possible subfamilies to be able manage them (it is not possible to use CSS font style operators when a valid subfamily can be "naughty crocodile"). So in a later spec version they restricted the possible subfamilies to a specific list (defined in their WWS whitepaper)

Right now fontconfig does not check if fonts conform to spec 2. even though it is known such fonts are application-unfriendly (ms transforms the font subfamily names with heuristics to make sure apps have a name conforming to 2. even if the font itself is broken). So it lets any font subfamily pass in a merged font. But what is broken is the font files, not fontconfig.

Also, some apps like openoffice.org were written with assumptions such as "only n,r,b,bi styles can exist". They are slowly being fixed but in the meanwhile they are broken with modern fonts. And the fix is not to hide modern fonts from them in fontconfig, the fix is to change those apps. Since only n,r,b,bi styles existed for a long time, finding the problem bits of code and fixing them is slow

To understand this mess you need to read
http://nim.fedorapeople.org/Understanding%20fonts%20and%20fontconfig.tar.gz
http://blogs.msdn.com/text/attachment/2249036.ashx
http://blogs.adobe.com/typblography/typotechnica2007/Font%20names.pdf
http://blogs.adobe.com/typblography/atypi2006/CSS%20&%20OT%2015.pdf

Executive summary:
1. If a font does not expose a WWS-compliant style name it is broken today (no funny names, no deviation even if it existed in the past even in some widely used proprietary fonts)
2. If an app can not use a font with a WWS-compliant style the app is broken (n,r,b,bi is not a reasonable asumption anymore)
3. It would be nice if fontconfig changed style names to be WWS-compliant, but it is a workaround at best and fontconfig is usually not the broken bit (it's either 1. or 2.)

PS The same fonts won't behave the same way in all apps because every app does not read the same font naming metadata. Fontconfig reads the most recent one (as defined in the OpenType spec). If the font author put correct info in older naming layers, but didn't put correct info in the recent fields (because he didn't have any modern app to test with), the font will work in old apps but not in modern apps (such as those that use modern fontconfig versions)
Comment 11 Sam Liddicott 2010-03-24 03:03:25 UTC
I accept Nicolas's summary of the situation.

The question remaining is not whether or not fontconfig is to blame but:
1. if fontconfig can do anything to make things better (like ms did)
2. if fontconfig should do anything to make things better

or if we should just wait a few years until all the fonts and programs are fixed
Comment 12 Julian Sikorski 2010-03-24 03:14:17 UTC
Is Arial Narrow case #1 then? The current behaviour is sort of confusing, and I think it is pretty obvious now that the font is not going to get fixed. So, my question is, are there any plans to do anything about bug #18725? It's been almost two years...
Comment 13 Julian Sikorski 2010-03-24 03:21:15 UTC
Or maybe there is a fixed Arial Narrow font? I just noticed that the font is available from linotype, but I'm rather unwilling to pay 149 Swiss francs just to check. Did anyone try to use the newer version?
Comment 14 Nicolas Mailhot 2010-03-24 10:00:20 UTC
The version of Arial Narrow that a lot of people have (the version MS allowed to be redistributed before changing its mind) is clearly case #1. They probably fixed later versions (or relied on their renaming heuristic to hide this fact, reading the MS whitepaper Arrial Narrow almost certainly was one of their primary test cases for heuristical renaming)

But anyway yes there is no good solution short of fixing fonts and apps (an heuristic, by definition, is not a safe solution, it sort of works in most cases and fails horribly in others). And that won't happen by complaining in fontconfig's bugzilla. The people that write font and apps do not read it. If you find a problem font complain at its issuer. If you find a problem app complain at its author. fontconfig can not possibly hide 100% the fact that the OpenType spec has changed over the years. For one thing, the changes in the spec were done because font authors and app authors requested them. So hiding them is punishing good apps and fonts (that use the new spec properly) to avoid fixing bad fonts and bad apps (that try to ignore requirements changed)
Comment 15 Julian Sikorski 2010-03-24 12:21:48 UTC
Reading the #18725 it seems like you were for adding heuristics or some sort of matching tables in the past.
I understand your argument that heuristics is bound to miserably fail at some point, but why providing a way to work around broken fonts would be bad? It  might delay fixing of fonts, but given that the spec was requested by font authors, I would assume that the ones that cared already did their job. Believing that all fonts will eventually be fixed is unrealistic.
Taking ACPI as an example, there are quirks provided so that laptops no longer supported by manufacturer can suspend.
Comment 16 Nicolas Mailhot 2010-03-25 04:20:42 UTC
This is fine in theory but who is going to write the quirks? We don't even have correct fontconfig configuration for all non-broken fonts in Fedora for example, let alone broken out-of-distribution fonts (and people have failed to find the correct settings for CJK for years).

If you want to write quirks, fontconfig has public documentation about its config syntax, we've added some fedora-side in fontpackages-devel, etc. So, nothing here to stop you now. But it is much easier for a font author to fix its font than to try to intercept all the ways applications query font metadata and fix it up at this level. (because fonts formats have accumulated over the years *many* different naming layers and you can't guess beforehand which one an app will use, or worse which *mix* of accesses it's going to try, assuming they are consistent when the data font side is not).

And it remains that the apps people complain of most (openoffice.org, firefox, inkscape, etc) have broken font handling (as in, they've implemented 80% of what's needed and someone who cares will do the rest) that shows breakage as soon as they get fed non-trivial fonts. And the breakage is *not* in fontconfig it's in the use of the library application-side. So it's *useless* to try to fix fonts of fake font characteristics at fontconfig level if you don't get the apps fixed first.

Complex fonts work in Adobe apps for example because Adobe did its work (1. released complex fonts *and* 2. added needed bits ui-side so the new features could be accessed). You're asking to avoid doing 2. and avoid fixing 1. when it has bugs and somehow magically get the same result as in Adobe apps through some fontconfig miracle. It's not going to happen. Fontconfig is good, but not that good.

Comment 17 Sam Liddicott 2010-03-25 04:36:39 UTC
Apart from this long rambling discussion spanning 2 years, is there a concise source of information to which we might refer developers of packages from which they can learn how to implement the missing 20%.

It would be easy to close this bug as wont-fix if there is a simple message to convey on what needed to be done in the applications.
Comment 18 Nicolas Mailhot 2010-03-25 04:55:18 UTC
The reference documentation on modern font naming is
http://blogs.msdn.com/text/attachment/2249036.ashx

It's not short or well written but that's the best we have. It would be nice is someone wrote a clear summary of it (I've started in http://nim.fedorapeople.org/Understanding%20fonts%20and%20fontconfig.tar.gz but I do not seem to find the time to finish it)

Apps need to :
1. recognize fonts that use a style conformant to the WWS model defined in this paper (not "read the WWS name" but "check whatever naming layer they use conform to the model). Probably both the canonical WWS style name and the aliases MS identified in its paper
4. provide means for users to pass from one of the styles to another (as in, bold-er, wide-er, etc)

And *then* when apps are able to use WWS fonts properly people can think of band-aids to process non-WWS fonts.

As long as apps provide a "bold" button but have no idea how to manage all the weight variants defined in the whitepaper for example there's little hope
Comment 19 Julian Sikorski 2010-04-29 11:39:35 UTC
Accidentally, I found a workaround for my Arial Narrow problem. I learned that there is a Liberation Sans Narrow font available. For some reason it shows as a separate font, not as a Liberation Sans variant. As a result, I was able to set up substitution table in OpenOffice.org.
Any ideas why this font is not merged with Liberation Sans proper?
Comment 20 GitLab Migration User 2018-08-20 21:45:44 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/fontconfig/fontconfig/issues/38.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.