Detailed font reports using fontreport tool

Google i18n team developed a tool to create detailed report of fonts. The tool named fontreport, produces a multi page PDF with Unicode coverage of the font, what glyphs are in it, what Open Type features it supports, available ligatures, and glyph substitutions. Optionally the tool can also create plain text reports. The PDF is generated using TeX.

Manjari font report generated using fontreport tool

I found it very useful to create report for a dozen of fonts I maintain with Swathantha Malayalam Computing community. Sharing the reports it created:

Font reports(PDF):

  1. Rachana Regular
  2. Rachana Bold
  3. Meera
  4. Manjari Regular
  5. Manjari Bold
  6. Manjari Thin
  7. Dyuthi
  8. Chilanka
  9. Karumbi
  10. AnjaliOldLipi
  11. Keraleeyam
  12. Uroob

Fontconfig language matching

I had to spend a few hours to debug a problem about fontconfig not identifiying a font for a language. Following the tradition of sharing the knowledge you acquired in hard way, let me note it down here for search engines.

The font that I am designing now has 3 style variants, thin, regular and bold. All has same family name. So if you set this family for whatever purpose, depending on context, thin, regular or bold versions will be picked up. Regular is expected by default. Also when you pick the font from font selectors, you would expect, regular being selected by default.

The problem I was facing is, instead of Regular, Bold was getting selected as default. In font selectors, Bold was listed first.

In GNU/Linux systems, this font matching and selection is done by fontconfig. I started with fc-match

$ fc-match MyFont
MyFontBold.otf: "MyFont" "Bold"

So that confirms the problem. After fiddling with os/2 properties , asking in fontconfig mailing list, and reading fontconfig documentation, I found that the lang property fontconfig calculates from Regular variant of font does not include ‘en’

$ fc-list MyFont : family : style : lang 
MyFont:style=Bold:lang=aa|ay|bi|br|ch|en|es|eu|fj|fur|gd|gl|gv|ho|ia|id|ie|io|it|mg|ml|nl|nr|nso|oc|om|pt|rm|so|sq|ss|st|sw|tl|tn|ts|uz|vo|xh|yap|zu|an|fil|ht|jv|kj|kwm|li|ms|ng|pap-an|pap-aw|rn|r
w|sc|sg|sn|su|za 
MyFont:style=Regular:lang=aa|ay|bi|br|ch|da|de|es|et|eu|fi|fj|fo|fur|fy|gl|ho|ia|id|ie|io|is|it|ki|lb|mg|ml|nb|nds|nl|nn|no|nr|nso|ny|om|rm|sma|smj|so|ss|st|sv|sw|tl|tn|ts|uz|vo|vot|xh|yap|zu|an|f
il|ht|jv|kj|kwm|li|ms|na|ng|pap-an|pap-aw|rn|rw|sc|sg|sn|su|za

I tried to find how fontconfig calculates the languages supported by a font. The minimum set of code points to be included in a font so that fontconfig declare that it supports a given language is defined in the fontconfig library. You can find them in source code. For example, mandatory code points(glyphs that match to it) to be present for English is defined in en.orth file. I cross checked each code points and one was indeed missing from my regular font variant, but bold version had everything. When I added it, all started working normally.

Later fontconfig developer Akira TAGOH told me that I can also use fc-validate to check the language coverage

$ fc-validate --lang=en MyFont.otf
MyFont.otf:0 Missing 1 glyph(s) to satisfy the coverage for en language

And after adding the missing glyph

$ fc-validate --lang=en MyFont.otf
MyFont.otf:0 Satisfy the coverage for en language

And now fc-match list Regular as default style

$ fc-match MyFont
MyFont.otf: "MyFont" "Regular"

New handwriting style font for Malayalam: Chilanka

A new handwriting style font for Malayalam is in development. The font is named as “Chilanka”(ചിലങ്ക).

This is a alpha version release. Following is a sample rendering.

More samples here.

You may try the font using this edtiable page http://smc.org.in/downloads/fonts/chilanka/tests/ -It has the font embedded

Download the latest version: http://smc.org.in/downloads/fonts/chilanka/Chilanka.ttf

Chilanka/ചിലങ്ക is a musical anklet

A brief note on the workflow I used for font development is as follows

  1. Prepared a template svg in Inkscape that has all guidelines and grid setup.
  2. Draw the glyphs. This is the hardest part. For this font, I used bezier tool of inkscape. SVG with stroke alone is saved. Did not prepare outline in Inkscape, this helped me to rework on the drawing several times easily. To visualize how the stroke will look like in outlined version, I set stroke width as 130, with rounded end points. All SVGs are version tracked. SVGs are saved as inkscape svgs so that I can retain my guidelines and grids.
  3. In fontforge, import this svgs and create the outline using expand stroke, with stroke width 130, stroke height 130,  pen angle 45 degree, line cap and line join as round.
  4. Simplify the glyph automatically and manually to reduce the impact of conversion of Cubic bezier to quadratic bezier.
  5. Metrics tuning. Set both left and right bearings as 100 units(In general, there are glyph specfic tuning)
  6. The opentype tables are the complex part. But for this font, it did not take much time since I used SMC’s already existing well maintained feature tables. I could just focus on design part.
  7. Test using test scripts

Some more details:

  • Design: Santhosh Thottingal
  • Technology: Santhosh Thottingal and Kavya Manohar
  • Total number of glyphs: 676. Includes basic latin glyphs.
  • Project started on September 15, 2014
  • Number of svgs prepared: 271
  • Em size: 2048. Ascend: 1434. Descend: 614
  • 242 commits so far.
  • Latest version: 1.0.0-alpha.20141027
  • All drawings are in inkscape. No paper involved, no tracing.

Thanks for all my friends who are helping me testing and for their encouragement.
Stay tuned for first version announcement 🙂

(Cross posted from http://blog.smc.org.in/new-handwriting-style-font-for-malayalam-chilanka/ )

Meera Tamil font in Ubuntu Trusty Tahr

Ubuntu Trusty Tahr is going to be released on April 17th 2014.Meera Tamil

Meera Tamil font, a free licensed unicode font for Tamil will be available in this release.

The font is already available in Debian. In both Ubuntu and Debian you can install the font by

sudo apt-get install fonts-meera-taml

Thanks Vasudev for packaging it for Debian.

Spurious glyphs in NotoSansMalayalam

NotoSansMalayalam is a font released by Google internationalization team under noto project. I was checking the glyphs of Malayalam and noted a number of spurious glyphs in the font

notosansml-unwantedglyphs

It is interesting because the font attempted to provide a minimal Malayalam font with reduced glyph set. While attempting that about 10% of the glyphs are either non-existing Malayalam glyphs(Glyphs with dot under consonants) or rarely used glyphs(Glyphs with U+0D62 MALAYALAM VOWEL SIGN VOCALIC L)

Bug filed. It was learned that the last set of glyphs with dots above and below are for Carnatic music notations. They are not defined by unicode, but font used a custom way of representing them with a ligature formed by U+0307.

Collaboratively edited documentation for Indic font developers

One of the integral building blocks for providing multilingual support for digital content are fonts. In current times, OpenType fonts are the choice. With the increasing need for supporting languages beyond the Latin script, the TrueType font specification was extended to include elements for the more elaborate writing systems that exist. This effort was jointly undertaken in the 1990s by Microsoft and Adobe. The outcome of this effort was the OpenType Specification – a successor to the TrueType font specification.

JanaSanskritSans_ddhrya
The Devanagari ddhrya-ligature, as displayed in the
JanaSanskritSans font.

Fonts for Indic languages had traditionally been created for the printing industry. The TrueType specification provided the baseline for the digital fonts that were largely used in desktop publishing. These fonts however suffered from inconsistencies arising from technical shortcomings like non-uniform character codes. These shortcomings made the fonts highly unreliable for digital content and their use across platforms. The problems with character codes were largely alleviated with the gradual standardization through modification and adoption of Unicode character codes. The OpenType Specification additionally extended the styling and behavior for the typography.

The availability of the specification eased the process of creating Indic language fonts with consistent typographic behaviour as per the script’s requirement. However, disconnects between the styling and technical implementation hampered the font creation process. Several well-stylized fonts were upgraded to the new specification through complicated adjustments, which at times compromised on their aesthetic quality. On the other hand, the technical adoption of the specification details was a comparatively new know-how for the font designers. To strike a balance, an initiative was undertaken by the a group of font developers and designers to document the knowledge acquired from the hands own experience for the benefit of upcoming developers and designers in this field.

glyph-fontforge-meera
Glyphs inside Meera font

The outcome of the project will be an elaborate, illustrated guideline for font designers. A chapter will be dedicated to each of the Indic scripts – Bengali, Devanagari, Gujarati, Kannada, Malayalam, Odia, Punjabi, Tamil and Telugu. The guidelines will outline the technical representation of the canonical aspects of these complex scripts. This is especially important when designing for complex scripts where the shape or positioning of a character depends on its relation to other characters.

This project is open for participation and contributors can commit directly on the project repository.

New version of Malayalam fonts released

Swathanthra Malayalam Computing project announced the release of new version of Malayalam unicode fonts this week. In this version, there are many improvements for popular Malayalam fonts Rachana and Meera. Dyuthi font has some bug fixes. I am listing the changes below.

  1. Meera font was small compared to other fonts. This was not really a problem in Gnome environment since fontconfig allows you to define a scaling factor to match other font size. But it was an issue in Libreoffice, KDE and mainly in Windows where this kind of scaling feature does not work. Thanks to P Suresh for a rework on glyphs and fixing this issue.
  2. Rachana, Meera and Dyuthi had wrong glyphs used as placeholder glyphs. Bugs like these are fixed.
  3. Virama 0D4D had a wrong LSB that cause the cursor positioning and glyph boundary go wrong. Fixed that bug
  4. Atomic Chilu code points introduced in Unicode 5.1 was missing in all the fonts that SMC maintained because of the controversial decision by Unicode and SMC’s stand against that. Issues still exist, but content with code point is present, to avoid any difficulties to users, added those characters to Meera and Rachana fonts.
  5. Rupee Symbols added to Meera and Rachana. Thanks to Hiran for designing Sans and Serif glyphs for Rupee.
  6. Dot Reph(0D4E) – The glyphs for this was already present in Meera but unmapped to any unicode point. GSUB Lookup tables added to the glyphs according to unicode specification.

For a more detailed change description see this mail thread. There are some minor changes as well.

Thanks to Hussain K H (designer of both Meera and Rachana) , P Suresh, Hiran for their valuable contribution. And thanks to SMC community and font users for using the fonts and reporting bugs. We hope that we can bring this new version in your favorite GNU/Linux distros soon. Wikimedia’s WebFonts extension uses Meera font and the font will be updated there soon. Next release of GNU Freefont is expected to update Malayalam glyphs using Meera and Rachana for freefont-sans and freefont-serif font respectively. We plan to update other fonts we maintain also with these changes in next versions. There are still some glyphs missing in these fonts with respect to the latest unicode version.

 

SVG Fonts

This post is some notes on the current state of SVG Fonts.

SVG is not a webfont format. The purpose of SVG fonts is to be embedded inside of SVG documents  (or linked to them), similar to the way you would embed standard  TrueType or OpenType fonts in a PDF.  SVG fonts are text files that contain the glyph outlines represented  as standard SVG elements and attributes, as if they were single vector  objects in the SVG image. Unlike EOT, WOFF, TTF formats , SVG is plain text uncompressed file.

Even though it is not webfont format, some browsers will accept svg in the @fontface css3 declaration.

Firefox and IE does not support SVG Fonts. Here is the bug on Mozilla bugzilla about this with a lengthy discussion –https://bugzilla.mozilla.org/show_bug.cgi?id=119490  This is one of the reason why Firefox does not score ACID3 test – http://www.itcode.org/why-firefox-is-not-scoring-100-in-the-acid3-test . Developers argue that WOFF is sufficient and SVG Fonts does not give any advantage.  Support for SVG Fonts in the web development and font communities has been declining for some time. There’s already been discussion without objection of dropping SVG fonts from the Acid3 test. The community has put forth a proposal in the SVG Working Group to give SVG Fonts optional status.

Browser Support Matrix for SVG Fonts http://caniuse.com/#feat=svg-fonts  IE and FF do not support it. Webkit based browsers support it – Chrome, Epiphany. For browsers not supporting svg features natively there is a flash based javascript library named svgweb http://code.google.com/p/svgweb/

Limitations of svg fonts:

  • Not all of the opentype features are available in SVG specification
  • For example Indic fonts require many opentype features for correct rendering – see opentype spec of Malayalam http://www.microsoft.com/typography/otfntdev/malayot/shaping.aspx
  • Even though SVG Fonts are support is available in some browsers, practically they cannot render SVG fonts for complex scripts such as Indic – Here is a sample svg file with Meera font defined in it – http://thottingal.in/tests/svg/Meera-fontembedding.svg. As you can see, rendering is wrong.
  • The main drawback to SVG fonts is there is no provision for font-hinting. The SVG standard states: “SVG  fonts contain unhinted font outlines. Because of this, on many  implementations there will be limitations regarding the quality and  legibility of text in small font sizes. For increased quality and  legibility in small font sizes, content creators may want to use an  alternate font technology, such as fonts that ship with operating  systems or an alternate WebFont format. – http://www.w3.org/TR/SVG/fonts.html”

There is an alternate proposal to use opentype features of the font and use svg just for the glyphs https://wiki.mozilla.org/SVGOpenTypeFonts

Fontforge can be used for creating SVG Fonts. But the created SVG font works only for simple scripts like Latin. Fails to export GPOS/GSUB tables to the SVG- bug report – http://sourceforge.net/mailarchive/message.php?msg_id=27964229 GSUB issue can be solved either by handcoding the unicode sequences for glyphs or by writing an external script. But , more important opentype features- Vowel sign(matra) reordering issues persists.

Eventhough svg fonts by defining font data inside svg itself does not seem to have much interest from developers, using webfonts inside for svg has some importance. Just like web pages, webfonts can be used to render the text inside the svg. The webfont format depends on the browser. Example: http://thottingal.in/tests/svg/Dyuthi-Webfont.svg (Have a look at the source code of the file)

Creating a new Language ecosystem- Sourashtra as example

Sourashtra is a language spoken by Sourashtra  people living in South Tamilnadu and Gujarat of India. Originated from Brahmi and then Grandha, this language is mother tongue for half a million people. But most of them are not familiar with the script of this language. Very few people knows reading and writing on Sourashtra script. Sourashtra has a ISO 639-3 language code saz and  Unicode range  U+A880 – U+A8DF

Recently Sourashtra wikipedia project was started in the wikimedia incubator : http://incubator.wikimedia.org/wiki/Wp/saz and Mediawiki localization started in translatewiki Since the language did not had any proper fonts or input tools, this was not going well.

When we add a  new language support in Mediawiki or start a new language wikipedia,  we need to develop the language technology ecosystem to support its growth. This ecosystem comprises of Unicode code points for the script, proper fonts, rendering support,  input tools, availability of these fonts and input tools in operating systems or alternate ways to get it working in operating system etc.

Sourashtra language had a unicode font developed by Prabu M Rengachari, named ‘Sourashtra’ itself. The font had problems with browsers/operating systems. We fixed to make it work properly. The font was not licensed properly. Prabu agreed to release it in GNU GPLV3 license with font exception. He also agreed to rename the font to another name other than the script name itself.

The font was renamed to Pagul, meaning ‘Footstep’ in Sourashtra and hosted in sourceforge

Once we have a font with proper license, we wanted it to be available in operating systems. I filed a packaging request in Debian. Vasudev Kamath of Debian India Team packaged it and now it is available in debian unstable(sid).  Parag Nemade of Fedora India packaged the font for Fedora and will be avialable in upcoming Fedora 15.

To add a new language support in operating system, we need a locale definition. In GNU Linux this is GLibc locale definition. With the help of Prabu, I prepared the saz_IN locale file for glibc, and filed as bug report to add to glibc. I hope, soon it will be part of Glibc.

Well, all of these was possible since it was GNU/Linux or Free software. Things are a bit difficult on the other side, proprietary operating system world. There is nothing we can do with those operating systems. Since there is no ‘market’ for these minority language, it won’t come to the priority of those companies to add support for these languages. Users will see squares or question marks when they visit sourashtra wikipedia.

We are working on a solution for this, not only for sourashtra, but a common solution for all languages. We are developing a webfonts extension for Mediawiki to provide font embedding in wiki pages to avoid the necessity of having fonts installed in user’s computers. The extension is in development and one can preview it in my test wiki. For Sourashtra, we added webfonts support(preview) .

Input tools needs to be developed and packaged. For mediaiwki, with the help of Narayam extension, we can easily add this support.

With the silpa project, I added a server side, PDF/PNG/SVG rendering support for Sourashtra as well.