Blogs

In the previous blog post I explained my efforts to modernize SFST. Since I published SFST python binding and modernized to make it compile in all operating systems, next step was to drop HFST dependency of mlmorph and use the new version of SFST 1.5.0. mlmorph 1.3.0 has no dependency on HFST and all installation problems in different operating systems and python versions are solved now. Latest version is also available in pypi.org. It can be installed using pip install mlmorph. You can try it in the collab notebook.

The Malayalam morphology analyser(mlmorph) I am actively developing since 2016 is based on Stuttgart Finite State Transducer(SFST) formalism. SFST was developed by Dr Helmut Schmid. Since SFST had minimal developer apis and no functional python APIs, I was using Helsinki Finite State Transducer(HFST) toolkit. HFST has python binding and better tooling and development support. HFST has SFST backend so there was no issue in using SFST formalism. But HFST development has slowed down in past years. In 2017, I had to switch back to SFST for compiling my automata because HFST’s compile support for SFST was broken. That complicated my build system, but still I was able to use the python binding of HFST. But the development and release cycles of HFST further slowed down and python bindings were not released for python 3.8 or later versions. So mlmorph was stuck with python 3.8 and this was a big pain point since developers trying to use mlmorph found it not working in their python versions. It became more serious when Ubuntu and such popular distros moved to later python versions as default ones. I tried to contact the upstream developers. They were trying to help, but there were no active developers. Helping HFST to release python bindings was not easy because of very complicated build system they have. ...

A new version of Manjari Malayalam typeface is available now. Version 2.000 comes with a few bugfixes and glyph additions. Most of the changes are based on a review by Google fonts team. New version is available at SMC website for preview and download. Changes Design correction for Malayalam number 10 to better match the attestations (0d6b0fc2) Avoid glyphs that use components that are components themself (6fe9b35d) Add ligatures glyphs for ൻ + ് + റ version of nta (2c39ae08) Review and update diacritics alignment (c498245d) Rename License.txt to OFL.txt (a483e86c and 40411ef0) Revert “Use consistant hhea ascender and descender” and disable vertical metrics check in fontbakery (832e54e3 and 05a899d1) GDEF: Change the Mark glyphs to non-spacing and add more mark glyphs (550d0515) Improve fractions glyphs for consistant stroke widths (b6e70bb5) Fix points out of bounds issues (285417c6) Add ക്ട്ര ligature glyph (186fdb4b) Add more tests for kerning and latin (9d900e58) CI: Exclude name check in fontbakery (c28d095f) Multiple build toolchain updates The latest version is not yet available at Google fonts. But I hope it will be updated soon. ...

I prepared a web frontend for Tesseract OCR to do optical character recognition for Malayalam - https://ocr.smc.org.in This application uses Tesseract.js, Javascript port of Tesseract. You can use images with English or Malayalam content. Use the editor and the spellchecker for proofreading the text recognized. Your image does not leave your browser since the recognition is done in browser and does not use any remote servers. ...

In Malayalam, the Ya, Va and Ra consonant signs when appeared together has an interesting problem. The Ra sign(്ര also known as reph) is prebase sign, meaning, it goes to left side of the consonant or conjunct to which it applies. The Ya sign(്യ) and Va sign(്വ) are post base, meaning it goes to the right side of consonant or conjunct to which it applies. So, after a consonant or conjunct, if Ra sign and Ya sign is present, Ra sign goes to left and Ya sign remain to the right. This is same, irrespective of the order of ya and ra signs. ...

SMC had started a machine translation service at translate.smc.org.in for English-Malayalam. This system uses huggingface transformers with OpusMT language models for translation. OPUS MT provides pre-trained neural translation models trained on OPUS data. These models can seamlessly run with the OPUS-MT transation servers that can be installed from our OPUS-MT github repository. The translation service is powered by Marian Neural MT engine The quality of the machine translation depends on the availability of parallel corpus. Data contributions by actual users are more than welcome. If you are interested in contributing translated documents, please use Opus repository to upload translated documents in various formats. This could also be translated webpages. Anyone can create an account and start uploading data. You may also help us uploading additional training data at https://translate.ling.helsinki.fi/

In my previous blog post, I wrote about an experiment of using SVG path animation to help learn malayalam letter writing. That prototype application was well received by many people and that encouraged me to host it as a proper application. Now, the Malayalam learning application is available at https://learn.smc.org.in Source code: https://gitlab.com/smc/mlmash I added all letters of Malayalam there. Added a few common ligatures too. Kavya helped to record and add pronunication of these letters with couple of examples. ...

I wanted to make an educational typeface with writing directions in each glyphs. Something like this: But considering the effort it takes I was bit confused whether it is really necessary to have a typeface or just images like this will suffice. Recently, I read about SVG path animations and I thought animating the path inside each letters will be more helpful than static image with drawing directions. The Chilanka and Manjari typefaces I designed have SVG images with strokes as master designs and in most of the cases, the stroke path directions are the writing directions. That makes my job easy as I can just reuse all those SVGs and apply path animations. ...

https://spell.toolforge.org/ is a webservice providing spellcheck web API for 90 languages. I wrote this service hoping that it can be potentially integrated to wikipedia editor to help contributors. The spellchecker backend is hunspell for majority of the languages. It can also proxy similar webservices to provide a single interface. For Malayalam language it uses such an external web API. This Express based nodejs service interfaces to hunspell using nodehun. API GET spellcheck/:language/:word: Check the word in the given language for spelling mistakes. Example POST spellcheck/:language: Check the words in the text passed for spelling mistakes. The posted body should be json with “text” key containing text. GET list/languages: List available languages. Example Example As an example, I created a TinyMCE Editor with spellchecker capability using this API ...

docs.thottingal.in is my personal knowledge space I try to document and share everything I know about this world in form of an online GitBook hosted on GitHub. What is a digital garden? In basic terms, it is a different format for written content on the web. It’s about moving away from blog posts ordered by dates and categories, into more of an interlinked web of notes. Blog posts are articles at a point in time, while here it is living document. Digital gardens are also known as Zettelkasten or Second Brain. ...