English Malayalam Translation using OpusMT

 Posted on November 1, 2020  |  Santhosh Thottingal

SMC had started a machine translation service at translate.smc.org.in for English-Malayalam. This system uses huggingface transformers with OpusMT language models for translation. OPUS MT provides pre-trained neural translation models trained on OPUS data. These models can seamlessly run with the OPUS-MT transation servers that can be installed from our OPUS-MT github repository. The translation service is powered by Marian Neural MT engine The quality of the machine translation depends on the availability of parallel corpus. [Read More]
malayalam  machine-translation 

Web application for learning Malayalam writing

 Posted on November 1, 2020  |  Santhosh Thottingal

In my previous blog post, I wrote about an experiment of using SVG path animation to help learn malayalam letter writing. That prototype application was well received by many people and that encouraged me to host it as a proper application. Now, the Malayalam learning application is available at https://learn.smc.org.in Source code: https://gitlab.com/smc/mlmash I added all letters of Malayalam there. Added a few common ligatures too. Kavya helped to record and add pronunication of these letters with couple of examples. [Read More]
malayalam  learning  svg 

Animated SVGs for learning Malayalam writing

 Posted on October 27, 2020  |  Santhosh Thottingal

I wanted to make an educational typeface with writing directions in each glyphs. Something like this: But considering the effort it takes I was bit confused whether it is really necessary to have a typeface or just images like this will suffice. Recently, I read about SVG path animations and I thought animating the path inside each letters will be more helpful than static image with drawing directions. The Chilanka and Manjari typefaces I designed have SVG images with strokes as master designs and in most of the cases, the stroke path directions are the writing directions. [Read More]
malayalam  learning  svg 

A spellchecker webservice supporting 90 languages

 Posted on October 26, 2020  |  Santhosh Thottingal

https://spell.toolforge.org/ is a webservice providing spellcheck web API for 90 languages. I wrote this service hoping that it can be potentially integrated to wikipedia editor to help contributors. The spellchecker backend is hunspell for majority of the languages. It can also proxy similar webservices to provide a single interface. For Malayalam language it uses such an external web API. This Express based nodejs service interfaces to hunspell using nodehun. API GET spellcheck/:language/:word: Check the word in the given language for spelling mistakes. [Read More]
web-api  spellchecker 

Digital garden

 Posted on October 25, 2020  |  Santhosh Thottingal

docs.thottingal.in is my personal knowledge space I try to document and share everything I know about this world in form of an online GitBook hosted on GitHub. What is a digital garden? In basic terms, it is a different format for written content on the web. It’s about moving away from blog posts ordered by dates and categories, into more of an interlinked web of notes. Blog posts are articles at a point in time, while here it is living document. [Read More]
personal  learning  writing 

Foreign word detection in mlmorph

 Posted on October 24, 2020  |  Santhosh Thottingal

The test corpus for Malayalam Morphological analysis has many foreign words. They are either written in a non-Malayalam script or written in Malayalam. For example, “ഇലക്ട്രിസിറ്റി”, “ഡോക്സ്”, “ഇന്റർമീഡിയറ്റ്”, “അബ്സ്ട്രാക്റ്റ്”, “ഇല്ലസ്ടേഷൻ”, “ഇല്ലിറ്ററേറ്റ്”, “റെക്കോർഡ്”, “procrastination”, “唐宸禹” - These are all foreign words and it is useless to analyse them using mlmorph. Since mlmorph works based on a root word lexicon, it is practically impossible to have them in lexicon. So there should be a way to identify the words easily and tag them as FW - Foreign word Part of speech. [Read More]
mlmorph  foreign-word-detection 

What is a good input method?

 Posted on September 21, 2020  |  Santhosh Thottingal

As more and more people enter to the Malayalam digital world, the issue of not having any formal training for Malayalam computing is more observed. People sometimes just search for input methods in web, ask friends or use whatever coming with the devices they have. Since I myself is an author of two input methods, sometimes people ask me too. This essay is about the characteristics of a good input method to help people make the right choice. [Read More]
input-methods  handwriting 

Stuttgart Finite State Transducer(SFST) formalism support for VS Code

 Posted on September 11, 2020  |  Santhosh Thottingal

I just published a VS Code language extension to support syntax highlighting for Stuttgart Finite State Transducer (SFST) formalism to VS Code.

I learned how to write a language extension when I attempted the opentype feature file support. So I thought of applying that learning to SFST which I regularly use for the Malayalam morphology analyser project.

vscode  sfst  mlmorph 

Opentype feature file support for VS Code

 Posted on September 10, 2020  |  Santhosh Thottingal

I just published a VS Code language extension to support OpenType feature files in the Adobe “AFDKO” format. The extension provides syntax highlighting and code snippet support. (Screenshot From Amiri font) The syntax highlighting patterns for AFDKO is based on the opentype-feature-bundle for Atom Editor by Kennet Ormandy which is based upon Brook Elgie’s original Textmate bundle. The code snippets are based on the snippets prepared by Simon Cozens for AFDKO-SublimeText [Read More]
vscode  afdko 

Malayalam Spellchecker version 1.1.1 released

 Posted on August 29, 2020  |  Santhosh Thottingal

A new version of Malayalam spell checker based on mlmorph is available as python library. Install the library $ pip install mlmorph_spellchecker Sample usage >>> from mlmorph_spellchecker import SpellChecker >>> spellchecker = SpellChecker() >>> word = "ഉച്ഛാരണം" >>> spellchecker.spellcheck(word) False >>> spellchecker.candidates(word) ['ഉച്ചാരണം'] >>> spellchecker.spellcheck("ചിത്രകാരൻ") True The new version adds a database of commonly mistaken words of Malayalam for quick checks and correction. If the given word is present in that common list, spellcheck result and correction suggestions will be based on that database. [Read More]