Santhosh Thottingal » Community

New version of Malayalam fonts released

Santhosh — Sat, 10 Mar 2012 13:24:05 +0000

Swathanthra Malayalam Computing project announced the release of new version of Malayalam unicode fonts this week. In this version, there are many improvements for popular Malayalam fonts Rachana and Meera. Dyuthi font has some bug fixes. I am listing the changes below.

Meera font was small compared to other fonts. This was not really a problem in Gnome environment since fontconfig allows you to define a scaling factor to match other font size. But it was an issue in Libreoffice, KDE and mainly in Windows where this kind of scaling feature does not work. Thanks to P Suresh for a rework on glyphs and fixing this issue.
Rachana, Meera and Dyuthi had wrong glyphs used as placeholder glyphs. Bugs like these are fixed.
Virama 0D4D had a wrong LSB that cause the cursor positioning and glyph boundary go wrong. Fixed that bug
Atomic Chilu code points introduced in Unicode 5.1 was missing in all the fonts that SMC maintained because of the controversial decision by Unicode and SMC’s stand against that. Issues still exist, but content with code point is present, to avoid any difficulties to users, added those characters to Meera and Rachana fonts.
Rupee Symbols added to Meera and Rachana. Thanks to Hiran for designing Sans and Serif glyphs for Rupee.
Dot Reph(0D4E) – The glyphs for this was already present in Meera but unmapped to any unicode point. GSUB Lookup tables added to the glyphs according to unicode specification.

For a more detailed change description see this mail thread. There are some minor changes as well.

Thanks to Hussain K H (designer of both Meera and Rachana) , P Suresh, Hiran for their valuable contribution. And thanks to SMC community and font users for using the fonts and reporting bugs. We hope that we can bring this new version in your favorite GNU/Linux distros soon. Wikimedia’s WebFonts extension uses Meera font and the font will be updated there soon. Next release of GNU Freefont is expected to update Malayalam glyphs using Meera and Rachana for freefont-sans and freefont-serif font respectively. We plan to update other fonts we maintain also with these changes in next versions. There are still some glyphs missing in these fonts with respect to the latest unicode version.

Malayalam Wikisource Offline version

Santhosh — Sat, 11 Jun 2011 09:11:38 +0000

Malayalam Wikisource community today released the first offline version of Malayalam wikisource during the 4th annual wiki meetup of Malayalam wikimedians. To the best of our knowledge, this is the first time a wikisource project release its offline version. Malayalam wiki community had released the first version of Malayalam wikipedia one year back.

Releasing the offline version of a wikisource is a challenging project. The technical aspects of the project was designed and implemented by myself. So let me share the details of the project.

As you know a Wikisource contains lot of books, and each book varies in its size, it is divided to chapters or sections. There is no common pattern for books. Each having its own structure. A novel presentation is different from a collection of poems from a Poet. Wikisource also has religious books like Bible, Quran, Bhagavat Geeta, Ramayana etc. Since books are for continuous reading for a long time, the readabilty and how we present the lengthy chapters in screen also matters. Offline wikipedia tools for example, Kiwix does not do any layout modification of the content and present as it is shown in wikipedia/wikisource. The tool we wrote last year for Malayalam wikipedia offline version also present scrollable vertical content in the screen. Both are not configurable to give different presentation styles depending on the nature of the book.

What we wanted is a book reader kind of application interface. Readers should be able to easily navigate to books, chapters. The chapter content will be very lengthy. For a long time reading of this content, a lengthy vertically scrolled text is not a good idea. We also need to take care of the width of the lines. If each line spans 80-90% of the screen, especially for a wide screen monitor, it is a strain for neck and eyes.

Screenshot of Offline version. Click to enlarge

The selection of books for the offline version was done by the active wikimedians at Wiksource. Some of the selected books was proof read by many volunteers within the last 2 weeks.

The tools used for extracting htmls were adhoc and adapted to meet the good presentation of each book. So there is nothing much to reuse here. Extracting the html and then taking the content part alone using pyquery and removing some unwanted sections from html- basically this is what our scripts did. The content is added to predefined HTML templates with proper CSS for the UI. CSS3 multicolumn feature was used for book like interface. Since IE did not implement this standard even in IE9, for that browser the book like interface was not provided. Chrome browser with version less than 12 could not support, because of these bugs: http://code.google.com/p/chromium/issues/detail?id =45840 and http://code.google.com/p/chromium/issues/detail?id =78155. For easy navigation, mouse wheel support and page navigation buttons provided. For solving non-availability of required fonts, webfonts were integrated with a selection box to select favorite font. Reader can also select the font size to make the reading comfortable.

Why static html? The variety of platforms and other versions we need to support, necessity to have webfonts, complex script rendering, effort to develop and customize UI, relatively small size of the data, avoiding any installation of software in users system etc made us to choose static html+ jquery + css as the technology choice. The downside is we could not provide full text search.

Apart from the wikisource, we also included a collection of copyleft of images from wikimedia commons. Thanks to Nishan Naseer, for preparing a gallery application using jquery. We selected 4 categories from Commons which are related to Kerala. We hope everybody will like the pictures and it will give a small introduction to Wikimedia Commons.

Even though the python scripts are not ready to reuse in any projects, if anybody want to have a look at it, please mail me. I am not putting it in public since the script does not make sense outside the context of each book and its existing presentation in Malayalam wikisource.

The CD image is available for download here and one can also browse the CD content here.

Thanks to Shiju Alex for coordinating this project. And thanks to all Malayalam wikisource volunteers for making this happen. We have included poems, folk songs, devotional songs, novel, grammar book, tales, books on Hinduism, Islam-ism, Christianity, Communism, Philosophy. With this release, it becomes the biggest offline digital archive of Malayalam books.

Mediawiki Berlin hackathon

Santhosh — Tue, 17 May 2011 16:16:36 +0000

I am just back from Mediawiki Berlin Hackathon. On May 13 to 15, Mediawiki developers attended the hackathon and squashed many bugs and discussed many features. Members of language committee had its first real-life meeting in parallel with hackathon. It was a nice event, learned a lot, talked to many awesome hackers and linguists.

Milos Rancic has written a summary of the discussions happened during language committee meeting here : http://lists.wikimedia.org/pipermail/foundation-l/2011-May/065537.html
Niklas Laxström and Siebrand reviewed the WebFonts extension and enabled at translatewiki.net. Fixed a few bugs that Niklas reported on the extension.
Purodha Blissenbach was very much interested in the WebFonts and Narayam extensions, we discussed some of the features we need to add. We have it here: Bug 28900 , Bug 28999 and Bug 29000

Indic Language Computing Workout, Pune

Santhosh — Mon, 23 Aug 2010 11:29:15 +0000

On 22nd August, I conducted a workout session with Praveen on Indic Language Computing at Red Hat Office, Pune. The plan was to solve some of the issues in Devanagari support for the encoding converter Payyans. But most of the time was spent on Introducing the concepts of Indic language computing to participants. Project Silpa was also introduced and demonstrated. Students from College of Engg, Pune and other colleges attended. Red Hat sponsored the venue at their office. It was very interesting to interact with energetic and enthusiastic students.

Wikimania 2010, Poland

Santhosh — Sat, 17 Jul 2010 09:25:44 +0000

I left Chennai on Wednesday(8th) and reached Frankfurt airport on Thursday morning. Rest of the people from India for wikimania- Shiju Alex, Tinu Cherian, Srinivas Gunta, Arjun Rao were already reached the airport and I joined them. We reached Gdansk Airport by 12.30 PM. Our accommodation was at a students hostel of Gdansk University. Language was a big issue since most of the people does not understand English and only know Polish Language. The lady at the reception of the hostel we stayed was using Google translate tool for communicating with us. The Gdansk city is a very beautiful city with streets of big brick made tall buildings.

The conference started on Friday morning. Sue Gardner, Executive Director of the Wikimedia Foundation. talked about the strategies of foundation, and it followed by a QnA with wikimedia board members. We presented the Malayalam CD to Sue Gardner. She remembered the International free software conference she attended at Trivandum on 2008 december.
Our workshop on offline wikipedia versions started at 2.30. Martin Walked introduced the workshop to participants. Manuel Schneider from German wikipedia explained the Openzim format for offline compressed storage of wikipedia and the available readers on desktop computers and mobile phones. Shiju Alex introduced the Malayalam wikipedia offline verision 1.0. I talked about the issues and solutions for providing an offline version, particularly a non-latin complex script wiki to users in CD ROM or DVD. I demonstrated sample offline wikis on Hebru, Tamil, Polish, English, Japanese with the wiki2cd tool. There were a couple of questions on wiki2cd and openzim. Kul Takanao Wadhwa and Tomasz Finc from wikimedia foundation who are focusing on offline wiki projects attended the workshop and we had a discussion after the talk.

Offline wikipedia people: myself, Shiju Alex, Martin Walker, Manuel Schneider

The offline wiki workshop continued with Pediapress team. They are the people behind the recently added export book/PDF feature of wikipedia. Unfortunately this feature require lots of improvements to get work with Indian scripts.
We met Gerald M, who focus on language support of wikis. He is such a person with amazing passion towards our local language wikipedias. We discussed on the technical issues of the local language wikis. Siebrand joined the discussion and he pointed out some improvements in wiki2cd software.

Discussion with Siebrand on wiki2cd improvements. From left 2 right: Tinu Cherian, myself, Gerard M, Siebrand

On the second day I met Volker Haas, the developer of PDF export/books feature of wikipedia. The library used by the extension of creating PDFs is Reportlab. But it does not support complex scripts such as Indic or Arabic. We have a long discussion on possible solutions. Discussed the Reportlab code. the mwlib code, and the s/w which I am writing now a days to provide complex script pdf rendering APIs. We will continue to try out some of the options we have to solve this issue soon.

Martin Walker, who presented the Article Selection process of English wikipedia along with us in the workshop invited me and Shiju for a dinner with his family. And we went for that.

The third day started with plenary session by Jimmy Wales. He talked about small language wikipedia and the issues faced by them. He emphasized the need for offline versions of wikipedia to reach more number of people and talked about the Malayalam Wikipedia offline version.

Jimmy Wales with Malayalam wikipedia CD

Mayooranathan from Tamil wikipedia presented the issues and statistics of Tamil Wikipedia Community. On Monday and Tuesday, we spent time by roaming around the Old Town of Gdansk. Visited St. Marys Church , the biggest brick made church in the world. We climbed the 400 steps of the tower of the church. From the top of the chruch, one can see the entire city. We went to the Baltic sea beach -Westerplatte on a boat and visited Wisłoujście Fortress

* Creating Malayalam Wikipedia CD: http://shijualex.wordpress.com/2010/04/24/creating-malayalam-wikipedia-cd/
* Wiki2cd: http://thottingal.in/blog/2010/04/17/mlwikioncd/
* Wikipedia Sign post: http://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2010-04-19/News_and_notes#Briefly
* Gerard’s Blog: http://ultimategerardm.blogspot.com/2010/04/best-of-malayalam-wikipedia.html
* Gerard’s Blog: http://ultimategerardm.blogspot.com/2010/04/cd-dowloaded-4390-in-10-days.html
* Gerard’s Blog: http://ultimategerardm.blogspot.com/2010/07/malayalamwikipedia-success-story.html

Attending Wikimania 2010

Santhosh — Tue, 06 Jul 2010 09:32:45 +0000

I will be attending Wikimania 2010, Gdansk, Poland. This annual international conference of the Wikimedia community is from July 9 to July 11.

I will be presenting wik2cd, the tool I wrote for Malayalam wikipedia version 1.0 there in a joint workshop with wikipedia offline developers. I will be joining with Manuel Schneider, Shiju Alex, Martin Walker in the workshop titled: Creating offline version of Wiki content – Solutions and Challenges. Apart from this, I will be meeting pediapress team, the team behind wikipedia’s latest PDF/Book export feature. There are some issues in this tool for working with Indic languages, mainly because of the PDF rendering engine not capable of rendering complex scripts.

Thanks to Wikimedia foundation for granting me a scholarship to cover travel expenses.

Conferences : FOSS.IN and NCIDEEE

Santhosh — Sat, 28 Nov 2009 05:14:16 +0000

FOSS.IN 2009 starts on 1st December. I wanted to attend all 5 days but I have another conference on Dec 1st to 3rd at Chennai. I am attending National Conference on ICTs for the differently- abled/under privileged communities in Education, Employment and Entrepreneurship 2009 – (NCIDEEE 2009) at Loyola College, Chennai. So I will miss the first 3 days of foss.in.
We have a workout on Project Silpa during foss.in. I am also planning to have a workout with Debayan and Jinesh to get his tesseract-indic OCR work with Malayalam.

See you at foss.in!

Openoffice Indic Regional Language group

Santhosh — Wed, 27 May 2009 03:48:00 +0000

We just formed Indic Regional Language group for Openoffice. This is as per the Openoffice Native Language Consortium Plans. The objectives of such groups can be read from here. Basically the group is meant for better coordination among Indic languages to make Openoffice experience in our language better.
The announcement of this group is here

Thanks to Charles-H. Schulz, we got a mailing list indic@native-lang.openoffice.org. To subscribe login to http://native-lang.openoffice.org

We just started, and I will soon setup a wiki page there. To start with , I will collect the list of issues pending for Indian languages from people from various languages and will find out people from various languages as point of contacts. Feel free to contact me for anything related to Openoffice in your language.

Update: June 3, 2009: This is our wiki page

Localization: What are we missing?

Santhosh — Thu, 26 Mar 2009 05:19:00 +0000

[This blog post is kind of self criticism and written not forgetting the valuable contribution that l10n communities are doing. ]
Some observations on the Localized desktops in Indian Languages
* Not all localization team members try the application that he/she translate at least once before working on the PO file. Result: If somebody does the localization without understanding what the application does and try the en_US interface, he/she miss the context of the strings. An example I have seen : the string “Querying” was translated to xx_IN language string which means “Questioning” instead of the required string corresponding to “Searching”. Sometimes we miss to understand how much space the string is going to take in the screen and we translate a small English word to a long xx_IN string to make the meaning clear. Result: Ugly interface.

Tamil gedit from Ubuntu 8.10(Click to enlarge)
* Not all localization team members *try* the application that he/she translated after completing the PO file or even after the application is released. This happens when he/she translates many applications(sometimes if it is part his/her job).
* Practically, there is no process called *testing* localized desktop in our SDLC. L10N members translates a PO file and sometimes he/she translates it as text file rather than a user interface. It is must that we should bring some process to make sure that the localized desktop is tested for usability, contextually correct translation, spelling mistakes, wrong short cut keys, fuzzy strings , non translated strings in main interface etc etc.
* Since the ratio between the total number of applications in a desktop environment and number of team members is very less, we end up in translating one application by many people. Result: inconsistent translation and no ownership for ensuring the translation quality. Ramadoss from Tamil team was suggesting that ideally , for each application there should be a person from each language , who is responsible for timely translation, testing. He can take more than one application responsibility but not more than say, 10. Practically, this requires a big l10n community per language and unfortunately we don’t have it as of now.
* Peer review, one of the important and mandatory process in l10n is not happening properly when the release date is approaching. L10N communities often try to meet the percentage of completion somehow. IMHO, the new l10n tools frameworks often miss to give importance for peer review in the workflow they design. FOSS community, being inclusive in nature often welcomes new l10n contributors. I have seen many members improving their l10n skills after making the corrections as per the review comments from others. When a new l10n workflow allows every contributor to submit their translated PO file without the peer review from community, the ultimate result is very bad user interface. We have seen this many times with Rosetta translations of Ubuntu. Everybody going there tries out the Rosetta “features” and leave few strings “translated” there. And Ubuntu takes those strings for their immediate release. Upstream translations are never taken on time or the “translated” strings are never submitted to upstream. Result: Very bad localized desktop with many spelling mistakes, inconstant translations etc.. We ml_IN team used to watch who is “contributing” through Rosetta and make him work with the community. I hope the new translation frameworks will give sufficient attention to this problem. If we are not keeping a balance between newbie translation and quality assuarance , our localized desktops will not improve.

Again Tamil gedit, but from Debian Lenny. Compare it with the Ubuntu version shown above(Click to enlarge)

* User feedback: The number of users who use the desktop in their own mother tongue, even though the % of translation is more than 80% for many languages, is very less. IMHO, It is because of a ‘dependency conflict’ of the following things
a) A person who is not good in English
b) A person want to use computer in mother language for some “purpose”
c) A person who is capable of spending Rs ~20K for a computer
Most of the cases, there is a conflict between any 2 of the following and that ends up in a) Person use his desktop in english b) Person not using computer at all. I am sure that if there are good number of users, we will not end up in interfaces I showed in the screen shots.
* One inconsistency I noticed across localized desktops is regarding the shortcut keys/accelerator keys. Some languages use English short cut keys and give at the end of the word in Brackets for eg: അടയ്ക്കുക (C). As you can see in screen shots sometimes we have small letter and sometimes capital letter for that. Some languages use letters in xx_IN itself. But there is no consistency. For Control and Alt keys, some language translate them, some others keep it in English. What is the problem with English short cut key? For using English short cut key , the user should be using English layout keyboard. For shortcut keys in xx_IN, one should be using xx_IN keyboard layout. For a user(assume that he use xx_IN desktop since he is not good in English) typing in xx_IN in gedit using xx_IN keyboard, is it possible to use the short cut keys if we give in English? Are we expecting that for using short cut key while typing the document, he change switch his keyboard layout ? (btw, anybody noticed that Apple doesn’t use Accelerator keys in its OS?)

bn_IN gedit in Ubuntu 8.10(Click to enlarge)

ml_IN gnome dictionary client in Ubuntu 8.10(Click to enlarge)

Suggestion/Ideas are welcome… How can we make our localized desktop more beautiful and user friendly?

Updates…

Santhosh — Sun, 11 Jan 2009 18:09:00 +0000

Praveen prepared videos from the matrix screen savers in 6 languages
This video is translated to Malayalam. For those who are interested in how to do that refer this
I prepared the glibc collation table for Malayalam . But still some more bugs to be fixed
We friends are working on adding Saka year system to KDE calendar system and it is almost ready . And here is the video : Saka calendar in KDE
Dict based english-malayalam dictionary is in developement and we are ready for a beta release. Rajeesh did a woderful job in preparing it
KDE Malayalam team is doing great job
Worked on hunspell to handle the agglutinative nature of Indian languages and found some problems with hunspell. Hunspell developer Nemth is looking into that
Sujith added Malayalam support to KLetters
Planning to attend fossmeet @ Model Engineering College, Cochin