#!/usr/bin/env python
# -*- coding: utf-8 -*-
ml_string=u"സന്തോഷ്  हिन्दी"
for ch in ml_string:
    if(ch.isalpha()):
        print ch

gives this output

സ
ന
ത
ഷ
ह
न
द

And fails for all mathra signs of Indian languages. This is a known bug in glibc.
Does anybody know whether python internally use glibc functions for this basic string operations or use separate character database llke QT does?

The bottom half of the image is the source code. We can clearly see that the closing bold tag is placed in between the word instead of putting at the end of the word. As a result, the word is rendered wrong in the page.
This happens for all languages which use ZWJ, ZWNJ, ZWS etc. It breaks the word just before the zwnj/zwj and puts the end of bold tag to highlight the search result..

I showed this to Gopal and told me that he filed a bug on that.

Loop through the chars of the word, until the current char is not a letter/ anymore.

And for this , it use the QChar::.isLetter() function. This functions fails for Matra signs of our languages.

A screenshot from a text area in Konqueror:

For example

#include <QtCore/QString>
#include <stdlib.h>
int main(){
	QChar letter ;
	letter = 'அ';
	fprintf(stdout,"%d\n", letter.isLetter());
	letter = 'ी';
	fprintf(stdout,"%d\n", letter.isLetter());
}

In this program, you will get true as output for அ and false for ी.

When I showed this to Sayamindu during foss.in , he showed me a bug in glibc . Eventhough the bug is about Bengali, it is applicable for all languages. It is assigned to Pravin Satpute and he told me that he got a solution and will be submitting soon to glibc.

But I am wondering why this bug in KDE unnoticed so far? Nobody used spellcheck for Indian languages in KDE?!

Let me explain why this is not happening in GNOME spellchecker if this is a glibc bug. In gnome, this word splitting will be done in application itself using gtk_text_iter_* and these iteration through words are done by pango words boundary detection algorithms.

Filed a bug in KDE to track it.

1. Bug 434044 – The tokenization of words for spellcheck is wrong when there is a ZWJ/ZWNJ/ZWS in the word. – Reported: 2008-05-16 07:49 PDT by Santhosh Thottingal
2. Bug 318040 – Spell checker flags words containing full stops (periods) Reported: 2005-11-28 12:45 PDT by Joseph Wright

Ok, 9.7 GB. so who is writing to messages?
$tail -n 100 messages
This gave me some hint. Some sample lines from messages file:
May 27 20:39:23 thottingal gdm-simple-slave[2523]: DEBUG: GdmSignalHandler: Adding handler 5: signum=8 0x804c520
May 27 20:39:23 thottingal gdm-simple-slave[2523]: DEBUG: GdmSignalHandler: Registering for 8 signals
May 27 20:39:23 thottingal gdm-simple-slave[2523]: DEBUG: GdmSignalHandler: Adding handler 6: signum=1 0x804c520
May 27 20:39:23 thottingal gdm-simple-slave[2523]: DEBUG: GdmSignalHandler: Registering for 1 signals

GDM was writing all debug messages to the /var/log/messages. can somebody help me to figure out what is wrong with my GDM?
(the [debug] section of /etc/gdm/custom.conf is empty)

I found this when I was trying to prepare a Malayalam spellcheck extension for firefox(Hunspell wordlist). Still many languages do not have the affix rules in place for aspell/hunspell and it makes the spellcheck less efficient particularly for highly inflected/agglutinated languages like Malayalam.

Thanks to Németh László, Hunspell developer for helping me to figure out the problem