What is the frequency of characters in a typical Dhivehi writing? What is the most commonly used Thaana akuru/fili in Dhivehi? Is there a general pattern of akuru and fili to be expected in any given Dhivehi document?

These questions, and especially the latter, kindled my curiosity yesterday and had me off to explore a little bit. Although seemingly trivial and of no practical use, these are serious questions that probe into the finer details of Dhivehi and help produce computational models of Dhivehi - which have practical applications. Even the generalizations and patterns that result from the simplest statistical analysis transcend the (quirks of) individual writing and give a broader picture of what a language is really like. For example, I'm employing a statistical fingerprint of Dhivehi that was generated during this little exercise as part of an experimental procedure that identifies (the presence of Dhivehi) content in web pages. It takes advantage of the fact that the fingerprint for Dhivehi and that for English are dramatically different thus allowing a computer program to discern the type of content it is dealing with - all without really "understanding" a language.

I conducted the analysis on a dataset consisting of ~5000 Dhivehi articles from Haveeru Daily and ~7000 Dhivehi articles from Jazeera Daily. They may not represent the whole varieties of Dhivehi literature available but I think they are a very good approximation - especially of Dhivehi web content which is what I was mostly interested in. My focus was on the individual character level and ran basic mean, mode, variance, standard deviation and frequency calculations with a further character correlation analysis. Despite these being quite simple analyses, I don't think anyone's ever explored as much before and hence the following should make for (exciting!) new information.

Enjoy

The following goes out to the elderly and all those who aren't versed in English, to all those who are disadvantaged by the sorry state of science in Dhivehi and to all those who were misinformed and deceived. Hope this helps.

(Download the article as a PDF formatted for print)

I woke up this morning to find in my email inbox a scan of a document titled "Miureekhah Hulhagun Iru Araifi" (translated: "The Sun rose from the West in Mars"), authored by someone operating under the name of "Abu Osama" and bearing the stamp of approval by the Islaamee Kathakaa Behey Mathee Majlis (pages 1 2 3). I assumed that it was just a chain-mail doing the rounds and went off to run a few errands only to return in the evening to find yet another email in my inbox - this time from a close buddy who had spotted the (same!) article appearing in today's issue of Haveeru Daily (view clipping). It was then that I realized that the document I read earlier was a serious piece of literature produced by someone, or some religious entity, as part of organised religious evangelism.

The document was shock full of factual errors and scientific errors that I just had to take it seriously - especially since these very "errors" acted as the basis for their evangelical argument. Could they have been so scientifically illiterate?! This whole story of the Sun rising from the West and the supposed evidence for it by events on Mars is just ludicrous. I'd laugh my belly off if I weren't so worried about the sheer lack of scientific understanding and the lack of skepticism. Anyway, I'd like to go over the document in question and set the science straight...

Claims in the document
The basic claim in the "Miureekhah Hulhagun Iru Araifi" article is that [1] the planet Mars had slowed down in its trek around the Sun, [2] came to a complete stop on 30th July 2005 and then [3] resumed its trek around the sun - [4] in the opposite direction - [5] in August/September 2005. The document attributes these observations to "Retrograde motion". The document also claims that [6] this encounter meant that the Sun had started to rise from the West on Mars instead of the usual East. It further claims that [7] the same would happen to Earth someday. They then interject a Hadith that claims the Sun rising from the West as being a sign that the end is near and that once it happens no one could repent or achieve salvation. The document also claims, on the authority of religious scholars, that [8] the Sun would rise in the West on Earth once only - for one day only - and things would continue as normal thereafter.

The document then uses these claims as a premise to attempt validate, justify and promote a number of Hadith regarding the end of the world and judgement day to call upon people to stay steadfast to Islam, to repent and to call upon non-believers to the faith.

The document then gives an invalid URL to an article on space.com in an attempt to create a smokesreen to make it look more credible.

Analysis and Explanations
It is important to note that the article presents an incorrect definition of what a "Retrograde" really is. A planetary body is in retrograde motion when it moves in a direction dissimilar, i.e. opposite, to another planetary body within a system. The problem at hand deals with a specific type of retrograde motion that occurs when the motion of planets is viewed relative to the Earth - known as an Apparent Retrograde - and is applicable to all planets that lie beyond Earth in the Solar System: Mars, Jupiter, Saturn, Uranus and Neptune.

Perhaps the best way to understand and help imagine the interactions betweens the planets involved in retrogrades is to watch the simulation available here. Run the simulation on that page and watch carefully how the movement of Earth and Mars in their own elliptical paths results in changes in what we see in the sky - with such amazing things as Mars apparently slowing down, stopping entirely, changing direction and then resuming as normal. They say a picture is worth a thousand words but I tell you, this simulation is invaluable. It is important to recognize that these "Mars retrograde" events are cyclical events that occur once every 26 months and has been taking place since before the earliest humans.

- The article's false claim [1] thus acquires a whole new meaning - it is the apparent behaviour of Mars that we see that is actually slowing down and not Mars itself.
- The dates given in claims [2] and [5] are false - in the 2003 Mars retrograde, Mars reached the first stationary point on 30th July, went through the loop back and reversed direction after reaching the stationary point on 29 September. In the 2005 Mars retrograde, Mars reached the first stationary point around 1 October and went through the loopback till it reached the second stationary point on 10 December and reversed direction again. A nice sketch of the movement of Mars, including the key dates, for both these years is available here.
- As said before, even during a retrograde, despite how we perceive it, Mars never actually stops or moves in any special way other than its usual motion along the elliptic orbit around the sun - thus claims [2], [3] and [4] are false if taken in the literal sense.
- Since Mars never budges from its usual orbit path nor its usual rotation direction, unlike as claimed in [6], there never is even a moment of time that the Sun rises from the West in Mars. The Martian sunrise and sunsets continue as they have been doing for the billions of years it has been in existence.
- The article quite correctly quotes scientists as saying that the retrograde happens for all planets. But the quote is placed out of context amongst the article's drive toward making it seem that Mars switched directions and had the Sun rise from its West. Truth of the matter is that scientists are actually referring to the planets other than Mars that are also affected by Apparent Retrogrades - ie Jupiter, Saturn, Uranus and Neptune - and does not include Earth in it. However, since Apparent Retrogrades depend on the position of the viewer, Earth would also be subject to the sort of retrograde illusions as Mars if we were to observe things from Venus or Mercury and there'd come a time every year when Earth retrogrades and the entire Earth would seem to wobble a bit as it scurries backward and then forward again. But when talking with respect to Earth, claims [7] and [8] just fall flat.

It is worth noting that one of the main tasks of the article seems to be to suggest that this is a one off event that demands our immediate attention. However, this seems absurd a suggestion given that Mars retrogrades are cyclical events that have even been documented carefully by ancient Greeks and the Mayans and that a good part of humanity has known, predicted and observed it for centuries. What more, we are currently right in the middle of such a retrograde - Mars reached its first stationary point on 15 November 2007 and has been trekking through the loopback all of December and will continue to do so for the entire January till it reaches its second stationary point on 30 January 2008.

Scientifically, there's absolutely no reason to think that Earth will change its direction of rotation or direction of orbit around the sun - hence the direction of the Sun rise - EVER! The angular momentum of earth's current spin would keep the Earth turning the way it does now until opposing forces slow it down to a complete standstill - which happens to be millions of years away. The only scenario where the Earth MIGHT change its direction of rotation is if a there is a collision with a very (very!!) large foreign object such as a very large meteor and that too only under particular conditions that depend heavily on the location of impact and speed of the object - but this act would almost certainly wipe out every single living thing on Earth and so there'd be no point to this issue of Sun rising from West. Another occasion that might allow for the Sun to rise from West - if all you have is a compass - is if the Earth flips the polarity of its magnetic field. The Earth could and IS presently slowly changing its magnetic polarization - so if/when a polarity flip occurs, what is currently shown as "West" on a magnetic compass would then register as "East". But the cardinal directions are NOT defined by the Earth's magnetic field and the Sun would still continue to rise from the same direction it has always risen - the only thing that would have then changed would probably be the compasses to make up for the switch in pole polarity.

After thoughts
I hope the above deconstructs the myth/hoax published in Haveeru and popularised in Male'. It is sad to see that people, especially with the blessings of the Islaamee Kanthakaa Behey Mathee Majlis, engage in fear-mongering and scare tactics to psychologically intimidate people into religion. Surely there are more honest means than this?

Mars has been the subject of many other hoaxes recently and it seems to be able to consistently dupe a lot of people. It is easy to be duped and had for unless one practices healthy skepticism and read up. Meanwhile, go out and have a look at Mars shining bright in the sky - that's what I was doing with a telescope during the 2003 Mars retrograde when the red planet was supposed to be the closest to Earth it will be for years and years to come...

PS: By all means, feel free to send this article to your friends who've been taken in by the hoax.



UPDATE (25/01/2007): It seems the exact same hoax article was also disseminated by the Maldivian Islamic political party Adaalath Party on on their website. Sigh.

I was watching a presentation tonight titled "African fractals, in buildings and braids" (on TEDTalks) and was totally absorbed in it when an African game board shown in it caught my eye - the board looked eerily similar to something I knew: an Ovvalhugondi. I had always been under the impression that "Ovvalhu" was a distinctly Maldivian game but just like many other supposed Maldivian games of the likes of "Koraa" and "Baibalaa", I wondered if Ovvalhu too was just another foreign game that had been absorbed into our culture. Anyway, I was compelled to look up more about the mentioned African origin game called "Mancala".

Finding out more about Mancala was a much easier task than I thought. Rather than it being an obscure game played in a lone part of Africa that had little mention in any literature, Mancala was literally something of a global phenomenon that had many a mention of it, played all over the world and had dozens of online shops selling the boards. There even were online versions of the game! Mancala or Manqala in Arabic, basically refers to a class of games that all have similar game play - the objective always being to capture more "stones" than the opponent. There are a number of variants (see Wikipedia's list), as adopted by different countries or areas, that differ in the finer details of how its played. Apparent differences obviously include the number of pits in the board and the number of "stones" used in play.

I might be wrong (very wrong, infact) but from what I read I suspect that Ovvalhu takes after the version played in South India or possibly the version played in Ghana. Ovvalhu may not be our national game but Ghana's Mancala variant called "Oware" is supposed to be their national game. I found it amusing that the names sounded similar but that might just be mere coincidence(?!). A similarity that certainly is not a coincidence is that Maldivians also used to play Ovvalhugondi with "Laagulha" (picture), ie. the seed of "Kashikumburu", which is what Oware is supposedly played with (in the Caribbean atleast).

What was even more interesting, to me, was to learn that Mancala (or atleast some variants of it) had been analysed using combinatorial game theory. The game of Awari was tackled by two Dutch scientists who generated the entire state-space for the game - mounting upto almost 9 billion positions - and cracked the perfect play for the game (here's their paper). Perfect play is a game theory term for a strategy or set of moves that guarantees a certain outcome in a game - a win or a draw at the least - if the game allows so, mathematically. I have no interest in Ovvalhu but I find such computational challenges almost erotic. I'm very much tempted to attempt analyzing Ovvalhu for a perfect play as well, so I've added it to my list of future boredom-killer projects.

Anyway, though it is pretty conclusive that Ovvalhu is not a Dhivehi game, I think it is interesting to learn that it certainly is one with an exciting history and background!

As much as I would like the Dhivehi language to die and rot away, it seems it won't happen, atleast for a while. The (relatively) newly minted freedom to publish newspapers and the growth of web-based news sites may have poised Dhivehi for a serious revival of the language. The revival probably isn't so much in terms of improvements in the vocabulary or other more linguistics related changes but rather a revival in terms of the amount of information now being pumped out in Dhivehi - and in my opinion, that's a great start.

A (if not THE) point worth noting here is that much of this new information is being produced - and published - by digital means. Most government authorities now have web portals and an increasing number of them maintain them diligently. Most, if not all, newspapers and magazines also seem to maintain web portals with their content being made available online on the web. This modern revival thus presents a very interesting and a very much modern set of problems (to geeks like me atleast ) :- accessing it. It is probably the first time in Maldivian history that a "dhivehi search engine" makes practical sense.

Now, I am aware that Google and other search engines can be used to search for Dhivehi and I'm also aware that there are a few local operations that purport/aspire to be Maldivian search engines but they all share important shortcomings. These shortcomings are mostly inherent to the various methods of writing Thaana as used on the World Wide Web.

Say you want to search for the word "rayyithunge". Typing that into a search engine would bring an entirely different set of results from typing in "rwacyituncge" or "ރައްޔިތުންގެ" - both of which are alternative forms of representing the same thing in Dhivehi. The different set of results arise because of the differences in the representation schemes used on the different sites. A search with the phrase "rayyithunge" would bring in results with pages that seem to mostly contain English and that's because "rayyithunge" is Dhivehi "Latin"ised into English so that we could use standard English characters to write Dhivehi words. People commonly use such Latinised Dhivehi when writing emails or chatting - say "haalu kihineththa" etc. Meanwhile, a search with the phrase "rwacyituncge" results in a listing of content from sites like Haveeru and Miadhu who use standard ASCII coupled with custom Dhivehi fonts with the characters mapped. If you try copy-pasting something written on the Haveeru page you'd see that it comes out as a seemingly meaningless jumble of letters. Lastly, a search with the phrase "ރައްޔިތުންގެ would bring in results from sites like Minivan Daily and Sangu Daily who use Unicode to display Dhivehi. Anyway, the technical explanations aside, the point is that Dhivehi search is (currently) a messy enterprise.

The solution to this problem can (seem to) be pretty simple. A custom search interface could be made to simply take the search query from a user and convert it into the three different representation schemes and then spawn search a search for each representation phrase on any of the existing search engines. This would work just fine... until you run into peculiar problems related to Latinised and ASCII Dhivehi schemes. Take for example the word "ފަލަ" Latinised into "fala" - a search on the word would result in almost entirely non-Dhivehi results totally unrelated to what we really want. Similarly, a search on the ASCII'ed phrase "Oled" (which is the word "ދެލޯ") would result in a large number of non-Dhivehi results with no bearing on what we wanted. These problems occur because Latinised and ASCII Dhivehi representations can result in text that have meaning in English as well - such as the case of "Oled" as above which happens to be a popular technical term in English.

A more sophisticated approach to the search problem probably could successfully iron out (most of) these quirks. An ideal solution would be to do away with the existing search engines such as Google, despite their awesomeness, and develop a custom search engine. A custom engine would allow for the recognition of the various representation schemes used and the subtle differences between them. A search phrase entered on such an engine would perhaps standardize the phrase and search through a standardized index to return results that are a better mirror of the Dhivehi content that is out there. Such a custom search engine could bundle in extra Dhivehi-related facilities such as conversions to allow for lack of (particular) fonts as used on sites and spelling correction among others.

So, perhaps the question now is, is there a real need for a Dhivehi search engine yet? When should a Maldivian "Google" be born?