Anyone developing custom code for a Thaana application or website will undoubtedly run into many interesting challenges and issues. Judging by how often I get emails asking for help with getting MySQL to play nice when using Unicode Thaana, I'd say it's one of the very first issues many (newbie?) web developers face.

Here are some of the basics that one should keep in mind when attempting to use Unicode Thaana and MySQL.

Charset
It is very important that the correct character set be used when using Unicode Thaana for MySQL to correctly recognize the text. If you are going nuts over your code returning "???" or junk characters after trying to store Thaana text into a MySQL database, then it most certainly is because of an incorrect charset being used.

I recommend issuing the following SQL commands upon successfully establishing a connection with the MySQL server.

SET NAMES utf8;
SET CHARACTER SET utf8;

Read up the MySQL reference page on connection charsets and collations for more details.

Collation
MySQL does not have special language specific collation rules for Thaana - and there is no need for such far as I am aware. I recommend using the "utf8_unicode_ci" collation when creating tables and fields in MySQL.

Refer to the MySQL reference page Unicode character sets for more details.

Happy databasing

Here is a minor update to my previously released Thaana Conversions class for PHP. This new version adds the function convertUtf8ToEntities() which I had forgotten to include in the previous public release.

The Thaana Conversions class for PHP provides a number of useful functions for the conversion and transliteration of text between various Thaana representation formats.

Functions listing:
- convertUtf8ToUnicodeIntegers()
- convertUtf8ToAscii()
- convertUtf8ToEntities()
- convertEntitiesToUnicodeIntegers()
- convertEntitiesToUtf8
- convertEntitiesToAscii()
- convertUnicodeIntegersToUtf8()
- convertUnicodeIntegersToEntities()
- convertUnicodeIntegersToAscii()
- convertAsciiToUtf8()
- convertAsciiToUnicodeEntities()
- convertAsciiToUnicodeIntegers()

Requires:
PHP 5

License:
Open Source MIT License

Usage:

<?php
// Load the class
require 'thaana_conversions.obj.php';

// Initialize the Thaana object
$thaana = new Thaana_Conversions();

// Example: Converting Thaana expressed as HTML entities to ASCII
echo $thaana->convertEntitiesToAscii('&#1931;&#1960;&#1928;&#1964;&#1920;&#1960;');

// Example: Converting ASCII Thaana to UTF-8
echo $thaana->convertAsciiToUtf8('rWacje');
?>

Download:
- Thaana_Conversions.zip (v0.3, 4KB)

Drop me a line if you run into trouble with any of the functionality or have comments/queries. Enjoy

Update (29-Jan-2009): This version is now superseded by the v0.4 release.

Earlier this week, Mozilla Labs released a very interesting (and useful) new extension for Firefox called "Ubiquity". It basically taps into the services provided by web services to integrate them better into the browsing experience. For an introduction to Ubiquity, it'd be best to read the post on the Mozilla Labs blog where the product was launched - the video featured in the post presents quite impressive practical enhancements to the browsing experience made possible by the product.

Radheef for Ubiquity
Radheef for Ubiquity adds the verb 'bas' to Ubiquity to facilitate quick and easy look up of definitions of Dhivehi words from any Thaana web page using the Radheef. To lookup a word you find on a Thaana web page, you can either select the word and bring up Ubiquity. Alternatively, you can type the word in to the Ubiquity command window. The definition results are shown immediately within the Ubiquity command bubble.

To install this, visit the Radheef for Ubiquity page after Ubiquity is installed and follow the normal verb installation steps.

Screenshots:


Happy browsing

Maldivians who use Firefox would be aware that certain Dhivehi websites, such as Miadhu Online, no longer display the Thaana fonts correctly when they switched to the recently released version 3 of the popular browser. I would like to review the issue for the benefit of Maldivian web developers and put forward some solutions that could be used. Further, I would also like to make available a fix that ordinary web users can themselves use until website owners (or the Firefox developers) fix the issue.

Problem description

Firefox 3.x series (and the 2.x series as well, to a lesser extent) fails in correctly displaying Thaana in web pages when certain non-Unicode Thaana fonts are applied to the elements using CSS. The same pages, however, render correctly without issue with Internet Explorer, Safari and Opera.

DOCTYPE - One contributing factor seems to be the DOCTYPE of the page. My guess is that this issue may have something to do with quirksmode rendering or standards compliance. The lack of a DOCTYPE in the markup gives correct rendering of the Thaana fonts on the page. However, omission of the DOCTYPE cannot and should not be considered a solution as DOCTYPE is required for most page markup and browsers need the correct DOCTYPE specification to correctly render modern pages.

Font - Another factor seems to be the font file used. The Thaana characters fail to be rendered correctly when almost all of the commonly used Thaana fonts, such as A_Faseyha, A_Waheed and A_Randhoo, are used. However, some fonts do work without issue - A_Ilham for example.

Here are some demo pages to highlight the issues. Each of the pages has three lines of Thaana - first of which is Thaana text enclosed in a font tag specifying a (problematic) Thaana font, the second is a H3 headline which has the font family set to a (problematic) Thaana font using CSS alone, the third is again a H3 headline which has the font family is set to a (problematic) Thaana font using CSS but has the text placed inside a font tag and finally the fourth line has a H3 headline whose font family is set to a (working) Thaana font using CSS alone.
View Thaana on page with: no DOCTYPE, HTML 4.01 DOCTYPE and XHTML 1.0 DOCTYPE.

Developer's fix

There are two definite solutions that can be easily applied by web developers.

Solution 1: Add HTML Font tags around any and all text that is to be displayed in Thaana. Specify the font to be used within the "face" attribute of the Font tags as usual. The flip-side of this method is that it results in a significant increase in page size. Haveeru News seems to have addressed the problem using this method. Here's a example:

bwlimIhunc aufulunc 

should be transformed into 

bwlimIhunc aufulunc

Solution 2: Change font used in the CSS definition to "A_Ilham". It is, perhaps, not as clean and pretty as "A_Faseyha" but until there is a fix to Firefox it will have to do.

A further alternative solution would be for the site owners and developers to take this occasion to shift to Unicode Thaana. It is much more reliable and is the currently recommended method of displaying Thaana on the web. Jazeera Daily, Haama Daily and MvHeadlines, to name a few, are all using Unicode for text display and entry. You can utilize the PHP-based Thaana Conversions class I released to convert the existing non-Unicode Thaana text to Unicode - and you can do such conversion on-the-fly on page requests.

User's fix

I wrote a quick bookmarklet-based solution several weeks ago for my use after getting annoyed with having to open Internet Explorer to view pages from sites affected by this bug. This solution will, or rather should, work on any affected site and on any computer.

Simply right click on this link - Jaa's Thaana Fix - and select "Bookmark this link" from the drop-down menu. Alternatively, you can drag and drop the link onto your bookmarks toolbar. When you are on a page that is messed up by the bug, such as Miadhu Online, Vaikaradhoo Live or Kavaasaa, click the "Thaana fix" link on your Bookmarks menu or toolbar. You will need to do this for each page you view.

Happy reading

I released my Javascript Unicode Keyboard Handler for Thaana early this year as open-source software so that web developers producing Dhivehi websites can allow users to type Thaana straight into text entry fields without forcing them to switch keyboard using the relevant features on the user's computer operating system. The code has since made its way into many different Dhivehi websites. However, the code I released then was mostly as-is from it's original version which I had written back in 2003 which, sadly, means that its behavior could be a little bit unpredictable with certain modern browsers - especially Opera and Safari.

I've now rewritten the code with the intent of producing cleaner, easier-to-use code that works without fail on all modern browsers. This version is (more or less!) guaranteed to work, and has been tested, on Firefox 2+, Opera 9+, Internet Explorer 6+ and Safari 2+ and has also been tested on Windows, Mac and Linux operating systems.

I am a big fan of separating code from design, so in keeping with that ideal this new version uses a more modern way of assigning the Thaana keyboard functionality in favour of inline javascript event handling used by the previous version (look below for an example). Since everything needs a spunky name I've also changed the old name to the more descriptive "Javascript Thaana Keyboard", which future versions of the script will maintain.

As before, it is being released under the MIT License, which allows its use in both personal and commercial applications as long as the copyright and license permission notice remains intact - so what the guy at basfoiy.com has done is a definite no-no.

Usage:

1. Link the file in the HEAD section of the page:

2. For any text input element (i.e INPUTs or TEXTAREAs), assign them the class name "thaanaKeyboardInput". You can assign further classes to the elements without ill-effect, if needed.

3. Using CSS, set any Unicode-compatible Dhivehi font (and size) to be used for the fields. You can easily do that by adding a class definition for the "thaanaKeyboardInput" class or by any other method of your choice.

4. The Thaana functionality would be automatically applied to any elements with the required class name when the page is loaded!

Demo:

Check out the demonstration and testing page here.

Download:

- original full source version (7.34 KB)
- minified version (2.01 KB)
I recommend you use the minified version.

As always, drop a line here if you use it and/or have problems or suggestions. Enjoy.

Update (20-Oct-2008): This version is now superseded by the new and improved v4.0.

Here is a short guide to using Thaana to make posts in Dhivehi using the free WordPress blogging service. I regularly get emails from different people asking me for help on this topic, so I hope this is helpful to all such people .

The solution presented here can be used on the free Wordpress blogging accounts available from WordPress.com and also on custom installs of WordPress. Please be aware, however, that there are much better ways to setup WordPress for Dhivehi posting if you have a custom install of WordPress or have a paid WordPress account.

Requirements

You need to have the Dhivehi keyboard installed on your computer. If you already type Dhivehi, say using MS Word or OpenOffice, then you have this already. See this post by Fayid if you need help with getting the Dhivehi keyboard installed.

Steps

1. Login to WordPress and click on the "Write" tab to start a new post.


2. In the post editor area, find and click the "HTML" tab to switch the view to HTML mode.


3. Copy and paste the following code into the editing area. It contains the bare minimum HTML/CSS needed to correctly display Dhivehi on all browsers supporting right-to-left text display in Unicode.

Test

4. Tinker around with the font family and font size settings if you know some CSS (or are feeling adventurous enough!). Faruma is probably the most decent Dhivehi Unicode font and is installed on most computers - hence is listed as the first preference. Different people like different font sizes for Dhivehi but 14px and 16px, I think, tend to be the easiest on the eye.

5. Click the "Visual" tab to switch the view back to the normal WYSIWYG mode.


6. Switch the keyboard to "Divehi" on the computer.


7. Select or delete the "Test" text on the post editor and start writing in Dhivehi.


That's all it takes!

Notes:
Unfortunately, there is no way to apply proper formatting to the post title with the free accounts on WordPress so you will have to settle for the out-of-alignment title display.

Video guide:
I thought it'd be fun to make a video guide for this and here is what thus materialized: View good quality | View crap quality (Youtube)

Good luck with the blogging!

Here is a PHP 5 class that provides a drop-in function replacement/equivalent for the built-in PHP date() function to output formatted dates in Thaana/Dhivehi. It follows the standard method of writing Gregorian dates in Thaana by using transliterations of the English month names and using the native Dhivehi names for the week days. It accepts all the usual formatting arguments permitted by the original date() function thus allowing the same degree of formatting freedom as the original. The output returned from the function uses ASCII Thaana and, if needed, can then be converted to Unicode/UTF-8 by using the Thaana Conversions class. This class does not support Hijri dates (yet).

The class is being released under the Open Source MIT License.

Functions exposed

format()
Returns a Dhivehi date string formatted according to the given format string using the given integer timestamp

Usage

<?php
// Load class include
require 'thaana_date.obj.php';

// Format date
$thaanatoday = Thaana_Date::format('j M Y', time());
?>

Download

- Thaana_Date.zip (v0.2, 1.4KB)

Drop me a line if you have comments/queries. Enjoy