為何從斯里蘭卡政府公報複製的僧伽羅文會變成「Y%S ,xld」，以及如何轉換為 Unicode

Copy a line of Sinhala from a Sri Lankan government gazette PDF and paste it somewhere. You don't get Sinhala. You get this:

Y%S ,xld m%cd;dka;%sl iudcjd§ ckrcfha .eiÜ m;%h

That line is the gazette's own heading, ශ්‍රී ලංකා ප්‍රජාතාන්ත්‍රික සමාජවාදී ජනරජයේ ගැසට් පත්‍රය. The cause is that the text was typed in FM Abhaya, FM Malithi or another "FM" font.

I wrote a converter for this. Here is how these fonts work and what it takes to turn them into proper Unicode.

The FM fonts are glyph fonts, not Sinhala fonts

The FM fonts come from before Unicode Sinhala was usable everywhere. They don't contain Sinhala characters at all. They draw Sinhala shapes over ordinary ASCII and Latin-1 codes:

So the file stores l, and only the font makes it look like ක. Lose the font and you see Latin letters. Search engines, screen readers and translation tools also only ever see the Latin letters.

Step 1: a character map

The first part is a big lookup table. Consonants, independent vowels, dependent vowel signs, punctuation:

const CONS = { l: "ක", L: "ඛ", ".": "ග", p: "ච", c: "ජ", g: "ට", ";": "ත", o: "ද", k: "න",
               m: "ප", n: "බ", u: "ම", h: "ය", r: "ර", ",": "ල", j: "ව", Y: "ශ", i: "ස", y: "හ" /* … */ };
const SIGNS = { d: "ා", e: "ැ", E: "ෑ", s: "ි", S: "ී", q: "ු", Q: "ූ", a: "්" /* … */ };
const VOWELS = { w: "අ", b: "ඉ", W: "උ", t: "එ", T: "ඔ" /* … */ };

If the fonts were only that, a replace per character would do. They aren't, for three reasons.

Step 2: FM text is in visual order

In Sinhala, the kombuva (ෙ) is written to the left of the consonant it belongs to. FM text stores what you see, so the kombuva comes first. Unicode stores the logical order: consonant first, then the vowel sign.

A two-part vowel like ො (kombuva + aela-pilla) is worse: FM types the kombuva before the consonant and the aela-pilla after it, but in Unicode ො is one code point after the consonant.

FM:       f  m  d  ;          (ෙ  ප  ා  ත)
Unicode:  ප  ො  ත             "පොත" (book)

So when the converter meets an f, it reads ahead: the consonant, then any signs typed after it, and decides which vowel the kombuva makes together with those signs:

// signs = the vowel-sign codes typed after the consonant
function kombuVowel(signs) {
  const [a, b] = signs;
  if (a === "d" && (b === "a" || b === "A")) return { vowel: "ෝ", rest: signs.slice(2) }; // f + C + d + a
  if (a === "d") return { vowel: "ො", rest: signs.slice(1) };                              // f + C + d
  if (a === "a" || a === "A") return { vowel: "ේ", rest: signs.slice(1) };                 // f + C + a
  if (a === "!") return { vowel: "ෞ", rest: signs.slice(1) };
  return { vowel: "ෙ", rest: signs };                                                       // plain kombuva
}

Two kombuvas in a row (ff) are the kombu deka, ෛ.

Step 3: rakaransaya and yansaya need a ZWJ

The little ර under a letter (rakaransaya, as in ප්‍ර) and the ය joined after one (yansaya, as in ක්‍ය) are one keystroke each in FM: % and H. In Unicode they are three characters: al-lakuna, a zero-width joiner (U+200D), then ර or ය. Without the ZWJ you get a visible al-lakuna and a separate letter instead of the joined form.

const ZWJ = "‍";
const RAKAR = "්" + ZWJ + "ර";
const YANS  = "්" + ZWJ + "ය";

They also have an ordering problem of their own: typists put the vowel sign before or after the %. Both of these turn up in real gazette text, and both are ශ්‍රී:

Y%S   and   YS%

So after a consonant the converter collects the rakaransaya/yansaya and the vowel signs separately, in any typed order, and writes them back out in Unicode order: consonant, then ්‍ර, then the vowel sign.

Step 4: ligatures

Some consonant + vowel pairs are a single glyph in FM, because the shape changes (§ is දී, ß is රි, ÷ is ඳු), and so are some conjuncts (Ë is ක්‍ෂ). Those map to several Unicode characters.

A trap here: a ligature can end in a sign, and a kombuva can come before it. ï is ම් with the al-lakuna built in, so fï should be මේ. If the ligature is treated as finished text, the kombuva can't combine with its al-lakuna and you get the wrong vowel. The fix was to split a ligature's trailing sign back off and treat it like a typed sign, so the kombuva logic above sees it.

The small things that bite

• ර + ු is drawn differently. On ර the ු and ූ look like ැ and ෑ, so FM types රු as r + e and රූ as r + E. A naive map gives you රැ.
• ¾ is not a reph. It looks like one, but it is just the inline ර් glyph and stays where it is, so f¾ is රේ and ld¾hd,h is කාර්යාලය.
• Normalise first. Several FM codes are Latin-1 letters with accents (ú is වි), and the same letter can arrive precomposed or as u + a combining accent. Calling input.normalize("NFC") before mapping makes both forms hit the same table entry.
• English words get converted too. In an FM font every Latin letter is a Sinhala letter, so there is no safe way to skip English in mixed text. Paste Sinhala only. To avoid double-converting text that is already Unicode, I count Sinhala characters against Latin letters first.

Testing with real documents

Gazette PDFs make good test data. Their text layer is the FM codes and the printed page shows the Sinhala, so every line gives you an input and its expected output from a real document, not from my own typing:

import test from "node:test";
import assert from "node:assert/strict";

const Z = "‍";
test("Gazette heading: FM text layer to the printed Sinhala", () => {
  assert.equal(fmToUnicode("Y%S ,xld"), `ශ්${Z}රී ලංකා`);
  assert.equal(fmToUnicode("YS% ,xld"), `ශ්${Z}රී ලංකා`); // sign typed before the rakaransaya
  assert.equal(fmToUnicode("fmd;"), "පොත");
  assert.equal(fmToUnicode("isxy,"), "සිංහල");
});

Try it

The converter runs in the browser (nothing is uploaded): Sinhala FM to Unicode converter. Paste FM Abhaya text and you get Unicode you can put in a website, email or document.

If you find FM text it gets wrong, I'd like to see it. Leave it in the comments.

This article was written with the help of AI. The code, numbers and sources come from my own project and were checked against it before publishing.