# Low-Resource Transliteration for Roman-Urdu and Urdu Using Transformer-Based Models

arXiv: https://arxiv.org/abs/2503.21530 · HTML: https://mlsanity.com/paper/2503.21530/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 25 references that are in the map.
- First posted on arXiv 27 Mar 2025.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://mlsanity.com/paper/1812.10464/) (2018)
- [The IIT Bombay English-Hindi Parallel Corpus](https://mlsanity.com/paper/1710.02855/) (2017)
- [A Survey of Code-switched Speech and Language Processing](https://mlsanity.com/paper/1904.00784/) (2019)
- [Processing South Asian Languages Written in the Latin Script: the Dakshina Dataset](https://mlsanity.com/paper/2007.01176/) (2020)
- [CLSRIL-23: Cross Lingual Speech Representations for Indic Languages](https://mlsanity.com/paper/2107.07402/) (2021)
- [Generalists vs. Specialists: Evaluating Large Language Models for Urdu](https://mlsanity.com/paper/2407.04459/) (2024)

## Built on by (most cited first, 1 of 1)

- [Evaluating Large Language Models on Urdu Idioms](https://mlsanity.com/paper/2510.17460/) (2025)

## Abstract (arXiv)

As the Information Retrieval (IR) field increasingly recognizes the importance of inclusivity, addressing the needs of low-resource languages remains a significant challenge. Transliteration between Urdu and its Romanized form, Roman Urdu, remains underexplored despite the widespread use of both scripts in South Asia. Prior work using RNNs on the Roman-Urdu-Parl dataset showed promising results but suffered from poor domain adaptability and limited evaluation. We propose a transformer-based approach using the m2m100 multilingual translation model, enhanced with masked language modeling (MLM) pretraining and fine-tuning on both Roman-Urdu-Parl and the domain-diverse Dakshina dataset. To address previous evaluation flaws, we introduce rigorous dataset splits and assess performance using BLEU, character-level BLEU, and CHRF. Our model achieves strong transliteration performance, with Char-BLEU scores of 96.37 for Urdu->Roman-Urdu and 97.44 for Roman-Urdu->Urdu. These results outperform both RNN baselines and GPT-4o Mini and demonstrate the effectiveness of multilingual transfer learning for low-resource transliteration tasks.