# Aksharantar: Open Indic-language Transliteration datasets and models for the Next Billion Users

arXiv: https://arxiv.org/abs/2205.03018 · HTML: https://mlsanity.com/paper/2205.03018/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 79 references that are in the map.
- First posted on arXiv 6 May 2022.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [XNLI: Evaluating Cross-lingual Sentence Representations](https://mlsanity.com/paper/1809.05053/) (2018)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation](https://mlsanity.com/paper/1611.04558/) (2016)
- [Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges](https://mlsanity.com/paper/1907.05019/) (2019)
- [MuRIL: Multilingual Representations for Indian Languages](https://mlsanity.com/paper/2103.10730/) (2021)
- [Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages](https://mlsanity.com/paper/2104.05596/) (2021)
- [When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models](https://mlsanity.com/paper/2010.12858/) (2020)
- [The IIT Bombay English-Hindi Parallel Corpus](https://mlsanity.com/paper/1710.02855/) (2017)
- [Transfer Learning across Low-Resource, Related Languages for Neural Machine Translation](https://mlsanity.com/paper/1708.09803/) (2017)
- [Processing South Asian Languages Written in the Latin Script: the Dakshina Dataset](https://mlsanity.com/paper/2007.01176/) (2020)
- [Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Loss](https://mlsanity.com/paper/1604.05529/) (2016)
- [Exploiting Language Relatedness for Low Web-Resource Language Model Adaptation: An Indic Languages Study](https://mlsanity.com/paper/2106.03958/) (2021)
- [Design Challenges in Named Entity Transliteration](https://mlsanity.com/paper/1808.02563/) (2018)

## Built on by (most cited first, 19 of 19)

- [The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants](https://mlsanity.com/paper/2308.16884/) (2023)
- [IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languages](https://mlsanity.com/paper/2403.06350/) (2024)
- [RomanSetu: Efficiently unlocking multilingual capabilities of Large Language Models via Romanization](https://mlsanity.com/paper/2401.14280/) (2024)
- [RomanLens: The Role Of Latent Romanization In Multilinguality In LLMs](https://mlsanity.com/paper/2502.07424/) (2025)
- [LAHAJA: A Robust Multi-accent Benchmark for Evaluating Hindi ASR Systems](https://mlsanity.com/paper/2408.11440/) (2024)
- [From Human Judgements to Predictive Models: Unravelling Acceptability in Code-Mixed Sentences](https://mlsanity.com/paper/2405.05572/) (2024)
- [Pretraining Language Models Using Translationese](https://mlsanity.com/paper/2403.13638/) (2024)
- [Bhaasha, Bhasa, Zaban: A Survey for Low-Resourced Languages in South Asia -- Current Stage and Challenges](https://mlsanity.com/paper/2509.11570/) (2025)
- [Script Gap: Evaluating LLM Triage on Indian Languages in Native vs Romanized Scripts in a Real World Setting](https://mlsanity.com/paper/2512.10780/) (2025)
- [IndicJR: A Judge-Free Benchmark of Jailbreak Robustness in South Asian Languages](https://mlsanity.com/paper/2602.16832/) (2026)
- [IndoNLP 2025: Shared Task on Real-Time Reverse Transliteration for Romanized Indo-Aryan languages](https://mlsanity.com/paper/2501.05816/) (2025)
- [Improving Informally Romanized Language Identification](https://mlsanity.com/paper/2504.21540/) (2025)
- [X-MuTeST: A Multilingual Benchmark for Explainable Hate Speech Detection and A Novel LLM-consulted Explanation Framework](https://mlsanity.com/paper/2601.03194/) (2026)
- [NepTam: A Nepali-Tamang Parallel Corpus and Baseline Machine Translation Experiments](https://mlsanity.com/paper/2603.14053/) (2026)
- [Share What You Already Know: Cross-Language-Script Transfer and Alignment for Sentiment Detection in Code-Mixed Data](https://mlsanity.com/paper/2402.04542/) (2024)
- [Beyond Specialization: Benchmarking LLMs for Transliteration of Indian Languages](https://mlsanity.com/paper/2505.19851/) (2025)
- [ILID: Native Script Language Identification for Indian Languages](https://mlsanity.com/paper/2507.11832/) (2025)
- [NADIR: Differential Attention Flow for Non-Autoregressive Transliteration in Indic Languages](https://mlsanity.com/paper/2601.12389/) (2026)
- [Indi-RomCoM: Code-Mixed Benchmark for Evaluating LLMs on Romanized Indic-English Instructions](https://mlsanity.com/paper/2606.30790/) (2026)

## Abstract (arXiv)

Transliteration is very important in the Indian language context due to the usage of multiple scripts and the widespread use of romanized inputs. However, few training and evaluation sets are publicly available. We introduce Aksharantar, the largest publicly available transliteration dataset for Indian languages created by mining from monolingual and parallel corpora, as well as collecting data from human annotators. The dataset contains 26 million transliteration pairs for 21 Indic languages from 3 language families using 12 scripts. Aksharantar is 21 times larger than existing datasets and is the first publicly available dataset for 7 languages and 1 language family. We also introduce the Aksharantar testset comprising 103k word pairs spanning 19 languages that enables a fine-grained analysis of transliteration models on native origin words, foreign words, frequent words, and rare words. Using the training set, we trained IndicXlit, a multilingual transliteration model that improves accuracy by 15% on the Dakshina test set, and establishes strong baselines on the Aksharantar testset introduced in this work. The models, mining scripts, transliteration guidelines, and datasets are available at https://github.com/AI4Bharat/IndicXlit under open-source licenses. We hope the availability of these large-scale, open resources will spur innovation for Indic language transliteration and downstream applications. We hope the availability of these large-scale, open resources will spur innovation for Indic language transliteration and downstream applications.