# Exploring the Role of Transliteration in In-Context Learning for Low-resource Languages Written in Non-Latin Scripts

arXiv: https://arxiv.org/abs/2407.02320 · HTML: https://mlsanity.com/paper/2407.02320/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 27 references that are in the map.
- First posted on arXiv 2 Jul 2024.

## Builds on (most cited first, 16 of 16)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model](https://mlsanity.com/paper/2402.07827/) (2024)
- [SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects](https://mlsanity.com/paper/2309.07445/) (2023)
- [Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages](https://mlsanity.com/paper/2305.12182/) (2023)
- [Do Multilingual Language Models Think Better in English?](https://mlsanity.com/paper/2308.01223/) (2023)
- [Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpaca](https://mlsanity.com/paper/2309.08958/) (2023)
- [Revisiting Machine Translation for Cross-lingual Classification](https://mlsanity.com/paper/2305.14240/) (2023)
- [Understanding Cross-Lingual Alignment -- A Survey](https://mlsanity.com/paper/2404.06228/) (2024)
- [Role of Language Relatedness in Multilingual Fine-tuning of Language Models: A Case Study in Indo-Aryan Languages](https://mlsanity.com/paper/2109.10534/) (2021)
- [Taxi1500: A Multilingual Dataset for Text Classification in 1500 Languages](https://mlsanity.com/paper/2305.08487/) (2023)
- [Romanization-based Large-scale Adaptation of Multilingual Language Models](https://mlsanity.com/paper/2304.08865/) (2023)
- [TransMI: A Framework to Create Strong Baselines from Multilingual Pretrained Language Models for Transliterated Data](https://mlsanity.com/paper/2405.09913/) (2024)
- [Crosslingual Retrieval Augmented In-context Learning for Bangla](https://mlsanity.com/paper/2311.00587/) (2023)

## Built on by (most cited first, 8 of 8)

- [How Transliterations Improve Crosslingual Alignment](https://mlsanity.com/paper/2409.17326/) (2024)
- [Multimodal In-context Learning for ASR of Low-resource Languages](https://mlsanity.com/paper/2601.05707/) (2026)
- [Enhancing Cross-Lingual Transfer through Reversible Transliteration: A Huffman-Based Approach for Low-Resource Languages](https://mlsanity.com/paper/2509.17493/) (2025)
- [One Script Instead of Hundreds? On Pretraining Romanized Encoder Language Models](https://mlsanity.com/paper/2601.05776/) (2026)
- [Code Mixologist : A Practitioner's Guide to Building Code-Mixed LLMs](https://mlsanity.com/paper/2602.11181/) (2026)
- [The Latin Substrate: How Language Models Represent and Mediate Script Choice](https://mlsanity.com/paper/2605.31363/) (2026)
- [CyrillicQA: The Influence of Phonetically Encoded Secret Language on LLM Performance](https://mlsanity.com/paper/2608.21462/) (2026)
- [5-Dialects-BN: Unmasking the Impact of Transliteration on Bangla Dialectal LLMs](https://mlsanity.com/paper/2609.09964/) (2026)

## Abstract (arXiv)

Decoder-only large language models (LLMs) excel in high-resource languages across various tasks through few-shot or even zero-shot in-context learning (ICL). However, their performance often does not transfer well to low-resource languages, especially those written in non-Latin scripts. Inspired by recent work that leverages transliteration in encoder-only models, we investigate whether transliteration is also effective in improving LLMs' performance for low-resource languages written in non-Latin scripts. To this end, we propose three prompt templates, where the target-language text is represented in (1) its original script, (2) Latin script, or (3) both. We apply these methods to several representative LLMs of different sizes on various tasks including text classification and sequential labeling. Our findings show that the effectiveness of transliteration varies by task type and model size. For instance, all models benefit from transliterations for sequential labeling (with increases of up to 25%).