mlsanityData to 7 Oct 2026

Exploring the Role of Transliteration in In-Context Learning for Low-resource Languages Written in Non-Latin Scripts

arXiv 2407.02320 ↗First posted 2 Jul 2024cs.CL cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Decoder-only large language models (LLMs) excel in high-resource languages across various tasks through few-shot or even zero-shot in-context learning (ICL). However, their performance often does not transfer well to low-resource languages, especially those written in non-Latin scripts. Inspired by recent work that leverages transliteration in encoder-only models, we investigate whether transliteration is also effective in improving LLMs' performance for low-resource languages written in non-Latin scripts. To this end, we propose three prompt templates, where the target-language text is represented in (1) its original script, (2) Latin script, or (3) both. We apply these methods to several representative LLMs of different sizes on various tasks including text classification and sequential labeling. Our findings show that the effectiveness of transliteration varies by task type and model size. For instance, all models benefit from transliterations for sequential labeling (with increases of up to 25%).

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 16 of its 27 references that are in the map. First posted on arXiv 2 Jul 2024.

Builds on

16 of 27 references.

References · 16

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  4. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  5. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  6. 2023SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialectscited by 149
  7. 2023Glot500: Scaling Multilingual Corpora and Language Models to 500 Languagescited by 116
  8. 2023Do Multilingual Language Models Think Better in English?cited by 87
  9. 2023Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpacacited by 60
  10. 2023Revisiting Machine Translation for Cross-lingual Classificationcited by 40
  11. 2024Understanding Cross-Lingual Alignment -- A Surveycited by 34
  12. 2021Role of Language Relatedness in Multilingual Fine-tuning of Language Models: A Case Study in Indo-Aryan Languagescited by 24
4 more
  1. 2023Taxi1500: A Multilingual Dataset for Text Classification in 1500 Languagescited by 23
  2. 2023Romanization-based Large-scale Adaptation of Multilingual Language Modelscited by 21
  3. 2024TransMI: A Framework to Create Strong Baselines from Multilingual Pretrained Language Models for Transliterated Datacited by 9
  4. 2023Crosslingual Retrieval Augmented In-context Learning for Banglacited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 6.

8 papers.

  1. 2024How Transliterations Improve Crosslingual Alignmentcited by 6
  2. 2025Enhancing Cross-Lingual Transfer through Reversible Transliteration: A Huffman-Based Approach for Low-Resource Languagescited by 1
  3. 2026Multimodal In-context Learning for ASR of Low-resource Languagescited by 1
  4. 2026One Script Instead of Hundreds? On Pretraining Romanized Encoder Language Modelscited by 1
  5. 2026Code Mixologist : A Practitioner's Guide to Building Code-Mixed LLMscited by 0
  6. 2026The Latin Substrate: How Language Models Represent and Mediate Script Choicecited by 0
  7. 2026CyrillicQA: The Influence of Phonetically Encoded Secret Language on LLM Performancecited by 0
  8. 20265-Dialects-BN: Unmasking the Impact of Transliteration on Bangla Dialectal LLMscited by 0