mlsanityData to 7 Oct 2026

Teaching Large Language Models an Unseen Language on the Fly

arXiv 2402.19167 ↗First posted 29 Feb 2024cs.CLcited by 29

Not on the map. Why?

Abstract (arXiv)

Existing large language models struggle to support numerous low-resource languages, particularly the extremely low-resource ones, for which there is minimal training data available for effective parameter updating. We thus investigate whether LLMs can learn a new language on the fly solely through prompting. To study this question, we collect a research suite for Zhuang, a language supported by no LLMs currently. We introduce DiPMT++, a framework for adapting LLMs to unseen languages by in-context learning. Using a dictionary and 5K parallel sentences only, DiPMT++ significantly enhances the performance of GPT-4 from 0 to 16 BLEU for Chinese-to-Zhuang translation and achieves 32 BLEU for Zhuang-to-Chinese translation. We also validate the effectiveness of our framework on Kalamang, another unseen language. Furthermore, we demonstrate the practical utility of DiPMT++ in aiding humans in translating completely unseen languages, which could contribute to the preservation of linguistic diversity.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 25 of its 42 references that are in the map. First posted on arXiv 29 Feb 2024.

Builds on

25 of 42 references.

References · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  5. 2023Qwen Technical Reportcited by 3,273
  6. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  7. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  8. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  9. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
  10. 2017Word Translation Without Parallel Datacited by 480
  11. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
  12. 2023MEGA: Multilingual Evaluation of Generative AIcited by 291
13 more
  1. 2023Prompting Large Language Model for Machine Translation: A Case Studycited by 247
  2. 2022In-context Examples Selection for Machine Translationcited by 172
  3. 2022Prompting PaLM for Translation: Assessing Strategies and Performancecited by 144
  4. 2020UNKs Everywhere: Adapting Multilingual Language Models to New Scriptscited by 113
  5. 2022A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translationcited by 97
  6. 2023The unreasonable effectiveness of few-shot learning for machine translationcited by 91
  7. 2022BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Promptingcited by 90
  8. 2023Dictionary-based Phrase-level Prompting of Large Language Models for Machine Translationcited by 74
  9. 2023A Benchmark for Learning to Translate a New Language from One Grammar Bookcited by 74
  10. 2023PLUG: Leveraging Pivot Language in Cross-Lingual Instruction Tuningcited by 39
  11. 2023Romanization-based Large-scale Adaptation of Multilingual Language Modelscited by 21
  12. 2023InstructAlign: High-and-Low Resource Language Alignment via Continual Crosslingual Instruction Tuningcited by 21
  13. 2023SCALE: Synergized Collaboration of Asymmetric Language Translation Enginescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 14; 2026: 8.

29 papers.

Sort:

By area:

  1. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 367
  2. 2024Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book?cited by 23
  3. 2024Shortcomings of LLMs for Low-Resource Translation: Retrieval and Understanding are Both the Problemcited by 17
  4. 2025Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on Manchucited by 12
  5. 2025Overcoming Data Scarcity in Generative Language Modelling for Low-Resource Languages: A Systematic Reviewcited by 8
  6. 2024Evaluating Large Language Models along Dimensions of Language Variation: A Systematik Invesdigatiom uv Cross-lingual Generalizationcited by 7
  7. 2025LingGym: How Far Are LLMs from Thinking Like Field Linguists?cited by 6
  8. 2025It's All About In-Context Learning! Teaching Extremely Low-Resource Languages to LLMscited by 5
  9. 2025MiLiC-Eval: Benchmarking Multilingual LLMs for China's Minority Languagescited by 4
  10. 2025The Gold Medals in an Empty Room: Diagnosing Metalinguistic Reasoning in LLMs with Camlangcited by 3
  11. 2025Fine-Tuning LLMs for Low-Resource Dialect Translation: The Case of Lebanesecited by 2
  12. 2025Compensating for Data with Reasoning: Low-Resource Machine Translation with LLMscited by 2
17 more
  1. 2024SCOI: Syntax-augmented Coverage-based In-context Example Selection for Machine Translationcited by 1
  2. 2024Shaping the Future of Endangered and Low-Resource Languages -- Our Role in the Age of LLMs: A Keynote at ECIR 2024cited by 1
  3. 2024From Priest to Doctor: Domain Adaptation for Low-Resource Neural Machine Translationcited by 1
  4. 2025DialUp! Modeling the Language Continuum by Adapting Models to Dialects and Dialects to Modelscited by 1
  5. 2025Prompt and circumstance: A word-by-word LLM prompting approach to interlinear glossing for low-resource languagescited by 1
  6. 2026Multimodal In-context Learning for ASR of Low-resource Languagescited by 1
  7. 2026AFRILANGTUTOR: Advancing Language Tutoring and Culture Education in Low-Resource Languages with Large Language Modelscited by 1
  8. 2025Multilingual Encoder Knows more than You Realize: Shared Weights Pretraining for Extremely Low-Resource Languagescited by 0
  9. 2025Automated Annotation of Evolving Corpora for Augmenting Longitudinal Network Data: A Framework Integrating Large Language Models and Expert Knowledgecited by 0
  10. 2025CUTE: A Multilingual Dataset for Enhancing Cross-Lingual Knowledge Transfer in Low-Resource Languagescited by 0
  11. 2025On Non-interactive Evaluation of Animal Communication Translatorscited by 0
  12. 2026Rashid: A Cipher-Based Framework for Exploring In-Context Language Learningcited by 0
  13. 2026An Empirical Study of Many-Shot In-Context Learning for Machine Translation of Low-Resource Languagescited by 0
  14. 2026Efficient Low-Resource Language Adaptation via Multi-Source Dynamic Logit Fusioncited by 0
  15. 2026A Komi-Yazva--Russian Parallel Corpus and Evaluation Protocol for Zero- and Few-Shot LLM Translationcited by 0
  16. 2026Reinforcement Learning Elicits Contextual Learning of Unseen Language Translationcited by 0
  17. 2026A Factorial Study of Synthetic Data Generation for Low-Resource Machine Translation using Grammar Bookscited by 0