mlsanityData to 14 Oct 2026

LLMs Are Few-Shot In-Context Low-Resource Language Learners

arXiv 2403.16512 ↗First posted 25 Mar 2024cs.CL cs.AIcited by 68

Not on the map. Why?

Abstract (arXiv)

In-context learning (ICL) empowers large language models (LLMs) to perform diverse tasks in underrepresented languages using only short in-context information, offering a crucial avenue for narrowing the gap between high-resource and low-resource languages. Nonetheless, there is only a handful of works explored ICL for low-resource languages with most of them focusing on relatively high-resource languages, such as French and Spanish. In this work, we extensively study ICL and its cross-lingual variation (X-ICL) on 25 low-resource and 7 relatively higher-resource languages. Our study not only assesses the effectiveness of ICL with LLMs in low-resource languages but also identifies the shortcomings of in-context label alignment, and introduces a more effective alternative: query alignment. Moreover, we provide valuable insights into various facets of ICL for low-resource languages. Our study concludes the significance of few-shot in-context information on enhancing the low-resource understanding quality of LLMs through semantically relevant information by closing the language gap in the target language and aligning the semantics between the targeted low-resource and the high-resource language that the model is proficient in. Our work highlights the importance of advancing ICL research, particularly for low-resource languages. Our code is publicly released at https://github.com/SamuelCahyawijaya/in-context-alignment

Key facts. Cited by 68 papers in the map (data through 14 Oct 2026). Builds on 42 of its 62 references that are in the map. First posted on arXiv 25 Mar 2024.

Builds on

42 of 62 references.

References · 42

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  5. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  6. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  7. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  8. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  9. 2018Experience Replay for Continual Learningcited by 1,095
  10. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
  11. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  12. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
30 more
  1. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  2. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 593
  3. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  4. 2022Dataless Knowledge Fusion by Merging Weights of Language Modelscited by 387
  5. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 311
  6. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  7. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 223
  8. 2020The Multilingual Amazon Reviews Corpuscited by 143
  9. 2019CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairscited by 136
  10. 2021XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyondcited by 114
  11. 2020IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understandingcited by 109
  12. 2021Language Models are Few-shot Multilingual Learnerscited by 92
  13. 2022BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Promptingcited by 90
  14. 2023Multilingual LLMs are Better Cross-lingual In-context Learners with Alignmentcited by 71
  15. 2022NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languagescited by 68
  16. 2023BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfercited by 68
  17. 2022One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesiacited by 66
  18. 2023Multilingual Large Language Models Are Not (Yet) Code-Switcherscited by 65
  19. 2021IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generationcited by 57
  20. 2021Are Multilingual Models Effective in Code-Switching?cited by 56
  21. 2023Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLUcited by 53
  22. 2023Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languagescited by 51
  23. 2023Multi-lingual and Multi-cultural Figurative Language Understandingcited by 47
  24. 2022IndicNLG Benchmark: Multilingual Datasets for Diverse NLG Tasks in Indic Languagescited by 42
  25. 2022NusaCrowd: Open Source Initiative for Indonesian NLP Resourcescited by 38
  26. 2023MasakhaNEWS: News Topic Classification for African languagescited by 31
  27. 2022Bloom Library: Multimodal Datasets in 300+ Languages for a Variety of Downstream Taskscited by 28
  28. 2019Low-Resource Corpus Filtering using Multilingual Sentence Embeddingscited by 25
  29. 2024LinguAlchemy: Fusing Typological and Geographical Elements for Unseen Language Generalizationcited by 10
  30. 2023The Obscure Limitation of Modular Multilingual Language Modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 17; 2025: 39; 2026: 12.

68 papers.

Sort:

By area:

  1. 2024Adapting Multilingual LLMs to Low-Resource Languages using Continued Pre-training and Synthetic Corpuscited by 24
  2. 2025Learn Globally, Speak Locally: Bridging the Gaps in Multilingual Reasoningcited by 24
  3. 2024Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book?cited by 23
  4. 2025Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directionscited by 23
  5. 2024High-Dimension Human Value Representation in Large Language Modelscited by 20
  6. 2024Can we teach language models to gloss endangered languages?cited by 14
  7. 2025Compositional Translation: A Novel LLM-based Approach for Low-resource Machine Translationcited by 14
  8. 2024RealTCD: Temporal Causal Discovery from Interventional Data with Large Language Modelcited by 13
  9. 2024Transcending Language Boundaries: Harnessing LLMs for Low-Resource Language Translationcited by 12
  10. 2025Instructing Large Language Models for Low-Resource Languages: A Systematic Study for Basquecited by 12
  11. 2025One Tokenizer To Rule Them All: Emergent Language Plasticity via Multilingual Tokenizerscited by 12
  12. 2024MINERS: Multilingual Language Models as Semantic Retrieverscited by 11
38 more
  1. 2025The Translation Barrier Hypothesis: Multilingual Generation with Large Language Models Suffers from Implicit Translation Failurecited by 10
  2. 2025Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asiacited by 9
  3. 2026CURE-Med: Curriculum-Informed Reinforcement Learning for Multilingual Medical Reasoningcited by 9
  4. 2025Zero-shot Meta-learning for Tabular Prediction Tasks with Adversarially Pre-trained Transformercited by 8
  5. 2024Evaluating Large Language Models along Dimensions of Language Variation: A Systematik Invesdigatiom uv Cross-lingual Generalizationcited by 7
  6. 2024Towards Incremental Learning in Large Language Models: A Critical Reviewcited by 6
  7. 2025Enhancing Cross-task Transfer of Large Language Models via Activation Steeringcited by 6
  8. 2025It's All About In-Context Learning! Teaching Extremely Low-Resource Languages to LLMscited by 6
  9. 2025Bhaasha, Bhasa, Zaban: A Survey for Low-Resourced Languages in South Asia -- Current Stage and Challengescited by 6
  10. 2024XAMPLER: Learning to Retrieve Cross-Lingual In-Context Examplescited by 5
  11. 2024PromptRefine: Enhancing Few-Shot Performance on Low-Resource Indic Languages with Example Selection from Related Example Bankscited by 5
  12. 2025SampleLLM: Optimizing Tabular Data Synthesis in Recommendationscited by 4
  13. 2025Contextualizing Search Queries In-Context Learning for Conversational Rewriting with LLMscited by 4
  14. 2025From Curated Data to Scalable Models: Continual Pre-training of Dense and MoE Large Language Models for Tibetancited by 4
  15. 2025Aligning Large Language Models to Low-Resource Languages through LLM-Based Selective Translation: A Systematic Studycited by 4
  16. 2024Generating bilingual example sentences with large language models as lexicography assistantscited by 3
  17. 2025Semantic-Aware Contrastive Fine-Tuning: Boosting Multimodal Malware Classification with Discriminative Embeddingscited by 3
  18. 2025Gradual Code-Switching as Inference-Time Cross-Lingual Representational Alignment for LLMscited by 3
  19. 2024The Rise and Down of Babel Tower: Investigating the Evolution Process of Multilingual Code Large Language Modelcited by 2
  20. 2025LSC-Eval: A General Framework to Evaluate Methods for Assessing Dimensions of Lexical Semantic Change Using LLM-Generated Synthetic Datacited by 2
  21. 2025High-Dimensional Interlingual Representations of Large Language Modelscited by 2
  22. 2025An evaluation of LLMs and Google Translate for translation of selected Indian languages via sentiment and semantic analysescited by 2
  23. 2025In-context Language Learning for Endangered Languages in Speech Recognitioncited by 2
  24. 2025Compensating for Data with Reasoning: Low-Resource Machine Translation with LLMscited by 2
  25. 2025How and Where to Translate? The Impact of Translation Strategies in Cross-lingual LLM Promptingcited by 2
  26. 2025Linguistic Neuron Overlap Patterns to Facilitate Cross-lingual Transfer on Low-resource Languagescited by 2
  27. 2025NormGenesis: Multicultural Dialogue Generation via Exemplar-Guided Social Norm Modeling and Violation Recoverycited by 2
  28. 2025Steering LLMs toward Korean Local Speech: Iterative Refinement Framework for Faithful Dialect Translationcited by 2
  29. 2026Multimodal In-context Learning for ASR of Low-resource Languagescited by 2
  30. 2024Vision-Braille: A Curriculum Learning Toolkit and Braille-Chinese Corpus for Braille Translationcited by 1
  31. 2025DialUp! Modeling the Language Continuum by Adapting Models to Dialects and Dialects to Modelscited by 1
  32. 2025Are Multilingual Language Models an Off-ramp for Under-resourced Languages? Will we arrive at Digital Language Equality in Europe in 2030?cited by 1
  33. 2025A Case Study of Cross-Lingual Zero-Shot Generalization for Classical Languages in LLMscited by 1
  34. 2025TopXGen: Topic-Diverse Parallel Data Generation for Low-Resource Machine Translationcited by 1
  35. 2025CUTE: A Multilingual Dataset for Enhancing Cross-Lingual Knowledge Transfer in Low-Resource Languagescited by 1
  36. 2025Qomhra: A Bilingual Irish and English Large Language Modelcited by 1
  37. 2026Reasonably reasoning AI agents can avoid game-theoretic failures in zero-shot, provablycited by 1
  38. 2026Polite on the Surface, Broken in Practice: A Curated Dataset for Fixing Generation and Register Failures in Low-Resource Bangla Text Generationcited by 1

50 most cited of 68 All 68 (JSON)