mlsanityData to 14 Oct 2026

Balanced Multi-Factor In-Context Learning for Multilingual Large Language Models

arXiv 2502.11495 ↗First posted 17 Feb 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Multilingual large language models (MLLMs) are able to leverage in-context learning (ICL) to achieve high performance by leveraging cross-lingual knowledge transfer without parameter updates. However, their effectiveness is highly sensitive to example selection, particularly in multilingual settings. Based on the findings of existing work, three key factors influence multilingual ICL: (1) semantic similarity, (2) linguistic alignment, and (3) language-specific performance. However, existing approaches address these factors independently, without explicitly disentangling their combined impact, leaving optimal example selection underexplored. To address this gap, we propose balanced multi-factor ICL (BMF-ICL), a method that quantifies and optimally balances these factors for improved example selection. Experiments on mCSQA and TYDI across four MLLMs demonstrate that BMF-ICL outperforms existing methods. Further analysis highlights the importance of incorporating all three factors and the importance of selecting examples from multiple languages.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 54 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

38 of 54 references.

References · 38

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  6. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  7. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  8. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  9. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  10. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  11. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  12. 2016Bag of Tricks for Efficient Text Classificationcited by 876
26 more
  1. 2019How multilingual is Multilingual BERT?cited by 645
  2. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  3. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 593
  4. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 530
  5. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 488
  6. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 469
  7. 2021True Few-Shot Learning with Language Modelscited by 344
  8. 2016FastText.zip: Compressing text classification modelscited by 335
  9. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 295
  10. 2022Leveraging Large Language Models for Multiple Choice Question Answeringcited by 203
  11. 2024When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboardscited by 137
  12. 2023Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languagescited by 97
  13. 2021Language Models are Few-shot Multilingual Learnerscited by 92
  14. 2023Do Multilingual Language Models Think Better in English?cited by 88
  15. 2023Multilingual LLMs are Better Cross-lingual In-context Learners with Alignmentcited by 71
  16. 2024LLMs Are Few-Shot In-Context Low-Resource Language Learnerscited by 68
  17. 2022Gender Bias in Masked Language Models for Multiple Languagescited by 60
  18. 2023Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Promptscited by 45
  19. 2024Social Bias Evaluation for Large Language Models Requires Prompt Variationscited by 40
  20. 2022Speaking Multiple Languages Affects the Moral Bias of Language Modelscited by 34
  21. 2021Analysing The Impact Of Linguistic Features On Cross-Lingual Transfercited by 16
  22. 2023Evaluating Gender Bias of Pre-trained Language Models in Natural Language Inference by Considering All Labelscited by 16
  23. 2024Breaking the Language Barrier: Can Direct Inference Outperform Pre-Translation in Multilingual LLM Applications?cited by 12
  24. 2023How You Prompt Matters! Even Task-Oriented Constraints in Instructions Affect LLM-Generated Text Detectioncited by 8
  25. 2024A Little Leak Will Sink a Great Ship: Survey of Transparency for Large Language Models from Start to Finishcited by 6
  26. 2024Eagle: Ethical Dataset Given from Real Interactionscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Align, Generate, Learn: A Novel Closed-Loop Framework for Cross-Lingual In-Context Learning
  2. 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias
  3. 2024Multilingual Large Language Models: A Systematic Survey
  4. 2025Refract ICL: Rethinking Example Selection in the Era of Million-Token Models
  5. 2024Large Language Models Know What Makes Exemplary Contexts
  6. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers
  7. 2024Pruning Multilingual Large Language Models for Multilingual Inference
  8. 2024What Drives Performance in Multilingual Language Models?