mlsanityData to 14 Oct 2026

Multilingual LLMs are Better Cross-lingual In-context Learners with Alignment

arXiv 2305.05940 ↗First posted 10 May 2023cs.CLcited by 71

Not on the map. Why?

Abstract (arXiv)

In-context learning (ICL) unfolds as large language models become capable of inferring test labels conditioned on a few labeled samples without any gradient update. ICL-enabled large language models provide a promising step forward toward bypassing recurrent annotation costs in a low-resource setting. Yet, only a handful of past studies have explored ICL in a cross-lingual setting, in which the need for transferring label-knowledge from a high-resource language to a low-resource one is immensely crucial. To bridge the gap, we provide the first in-depth analysis of ICL for cross-lingual text classification. We find that the prevalent mode of selecting random input-label pairs to construct the prompt-context is severely limited in the case of cross-lingual ICL, primarily due to the lack of alignment in the input as well as the output spaces. To mitigate this, we propose a novel prompt construction strategy -- Cross-lingual In-context Source-Target Alignment (X-InSTA). With an injected coherence in the semantics of the input examples and a task-based alignment across the source and target languages, X-InSTA is able to outperform random prompt selection by a large margin across three different tasks using 44 different cross-lingual pairs.

Key facts. Cited by 71 papers in the map (data through 14 Oct 2026). Builds on 23 of its 32 references that are in the map. First posted on arXiv 10 May 2023.

Builds on

23 of 32 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  8. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  9. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  10. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  11. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  12. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
11 more
  1. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
  2. 2022Transformers learn in-context by gradient descentcited by 732
  3. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 593
  4. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 326
  5. 2022In-context Examples Selection for Machine Translationcited by 172
  6. 2020The Multilingual Amazon Reviews Corpuscited by 143
  7. 2021Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learnerscited by 115
  8. 2022The Geometry of Multilingual Language Model Representationscited by 95
  9. 2021Language Models are Few-shot Multilingual Learnerscited by 92
  10. 2020Zero-Shot Cross-Lingual Transfer with Meta Learningcited by 66
  11. 2022XRICL: Cross-lingual Retrieval-Augmented In-Context Learning for Cross-lingual Text-to-SQL Semantic Parsingcited by 33

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 10; 2024: 24; 2025: 24; 2026: 13.

71 papers.

Sort:

By area:

  1. 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 208
  2. 2024How do Large Language Models Handle Multilingualism?cited by 150
  3. 2024Large Language Models Meet NLP: A Surveycited by 103
  4. 2023Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languagescited by 97
  5. 2025A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Modelscited by 88
  6. 2024LLMs Are Few-Shot In-Context Low-Resource Language Learnerscited by 68
  7. 2023Multilingual Large Language Models Are Not (Yet) Code-Switcherscited by 65
  8. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 51
  9. 2023MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learningcited by 46
  10. 2024Language Model Alignment in Multilingual Trolley Problemscited by 46
  11. 2025Emergent Abilities in Large Language Models: A Surveycited by 37
  12. 2024AdaMergeX: Cross-Lingual Transfer with Large Language Models via Adaptive Adapter Mergingcited by 34
38 more
  1. 2024Understanding Cross-Lingual Alignment -- A Surveycited by 34
  2. 2023Improving In-context Learning of Multilingual Generative Language Models with Cross-lingual Alignmentcited by 33
  3. 2024Unveiling Linguistic Regions in Large Language Modelscited by 22
  4. 2023From Classification to Generation: Insights into Crosslingual Retrieval Augmented ICLcited by 17
  5. 2024Teaching LLMs to Abstain across Languages via Multilingual Feedbackcited by 14
  6. 2025HyperTree Planning: Enhancing LLM Reasoning via Hierarchical Thinkingcited by 14
  7. 2024The Impact of Demonstrations on Multilingual In-Context Learning: A Multidimensional Analysiscited by 13
  8. 2024Language Models can Exploit Cross-Task In-context Learning for Data-Scarce Novel Taskscited by 13
  9. 2025Mimic In-Context Learning for Multimodal Taskscited by 12
  10. 2024Analyzing and Adapting Large Language Models for Few-Shot Multilingual NLU: Are We There Yet?cited by 11
  11. 2024MINERS: Multilingual Language Models as Semantic Retrieverscited by 11
  12. 2025Steering into New Embedding Spaces: Analyzing Cross-Lingual Alignment Induced by Model Interventions in Multilingual Language Modelscited by 11
  13. 2024Strategic Demonstration Selection for Improved Fairness in LLM In-Context Learningcited by 10
  14. 2024Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Modelscited by 9
  15. 2023On the Opportunities of Green Computing: A Surveycited by 7
  16. 2023Crosslingual Retrieval Augmented In-context Learning for Banglacited by 7
  17. 2023Lever LM: Configuring In-Context Sequence to Lever Large Vision Language Modelscited by 7
  18. 2023Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual Generalizationcited by 6
  19. 2024A Three-Pronged Approach to Cross-Lingual Adaptation with Multilingual LLMscited by 6
  20. 2024DAWN-ICL: Strategic Planning of Problem-solving Trajectories for Zero-Shot In-Context Learningcited by 6
  21. 2025Multilingual Prompt Engineering in Large Language Models: A Survey Across NLP Taskscited by 6
  22. 2025Relic: Enhancing Reward Model Generalization for Low-Resource Indic Languages with Few-Shot Examplescited by 6
  23. 2025Enhancing Cross-task Transfer of Large Language Models via Activation Steeringcited by 6
  24. 2025It's All About In-Context Learning! Teaching Extremely Low-Resource Languages to LLMscited by 6
  25. 2024XAMPLER: Learning to Retrieve Cross-Lingual In-Context Examplescited by 5
  26. 2024SADL: An Effective In-Context Learning Method for Compositional Visual QAcited by 5
  27. 2024PromptRefine: Enhancing Few-Shot Performance on Low-Resource Indic Languages with Example Selection from Related Example Bankscited by 5
  28. 2025DICE: Dynamic In-Context Example Selection in LLM Agents via Efficient Knowledge Transfercited by 5
  29. 2025Document Intelligence in the Era of Large Language Models: A Surveycited by 5
  30. 2026Cross-Lingual Activation Steering for Multilingual Language Modelscited by 5
  31. 2025Aligning Large Language Models to Low-Resource Languages through LLM-Based Selective Translation: A Systematic Studycited by 4
  32. 2023Gender-specific Machine Translation with Large Language Modelscited by 3
  33. 2025Language Fusion for Parameter-Efficient Cross-lingual Transfercited by 3
  34. 2025Gradual Code-Switching as Inference-Time Cross-Lingual Representational Alignment for LLMscited by 3
  35. 2026Language Steering for Multilingual In-Context Learningcited by 3
  36. 2026Cultural Value Alignment Via Latent Activation Steering in Large Language Modelscited by 3
  37. 2024The Rise and Down of Babel Tower: Investigating the Evolution Process of Multilingual Code Large Language Modelcited by 2
  38. 2025Unveiling Effective In-Context Configurations for Image Captioning: An External & Internal Analysiscited by 2

50 most cited of 71 All 71 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Align, Generate, Learn: A Novel Closed-Loop Framework for Cross-Lingual In-Context Learning
  2. 2024Does In-Context Learning Really Learn? Rethinking How Large Language Models Respond and Solve Tasks via In-Context Learning
  3. 2024In-Context Language Learning: Architectures and Algorithms
  4. 2024How Far Can In-Context Alignment Go? Exploring the State of In-Context Alignment
  5. 2023Beyond Output Matching: Bidirectional Alignment for Enhanced In-Context Learning
  6. 2025On the Relationship Between the Choice of Representation and In-Context Learning
  7. 2024Revisiting In-context Learning Inference Circuit in Large Language Models
  8. 2025Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMs