# From Classification to Generation: Insights into Crosslingual Retrieval Augmented ICL

arXiv: https://arxiv.org/abs/2311.06595 · HTML: https://mlsanity.com/paper/2311.06595/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation.
- Builds on 30 of its 44 references that are in the map.
- First posted on arXiv 11 Nov 2023.

## Placed in

- Retrieval-augmented generation / [Generation and context integration](https://mlsanity.com/rag/generation-and-context-integration/)

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization](https://mlsanity.com/paper/2003.11080/) (2020)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [On the Cross-lingual Transferability of Monolingual Representations](https://mlsanity.com/paper/1910.11856/) (2019)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://mlsanity.com/paper/1812.10464/) (2018)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2010.08191/) (2020)
- [On Measuring Social Biases in Sentence Encoders](https://mlsanity.com/paper/1903.10561/) (2019)
- [XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages](https://mlsanity.com/paper/2106.13822/) (2021)
- [MEGA: Multilingual Evaluation of Generative AI](https://mlsanity.com/paper/2303.12528/) (2023)
- [SemEval-2017 Task 4: Sentiment Analysis in Twitter](https://mlsanity.com/paper/1912.00741/) (2019)
- [Multilingual LLMs are Better Cross-lingual In-context Learners with Alignment](https://mlsanity.com/paper/2305.05940/) (2023)
- [BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfer](https://mlsanity.com/paper/2305.14857/) (2023)
- [Discrete and Soft Prompting for Multilingual Models](https://mlsanity.com/paper/2109.03630/) (2021)
- [One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval](https://mlsanity.com/paper/2107.11976/) (2021)
- [Novelty Controlled Paraphrase Generation with Retrieval Augmented Conditional Prompt Tuning](https://mlsanity.com/paper/2202.00535/) (2022)
- [Zero-shot Cross-lingual Transfer of Prompt-based Tuning with a Unified Multilingual Prompt](https://mlsanity.com/paper/2202.11451/) (2022)
- [Cross-Lingual Retrieval Augmented Prompt for Low-Resource Languages](https://mlsanity.com/paper/2212.09651/) (2022)
- [Semantic-Oriented Unlabeled Priming for Large-Scale Language Models](https://mlsanity.com/paper/2202.06133/) (2022)

## Built on by (most cited first, 17 of 17)

- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2405.06211/) (2024)
- [Retrieval-Augmented Generation for AI-Generated Content: A Survey](https://mlsanity.com/paper/2402.19473/) (2024)
- [Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers](https://mlsanity.com/paper/2404.04925/) (2024)
- [Test-Time Learning for Large Language Models](https://mlsanity.com/paper/2505.20633/) (2025)
- [MMDocIR: Benchmarking Multimodal Retrieval for Long Documents](https://mlsanity.com/paper/2501.08828/) (2025)
- [Enhancing Retrieval and Managing Retrieval: A Four-Module Synergy for Improved Quality and Efficiency in RAG Systems](https://mlsanity.com/paper/2407.10670/) (2024)
- [Exploring the Security Threats of Knowledge Base Poisoning in Retrieval-Augmented Code Generation](https://mlsanity.com/paper/2502.03233/) (2025)
- [Adaptive Schema-aware Event Extraction with Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.08690/) (2025)
- [MemCollab: Cross-Model Memory Collaboration via Contrastive Trajectory Distillation](https://mlsanity.com/paper/2603.23234/) (2026)
- [Evaluating Implicit Bias in Large Language Models by Attacking From a Psychometric Perspective](https://mlsanity.com/paper/2406.14023/) (2024)
- [Enhancing Cross-task Transfer of Large Language Models via Activation Steering](https://mlsanity.com/paper/2507.13236/) (2025)
- [LLM-based Corroborating and Refuting Evidence Retrieval for Scientific Claim Verification](https://mlsanity.com/paper/2503.07937/) (2025)
- [Learning from Emptiness: De-biasing Listwise Rerankers with Content-Agnostic Probability Calibration](https://mlsanity.com/paper/2604.10150/) (2026)
- [DoTA-RAG: Dynamic of Thought Aggregation RAG](https://mlsanity.com/paper/2506.12571/) (2025)
- [The Virtuous Cycle: AI-Powered Vector Search and Vector Search-Augmented AI](https://mlsanity.com/paper/2603.09347/) (2026)
- [Mask-to-Correct^+: Leveraging Retriever Diversity for Masking-guided Faithful Fact Correction](https://mlsanity.com/paper/2605.18776/) (2026)

## Abstract (arXiv)

The remarkable ability of Large Language Models (LLMs) to understand and follow instructions has sometimes been limited by their in-context learning (ICL) performance in low-resource languages. To address this, we introduce a novel approach that leverages cross-lingual retrieval-augmented in-context learning (CREA-ICL). By extracting semantically similar prompts from high-resource languages, we aim to improve the zero-shot performance of multilingual pre-trained language models (MPLMs) across diverse tasks. Though our approach yields steady improvements in classification tasks, it faces challenges in generation tasks. Our evaluation offers insights into the performance dynamics of retrieval-augmented in-context learning across both classification and generation domains.