# In-context Learning with Retrieved Demonstrations for Language Models: A Survey

arXiv: https://arxiv.org/abs/2401.11624 · HTML: https://mlsanity.com/paper/2401.11624/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- Builds on 132 of its 175 references that are in the map.
- First posted on arXiv 21 Jan 2024.

## Builds on (most cited first, 30 of 132)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)

## Built on by (most cited first, 30 of 70)

- [LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristics](https://mlsanity.com/paper/2405.20132/) (2024)
- [What Makes Multimodal In-Context Learning Work?](https://mlsanity.com/paper/2404.15736/) (2024)
- [Visual Prompting in Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2409.15310/) (2024)
- [Beyond Examples: High-level Automated Reasoning Paradigm in In-Context Learning via MCTS](https://mlsanity.com/paper/2411.18478/) (2024)
- [Knowledge Mechanisms in Large Language Models: A Survey and Perspective](https://mlsanity.com/paper/2407.15017/) (2024)
- [UFO2: The Desktop AgentOS](https://mlsanity.com/paper/2504.14603/) (2025)
- [Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplars](https://mlsanity.com/paper/2405.16122/) (2024)
- [Mixture of In-Context Prompters for Tabular PFNs](https://mlsanity.com/paper/2405.16156/) (2024)
- [AMOR: A Recipe for Building Adaptable Modular Knowledge Agents Through Process Feedback](https://mlsanity.com/paper/2402.01469/) (2024)
- [TACO: Enhancing Multimodal In-context Learning via Task Mapping-Guided Sequence Configuration](https://mlsanity.com/paper/2505.17098/) (2025)
- [From Few to Many: Self-Improving Many-Shot Reasoners Through Iterative Optimization and Generation](https://mlsanity.com/paper/2502.00330/) (2025)
- [CEB: Compositional Evaluation Benchmark for Fairness in Large Language Models](https://mlsanity.com/paper/2407.02408/) (2024)
- [Retrieval Meets Reasoning: Even High-school Textbook Knowledge Benefits Multimodal Reasoning](https://mlsanity.com/paper/2405.20834/) (2024)
- [CTINexus: Automatic Cyber Threat Intelligence Knowledge Graph Construction Using Large Language Models](https://mlsanity.com/paper/2410.21060/) (2024)
- [Assessing Empathy in Large Language Models with Real-World Physician-Patient Interactions](https://mlsanity.com/paper/2405.16402/) (2024)
- [IC-Cache: Efficient Large Language Model Serving via In-context Caching](https://mlsanity.com/paper/2501.12689/) (2025)
- [Problem-Solving Logic Guided Curriculum In-Context Learning for LLMs Complex Reasoning](https://mlsanity.com/paper/2502.15401/) (2025)
- [Mimic In-Context Learning for Multimodal Tasks](https://mlsanity.com/paper/2504.08851/) (2025)
- [Improving Demonstration Diversity by Human-Free Fusing for Text-to-SQL](https://mlsanity.com/paper/2402.10663/) (2024)
- [Hyperband-based Bayesian Optimization for Black-box Prompt Selection](https://mlsanity.com/paper/2412.07820/) (2024)
- [Federated Retrieval Augmented Generation for Multi-Product Question Answering](https://mlsanity.com/paper/2501.14998/) (2025)
- [SPL: A Socratic Playground for Learning Powered by Large Language Model](https://mlsanity.com/paper/2406.13919/) (2024)
- [Divide, Reweight, and Conquer: A Logit Arithmetic Approach for In-Context Learning](https://mlsanity.com/paper/2410.10074/) (2024)
- [Demo-ICL: In-Context Learning for Procedural Video Knowledge Acquisition](https://mlsanity.com/paper/2602.08439/) (2026)
- [XAMPLER: Learning to Retrieve Cross-Lingual In-Context Examples](https://mlsanity.com/paper/2405.05116/) (2024)
- [Mixtures of In-Context Learners](https://mlsanity.com/paper/2411.02830/) (2024)
- [MICAS: Multi-grained In-Context Adaptive Sampling for 3D Point Cloud Processing](https://mlsanity.com/paper/2411.16773/) (2024)
- [Learning to Select In-Context Demonstration Preferred by Large Language Model](https://mlsanity.com/paper/2505.19966/) (2025)
- [One size doesn't fit all: Predicting the Number of Examples for In-Context Learning](https://mlsanity.com/paper/2403.06402/) (2024)
- [Why is prompting hard? Understanding prompts on binary sequence predictors](https://mlsanity.com/paper/2502.10760/) (2025)

## Abstract (arXiv)

Language models, especially pre-trained large language models, have showcased remarkable abilities as few-shot in-context learners (ICL), adept at adapting to new tasks with just a few demonstrations in the input context. However, the model's ability to perform ICL is sensitive to the choice of the few-shot demonstrations. Instead of using a fixed set of demonstrations, one recent development is to retrieve demonstrations tailored to each input query. The implementation of demonstration retrieval is relatively straightforward, leveraging existing databases and retrieval systems. This not only improves the efficiency and scalability of the learning process but also has been shown to reduce biases inherent in manual example selection. In light of the encouraging results and growing research in ICL with retrieved demonstrations, we conduct an extensive review of studies in this area. In this survey, we discuss and compare different design choices for retrieval models, retrieval training procedures, and inference algorithms.