# Dr.ICL: Demonstration-Retrieved In-context Learning

arXiv: https://arxiv.org/abs/2305.14128 · HTML: https://mlsanity.com/paper/2305.14128/

## Key facts

- Cited by 72 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 34 references that are in the map.
- First posted on arXiv 23 May 2023.

## Builds on (most cited first, 28 of 28)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [In-Context Retrieval-Augmented Language Models](https://mlsanity.com/paper/2302.00083/) (2023)
- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [Learning To Retrieve Prompts for In-Context Learning](https://mlsanity.com/paper/2112.08633/) (2021)
- [Large Dual Encoders Are Generalizable Retrievers](https://mlsanity.com/paper/2112.07899/) (2021)
- [Complexity-Based Prompting for Multi-Step Reasoning](https://mlsanity.com/paper/2210.00720/) (2022)
- [Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters](https://mlsanity.com/paper/2212.10001/) (2022)
- [Demystifying Prompts in Language Models via Perplexity Estimation](https://mlsanity.com/paper/2212.04037/) (2022)
- [Ask Me Anything: A simple strategy for prompting language models](https://mlsanity.com/paper/2210.02441/) (2022)
- [Compositional Exemplars for In-context Learning](https://mlsanity.com/paper/2302.05698/) (2023)
- [Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango](https://mlsanity.com/paper/2209.07686/) (2022)
- [UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation](https://mlsanity.com/paper/2303.08518/) (2023)
- [Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations](https://mlsanity.com/paper/2212.09865/) (2022)
- [Reordering Examples Helps during Priming-based Few-Shot Learning](https://mlsanity.com/paper/2106.01751/) (2021)
- [Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning](https://mlsanity.com/paper/2205.14704/) (2022)
- [Cross-Lingual Retrieval Augmented Prompt for Low-Resource Languages](https://mlsanity.com/paper/2212.09651/) (2022)
- [Understanding Finetuning for Factual Knowledge Extraction from Language Models](https://mlsanity.com/paper/2301.11293/) (2023)

## Built on by (most cited first, 30 of 72)

- [Efficient Large Language Models: A Survey](https://mlsanity.com/paper/2312.03863/) (2023)
- [A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems](https://mlsanity.com/paper/2504.09037/) (2025)
- [In-Context Learning with Iterative Demonstration Selection](https://mlsanity.com/paper/2310.09881/) (2023)
- [In-context Learning with Retrieved Demonstrations for Language Models: A Survey](https://mlsanity.com/paper/2401.11624/) (2024)
- [Sampling-Efficient Test-Time Scaling: Self-Estimating the Best-of-N Sampling in Early Decoding](https://mlsanity.com/paper/2503.01422/) (2025)
- [Learning to Retrieve In-Context Examples for Large Language Models](https://mlsanity.com/paper/2307.07164/) (2023)
- [Revisiting Demonstration Selection Strategies in In-Context Learning](https://mlsanity.com/paper/2401.12087/) (2024)
- [Large Language Model-Aware In-Context Learning for Code Generation](https://mlsanity.com/paper/2310.09748/) (2023)
- [Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMs](https://mlsanity.com/paper/2507.09477/) (2025)
- [Knowledge Boundary of Large Language Models: A Survey](https://mlsanity.com/paper/2412.12472/) (2024)
- [A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoning](https://mlsanity.com/paper/2501.02497/) (2025)
- [Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Models](https://mlsanity.com/paper/2310.00836/) (2023)
- [INMS: Memory Sharing for Large Language Model based Agents](https://mlsanity.com/paper/2404.09982/) (2024)
- [SurrealDriver: Designing LLM-powered Generative Driver Agent Framework based on Human Drivers' Driving-thinking Data](https://mlsanity.com/paper/2309.13193/) (2023)
- [ReLearn: Unlearning via Learning for Large Language Models](https://mlsanity.com/paper/2502.11190/) (2025)
- [Mitigating Lost-in-Retrieval Problems in Retrieval Augmented Multi-Hop Question Answering](https://mlsanity.com/paper/2502.14245/) (2025)
- [Beyond Static Summarization: Proactive Memory Extraction for LLM Agents](https://mlsanity.com/paper/2601.04463/) (2026)
- [Incremental Learning of Humanoid Robot Behavior from Natural Interaction and Large Language Models](https://mlsanity.com/paper/2309.04316/) (2023)
- [DemoRank: Selecting Effective Demonstrations for Large Language Models in Ranking Task](https://mlsanity.com/paper/2406.16332/) (2024)
- [On the Noise Robustness of In-Context Learning for Text Generation](https://mlsanity.com/paper/2405.17264/) (2024)
- [Cost-Effective In-Context Learning for Entity Resolution: A Design Space Exploration](https://mlsanity.com/paper/2312.03987/) (2023)
- [Assessing Empathy in Large Language Models with Real-World Physician-Patient Interactions](https://mlsanity.com/paper/2405.16402/) (2024)
- [TaskLAMA: Probing the Complex Task Understanding of Language Models](https://mlsanity.com/paper/2308.15299/) (2023)
- [Ambiguity-Aware In-Context Learning with Large Language Models](https://mlsanity.com/paper/2309.07900/) (2023)
- [In-Context Learning Demonstration Selection via Influence Analysis](https://mlsanity.com/paper/2402.11750/) (2024)
- [Is Relevance Propagated from Retriever to Generator in RAG?](https://mlsanity.com/paper/2502.15025/) (2025)
- [MDDial: A Multi-turn Differential Diagnosis Dialogue Dataset with Reliability Evaluation](https://mlsanity.com/paper/2308.08147/) (2023)
- [What Makes In-context Learning Effective for Mathematical Reasoning: A Theoretical Analysis](https://mlsanity.com/paper/2412.12157/) (2024)
- [Don't Take Things Out of Context: Attention Intervention for Enhancing Chain-of-Thought Reasoning in Large Language Models](https://mlsanity.com/paper/2503.11154/) (2025)
- [How Far Can In-Context Alignment Go? Exploring the State of In-Context Alignment](https://mlsanity.com/paper/2406.11474/) (2024)

## Abstract (arXiv)

In-context learning (ICL), teaching a large language model (LLM) to perform a task with few-shot demonstrations rather than adjusting the model parameters, has emerged as a strong paradigm for using LLMs. While early studies primarily used a fixed or random set of demonstrations for all test queries, recent research suggests that retrieving semantically similar demonstrations to the input from a pool of available demonstrations results in better performance. This work expands the applicability of retrieval-based ICL approaches by demonstrating that even simple word-overlap similarity measures such as BM25 outperform randomly selected demonstrations. Furthermore, we extend the success of retrieval-based ICL to instruction-finetuned LLMs as well as Chain-of-Thought (CoT) prompting. For instruction-finetuned LLMs, we find that although a model has already seen the training data at training time, retrieving demonstrations from the training data at test time yields better results compared to using no demonstrations or random demonstrations. Last but not least, we train a task-specific demonstration retriever that outperforms off-the-shelf retrievers.