mlsanityData to 14 Oct 2026

Dr.ICL: Demonstration-Retrieved In-context Learning

arXiv 2305.14128 ↗First posted 23 May 2023cs.CL cs.AIcited by 72

Not on the map. Why?

Abstract (arXiv)

In-context learning (ICL), teaching a large language model (LLM) to perform a task with few-shot demonstrations rather than adjusting the model parameters, has emerged as a strong paradigm for using LLMs. While early studies primarily used a fixed or random set of demonstrations for all test queries, recent research suggests that retrieving semantically similar demonstrations to the input from a pool of available demonstrations results in better performance. This work expands the applicability of retrieval-based ICL approaches by demonstrating that even simple word-overlap similarity measures such as BM25 outperform randomly selected demonstrations. Furthermore, we extend the success of retrieval-based ICL to instruction-finetuned LLMs as well as Chain-of-Thought (CoT) prompting. For instruction-finetuned LLMs, we find that although a model has already seen the training data at training time, retrieving demonstrations from the training data at test time yields better results compared to using no demonstrations or random demonstrations. Last but not least, we train a task-specific demonstration retriever that outperforms off-the-shelf retrievers.

Key facts. Cited by 72 papers in the map (data through 14 Oct 2026). Builds on 28 of its 34 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

28 of 34 references.

References · 28

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  5. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  6. 2022Emergent Abilities of Large Language Modelscited by 2,512
  7. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  8. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  9. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  10. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  11. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  12. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
16 more
  1. 2023In-Context Retrieval-Augmented Language Modelscited by 702
  2. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  3. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  4. 2021Large Dual Encoders Are Generalizable Retrieverscited by 456
  5. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 388
  6. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matterscited by 222
  7. 2022Demystifying Prompts in Language Models via Perplexity Estimationcited by 216
  8. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  9. 2023Compositional Exemplars for In-context Learningcited by 144
  10. 2022Text and Patterns: For Effective Chain of Thought, It Takes Two to Tangocited by 128
  11. 2023UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluationcited by 75
  12. 2022Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrationscited by 70
  13. 2021Reordering Examples Helps during Priming-based Few-Shot Learningcited by 50
  14. 2022Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learningcited by 36
  15. 2022Cross-Lingual Retrieval Augmented Prompt for Low-Resource Languagescited by 17
  16. 2023Understanding Finetuning for Factual Knowledge Extraction from Language Modelscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 14; 2024: 23; 2025: 25; 2026: 10.

72 papers.

Sort:

By area:

  1. 2023Efficient Large Language Models: A Surveycited by 186
  2. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 111
  3. 2023In-Context Learning with Iterative Demonstration Selectioncited by 74
  4. 2024In-context Learning with Retrieved Demonstrations for Language Models: A Surveycited by 70
  5. 2025Sampling-Efficient Test-Time Scaling: Self-Estimating the Best-of-N Sampling in Early Decodingcited by 63
  6. 2023Learning to Retrieve In-Context Examples for Large Language Modelscited by 60
  7. 2024Revisiting Demonstration Selection Strategies in In-Context Learningcited by 58
  8. 2023Large Language Model-Aware In-Context Learning for Code Generationcited by 43
  9. 2025Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMscited by 42
  10. 2024Knowledge Boundary of Large Language Models: A Surveycited by 40
  11. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  12. 2023Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Modelscited by 35
38 more
  1. 2024INMS: Memory Sharing for Large Language Model based Agentscited by 31
  2. 2023SurrealDriver: Designing LLM-powered Generative Driver Agent Framework based on Human Drivers' Driving-thinking Datacited by 24
  3. 2025ReLearn: Unlearning via Learning for Large Language Modelscited by 22
  4. 2025Mitigating Lost-in-Retrieval Problems in Retrieval Augmented Multi-Hop Question Answeringcited by 22
  5. 2026Beyond Static Summarization: Proactive Memory Extraction for LLM Agentscited by 21
  6. 2023Incremental Learning of Humanoid Robot Behavior from Natural Interaction and Large Language Modelscited by 19
  7. 2024DemoRank: Selecting Effective Demonstrations for Large Language Models in Ranking Taskcited by 17
  8. 2024On the Noise Robustness of In-Context Learning for Text Generationcited by 16
  9. 2023Cost-Effective In-Context Learning for Entity Resolution: A Design Space Explorationcited by 15
  10. 2024Assessing Empathy in Large Language Models with Real-World Physician-Patient Interactionscited by 13
  11. 2023TaskLAMA: Probing the Complex Task Understanding of Language Modelscited by 12
  12. 2023Ambiguity-Aware In-Context Learning with Large Language Modelscited by 11
  13. 2024In-Context Learning Demonstration Selection via Influence Analysiscited by 11
  14. 2025Is Relevance Propagated from Retriever to Generator in RAG?cited by 11
  15. 2023MDDial: A Multi-turn Differential Diagnosis Dialogue Dataset with Reliability Evaluationcited by 10
  16. 2024What Makes In-context Learning Effective for Mathematical Reasoning: A Theoretical Analysiscited by 7
  17. 2025Don't Take Things Out of Context: Attention Intervention for Enhancing Chain-of-Thought Reasoning in Large Language Modelscited by 7
  18. 2024How Far Can In-Context Alignment Go? Exploring the State of In-Context Alignmentcited by 6
  19. 2024Prompting with Phonemes: Enhancing LLMs' Multilinguality for Non-Latin Script Languagescited by 6
  20. 2025The Role of Diversity in In-Context Learning for Large Language Modelscited by 6
  21. 2025Relic: Enhancing Reward Model Generalization for Low-Resource Indic Languages with Few-Shot Examplescited by 6
  22. 2023ICXML: An In-Context Learning Framework for Zero-Shot Extreme Multi-Label Classificationcited by 5
  23. 2024Automatic Combination of Sample Selection Strategies for Few-Shot Learningcited by 5
  24. 2024XAMPLER: Learning to Retrieve Cross-Lingual In-Context Examplescited by 5
  25. 2024PromptRefine: Enhancing Few-Shot Performance on Low-Resource Indic Languages with Example Selection from Related Example Bankscited by 5
  26. 2025Learning to Select In-Context Demonstration Preferred by Large Language Modelcited by 5
  27. 2024One size doesn't fit all: Predicting the Number of Examples for In-Context Learningcited by 4
  28. 2024Self-Demos: Eliciting Out-of-Demonstration Generalizability in Large Language Modelscited by 4
  29. 2024Evaluating and Safeguarding the Adversarial Robustness of Retrieval-Based In-Context Learningcited by 4
  30. 2024DemoShapley: Valuation of Demonstrations for In-Context Learningcited by 3
  31. 2025Selecting Demonstrations for Many-Shot In-Context Learning via Gradient Matchingcited by 3
  32. 2023Improving Input-label Mapping with Demonstration Replay for In-context Learningcited by 2
  33. 2023Take One Step at a Time to Know Incremental Utility of Demonstration: An Analysis on Reranking for Few-Shot In-Context Learningcited by 2
  34. 2024Instruction Tuning with Retrieval-based Examples Ranking for Aspect-based Sentiment Analysiscited by 2
  35. 2025How Jailbreak Defenses Work and Ensemble? A Mechanistic Investigationcited by 2
  36. 2025MateICL: Mitigating Attention Dispersion in Large-Scale In-Context Learningcited by 2
  37. 2025Fine-Grained Emotion Recognition via In-Context Learningcited by 2
  38. 2024ParaICL: Towards Parallel In-Context Learningcited by 1

50 most cited of 72 All 72 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Does In-Context Learning Really Learn? Rethinking How Large Language Models Respond and Solve Tasks via In-Context Learning
  2. 2026Demonstrations, CoT, and Prompting: A Theoretical Analysis of ICL
  3. 2025Refract ICL: Rethinking Example Selection in the Era of Million-Token Models
  4. 2024Large Language Models Know What Makes Exemplary Contexts
  5. 2025On the Relationship Between the Choice of Representation and In-Context Learning
  6. 2023Self-ICL: Zero-Shot In-Context Learning with Self-Generated Demonstrations
  7. 2023How Many Demonstrations Do You Need for In-context Learning?
  8. 2024Demonstration Notebook: Finding the Most Suited In-Context Learning Example from Interactions