mlsanityData to 7 Oct 2026

Unified Demonstration Retriever for In-Context Learning

arXiv 2305.04320 ↗First posted 7 May 2023cs.CLcited by 141

Not on the map. Why?

Abstract (arXiv)

In-context learning is a new learning paradigm where a language model conditions on a few input-output pairs (demonstrations) and a test input, and directly outputs the prediction. It has been shown highly dependent on the provided demonstrations and thus promotes the research of demonstration retrieval: given a test input, relevant examples are retrieved from the training set to serve as informative demonstrations for in-context learning. While previous works focus on training task-specific retrievers for several tasks separately, these methods are often hard to transfer and scale on various tasks, and separately trained retrievers incur a lot of parameter storage and deployment cost. In this paper, we propose Unified Demonstration Retriever (UDR), a single model to retrieve demonstrations for a wide range of tasks. To train UDR, we cast various tasks' training signals into a unified list-wise ranking formulation by language model's feedback. Then we propose a multi-task list-wise ranking training framework, with an iterative mining strategy to find high-quality candidates, which can help UDR fully incorporate various tasks' signals. Experiments on 30+ tasks across 13 task families and multiple data domains show that UDR significantly outperforms baselines. Further analyses show the effectiveness of each proposed component and UDR's strong ability in various scenarios including different LMs (1.3B - 175B), unseen datasets, varying demonstration quantities, etc.

Key facts. Cited by 141 papers in the map (data through 7 Oct 2026). Builds on 44 of its 71 references that are in the map. First posted on arXiv 7 May 2023.

Builds on

44 of 71 references.

References · 44

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  8. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  9. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  10. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  11. 2017Billion-scale similarity search with GPUscited by 2,477
  12. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
32 more
  1. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  2. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  3. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  4. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  5. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  6. 2015Teaching Machines to Read and Comprehendcited by 1,039
  7. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  8. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  9. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  10. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 477
  11. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 413
  12. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
  13. 2019Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoningcited by 281
  14. 2022Active Example Selection for In-Context Learningcited by 195
  15. 2022In-context Examples Selection for Machine Translationcited by 172
  16. 2021Noisy Channel Language Model Prompting for Few-Shot Text Classificationcited by 163
  17. 2020MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmarkcited by 156
  18. 2022Synchromesh: Reliable code generation from pre-trained language modelscited by 156
  19. 2020DART: Open-Domain Structured Data Record to Text Generationcited by 144
  20. 2020Break It Down: A Question Understanding Benchmarkcited by 131
  21. 2020Task-Oriented Dialogue as Dataflow Synthesiscited by 122
  22. 2021Case-based Reasoning for Natural Language Queries over Knowledge Basescited by 107
  23. 2020Neural CRF Model for Sentence Alignment in Text Simplificationcited by 106
  24. 2023Finding Support Examples for In-Context Learningcited by 104
  25. 2018Abstractive Summarization of Reddit Posts with Multi-level Memory Networkscited by 102
  26. 2022On the Relation between Sensitivity and Accuracy in In-context Learningcited by 71
  27. 2019Semantic Noise Matters for Neural Natural Language Generationcited by 64
  28. 2023kNN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inferencecited by 47
  29. 2019Does It Make Sense? And Why? A Pilot Study for Sense Making and Explanationcited by 43
  30. 2022XRICL: Cross-lingual Retrieval-Augmented In-Context Learning for Cross-lingual Text-to-SQL Semantic Parsingcited by 32
  31. 2023Improving Contrastive Learning of Sentence Embeddings from AI Feedbackcited by 29
  32. 2022Beyond Opinion Mining: Summarizing Opinions of Customer Reviewscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 18; 2024: 62; 2025: 41; 2026: 20.

141 papers.

Sort:

By area:

  1. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  2. 2023Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learningcited by 235
  3. 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 150
  4. 2023Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communicationcited by 78
  5. 2023In-Context Learning with Iterative Demonstration Selectioncited by 74
  6. 2024In-context Learning with Retrieved Demonstrations for Language Models: A Surveycited by 69
  7. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 65
  8. 2023Learning to Retrieve In-Context Examples for Large Language Modelscited by 60
  9. 2024StreamBench: Towards Benchmarking Continuous Improvement of Language Agentscited by 44
  10. 2023PromptCBLUE: A Chinese Prompt Tuning Benchmark for the Medical Domaincited by 40
  11. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  12. 2023MoT: Memory-of-Thought Enables ChatGPT to Self-Improvecited by 37
38 more
  1. 2024Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplarscited by 37
  2. 2024ALoRA: Allocating Low-Rank Adaptation for Fine-tuning Large Language Modelscited by 36
  3. 2024Knowledge Boundary of Large Language Models: A Surveycited by 36
  4. 2024Efficient Prompt Optimization Through the Lens of Best Arm Identificationcited by 35
  5. 2025Emergent Abilities in Large Language Models: A Surveycited by 34
  6. 2023Prompt Optimization via Adversarial In-Context Learningcited by 33
  7. 2024Compromising Embodied Agents with Contextual Backdoor Attackscited by 29
  8. 2024CoMT: A Novel Benchmark for Chain of Multi-modal Thought on Large Vision-Language Modelscited by 29
  9. 2024PICLe: Eliciting Diverse Behaviors from Large Language Models with Persona In-Context Learningcited by 28
  10. 2024Let's Ask GNN: Empowering Large Language Model for Graph In-Context Learningcited by 21
  11. 2024MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuningcited by 19
  12. 2024RealCompo: Balancing Realism and Compositionality Improves Text-to-Image Diffusion Modelscited by 18
  13. 2025Make LVLMs Focus: Context-Aware Attention Modulation for Better Multimodal In-Context Learningcited by 18
  14. 2024DemoRank: Selecting Effective Demonstrations for Large Language Models in Ranking Taskcited by 17
  15. 2025GraphEval: A Lightweight Graph-Based LLM Framework for Idea Evaluationcited by 17
  16. 2024LLM-R2: A Large Language Model Enhanced Rule-based Rewrite System for Boosting Query Efficiencycited by 16
  17. 2024TRACE the Evidence: Constructing Knowledge-Grounded Reasoning Chains for Retrieval-Augmented Generationcited by 16
  18. 2024Retrieval-style In-Context Learning for Few-shot Hierarchical Text Classificationcited by 16
  19. 2023Cost-Effective In-Context Learning for Entity Resolution: A Design Space Explorationcited by 15
  20. 2024On the Noise Robustness of In-Context Learning for Text Generationcited by 15
  21. 2025AMAS: Adaptively Determining Communication Topology for LLM-based Multi-Agent Systemcited by 15
  22. 2023Overview of the PromptCBLUE Shared Task in CHIP2023cited by 14
  23. 2024Misconfidence-based Demonstration Selection for LLM In-Context Learningcited by 14
  24. 2023More Samples or More Prompts? Exploring Effective In-Context Sampling for LLM Few-Shot Prompt Engineeringcited by 13
  25. 2024The Impact of Demonstrations on Multilingual In-Context Learning: A Multidimensional Analysiscited by 13
  26. 2024Unearthing Large Scale Domain-Specific Knowledge from Public Corporacited by 12
  27. 2024Visual Prompt Selection for In-Context Learning Segmentationcited by 12
  28. 2025Mimic In-Context Learning for Multimodal Taskscited by 12
  29. 2023Ambiguity-Aware In-Context Learning with Large Language Modelscited by 11
  30. 2024In-Context Learning Demonstration Selection via Influence Analysiscited by 11
  31. 2024MINERS: Multilingual Language Models as Semantic Retrieverscited by 11
  32. 2024Improving Tool Retrieval by Leveraging Large Language Models for Query Generationcited by 11
  33. 2023Leveraging Code to Improve In-context Learning for Semantic Parsingcited by 10
  34. 2024NICE: To Optimize In-Context Examples or Not?cited by 9
  35. 2024Unraveling the Mechanics of Learning-Based Demonstration Selection for In-Context Learningcited by 9
  36. 2025PARA: Parameter-Efficient Fine-tuning with Prompt Aware Representation Adjustmentcited by 9
  37. 2024Se2: Sequential Example Selection for In-Context Learningcited by 8
  38. 2024PEDRO: Parameter-Efficient Fine-tuning with Prompt DEpenDent Representation MOdificationcited by 8

50 most cited of 141 All 141 (JSON)