mlsanityData to 7 Oct 2026

Learning to Retrieve Iteratively for In-Context Learning

arXiv 2406.14739 ↗First posted 20 Jun 2024cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

We introduce iterative retrieval, a novel framework that empowers retrievers to make iterative decisions through policy optimization. Finding an optimal portfolio of retrieved items is a combinatorial optimization problem, generally considered NP-hard. This approach provides a learned approximation to such a solution, meeting specific task requirements under a given family of large language models (LLMs). We propose a training procedure based on reinforcement learning, incorporating feedback from LLMs. We instantiate an iterative retriever for composing in-context learning (ICL) exemplars and apply it to various semantic parsing tasks that demand synthesized programs as outputs. By adding only 4M additional parameters for state encoding, we convert an off-the-shelf dense retriever into a stateful iterative retriever, outperforming previous methods in selecting ICL exemplars on semantic parsing datasets such as CalFlow, TreeDST, and MTOP. Additionally, the trained iterative retriever generalizes across different inference LLMs beyond the one used during training.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 31 of its 39 references that are in the map. First posted on arXiv 20 Jun 2024.

Builds on

31 of 39 references.

References · 31

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  7. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  8. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  9. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  10. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  11. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  12. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
19 more
  1. 2020On Layer Normalization in the Transformer Architecturecited by 906
  2. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  3. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  4. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  5. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 347
  6. 2022Active Example Selection for In-Context Learningcited by 195
  7. 2021Constrained Language Models Yield Few-Shot Semantic Parserscited by 162
  8. 2020MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmarkcited by 156
  9. 2023Compositional Exemplars for In-context Learningcited by 143
  10. 2022Diverse Demonstrations Improve In-context Compositional Generalizationcited by 136
  11. 2020Task-Oriented Dialogue as Dataflow Synthesiscited by 122
  12. 2021Few-Shot Semantic Parsing with Language Models Trained On Codecited by 50
  13. 2024The N+ Implementation Details of RLHF with PPO: A Case Study on TL;DR Summarizationcited by 49
  14. 2020Conversational Semantic Parsing for Dialog State Trackingcited by 40
  15. 2022Unobserved Local Structures Make Compositional Generalization Hardcited by 29
  16. 2021Finding needles in a haystack: Sampling Structurally-diverse Training Sets from Synthetic Data for Compositional Generalizationcited by 27
  17. 2022BenchCLAMP: A Benchmark for Evaluating Language Models on Syntactic and Semantic Parsingcited by 12
  18. 2023A Unified View of Evaluation Metrics for Structured Predictioncited by 11
  19. 2022Structurally Diverse Sampling for Sample-Efficient Training and Comprehensive Evaluationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 11; 2026: 1.

13 papers.

Sort:

By area:

  1. 2025TACO: Enhancing Multimodal In-context Learning via Task Mapping-Guided Sequence Configurationcited by 23
  2. 2024VideoICL: Confidence-based Iterative In-context Learning for Out-of-Distribution Video Understandingcited by 17
  3. 2025Optimizing Decomposition for Optimal Claim Verificationcited by 10
  4. 2025When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness Methods for LLMscited by 10
  5. 2025Improving Large Language Model Planning with Action Sequence Similaritycited by 7
  6. 2025Learning to Select In-Context Demonstration Preferred by Large Language Modelcited by 5
  7. 2025Topic Coverage-based Demonstration Retrieval for In-Context Learningcited by 3
  8. 2025Bridging Information Gaps with Comprehensive Answers: Improving the Diversity and Informativeness of Follow-Up Questionscited by 2
  9. 2025Reinforcing Compositional Retrieval: Retrieving Step-by-Step for Composing Informative Contextscited by 1
  10. 2025Foundational Design Principles and Patterns for Building Robust and Adaptive GenAI-Native Systemscited by 1
  11. 2025Think First, Assign Next (ThiFAN-VQA): A Two-stage Chain-of-Thought Framework for Post-Disaster Damage Assessmentcited by 1
  12. 2026Structured Semantic Information Helps Retrieve Better Examples for In-Context Learning Applied to Few-Shot Relation Extractioncited by 1
1 more
  1. 2025DecoupleSearch: Decouple Planning and Search via Hierarchical Reward Modelingcited by 0