mlsanityData to 14 Oct 2026

Learning to Rank for Multiple Retrieval-Augmented Models through Iterative Utility Maximization

arXiv 2410.09942 ↗First posted 13 Oct 2024cs.CL cs.IRcited by 14

On the map: Retrieval-augmented generation → Agentic RAG, agents and memory

Abstract (arXiv)

This paper investigates the design of a unified search engine to serve multiple retrieval-augmented generation (RAG) agents, each with a distinct task, backbone large language model (LLM), and RAG strategy. We introduce an iterative approach where the search engine generates retrieval results for the RAG agents and gathers feedback on the quality of the retrieved documents during an offline phase. This feedback is then used to iteratively optimize the search engine using an expectation-maximization algorithm, with the goal of maximizing each agent's utility function. Additionally, we adapt this to an online setting, allowing the search engine to refine its behavior based on real-time individual agents feedback to better serve the results for each of them. Experiments on datasets from the Knowledge-Intensive Language Tasks (KILT) benchmark demonstrates that our approach significantly on average outperforms baselines across 18 RAG models. We demonstrate that our method effectively ``personalizes'' the retrieval for each RAG agent based on the collected feedback. Finally, we provide a comprehensive ablation study to explore various aspects of our method.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation. Builds on 45 of its 64 references that are in the map. First posted on arXiv 13 Oct 2024.

Builds on

45 of 64 references.

References · 45

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  9. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  10. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  11. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  12. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
33 more
  1. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  2. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  3. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  4. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  5. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  6. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 692
  7. 2019Passage Re-ranking with BERTcited by 645
  8. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 573
  9. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
  10. 2023LaMP: When Large Language Models Meet Personalizationcited by 437
  11. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 426
  12. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 349
  13. 2018The Natural Language Decathlon: Multitask Learning as Question Answeringcited by 236
  14. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  15. 2022MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Textcited by 220
  16. 2018Online Learning: A Comprehensive Surveycited by 205
  17. 2016Unbiased Learning-to-Rank with Biased Feedbackcited by 157
  18. 2021End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answeringcited by 133
  19. 2022Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answeringcited by 127
  20. 2021KAT: A Knowledge Augmented Transformer for Vision-and-Languagecited by 126
  21. 2024Optimization Methods for Personalizing Large Language Models through Retrieval Augmentationcited by 99
  22. 2023Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Surveycited by 92
  23. 2024Evaluating Retrieval Quality in Retrieval-Augmented Generationcited by 90
  24. 2021End-to-End Training of Neural Retrievers for Open-Domain Question Answeringcited by 75
  25. 2022Retrieval-Enhanced Machine Learningcited by 47
  26. 2020Is Retriever Merely an Approximator of Reader?cited by 30
  27. 2024Stochastic RAG: End-to-End Retrieval-Augmented Generation through Expected Utility Maximizationcited by 29
  28. 2024Towards a Search Engine for Machines: Unified Ranking for Multiple Retrieval-Augmented Large Language Modelscited by 23
  29. 2023A Symmetric Dual Encoding Dense Retrieval Framework for Knowledge-Intensive Visual Question Answeringcited by 14
  30. 2024Large Language Models vs. Search Engines: Evaluating User Preferences Across Varied Information Retrieval Scenarioscited by 14
  31. 2024Retrieval-Enhanced Machine Learning: Synthesis and Opportunitiescited by 11
  32. 2023Pre-Training Multi-Modal Dense Retrievers for Outside-Knowledge Visual Question Answeringcited by 10
  33. 2023End-to-End Training of Neural Networks for Automotive Radar Interference Mitigationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 9; 2026: 4.

14 papers.

Sort:

By area:

  1. 2025Improving Retrieval-Augmented Generation through Multi-Agent Reinforcement Learningcited by 39
  2. 2025Personalized Text Generation with Contrastive Activation Steeringcited by 34
  3. 2025A Survey of Personalization: From RAG to Agentcited by 27
  4. 2025ExPerT: Effective and Explainable Evaluation of Personalized Long-Form Text Generationcited by 26
  5. 2025Towards AI Search Paradigmcited by 19
  6. 2025LLM-Based Multi-Agent Blackboard System for Information Discovery in Data Sciencecited by 15
  7. 2024Accelerating Retrieval-Augmented Generationcited by 14
  8. 2025LLM-Specific Utility for Retrieval-Augmented Generationcited by 4
  9. 2025CIIR@LiveRAG 2025: Optimizing Multi-Agent Retrieval Augmented Generation through Self-Trainingcited by 2
  10. 2026Improving User Privacy in Personalized Generation: Client-Side Retrieval-Augmented Modification of Server-Side Generated Speculationscited by 1
  11. 2025Plan-and-Refine: Diverse and Comprehensive Retrieval-Augmented Generationcited by 0
  12. 2026From Profiles to Steering Vectors: Global Sparse Priors and Local Semantic Calibration for Personalized Text Generationcited by 0
2 more
  1. 2026Preference Is Not Intervention: The Structure and Stability Boundaries of Reader-Specific Evidence Utilitycited by 0
  2. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MAIN-RAG: Multi-Agent Filtering Retrieval-Augmented Generation
  2. 2026A-RAG: Scaling Agentic Retrieval-Augmented Generation via Hierarchical Retrieval Interfaces
  3. 2025DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generation
  4. 2024A Collaborative Multi-Agent Approach to Retrieval-Augmented Generation Across Diverse Data
  5. 2026Agent-Orchestrated Adaptive RAG: A Comparative Study on Structured and Multi-Hop Retrieval
  6. 2025Towards Adaptive Memory-Based Optimization for Enhanced Retrieval-Augmented Generation
  7. 2024RAG Playground: A Framework for Systematic Evaluation of Retrieval Strategies and Prompt Engineering in RAG Systems
  8. 2025Direct Retrieval-augmented Optimization: Synergizing Knowledge Selection and Language Models