mlsanityData to 7 Oct 2026

CARROT: A Learned Cost-Constrained Retrieval Optimization System for RAG

arXiv 2411.00744 ↗First posted 1 Nov 2024cs.DB cs.CL cs.IRcited by 5

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated impressive ability in generation and reasoning tasks but struggle with handling up-to-date knowledge, leading to inaccuracies or hallucinations. Retrieval-Augmented Generation (RAG) mitigates this by retrieving and incorporating external knowledge into input prompts. In particular, due to LLMs' context window limitations and long-context hallucinations, only the most relevant "chunks" are retrieved. However, current RAG systems face three key challenges: (1) chunks are often retrieved independently without considering their relationships, such as redundancy and ordering; (2) the utility of chunks is non-monotonic, as adding more chunks can degrade quality; and (3) retrieval strategies fail to adapt to the unique characteristics of different queries. To overcome these challenges, we design a cost-constrained retrieval optimization framework for RAG. We adopt a Monte Carlo Tree Search (MCTS) based strategy to find the optimal chunk combination order, which considers the chunks' correlations. In addition, to address the non-monotonicity of chunk utility, instead of treating budget exhaustion as the termination condition, we design a utility computation strategy to identify the optimal chunk combination without necessarily exhausting the budget. Furthermore, we propose a configuration agent that predicts optimal configurations for each query domain, improving our framework's adaptability and efficiency. Experimental results demonstrate up to a 30% improvement over baseline models, highlighting the framework's effectiveness, scalability, and suitability. Our source code has been released at https://github.com/wang0702/CARROT.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 41 of its 68 references that are in the map. First posted on arXiv 1 Nov 2024.

Builds on

41 of 68 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024CRAG -- Comprehensive RAG Benchmark7 Jun
  2. 2024M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitions26 May

Other references · 39

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  4. 2020Supervised Contrastive Learningcited by 2,803
  5. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  6. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  7. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  8. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  9. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  10. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  11. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  12. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
27 more
  1. 2023Large Language Modelscited by 774
  2. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 519
  3. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
  4. 2023Active Retrieval Augmented Generationcited by 506
  5. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
  6. 2020Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlationcited by 420
  7. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 378
  8. 2024The Faiss librarycited by 370
  9. 2023Large Language Models are Effective Text Rankers with Pairwise Ranking Promptingcited by 302
  10. 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Modelscited by 273
  11. 2024G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answeringcited by 217
  12. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200
  13. 2024Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphscited by 116
  14. 2023Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakescited by 101
  15. 2023On Early Detection of Hallucinations in Factual Question Answeringcited by 54
  16. 2024Evaluating Large Language Models on Time Series Feature Understanding: A Comprehensive Taxonomy and Benchmarkcited by 38
  17. 2024Crafting Personalized Agents through Retrieval-Augmented Generation on Editable Memory Graphscited by 38
  18. 2023Approximation Rate of the Transformer Architecture for Sequence Modelingcited by 28
  19. 2023Lero: A Learning-to-Rank Query Optimizercited by 26
  20. 2024Towards Completeness-Oriented Tool Retrieval for Large Language Modelscited by 26
  21. 2018WikiPassageQA: A Benchmark Collection for Research on Non-factoid Answer Passage Retrievalcited by 19
  22. 2024RichRAG: Crafting Rich Responses for Multi-faceted Queries in Retrieval-Augmented Generationcited by 16
  23. 2024RoarGraph: A Projected Bipartite Graph for Efficient Cross-Modal Approximate Nearest Neighbor Searchcited by 12
  24. 2022DyRRen: A Dynamic Retriever-Reranker-Generator Model for Numerical Reasoning over Tabular and Textual Datacited by 10
  25. 2023Lightweight reranking for language model generationscited by 10
  26. 2024Multimodal Query Suggestion with Multi-Agent Reinforcement Learning from Human Feedbackcited by 9
  27. 2024An Interactive Multi-modal Query Answering System with Retrieval-Augmented Large Language Modelscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

  1. 2025A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Modelscited by 85
  2. 2025Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RLcited by 75
  3. 2025Atom-Searcher: Enhancing Agentic Deep Research via Fine-Grained Atomic Thought Rewardcited by 25
  4. 2025A2Search: Ambiguity-Aware Question Answering with Reinforcement Learningcited by 2
  5. 2026MARVEL: A Multi Agent-based Research Validator and Enabler using Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning
  2. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG
  3. 2024Toward Optimal Search and Retrieval for RAG
  4. 2024CoFE-RAG: A Comprehensive Full-chain Evaluation Framework for Retrieval-Augmented Generation with Enhanced Data Diversity
  5. 2024The Power of Noise: Redefining Retrieval for RAG Systems
  6. 2024Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models
  7. 2024Inference Scaling for Bridging Retrieval and Augmented Generation
  8. 2023Retrieval-Augmented Generation for Large Language Models: A Survey