mlsanityData to 14 Oct 2026

RAG-MCP: Mitigating Prompt Bloat in LLM Tool Selection via Retrieval-Augmented Generation

arXiv 2505.03275 ↗First posted 6 May 2025cs.AI cs.SEcited by 28

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

also in 1 more
Abstract (arXiv)

Large language models (LLMs) struggle to effectively utilize a growing number of external tools, such as those defined by the Model Context Protocol (MCP)citeIntroducingMCP, due to prompt bloat and selection complexity. We introduce RAG-MCP, a Retrieval-Augmented Generation framework that overcomes this challenge by offloading tool discovery. RAG-MCP uses semantic retrieval to identify the most relevant MCP(s) for a given query from an external index before engaging the LLM. Only the selected tool descriptions are passed to the model, drastically reducing prompt size and simplifying decision-making. Experiments, including an MCP stress test, demonstrate RAG-MCP significantly cuts prompt tokens (e.g., by over 50%) and more than triples tool selection accuracy (43.13% vs 13.62% baseline) on benchmark tasks. RAG-MCP enables scalable and accurate tool integration for LLMs.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 6 of its 16 references that are in the map. First posted on arXiv 6 May 2025.

Builds on

6 of 16 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Survey18 Dec

Other references · 5

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  3. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  4. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  5. 2024Enhancing Function-Calling Capabilities in LLMs: Strategies for Prompt Formats, Data Integration, and Multilingual Translationcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 17.

28 papers.

Sort:

By area:

  1. 2025Alita: Generalist Agent Enabling Scalable Agentic Reasoning with Minimal Predefinition and Maximal Self-Evolutioncited by 116
  2. 2025Step-GUI Technical Reportcited by 39
  3. 2025MCP-Zero: Active Tool Discovery for Autonomous LLM Agentscited by 33
  4. 2026More Skills, Worse Agents? Skill Shadowing Degrades Performance When Expanding Skill Librariescited by 12
  5. 2025Code2MCP: Transforming Code Repositories into MCP Servicescited by 10
  6. 2025Automated Composition of Agents: A Knapsack Approach for Agentic Component Selectioncited by 10
  7. 2025Alita-G: Self-Evolving Generative Agent for Agent Generationcited by 10
  8. 2026ToolChoiceConfusion: Causal Minimal Tool Filtering for Reliable LLM Agentscited by 9
  9. 2026How Many Tools Should an LLM Agent See? A Chance-Corrected Answercited by 7
  10. 2025Dynamic ReAct: Scalable Tool Selection for Large-Scale MCP Environmentscited by 6
  11. 2025Agentic Services Computingcited by 6
  12. 2026Capability Minimization as a Safety Primitive: Risk-Aware Causal Gating for Least-Privilege LLM Agentscited by 6
16 more
  1. 2026Self-Improvement of Large Language Models: A Technical Overview and Future Outlookcited by 5
  2. 2025TheMCPCompany: Creating General-purpose Agents with Task-specific Toolscited by 4
  3. 2026ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandboxcited by 4
  4. 2026Optimal Skill Selection for LLM Agents with Provable Bicriteria Guaranteescited by 2
  5. 2026Overwhelmed by Choice: Studying LLM Decision Making at Scalecited by 1
  6. 2025HuggingR4: A Progressive Reasoning Framework for Discovering Optimal Model Companionscited by 0
  7. 2025Z-Space: A Multi-Agent Tool Orchestration Framework for Enterprise-Grade LLM Automationcited by 0
  8. 2026MCP Server Architecture Patterns for LLM-Integrated Applicationscited by 0
  9. 2026Mitigating Errors in LLM-Generated Web API Invocations via Retrieval-Augmented Generation and Constrained Decodingcited by 0
  10. 2026Aleena: Alignment Agent for Research Software Engineering Collaborationscited by 0
  11. 2026Nexus: Depth-Adaptive KV-Cache Splicing and Retrieval-Decoupled Tool Routing for Agentic LLMs on Unified Memorycited by 0
  12. 2026Extracting Knowledge from Tools in LLM Agentscited by 0
  13. 2026Progressive Skill Discovery as Access Control for Tool-Using LLM Agents: Structural Governance through Role-Scoped Capability Deliverycited by 0
  14. 2026AnyAct: Universal Action for Self-Evolving Agentscited by 0
  15. 2026Which Metadata Matters for LLM Tool Selection in MCP? Evidence from Controlled Factor-Level Interventionscited by 0
  16. 2026Google Earth AI Geospatial Reasoning Agent: Capabilities and a Systematic Evaluationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Semantic Tool Discovery for Large Language Models: A Vector-Based Approach to MCP Tool Selection
  2. 2023Context Tuning for Retrieval Augmented Generation
  3. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG
  4. 2026Hybrid Semantic Tool Discovery for Enterprise MCP Gateway: Architecture and Implementation
  5. 2025MCP-RADAR: A Multi-Dimensional Benchmark for Evaluating Tool Use Capabilities in Large Language Models
  6. 2024Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach
  7. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wisely
  8. 2026PORTS: Preference-Optimized Retrievers for Tool Selection with Large Language Models