mlsanityData to 14 Oct 2026

Evaluation Report on MCP Servers

arXiv 2504.11094 ↗First posted 15 Apr 2025cs.IR cs.DBcited by 9

Not on the map. Why?

Abstract (arXiv)

With the rise of LLMs, a large number of Model Context Protocol (MCP) services have emerged since the end of 2024. However, the effectiveness and efficiency of MCP servers have not been well studied. To study these questions, we propose an evaluation framework, called MCPBench. We selected several widely used MCP server and conducted an experimental evaluation on their accuracy, time, and token usage. Our experiments showed that the most effective MCP, Bing Web Search, achieved an accuracy of 64%. Importantly, we found that the accuracy of MCP servers can be substantially enhanced by involving declarative interface. This research paves the way for further investigations into optimized MCP implementations, ultimately leading to better AI-driven applications and data retrieval solutions.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). First posted on arXiv 15 Apr 2025.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 2.

9 papers.

  1. 2025MCP-Zero: Active Tool Discovery for Autonomous LLM Agentscited by 33
  2. 2025MCPToolBench++: A Large Scale AI Agent Model Context Protocol MCP Tool Use Benchmarkcited by 32
  3. 2025MCP-AgentBench: Evaluating Real-World Language Agent Performance with MCP-Mediated Toolscited by 32
  4. 2025LiveMCP-101: Stress Testing and Diagnosing MCP-enabled Agents on Challenging Queriescited by 17
  5. 2025MCP-Flow: Facilitating LLM Agents to Master Real-World, Diverse and Scaling MCP Toolscited by 11
  6. 2025TheMCPCompany: Creating General-purpose Agents with Task-specific Toolscited by 4
  7. 2025Help or Hurdle? Rethinking Model Context Protocol-Augmented Large Language Modelscited by 3
  8. 2026AgentSelect: Benchmark for Narrative Query-to-Agent Recommendationcited by 3
  9. 2026Understanding How Enterprises Adopt the Model Context Protocol for LLM-Driven Software Engineeringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025LiveMCPBench: Can Agents Navigate an Ocean of MCP Tools?
  2. 2025A Measurement Study of Model Context Protocol Ecosystem
  3. 2026MCP Server Architecture Patterns for LLM-Integrated Applications
  4. 2025MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Servers
  5. 2025MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Servers
  6. 2025Network and Systems Performance Characterization of MCP-Enabled LLM Agents
  7. 2025MCPZoo: A Large-Scale Dataset of Runnable Model Context Protocol Servers for AI Agent
  8. 2025HumanMCP: A Human-Like Query Dataset for Evaluating MCP Tool Retrieval Performance