# Help or Hurdle? Rethinking Model Context Protocol-Augmented Large Language Models

arXiv: https://arxiv.org/abs/2508.12566 · HTML: https://mlsanity.com/paper/2508.12566/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 9 of its 14 references that are in the map.
- First posted on arXiv 18 Aug 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [Tool learning and function calling](https://mlsanity.com/agents/tool-learning-and-function-calling/)

## Builds on (most cited first, 9 of 9)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models](https://mlsanity.com/paper/2304.06364/) (2023)
- [Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directions](https://mlsanity.com/paper/2503.23278/) (2025)
- [MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploits](https://mlsanity.com/paper/2504.03767/) (2025)
- [MCP-RADAR: A Multi-Dimensional Benchmark for Evaluating Tool Use Capabilities in Large Language Models](https://mlsanity.com/paper/2505.16700/) (2025)
- [Enterprise-Grade Security for the Model Context Protocol (MCP): Frameworks and Mitigation Strategies](https://mlsanity.com/paper/2504.08623/) (2025)
- [Advancing Multi-Agent Systems Through Model Context Protocol: Architecture, Implementation, and Applications](https://mlsanity.com/paper/2504.21030/) (2025)
- [Beyond the Protocol: Unveiling Attack Vectors in the Model Context Protocol (MCP) Ecosystem](https://mlsanity.com/paper/2506.02040/) (2025)
- [Evaluation Report on MCP Servers](https://mlsanity.com/paper/2504.11094/) (2025)

## Built on by (most cited first, 3 of 3)

- [SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?](https://mlsanity.com/paper/2603.15401/) (2026)
- [GTM: Simulating the World of Tools for AI Agents](https://mlsanity.com/paper/2512.04535/) (2025)
- [MCP-GRANITE Benchmark: GRANularity Interface TEsting for MCP-Based LLM Agents](https://mlsanity.com/paper/2609.24161/) (2026)

## Abstract (arXiv)

The Model Context Protocol (MCP) enables large language models (LLMs) to access external resources on demand. While commonly assumed to enhance performance, how LLMs actually leverage this capability remains poorly understood. We introduce MCPGAUGE, the first comprehensive evaluation framework for probing LLM-MCP interactions along four key dimensions: proactivity (self-initiated tool use), compliance (adherence to tool-use instructions), effectiveness (task performance post-integration), and overhead (computational cost incurred). MCPGAUGE comprises a 160-prompt suite and 25 datasets spanning knowledge comprehension, general reasoning, and code generation. Our large-scale evaluation, spanning six commercial LLMs, 30 MCP tool suites, and both one- and two-turn interaction settings, comprises around 20,000 API calls and over USD 6,000 in computational cost. This comprehensive study reveals four key findings that challenge prevailing assumptions about the effectiveness of MCP integration. These insights highlight critical limitations in current AI-tool integration and position MCPGAUGE as a principled benchmark for advancing controllable, tool-augmented LLMs.