Model Context Protocol for Vision Systems: Audit, Security, and Protocol Extensions
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
The Model Context Protocol (MCP) defines a schema bound execution model for agent-tool interaction, enabling modular computer vision workflows without retraining. To our knowledge, this is the first protocol level, deployment scale audit of MCP in vision systems, identifying systemic weaknesses in schema semantics, interoperability, and runtime coordination. We analyze 91 publicly registered vision centric MCP servers, annotated along nine dimensions of compositional fidelity, and develop an executable benchmark with validators to detect and categorize protocol violations. The audit reveals high prevalence of schema format divergence, missing runtime schema validation, undeclared coordinate conventions, and reliance on untracked bridging scripts. Validator based testing quantifies these failures, with schema format checks flagging misalignments in 78.0 percent of systems, coordinate convention checks detecting spatial reference errors in 24.6 percent, and memory scope checks issuing an average of 33.8 warnings per 100 executions. Security probes show that dynamic and multi agent workflows exhibit elevated risks of privilege escalation and untyped tool connections. The proposed benchmark and validator suite, implemented in a controlled testbed and to be released on GitHub, establishes a reproducible framework for measuring and improving the reliability and security of compositional vision workflows.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 17 of its 36 references that are in the map. First posted on arXiv 26 Sep 2025.
Builds on
17 of 36 references.
Closest ancestors · 2
Same direction, newest first
- 2025A Large-Scale Evolvable Dataset for Model Context Protocol Ecosystem and Security Analysis30 Jun
- 2025We Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent Systems16 Jun
Other references · 15
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
- 2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 206
- 2023LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agentscited by 193
- 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 146
- 2025Magma: A Foundation Model for Multimodal AI Agentscited by 139
- 2021Seeing biodiversity: perspectives in machine learning for wildlife conservationcited by 130
- 2025Alita: Generalist Agent Enabling Scalable Agentic Reasoning with Minimal Predefinition and Maximal Self-Evolutioncited by 116
- 2023Enhancing Multimodal Compositional Reasoning of Visual Language Models with Generative Negative Miningcited by 25
- 2025Iterative Tool Usage Exploration for Multimodal Agents via Step-wise Preference Tuningcited by 19
- 2025We Urgently Need Privilege Management in MCP: A Measurement of API Usage in MCP Ecosystemscited by 15
- 2024VISION: A Modular AI Assistant for Natural Human-Instrument Interaction at Scientific User Facilitiescited by 10
3 more
- 2024Prompting Large Vision-Language Models for Compositional Reasoningcited by 6
- 2025Enhancing Clinical Decision Support and EHR Insights through LLMs and the Model Context Protocol: An Open-Source MCP-FHIR Frameworkcited by 5
- 2024Enhancing Compositional Generalization via Compositional Feature Alignmentcited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directions
- 2026Rethinking MCP Security: A Large-Scale Study of Runtime MCP Servers and Security Scanner Reliability
- 2026From Tool Connection to Execution Control: Benchmarking Security Invariants in MCP-Style Agent Runtimes
- 2026From Tool Orchestration to Code Execution: A Study of MCP Design Choices
- 2026A Taxonomy of Runtime Faults in Model Context Protocol Servers
- 2026MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Servers
- 2025MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploits
- 2026Real Faults in Model Context Protocol (MCP) Software: a Comprehensive Taxonomy