Visual Orchestration Tax in Agentic VLM Pipelines: Auditing and Certifying Visual Evidence Reuse
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Agentic VLM pipelines increasingly pass the same static visual evidence through multiple specialist agents and tools. This design creates an orchestration-level redundancy mode: semantically unchanged images are repeatedly reconstructed as image-conditioned requests at the VLM API boundary. We call this phenomenon visual orchestration tax and develop a measurement-to-certification framework for visual evidence reuse in agentic VLM pipelines. The audit side defines M1trace to count raw visual-evidence touches and M2 to measure structural touch redundancy, with query-level distributions, bootstrap confidence intervals, and paired quality tests. Across SeeingEye and MAMMQA on chart, document, general-VQA, and multi-modal-QA tasks, audits reveal 66.8-75.6% visual-evidence touch redundancy, and every audited query exceeds the predefined gate. The certification side introduces SharedVisCache, a contract-aware evidence reuse hook keyed by image content, preprocessing fingerprint, and encoder assumptions. On SeeingEye, contract validation certifies 75.0-75.5% repeated touches as reusable while preserving 350/350 output strings and DeltaM5=0. At the physical layer, certified hits reduce Fvision from 800 to 200 in ChartQA-200 trace replay and from 200 to 50 inside live SeeingEye translator-stage physical integration, preserving 800/800 replay strings and 200/200 integrated call outputs. The results position visual reuse as a measurable, behavior-preserving property of agent orchestration and define an agent-layer contract that makes backend prefix or token reuse semantically interpretable.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. First posted on arXiv 6 Oct 2026.
Builds on
No reference list yet.
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026VLM-in-Sandbox: Visual Workspaces for Agentic Visual Reasoning
- 2026VisualLeakBench: Reproducible Action-Boundary Propagation Failures in Vision-Language Agents
- 2026InSight: A Benchmark for Agentic Claim Verification in Interactive Visualizations
- 2026MedOpenClaw and MedFlowBench: Auditing Medical Agents in Full-Study Workflows
- 2026Route, Retrieve, Reflect, Repair: Self-Improving Agentic Framework for Visual Detection and Linguistic Reasoning in Medical Imaging
- 2025Model Context Protocol for Vision Systems: Audit, Security, and Protocol Extensions
- 2026VeriPhy: Agentic Physical Reasoning for World Model Evaluation and Refinement
- 2026LEDGER: Claim-to-Evidence Trace Graphs for Auditing LLM Agents