VisualLeakBench: Reproducible Action-Boundary Propagation Failures in Vision-Language Agents
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Vision-language agents increasingly consume screenshots, documents, and user interfaces before writing to memory, sending messages, or invoking external tools. We study a concrete failure mode in this setting: action-boundary propagation, where sensitive or unsafe visible text is copied from an image into downstream tool arguments. We present VisualLeakBench, a diversified 500-image benchmark spanning UI, chat, document, form, and dashboard scenes, and evaluate a stratified 100-image agent subset with four production VLM systems under two workflows: note capture and external handoff. At baseline, target strings are propagated into tool arguments in 78.8% of PII cases and 85.5% of rendered unsafe-text cases. Under a defensive system prompt, rendered unsafe-text propagation remains high at 52.6%, while PII tool propagation falls to 2.0%, largely by suppressing tool use rather than preserving utility. Rates are tool-surface dependent: search-like tools suppress PII propagation, but rendered unsafe text still crosses tool boundaries. We measure visual-to-tool propagation rather than downstream instruction execution. We additionally provide a labeled-target oracle upper-bound diagnostic that localizes most failures at the tool boundary while leaving response-side leakage as residual risk.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 23 of its 31 references that are in the map. First posted on arXiv 29 May 2026.
Builds on
23 of 31 references.
References · 23
- 2020Extracting Training Data from Large Language Modelscited by 2,144
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
- 2023GAIA: a benchmark for General AI Assistantscited by 1,084
- 2022Quantifying Memorization Across Neural Language Modelscited by 755
- 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 514
- 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
- 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 368
- 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 328
- 2023Visual Adversarial Examples Jailbreak Aligned Large Language Modelscited by 307
- 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
11 more
- 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 268
- 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Modelscited by 255
- 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attackscited by 208
- 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
- 2024Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Modelscited by 121
- 2024MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Modelscited by 51
- 2023PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Modelscited by 35
- 2026Enhancing Financial Report Question-Answering: A Retrieval-Augmented Generation System with Reranking Analysiscited by 20
- 2025AutoNeural: Co-Designing Vision-Language Models for NPU Inferencecited by 17
- 2026Sustainable Hybrid Document-Routed Retrieval for Financial RAG: Resolving the Robustness-Precision Trade-offcited by 13
- 2026VisualLeakBench: Auditing the Fragility of Large Vision-Language Models against PII Leakage and Social Engineeringcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026MM-ToolSandBox: A Unified Framework for Evaluating Visual Tool-Calling Agents
- 2026Adaptive Vision-Language Model Routing for Computer Use Agents
- 2026One Perturbation, Two Failure Modes: Probing VLM Safety via Embedding-Guided Typographic Perturbations
- 2026MLLMs Fail to Refuse when Using Tools Agentically
- 2025RedDiffuser: Auditing Multimodal Safety Failures in Vision-Language Models via Reinforced Diffusion
- 2026CUAAudit: Meta-Evaluation of Vision-Language Models as Auditors of Autonomous Computer-Use Agents
- 2026Beyond Attack Success Rate: Examining Trigger Leakage in Vision-Language Agentic Systems
- 2026Paint What You See: Benchmarking Dexterous Visual Tool Use in Multimodal Agents