mlsanityData to 14 Oct 2026

Beyond the Verdict: Evidence-Aligned Evaluation of Visual Prompt-Injection Guardrails

arXiv 2609.05535 ↗First posted 2 Sep 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Verdict-only evaluation does not reveal whether a vision-language model (VLM) used the visual evidence that should support its decision. We study this problem in web-agent guardrails, where a VLM judges whether on-screen text conflicts with a user instruction. We introduce Mind2Web-Injection, a benchmark of 9,954 instruction-screenshot pairs with instruction-relative labels, pixel-exact evidence boxes, and matched image-side counterfactuals. Across six VLMs, two models with nearly identical average precision differ ninefold in Evidence-Aligned Detection (EAD), the fraction of attacks both detected and correctly localized. To test whether a verdict depends on the command cited as evidence, we replace the instruction with one that endorses that command. Qwen3-VL-32B, the strongest open-weight localizer, returns aligned in only 58.7% of cases, whereas GPT-5.6-luna does so in 99.9%. To diagnose these failures, we propose two training-free interventions. ReadGate improves grounding without changing verdicts, while CmdCompare tests whether explicit instruction-command comparison resolves instruction-side inconsistency. These results motivate reporting verdict correctness, evidence localization, and counterfactual responsiveness separately.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 37 of its 40 references that are in the map. First posted on arXiv 2 Sep 2026.

Builds on

37 of 40 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Vision-LLMs Can Fool Themselves with Self-Generated Typographic Attacks1 Feb
  2. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov
  3. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun

Other references · 34

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2025Qwen3-VL Technical Reportcited by 2,793
  3. 2025Gemma 3 Technical Reportcited by 1,824
  4. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  5. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  6. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  7. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  8. 2023CogAgent: A Visual Language Model for GUI Agentscited by 704
  9. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 599
  10. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 567
  11. 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 563
  12. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 538
22 more
  1. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 530
  2. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 375
  3. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 364
  4. 2026Gemma 4 Technical Reportcited by 259
  5. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 172
  6. 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attackscited by 157
  7. 2024EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakagecited by 150
  8. 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attackscited by 143
  9. 2024ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Explorationcited by 103
  10. 2025PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Modelcited by 89
  11. 2024Get my drift? Catching LLM Task Drift with Activation Deltascited by 72
  12. 2024Contrastive Region Guidance: Improving Grounding in Vision-Language Models without Trainingcited by 64
  13. 2025WebInject: Prompt Injection Attack to Web Agentscited by 51
  14. 2025WAInjectBench: Benchmarking Prompt Injection Detections for Web Agentscited by 24
  15. 2025How Not to Detect Prompt Injections with an LLMcited by 12
  16. 2025Explanation-Driven Counterfactual Testing for Faithfulness in Vision-Language Model Explanationscited by 12
  17. 2026WebSentinel: Detecting and Localizing Prompt Injection Attacks for Web Agentscited by 11
  18. 2025Evaluating Reasoning Faithfulness in Medical Vision-Language Models using Multimodal Perturbationscited by 9
  19. 2026AlignSentinel: Alignment-Aware Detection of Prompt Injection Attackscited by 5
  20. 2026SnapGuard: Lightweight Prompt Injection Detection for Screenshot-Based Web Agentscited by 4
  21. 2026VISTAQA: Benchmarking Joint Visual Question Answering and Pixel-Level Evidencecited by 4
  22. 2026WebAgentGuard: A Reasoning-Driven Guard Model for Detecting Prompt Injection Attacks in Web Agentscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026One Perturbation, Two Failure Modes: Probing VLM Safety via Embedding-Guided Typographic Perturbations
  2. 2026Scenes as an Attack Surface: Uncovering Scene-Grounded Safety Failures in MLLMs
  3. 2024ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time
  4. 2025TAIJI: Textual Anchoring for Immunizing Jailbreak Images in Vision Language Models
  5. 2026Render Before Reading: Visual Rendering as a Prompt Injection Defense
  6. 2026GuardAlign: Test-time Safety Alignment in Multimodal Large Language Models
  7. 2025CAPTURE: Context-Aware Prompt Injection Testing and Robustness Enhancement
  8. 2026VisualLeakBench: Reproducible Action-Boundary Propagation Failures in Vision-Language Agents