mlsanityData to 14 Oct 2026

If you're waiting for a sign... that might not be it! Mitigating Trust Boundary Confusion from Visual Injections on Vision-Language Agentic Systems

arXiv 2604.19844 ↗First posted 21 Apr 2026cs.CV cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Recent advances in embodied Vision-Language Agentic Systems (VLAS), powered by large vision-language models (LVLMs), enable AI systems to perceive and reason over real-world scenes. Within this context, environmental signals such as traffic lights are essential in-band signals that can and should influence agent behavior. However, similar signals could also be crafted to operate as misleading visual injections, overriding user intent and posing security risks. This duality creates a fundamental challenge: agents must respond to legitimate environmental cues while remaining robust to misleading ones. We refer to this tension as trust boundary confusion. To study this behavior, we design a dual-intent dataset and evaluation framework, through which we show that current LVLM-based agents fail to reliably balance this trade-off, either ignoring useful signals or following harmful ones. We systematically evaluate 7 LVLM agents across multiple embodied settings under both structure-based and noise-based visual injections. To address these vulnerabilities, we propose a multi-agent defense framework that separates perception from decision-making to dynamically assess the reliability of visual inputs. Our approach significantly reduces misleading behaviors while preserving correct responses and provides robustness guarantees under adversarial perturbations. The code of the evaluation framework and artifacts are made available at https://anonymous.4open.science/r/Visual-Prompt-Inject.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 27 of its 43 references that are in the map. First posted on arXiv 21 Apr 2026.

Builds on

27 of 43 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024SceneTAP: Scene-Coherent Typographic Adversarial Planner against Vision-Language Models in Real-World Environments28 Nov
  2. 2024Multimodal Situational Safety8 Oct
  3. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov

Other references · 24

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  3. 2023Qwen Technical Reportcited by 3,344
  4. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  5. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  6. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  7. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
  8. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
  9. 2017Countering Adversarial Images using Input Transformationscited by 520
  10. 2022Visual Programming: Compositional visual reasoning without trainingcited by 516
  11. 2017Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networkscited by 486
  12. 2022Diffusion Models for Adversarial Purificationcited by 476
12 more
  1. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  2. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 364
  3. 2020A Self-supervised Approach for Adversarial Robustnesscited by 180
  4. 2024Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Makingcited by 144
  5. 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attackscited by 143
  6. 2024Dissecting Adversarial Robustness of Multimodal LM Agentscited by 120
  7. 2024A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systemscited by 77
  8. 2024BadRobot: Jailbreaking Embodied LLM Agents in the Physical Worldcited by 43
  9. 2025CHAI: Command Hijacking against embodied AIcited by 11
  10. 2025Benchmarking Vision-Language Models on Optical Character Recognition in Dynamic Video Environmentscited by 10
  11. 2023Agent as Cerebrum, Controller as Cerebellum: Implementing an Embodied LMM-based Agent on Dronescited by 9
  12. 2025Exploring Potential Prompt Injection Attacks in Federated Military LLMs and Their Mitigationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Security of Foundation-Model-Powered Embodied Agents: Attack Surfaces, Attacks, Defenses, and Evaluationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026TrustVLA: Mechanism-Guided Inference-Time Defense Against Vision-Language-Action Backdoors
  2. 2025Cognitive Inception: Agentic Reasoning against Visual Deceptions by Injecting Skepticism
  3. 2025Agentic Moderation: Multi-Agent Design for Safer Vision-Language Models
  4. 2025Seeing is Believing? Evaluating Vision-Language Model Susceptibility in Agent-to-Agent Multimodal Persuasion
  5. 2026Bad Seeing or Bad Thinking? Rewarding Perception for Multimodal Reasoning
  6. 2025BEAT: Visual Backdoor Attacks on VLM-based Embodied Agents via Contrastive Trigger Learning
  7. 2025Multimodal Safety Evaluation in Generative Agent Social Simulations
  8. 2026Beyond Attack Success Rate: Examining Trigger Leakage in Vision-Language Agentic Systems