mlsanityData to 14 Oct 2026

Safeguarding Vision-Language Models Against Patched Visual Prompt Injectors

arXiv 2405.10529 ↗First posted 17 May 2024cs.CV cs.AIcited by 27

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Large language models have become increasingly prominent, also signaling a shift towards multimodality as the next frontier in artificial intelligence, where their embeddings are harnessed as prompts to generate textual content. Vision-language models (VLMs) stand at the forefront of this advancement, offering innovative ways to combine visual and textual data for enhanced understanding and interaction. However, this integration also enlarges the attack surface. Patch-based adversarial attack is considered the most realistic threat model in physical vision applications, as demonstrated in many existing literature. In this paper, we propose to address patched visual prompt injection, where adversaries exploit adversarial patches to generate target content in VLMs. Our investigation reveals that patched adversarial prompts exhibit sensitivity to pixel-wise randomization, a trait that remains robust even against adaptive attacks designed to counteract such defenses. Leveraging this insight, we introduce SmoothVLM, a defense mechanism rooted in smoothing techniques, specifically tailored to protect VLMs from the threat of patched visual prompt injectors. Our framework significantly lowers the attack success rate to a range between 0% and 5.0% on two leading VLMs, while achieving around 67.3% to 95.0% context recovery of the benign images, demonstrating a balance between security and usability.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 28 of its 34 references that are in the map. First posted on arXiv 17 May 2024.

Builds on

28 of 34 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models26 Jul

Other references · 27

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  6. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  7. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  8. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
  9. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  10. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 776
  11. 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
  12. 2018Object Detection with Deep Learning: A Reviewcited by 528
15 more
  1. 2017Adversarial Patchcited by 517
  2. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 469
  3. 2023Pretraining Language Models with Human Preferencescited by 247
  4. 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 208
  5. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 172
  6. 2024Optimization-based Prompt Injection Attack to LLM-as-a-Judgecited by 143
  7. 2023PaLI-3 Vision Language Models: Smaller, Faster, Strongercited by 126
  8. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 120
  9. 2021PatchCleanser: Certifiably Robust Defense against Adversarial Patches for Any Image Classifiercited by 65
  10. 2023Robust Evaluation of Diffusion-Based Adversarial Purificationcited by 65
  11. 2022DensePure: Understanding Diffusion Models towards Adversarial Robustnesscited by 33
  12. 2021Towards Defending against Adversarial Examples via Attack-Invariant Featurescited by 25
  13. 2023DiffSmooth: Certifiably Robust Learning via Diffusion Models and Local Smoothingcited by 20
  14. 2023PatchCURE: Improving Certifiable Robustness, Model Utility, and Computation Efficiency of Adversarial Patch Defensescited by 11
  15. 2023Defending Against Physical Adversarial Patch Attacks on Infrared Human Detectioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 14; 2026: 9.

27 papers.

Sort:

By area:

  1. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 67
  2. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 67
  3. 2025VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agentscited by 45
  4. 2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoningcited by 42
  5. 2025Membership Inference Attacks Against Vision-Language Modelscited by 29
  6. 2024MirrorCheck: Efficient Adversarial Defense for Vision-Language Modelscited by 19
  7. 2025Poisoned-MRAG: Knowledge Poisoning Attacks to Multimodal Retrieval Augmented Generationcited by 12
  8. 2026Image-based Prompt Injection: Hijacking Multimodal LLMs through Visually Embedded Adversarial Instructionscited by 8
  9. 2024Zer0-Jack: A Memory-efficient Gradient-based Jailbreaking Method for Black-box Multi-modal Large Language Modelscited by 7
  10. 2024Defending LVLMs Against Vision Attacks through Partial-Perception Supervisioncited by 7
  11. 2025Seeing the Threat: Vulnerabilities in Vision-Language Models to Adversarial Attackcited by 5
  12. 2025SHIELD: Classifier-Guided Prompting for Robust and Safer LVLMscited by 5
15 more
  1. 2024Seeing is Deceiving: Exploitation of Visual Pathways in Multi-Modal Language Modelscited by 4
  2. 2026SnapGuard: Lightweight Prompt Injection Detection for Screenshot-Based Web Agentscited by 4
  3. 2025Safeguarding Vision-Language Models: Mitigating Vulnerabilities to Gaussian Noise in Perturbation-based Attackscited by 3
  4. 2026Concept-Based Dictionary Learning for Inference-Time Safety in Vision Language Action Modelscited by 3
  5. 2026WebAgentGuard: A Reasoning-Driven Guard Model for Detecting Prompt Injection Attacks in Web Agentscited by 3
  6. 2025SmoothGuard: Defending Multimodal Large Language Models with Noise Perturbation and Clustering Aggregationcited by 2
  7. 2025VSF-Med:A Vulnerability Scoring Framework for Medical Vision-Language Modelscited by 1
  8. 2025From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Modelscited by 1
  9. 2025SGM: Safety Glasses for Multimodal Large Language Models via Neuron-Level Detoxificationcited by 1
  10. 2026Extended to Reality: Prompt Injection in 3D Environmentscited by 1
  11. 2026VALD: Multi-Stage Vision Attack Detection for Efficient LVLM Defensecited by 1
  12. 2025Advancing Vision-based Human Action Recognition: Exploring Vision-Language CLIP Model for Generalisation in Domain-Independent Taskscited by 0
  13. 2026A Systematic Study of Cross-Modal Typographic Attacks on Audio-Visual Reasoningcited by 0
  14. 2026A Cross-Modal Prompt Injection Attack against Large Vision-Language Models with Image-Only Perturbationcited by 0
  15. 2026Devil in the Lens: Analyzing and Defending Physical Prompt Injection Against Vision-Language Models on Wearable Devicescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Patch is Enough: Naturalistic Adversarial Patch against Vision-Language Pre-training Models
  2. 2026Breaking the weakest link to evade vision language models
  3. 2024Towards Adversarially Robust Vision-Language Models: Insights from Design Choices and Prompt Formatting Techniques
  4. 2025Attention-Guided Patch-Wise Sparse Adversarial Attacks on Vision-Language-Action Models
  5. 2025Model-agnostic Adversarial Attack and Defense for Vision-Language-Action Models
  6. 2026Detect and Suppress: A Mechanistic Defense against Adversarial Patches in VLA Models
  7. 2025Transferable Adversarial Attacks on Black-Box Vision-Language Models
  8. 2024One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Models