mlsanityData to 7 Oct 2026

AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting

arXiv 2403.09513 ↗First posted 14 Mar 2024cs.CR cs.AIcited by 113

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

With the advent and widespread deployment of Multimodal Large Language Models (MLLMs), the imperative to ensure their safety has become increasingly pronounced. However, with the integration of additional modalities, MLLMs are exposed to new vulnerabilities, rendering them prone to structured-based jailbreak attacks, where semantic content (e.g., "harmful text") has been injected into the images to mislead MLLMs. In this work, we aim to defend against such threats. Specifically, we propose Adaptive Shield Prompting (AdaShield), which prepends inputs with defense prompts to defend MLLMs against structure-based jailbreak attacks without fine-tuning MLLMs or training additional modules (e.g., post-stage content detector). Initially, we present a manually designed static defense prompt, which thoroughly examines the image and instruction content step by step and specifies response methods to malicious queries. Furthermore, we introduce an adaptive auto-refinement framework, consisting of a target MLLM and a LLM-based defense prompt generator (Defender). These components collaboratively and iteratively communicate to generate a defense prompt. Extensive experiments on the popular structure-based jailbreak attacks and benign datasets show that our methods can consistently improve MLLMs' robustness against structure-based jailbreak attacks without compromising the model's general capabilities evaluated on standard benign tasks. Our code is available at https://github.com/rain305f/AdaShield.

Key facts. Cited by 113 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 52 of its 70 references that are in the map. First posted on arXiv 14 Mar 2024.

Builds on

52 of 70 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast13 Feb
  2. 2024Safety of Multimodal Large Language Models on Images and Texts1 Feb
  3. 2024Red Teaming Visual Language Models23 Jan
  4. 2024MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance5 Jan
  5. 2023OT-Attack: Enhancing Adversarial Transferability of Vision-Language Models via Optimal Transport Optimization7 Dec
  6. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models29 Nov
  7. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov
  8. 2023How Robust is Google's Bard to Adversarial Image Attacks?21 Sep
  9. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models26 Jul
  10. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun

Other references · 42

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  4. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  5. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  6. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  7. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  8. 2023A Survey on Multimodal Large Language Modelscited by 898
  9. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 891
  10. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  11. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  12. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
30 more
  1. 2023A Comprehensive Overview of Large Language Modelscited by 639
  2. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 617
  3. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
  4. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 545
  5. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  6. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 404
  7. 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 366
  8. 2024A Survey on Hallucination in Large Vision-Language Modelscited by 352
  9. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  10. 2023Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4Vcited by 291
  11. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 277
  12. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Modelscited by 223
  13. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
  14. 2022Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editingcited by 165
  15. 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Modelscited by 160
  16. 2023Silkie: Preference Distillation for Large Visual Language Modelscited by 128
  17. 2024Jailbreaking Attack against Multimodal Large Language Modelcited by 128
  18. 2024Attacks, Defenses and Evaluations for LLM Conversation Safety: A Surveycited by 124
  19. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 123
  20. 2023Skywork: A More Open Bilingual Foundation Modelcited by 110
  21. 2024On Prompt-Driven Safeguarding for Large Language Modelscited by 109
  22. 2023DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedbackcited by 99
  23. 2023How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defensescited by 65
  24. 2023A Challenger to GPT-4V? Early Explorations of Gemini in Visual Expertisecited by 49
  25. 2023JailGuard: A Universal Detection Framework for LLM Prompt-based Attackscited by 46
  26. 2023Chain of Thought Prompt Tuning in Vision Language Modelscited by 34
  27. 2023Large Language Models as Automated Aligners for benchmarking Vision-Language Modelscited by 10
  28. 2024Exploring the Limits of Zero Shot Vision Language Models for Hate Meme Detection: The Vulnerabilities and their Interpretationscited by 10
  29. 2023GPT-4V(ision) as A Social Media Analysis Enginecited by 6
  30. 2024PuriDefense: Randomized Local Implicit Adversarial Purification for Defending Black-box Query-based Attackscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 21; 2025: 53; 2026: 39.

113 papers.

Sort:

By area:

  1. 2025GuardReasoner: Towards Reasoning-based LLM Safeguardscited by 89
  2. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
  3. 2024FlipAttack: Jailbreak LLMs via Flippingcited by 73
  4. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  5. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 66
  6. 2024Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Charactercited by 61
  7. 2024ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Timecited by 57
  8. 2024Jailbreak Large Vision-Language Models Through Multi-Modal Linkagecited by 53
  9. 2024Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreakscited by 44
  10. 2024Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignmentcited by 42
  11. 2025VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agentscited by 42
  12. 2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoningcited by 38
38 more
  1. 2024IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselvescited by 34
  2. 2025Backdoor Cleaning without External Guidance in MLLM Fine-tuningcited by 34
  3. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 31
  4. 2024JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Modelscited by 28
  5. 2024Cross-modality Information Check for Detecting Jailbreaking in Multimodal Large Language Modelscited by 24
  6. 2024Adversarial Attacks of Vision Tasks in the Past 10 Years: A Surveycited by 22
  7. 2024The VLLM Safety Paradox: Dual Ease in Jailbreak Attack and Defensecited by 20
  8. 2024Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Securitycited by 18
  9. 2025Understanding and Rectifying Safety Perception Distortion in VLMscited by 17
  10. 2025Analyzing Finetuning Representation Shift for Multimodal LLMs Steeringcited by 15
  11. 2025Mirage in the Eyes: Hallucination Attack on Multi-modal Large Language Models with Only Attention Sinkcited by 15
  12. 2025FC-Attack: Jailbreaking Multimodal Large Language Models via Auto-Generated Flowchartscited by 15
  13. 2025Doxing via the Lens: Revealing Location-related Privacy Leakage on Multi-modal Large Reasoning Modelscited by 14
  14. 2024Securing Vision-Language Models with a Robust Encoder Against Jailbreak and Adversarial Attackscited by 13
  15. 2025Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Modelscited by 13
  16. 2025Learning to Steer: Input-dependent Steering for Multimodal LLMscited by 13
  17. 2025Attention! Your Vision Language Model Could Be Maliciously Manipulatedcited by 12
  18. 2025JALMBench: Benchmarking Jailbreak Vulnerabilities in Audio Language Modelscited by 11
  19. 2025SafeGRPO: Self-Rewarded Multimodal Safety Alignment via Rule-Governed Policy Optimizationcited by 11
  20. 2024SATA: A Paradigm for LLM Jailbreak via Simple Assistive Task Linkagecited by 10
  21. 2025JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Modelcited by 10
  22. 2025JPS: Jailbreak Multimodal Large Language Models with Collaborative Visual Perturbation and Textual Steeringcited by 10
  23. 2025OmniSafeBench-MM: A Unified Benchmark and Toolbox for Multimodal Jailbreak Attack-Defense Evaluationcited by 10
  24. 2025Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models?cited by 9
  25. 2025SafePTR: Token-Level Jailbreak Defense in Multimodal LLMs via Prune-then-Restore Mechanismcited by 9
  26. 2025Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoringcited by 9
  27. 2025SaFeR-VLM: Toward Safety-aware Fine-grained Reasoning in Multimodal Modelscited by 8
  28. 2024Defending LVLMs Against Vision Attacks through Partial-Perception Supervisioncited by 7
  29. 2025DAVSP: Safety Alignment for Large Vision-Language Models via Deep Aligned Visual Safety Promptcited by 7
  30. 2024Zer0-Jack: A Memory-efficient Gradient-based Jailbreaking Method for Black-box Multi-modal Large Language Modelscited by 6
  31. 2025HoliSafe: Holistic Safety Benchmarking and Modeling for Vision-Language Modelcited by 6
  32. 2025Steering Multimodal Large Language Models Decoding for Context-Aware Safetycited by 6
  33. 2025Implicit Jailbreak Attacks via Cross-Modal Information Concealment on Vision-Language Modelscited by 5
  34. 2025Seeing the Threat: Vulnerabilities in Vision-Language Models to Adversarial Attackcited by 5
  35. 2025Activation Steering Meets Preference Optimization: Defense Against Jailbreaks in Vision Language Modelscited by 5
  36. 2025Environmental Injection Attacks against GUI Agents in Realistic Dynamic Environmentscited by 5
  37. 2025SHIELD: Classifier-Guided Prompting for Robust and Safer LVLMscited by 5
  38. 2026Evolving Contextual Safety in Multi-Modal Large Language Models via Inference-Time Self-Reflective Memorycited by 5

50 most cited of 113 All 113 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency
  2. 2024textitMMJ-Bench: A Comprehensive Study on Jailbreak Attacks and Defenses for Multimodal Large Language Models
  3. 2025Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models
  4. 2026SafeSteer: A Decoding-level Defense Mechanism for Multimodal Large Language Models
  5. 2025Towards Robust Multimodal Large Language Models Against Jailbreak Attacks
  6. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing
  7. 2026Multi-Turn Adaptive Prompting Attack on Large Vision-Language Models
  8. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation