mlsanityData to 14 Oct 2026

JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Model

arXiv 2504.03770 ↗First posted 3 Apr 2025cs.CR cs.AIcited by 10

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Multimodal large language models (MLLMs) excel in vision-language tasks but also pose significant risks of generating harmful content, particularly through jailbreak attacks. Jailbreak attacks refer to intentional manipulations that bypass safety mechanisms in models, leading to the generation of inappropriate or unsafe content. Detecting such attacks is critical to ensuring the responsible deployment of MLLMs. Existing jailbreak detection methods face three primary challenges: (1) Many rely on model hidden states or gradients, limiting their applicability to white-box models, where the internal workings of the model are accessible; (2) They involve high computational overhead from uncertainty-based analysis, which limits real-time detection, and (3) They require fully labeled harmful datasets, which are often scarce in real-world settings. To address these issues, we introduce a test-time adaptive framework called JAILDAM. Our method leverages a memory-based approach guided by policy-driven unsafe knowledge representations, eliminating the need for explicit exposure to harmful data. By dynamically updating unsafe knowledge during test-time, our framework improves generalization to unseen jailbreak strategies while maintaining efficiency. Experiments on multiple VLM jailbreak benchmarks demonstrate that JAILDAM delivers state-of-the-art performance in harmful content detection, improving both accuracy and speed.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 39 of its 46 references that are in the map. First posted on arXiv 3 Apr 2025.

Builds on

39 of 46 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025HiddenDetect: Detecting Jailbreak Attacks against Large Vision-Language Models via Monitoring Hidden States20 Feb
  2. 2025Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency9 Jan
  3. 2024Effective and Efficient Adversarial Detection for Vision-Language Models via A Single Vector30 Oct
  4. 2024MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models11 Jun
  5. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting14 Mar
  6. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models29 Nov

Other references · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  4. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  5. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  6. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  7. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  8. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Modelscited by 382
  9. 2023Label-Free Concept Bottleneck Modelscited by 285
  10. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 230
  11. 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Modelscited by 167
  12. 2023Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuningcited by 154
21 more
  1. 2024Efficient Test-Time Adaptation of Vision-Language Modelscited by 141
  2. 2024Efficient Adversarial Training in LLMs with Continuous Attackscited by 113
  3. 2024GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysiscited by 96
  4. 2024Robust LLM safeguarding via refusal feature adversarial trainingcited by 64
  5. 2023JailGuard: A Universal Detection Framework for LLM Prompt-based Attackscited by 50
  6. 2024Exploring Diverse Methods in Visual Question Answeringcited by 45
  7. 2023Panoptic Scene Graph Generation with Semantics-Prototype Learningcited by 42
  8. 2024InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Modelscited by 42
  9. 2024AutoTrust: Benchmarking Trustworthiness in Large Vision Language Models for Autonomous Drivingcited by 35
  10. 2024AI Safety in Generative AI Large Language Models: A Surveycited by 33
  11. 2025Fraud-R1 : A Multi-Round Benchmark for Assessing the Robustness of LLM Against Augmented Fraud and Phishing Inducementscited by 32
  12. 2024DPU: Dynamic Prototype Updating for Multimodal Out-of-Distribution Detectioncited by 29
  13. 2023Improving Image Captioning Descriptiveness by Ranking and LLM-based Fusioncited by 21
  14. 2024Personalized Multimodal Large Language Models: A Surveycited by 19
  15. 2025Secure On-Device Video OOD Detection Without Backpropagationcited by 19
  16. 2024MetaOOD: Automatic Selection of OOD Detection Modelscited by 16
  17. 2025UniGuardian: A Unified Defense for Detecting Prompt Injection, Backdoor Attacks and Adversarial Attacks in Large Language Modelscited by 16
  18. 2023Temporal Coherent Test-Time Optimization for Robust Video Classificationcited by 15
  19. 2025Can Multimodal LLMs Perform Time Series Anomaly Detection?cited by 12
  20. 2024Rationale-guided Prompting for Knowledge-based Visual Question Answeringcited by 9
  21. 2024CMOOD: Concept-based Multi-label OOD Detectioncited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 10.

10 papers.

  1. 2026PROGRESSLM: Towards Progress Reasoning in Vision-Language Modelscited by 15
  2. 2026FORTIS: Benchmarking Over-Privilege in Agent Skillscited by 13
  3. 2026Multi-User Large Language Model Agentscited by 7
  4. 2026Visual Exclusivity Attacks: Automatic Multimodal Red Teaming via Agentic Planningcited by 4
  5. 2026Benchmarking and Improving Monitors for Out-Of-Distribution Alignment Failure in LLMscited by 4
  6. 2026Principled Steering via Null-space Projection for Jailbreak Defense in Vision-Language Modelscited by 2
  7. 2026Safe responses matter: Output-aware safety guardrail mitigate over-refusal in MLLMscited by 2
  8. 2026AISPA: User-Centric System Prompt Auditing for Large Language Model Applicationscited by 0
  9. 2026ThinkingGuard: Decoding Implicit Hazards via Step-by-Step Risk Attribution in Multimodal Large Language Modelscited by 0
  10. 2026Backdoor in the Loop: Compromising Agentic Search via Malicious Retrieverscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning to Detect Unseen Jailbreak Attacks in Large Vision-Language Models
  2. 2025Learning to Detect Unknown Jailbreak Attacks in Large Vision-Language Models
  3. 2026Fully Unleashing the Multimodal Attacker: Meta-Adaptive Jailbreaking of Vision-Language Models
  4. 2026Understanding and Defending VLM Jailbreaks via Jailbreak-Related Representation Shift
  5. 2024textitMMJ-Bench: A Comprehensive Study on Jailbreak Attacks and Defenses for Multimodal Large Language Models
  6. 2026SALLIE: Generation-Free Hidden-State Detection of Jailbreaks and Prompt Injections Across Text and Vision
  7. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks
  8. 2025JaiLIP: Jailbreaking Vision-Language Models via Loss Guided Image Perturbation