mlsanityData to 7 Oct 2026

HiddenDetect: Detecting Jailbreak Attacks against Large Vision-Language Models via Monitoring Hidden States

arXiv 2502.14744 ↗First posted 20 Feb 2025cs.CLcited by 29

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

The integration of additional modalities increases the susceptibility of large vision-language models (LVLMs) to safety risks, such as jailbreak attacks, compared to their language-only counterparts. While existing research primarily focuses on post-hoc alignment techniques, the underlying safety mechanisms within LVLMs remain largely unexplored. In this work , we investigate whether LVLMs inherently encode safety-relevant signals within their internal activations during inference. Our findings reveal that LVLMs exhibit distinct activation patterns when processing unsafe prompts, which can be leveraged to detect and mitigate adversarial inputs without requiring extensive fine-tuning. Building on this insight, we introduce HiddenDetect, a novel tuning-free framework that harnesses internal model activations to enhance safety. Experimental results show that {HiddenDetect} surpasses state-of-the-art methods in detecting jailbreak attacks against LVLMs. By utilizing intrinsic safety-aware patterns, our method provides an efficient and scalable solution for strengthening LVLM robustness against multimodal threats. Our code will be released publicly at https://github.com/leigest519/HiddenDetect.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 43 of its 52 references that are in the map. First posted on arXiv 20 Feb 2025.

Builds on

43 of 52 references.

Closest ancestors · 16

Same direction, newest first

  1. 2024RapGuard: Safeguarding Multimodal Large Language Models via Rationale-aware Defensive Prompting25 Dec
  2. 2024Cross-Modal Safety Mechanism Transfer in Large Vision-Language Models16 Oct
  3. 2024VLMGuard: Bootstrapping Malicious Prompt Detectors from Unlabeled Vision-Language Prompts in the Wild1 Oct
  4. 2024Cross-modality Information Check for Detecting Jailbreaking in Multimodal Large Language Models31 Jul
  5. 2024MirrorCheck: Efficient Adversarial Defense for Vision-Language Models13 Jun
  6. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks3 Apr
  7. 2024Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models14 Mar
  8. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation14 Mar
  9. 2024Safety of Multimodal Large Language Models on Images and Texts1 Feb
  10. 2024MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance5 Jan
  11. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models29 Nov
  12. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov
4 more
  1. 2023How Robust is Google's Bard to Adversarial Image Attacks?21 Sep
  2. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models26 Jul
  3. 2023Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs19 Jul
  4. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun

Other references · 27

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  7. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  8. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  9. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  10. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  11. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 545
  12. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
15 more
  1. 2023Detecting Language Model Attacks with Perplexitycited by 411
  2. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  3. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  4. 2024Safety Layers in Aligned Large Language Models: The Key to LLM Securitycited by 126
  5. 2024On Prompt-Driven Safeguarding for Large Language Modelscited by 109
  6. 2023How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMscited by 103
  7. 2023DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedbackcited by 99
  8. 2024GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysiscited by 87
  9. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editingcited by 77
  10. 2024Cross-modal Information Flow in Multimodal Large Language Modelscited by 65
  11. 2023Jailbreaking GPT-4V via Self-Adversarial Attacks with System Promptscited by 62
  12. 2023JailGuard: A Universal Detection Framework for LLM Prompt-based Attackscited by 46
  13. 2024SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steeringcited by 45
  14. 2023Misusing Tools in Large Language Models With Visual Adversarial Examplescited by 40
  15. 2024Unveiling the Backbone-Optimizer Coupling Bias in Visual Representation Learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 15; 2026: 14.

29 papers.

Sort:

By area:

  1. 2025VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agentscited by 42
  2. 2025MAGIC-VQA: Multimodal And Grounded Inference with Commonsense Knowledge for Visual Question Answeringcited by 18
  3. 2025JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Modelcited by 10
  4. 2025USB: A Comprehensive and Unified Safety Evaluation Benchmark for Multimodal Large Language Modelscited by 10
  5. 2025OmniSafeBench-MM: A Unified Benchmark and Toolbox for Multimodal Jailbreak Attack-Defense Evaluationcited by 10
  6. 2025Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoringcited by 9
  7. 2025Beyond Safe Answers: A Benchmark for Evaluating True Risk Awareness in Large Reasoning Modelscited by 7
  8. 2025MSR-Align: Policy-Grounded Multimodal Alignment for Safety-Aware Reasoning in Vision-Language Modelscited by 7
  9. 2025Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasetscited by 6
  10. 2025The Impact of Off-Policy Training Data on Probe Generalisationcited by 6
  11. 2025CrossGuard: Safeguarding MLLMs against Joint-Modal Implicit Malicious Attackscited by 4
  12. 2026ALERT: Zero-shot LLM Jailbreak Detection via Internal Discrepancy Amplificationcited by 3
17 more
  1. 2026Safe responses matter: Output-aware safety guardrail mitigate over-refusal in MLLMscited by 2
  2. 2025Dynamic Token Reweighting for Robust Vision-Language Modelscited by 1
  3. 2025Learning to Detect Unseen Jailbreak Attacks in Large Vision-Language Modelscited by 1
  4. 2025Learning to Detect Unknown Jailbreak Attacks in Large Vision-Language Modelscited by 1
  5. 2026Sparse Autoencoders as Plug-and-Play Firewalls for Adversarial Attack Detection in VLMscited by 1
  6. 2025Defending Large Language Models Against Jailbreak Exploits with Responsible AI Considerationscited by 0
  7. 2026CoT is Not the Chain of Truth: An Empirical Internal Analysis of Reasoning LLMs for Fake News Generationcited by 0
  8. 2026Detecting Jailbreak Attempts in Clinical Training LLMs Through Automated Linguistic Feature Extractioncited by 0
  9. 2026Adaptively Robust LLM Monitoring via Activation Watermarkingcited by 0
  10. 2026SALLIE: Generation-Free Hidden-State Detection of Jailbreaks and Prompt Injections Across Text and Visioncited by 0
  11. 2026StructBreak: Structural Cognitive Overload-Induced Safety Failures in MLLMscited by 0
  12. 2026DE-FIVE: Detecting Malicious Image Prompts via Fourier Features and Image Vector Embeddingscited by 0
  13. 2026Breaking Refusal in the First Half: A Mechanistic Study of the Prefill Jailbreakcited by 0
  14. 2026V-DEAL: Diagnosing Video Safety De-Calibration as an Understanding-Refusal Coupling Failurecited by 0
  15. 2026A Cross-Architecture Audit of Direction-Based Inference-Time Defences in Vision-Language Modelscited by 0
  16. 2026Measuring the Wrong Thing: Internal Harmfulness Scores Anti-Rank Successful Jailbreakscited by 0
  17. 2026ForeSight: Enhancing Risk Monitoring via Early Safety Signal Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024The Great Contradiction Showdown: How Jailbreak and Stealth Wrestle in Vision-Language Models?
  2. 2025JailBound: Jailbreaking Internal Safety Boundaries of Vision-Language Models
  3. 2026Jailbreaks on Vision Language Model via Multimodal Reasoning
  4. 2025Spot Risks Before Speaking! Unraveling Safety Attention Heads in Large Vision-Language Models
  5. 2026Safety-Potential Pruning for Enhancing Safety Prompts Against VLM Jailbreaking Without Retraining
  6. 2024Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreaks
  7. 2024UniGuard: Towards Universal Safety Guardrails for Jailbreak Attacks on Multimodal Large Language Models
  8. 2025Towards Effective MLLM Jailbreaking Through Balanced On-Topicness and OOD-Intensity