# HiddenDetect: Detecting Jailbreak Attacks against Large Vision-Language Models via Monitoring Hidden States

arXiv: https://arxiv.org/abs/2502.14744 · HTML: https://mlsanity.com/paper/2502.14744/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 43 of its 52 references that are in the map.
- First posted on arXiv 20 Feb 2025.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)
- LLM safety and security / [Vision-language and text-to-image safety](https://mlsanity.com/safety/vision-language-and-text-to-image-safety/)

## Builds on (most cited first, 30 of 43)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities](https://mlsanity.com/paper/2308.02490/) (2023)
- [Refusal in Language Models Is Mediated by a Single Direction](https://mlsanity.com/paper/2406.11717/) (2024)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [CogVLM: Visual Expert for Pretrained Language Models](https://mlsanity.com/paper/2311.03079/) (2023)
- [mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration](https://mlsanity.com/paper/2311.04257/) (2023)
- [XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models](https://mlsanity.com/paper/2308.01263/) (2023)
- [Detecting Language Model Attacks with Perplexity](https://mlsanity.com/paper/2308.14132/) (2023)
- [FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts](https://mlsanity.com/paper/2311.05608/) (2023)
- [Visual Adversarial Examples Jailbreak Aligned Large Language Models](https://mlsanity.com/paper/2306.13213/) (2023)
- [Emergent Linear Representations in World Models of Self-Supervised Sequence Models](https://mlsanity.com/paper/2309.00941/) (2023)
- [MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models](https://mlsanity.com/paper/2311.17600/) (2023)
- [Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models](https://mlsanity.com/paper/2307.14539/) (2023)
- [JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks](https://mlsanity.com/paper/2404.03027/) (2024)
- [How Robust is Google's Bard to Adversarial Image Attacks?](https://mlsanity.com/paper/2309.11751/) (2023)
- [Image Hijacks: Adversarial Images can Control Generative Models at Runtime](https://mlsanity.com/paper/2309.00236/) (2023)
- [Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models](https://mlsanity.com/paper/2403.09792/) (2024)
- [Safety Layers in Aligned Large Language Models: The Key to LLM Security](https://mlsanity.com/paper/2408.17003/) (2024)
- [Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs](https://mlsanity.com/paper/2307.10490/) (2023)
- [Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation](https://mlsanity.com/paper/2403.09572/) (2024)
- [MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance](https://mlsanity.com/paper/2401.02906/) (2024)
- [On Prompt-Driven Safeguarding for Large Language Models](https://mlsanity.com/paper/2401.18018/) (2024)
- [How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs](https://mlsanity.com/paper/2311.16101/) (2023)
- [DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedback](https://mlsanity.com/paper/2311.10081/) (2023)
- [GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysis](https://mlsanity.com/paper/2402.13494/) (2024)

## Built on by (most cited first, 30 of 33)

- [VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agents](https://mlsanity.com/paper/2506.02456/) (2025)
- [MAGIC-VQA: Multimodal And Grounded Inference with Commonsense Knowledge for Visual Question Answering](https://mlsanity.com/paper/2503.18491/) (2025)
- [OmniSafeBench-MM: A Unified Benchmark and Toolbox for Multimodal Jailbreak Attack-Defense Evaluation](https://mlsanity.com/paper/2512.06589/) (2025)
- [Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoring](https://mlsanity.com/paper/2512.12069/) (2025)
- [JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Model](https://mlsanity.com/paper/2504.03770/) (2025)
- [USB: A Comprehensive and Unified Safety Evaluation Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2505.23793/) (2025)
- [The Impact of Off-Policy Training Data on Probe Generalisation](https://mlsanity.com/paper/2511.17408/) (2025)
- [SARSteer: Safeguarding Large Audio-Language Models via Safe-Ablated Refusal Steering](https://mlsanity.com/paper/2510.17633/) (2025)
- [Beyond Safe Answers: A Benchmark for Evaluating True Risk Awareness in Large Reasoning Models](https://mlsanity.com/paper/2505.19690/) (2025)
- [MSR-Align: Policy-Grounded Multimodal Alignment for Safety-Aware Reasoning in Vision-Language Models](https://mlsanity.com/paper/2506.19257/) (2025)
- [Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets](https://mlsanity.com/paper/2505.02118/) (2025)
- [CrossGuard: Safeguarding MLLMs against Joint-Modal Implicit Malicious Attacks](https://mlsanity.com/paper/2510.17687/) (2025)
- [ALERT: Zero-shot LLM Jailbreak Detection via Internal Discrepancy Amplification](https://mlsanity.com/paper/2601.03600/) (2026)
- [Learning to Detect Unseen Jailbreak Attacks in Large Vision-Language Models](https://mlsanity.com/paper/2508.09201/) (2025)
- [Safe responses matter: Output-aware safety guardrail mitigate over-refusal in MLLMs](https://mlsanity.com/paper/2607.09697/) (2026)
- [Dynamic Token Reweighting for Robust Vision-Language Models](https://mlsanity.com/paper/2505.17132/) (2025)
- [Learning to Detect Unknown Jailbreak Attacks in Large Vision-Language Models](https://mlsanity.com/paper/2510.15430/) (2025)
- [Why Does Weak-OOD Help? A Further Step Towards Understanding Jailbreaking VLMs](https://mlsanity.com/paper/2511.08367/) (2025)
- [Adaptively Robust LLM Monitoring via Activation Watermarking](https://mlsanity.com/paper/2603.23171/) (2026)
- [Sparse Autoencoders as Plug-and-Play Firewalls for Adversarial Attack Detection in VLMs](https://mlsanity.com/paper/2605.07447/) (2026)
- [Defending Large Language Models Against Jailbreak Exploits with Responsible AI Considerations](https://mlsanity.com/paper/2511.18933/) (2025)
- [CoT is Not the Chain of Truth: An Empirical Internal Analysis of Reasoning LLMs for Fake News Generation](https://mlsanity.com/paper/2602.04856/) (2026)
- [Detecting Jailbreak Attempts in Clinical Training LLMs Through Automated Linguistic Feature Extraction](https://mlsanity.com/paper/2602.13321/) (2026)
- [SALLIE: Generation-Free Hidden-State Detection of Jailbreaks and Prompt Injections Across Text and Vision](https://mlsanity.com/paper/2604.06247/) (2026)
- [Revisiting JBShield: Breaking and Rebuilding Representation-Level Jailbreak Defenses](https://mlsanity.com/paper/2605.03095/) (2026)
- [Ellipsoid Control: A White-list Jailbreak Defense via Benign Latent Modeling](https://mlsanity.com/paper/2605.24552/) (2026)
- [StructBreak: Structural Cognitive Overload-Induced Safety Failures in MLLMs](https://mlsanity.com/paper/2605.25534/) (2026)
- [DE-FIVE: Detecting Malicious Image Prompts via Fourier Features and Image Vector Embeddings](https://mlsanity.com/paper/2606.22779/) (2026)
- [Breaking Refusal in the First Half: A Mechanistic Study of the Prefill Jailbreak](https://mlsanity.com/paper/2607.14147/) (2026)
- [V-DEAL: Diagnosing Video Safety De-Calibration as an Understanding-Refusal Coupling Failure](https://mlsanity.com/paper/2607.21151/) (2026)

## Abstract (arXiv)

The integration of additional modalities increases the susceptibility of large vision-language models (LVLMs) to safety risks, such as jailbreak attacks, compared to their language-only counterparts. While existing research primarily focuses on post-hoc alignment techniques, the underlying safety mechanisms within LVLMs remain largely unexplored. In this work , we investigate whether LVLMs inherently encode safety-relevant signals within their internal activations during inference. Our findings reveal that LVLMs exhibit distinct activation patterns when processing unsafe prompts, which can be leveraged to detect and mitigate adversarial inputs without requiring extensive fine-tuning. Building on this insight, we introduce HiddenDetect, a novel tuning-free framework that harnesses internal model activations to enhance safety. Experimental results show that {HiddenDetect} surpasses state-of-the-art methods in detecting jailbreak attacks against LVLMs. By utilizing intrinsic safety-aware patterns, our method provides an efficient and scalable solution for strengthening LVLM robustness against multimodal threats. Our code will be released publicly at https://github.com/leigest519/HiddenDetect.