What Intermediate Layers Know: Detecting Jailbreaks from Entropy Dynamics
On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks
also in 1 more
Abstract (arXiv)
Jailbreak attacks reveal a persistent weakness in aligned Large Language Models: carefully crafted prompts can elicit policy-violating responses despite safety training. While most defenses operate at the prompt or output level, it remains unclear how harmful intent is encoded within the model's internal representations. We investigate this question by analyzing token-level predictive entropy trajectories across layers of a frozen LLM using the logit lens. We find that static aggregate statistics of prompt-level entropy (e.g., mean, variance) carry little discriminative signal, whereas features capturing how entropy evolves across token positions, such as monotonic rank-based trend scores, are substantially more informative. Importantly, this signal is not uniform across model depth: it is concentrated in intermediate layers and degrades at the final layer, indicating that jailbreak-relevant structure is most pronounced in mid-network representations rather than at the output head. Across multiple models (Llama, Qwen, Gemma) and adversarial benchmarks, these entropy dynamics provide architecture-consistent separation without additional training. Together, our findings show that jailbreak behavior is reflected in structured intermediate uncertainty dynamics, clarifying both which entropy-derived features encode harmful intent and where in the network that signal is most pronounced.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 37 of its 45 references that are in the map. First posted on arXiv 23 Jun 2026.
Builds on
37 of 45 references.
Closest ancestors · 4
Same direction, newest first
- 2025Reasoned Safety Alignment: Ensuring Jailbreak Defense via Answer-Then-Check15 Sep
- 2025Single-pass Detection of Jailbreaking Input in Large Language Models21 Feb
- 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Models1 Sep
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
Other references · 33
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
- 2024Gemma: Open Models Based on Gemini Research and Technologycited by 916
- 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 839
- 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
- 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
- 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 469
- 2023Detecting Language Model Attacks with Perplexitycited by 431
- 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Modelscited by 223
- 2024Detecting Hallucinations in Large Language Model Generation: A Token Probability Approachcited by 42
- 2026AgentSentry: Mitigating Indirect Prompt Injection in LLM Agents via Temporal Causal Diagnostics and Context Purificationcited by 29
21 more
- 2023Random-Set Neural Networks (RS-NN)cited by 19
- 2022Epistemic Deep Learningcited by 15
- 2025Calibrating LLMs with Information-Theoretic Evidential Deep Learningcited by 15
- 2025LLM Jailbreak Detection for (Almost) Free!cited by 13
- 2025A Unified Evaluation Framework for Epistemic Predictionscited by 10
- 2025Efficient Input-level Backdoor Defense on Text-to-Image Synthesis via Neuron Activation Variationcited by 8
- 2025Epistemic Artificial Intelligence is Essential for Machine Learning Models to Truly 'Know When They Do Not Know'cited by 8
- 2025Credal and Interval Deep Evidential Classificationscited by 8
- 2022Robust Reinforcement Learning in Continuous Control Tasks with Uncertainty Set Regularizationcited by 7
- 2026PrivAct: Internalizing Contextual Privacy Preservation via Multi-Agent Preference Trainingcited by 7
- 2025Uncertainty-Aware Autonomous Vehicles: Predicting the Road Aheadcited by 5
- 2025Epistemic Deep Learning: Enabling Machine Learning Models to Know When They Do Not Knowcited by 4
- 2026Jailbreaking Leaves a Trace: Understanding and Detecting Jailbreak Attacks from Internal Representations of Large Language Modelscited by 4
- 2025Do Internal Layers of LLMs Reveal Patterns for Jailbreak Detection?cited by 3
- 2026Random-Set Graph Neural Networkscited by 3
- 2026Engineering Attack Vectors and Detecting Anomalies in Additive Manufacturingcited by 2
- 2026DF-LoGiT: Data-Free Logic-Gated Backdoor Attacks in Vision Transformerscited by 2
- 2026Integration of deep generative Anomaly Detection algorithm in high-speed industrial linecited by 2
- 2026Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoningcited by 2
- 2026BlackMirror: Black-Box Backdoor Detection for Text-to-Image Models via Instruction-Response Deviationcited by 1
- 2026Structured Credal Learningcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024JailbreakLens: Interpreting Jailbreak Mechanism in the Lens of Representation and Circuit
- 2026Mechanistic Interpretability of LLM Jailbreaks via Internal Attribution Graphs
- 2026Exploring Jailbreak Vulnerability Across Depth-Based Dynamic Inference in Large Language Models
- 2026Circuit Discovery Helps Detect LLM Jailbreaking: A Mechanistic Interpretability Study
- 2025NeuroBreak: Unveil Internal Jailbreak Mechanisms in Large Language Models
- 2024Defending against Jailbreak through Early Exit Generation of Large Language Models
- 2025GuardNet: Graph-Attention Filtering for Jailbreak Defense in Large Language Models
- 2026Defending Jailbreak Attacks on Large Language Models via Manifold Trajectory Kinetics