mlsanityData to 14 Oct 2026

JailbreakLens: Interpreting Jailbreak Mechanism in the Lens of Representation and Circuit

arXiv 2411.11114 ↗First posted 17 Nov 2024cs.CRcited by 22

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Despite the outstanding performance of Large language Models (LLMs) in diverse tasks, they are vulnerable to jailbreak attacks, wherein adversarial prompts are crafted to bypass their security mechanisms and elicit unexpected responses. Although jailbreak attacks are prevalent, the understanding of their underlying mechanisms remains limited. Recent studies have explained typical jailbreaking behavior (e.g., the degree to which the model refuses to respond) of LLMs by analyzing representation shifts in their latent space caused by jailbreak prompts or identifying key neurons that contribute to the success of jailbreak attacks. However, these studies neither explore diverse jailbreak patterns nor provide a fine-grained explanation from the failure of circuit to the changes of representational, leaving significant gaps in uncovering the jailbreak mechanism. In this paper, we propose JailbreakLens, an interpretation framework that analyzes jailbreak mechanisms from both representation (which reveals how jailbreaks alter the model's harmfulness perception) and circuit perspectives~(which uncovers the causes of these deceptions by identifying key circuits contributing to the vulnerability), tracking their evolution throughout the entire response generation process. We then conduct an in-depth evaluation of jailbreak behavior on five mainstream LLMs under seven jailbreak strategies. Our evaluation reveals that jailbreak prompts amplify components that reinforce affirmative responses while suppressing those that produce refusal. This manipulation shifts model representations toward safe clusters to deceive the LLM, leading it to provide detailed responses instead of refusals. Notably, we find a strong and consistent correlation between representation deception and activation shift of key circuits across diverse jailbreak methods and multiple LLMs.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 40 of its 47 references that are in the map. First posted on arXiv 17 Nov 2024.

Builds on

40 of 47 references.

Closest ancestors · 13

Same direction, newest first

  1. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Models26 Jun
  2. 2024Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis16 Jun
  3. 2024Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Models13 Jun
  4. 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden States9 Jun
  5. 2024CodeChameleon: Personalized Encryption Framework for Jailbreaking Large Language Models26 Feb
  6. 2023Causality Analysis for Evaluating the Security of Large Language Models13 Dec
  7. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily14 Nov
  8. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreaker6 Nov
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  10. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  11. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  12. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
1 more
  1. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 27

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2024Qwen2 Technical Reportcited by 2,120
  3. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  4. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  5. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  6. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  7. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  8. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
  9. 2019Are Sixteen Heads Really Better than One?cited by 816
  10. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  11. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  12. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
15 more
  1. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  2. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  3. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 401
  4. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
  5. 2023Language Models Represent Space and Timecited by 316
  6. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 209
  7. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 202
  8. 2023Generative Judge for Evaluating Alignmentcited by 169
  9. 2024Safety Layers in Aligned Large Language Models: The Key to LLM Securitycited by 134
  10. 2024AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformerscited by 120
  11. 2024Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmeticcited by 112
  12. 2024Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 92
  13. 2024Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neuronscited by 53
  14. 2023AtMan: Understanding Transformer Predictions Through Memory Efficient Attention Manipulationcited by 31
  15. 2023Attention Lens: A Tool for Mechanistically Interpreting the Attention Head Information Retrieval Mechanismcited by 22

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 15; 2026: 6.

22 papers.

Sort:

By area:

  1. 2024Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Modelscited by 23
  2. 2025Representation Engineering for Large-Language Models: Survey and Research Challengescited by 21
  3. 2025Chain-of-Thought Hijackingcited by 16
  4. 2025Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoringcited by 11
  5. 2025Jailbreak Attack Initializations as Extractors of Compliance Directionscited by 10
  6. 2025Using Mechanistic Interpretability to Craft Adversarial Attacks against Large Language Modelscited by 10
  7. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  8. 2025Backdoor Attribution: Elucidating and Controlling Backdoor in Language Modelscited by 10
  9. 2025Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMscited by 7
  10. 2025Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networkscited by 6
  11. 2025Neurons Speak in Ranges: Breaking Free from Discrete Neuronal Attributioncited by 5
  12. 2025Making Every Step Effective: Jailbreaking Large Vision-Language Models Through Hierarchical KV Equalizationcited by 4
10 more
  1. 2025ConceptGuard: Neuro-Symbolic Safety Guardrails via Sparse Interpretable Jailbreak Conceptscited by 2
  2. 2025Dual-Space Smoothness for Robust and Balanced LLM Unlearningcited by 2
  3. 2025NeuroBreak: Unveil Internal Jailbreak Mechanisms in Large Language Modelscited by 1
  4. 2026GUARD-SLM: Token Activation-Based Defense Against Jailbreak Attacks for Small Language Modelscited by 1
  5. 2026Safety Geometry Collapse in Multimodal LLMs and Adaptive Drift Correctioncited by 1
  6. 2025Matching Ranks Over Probability Yields Truly Deep Safety Alignmentcited by 0
  7. 2026SALLIE: Generation-Free Hidden-State Detection of Jailbreaks and Prompt Injections Across Text and Visioncited by 0
  8. 2026Why Do Large Language Models Generate Harmful Content?cited by 0
  9. 2026Why Do Aligned LLMs Remain Jailbreakable: Refusal-Escape Directions, Operator-Level Sources, and Safety-Utility Trade-offcited by 0
  10. 2026Why Are LLM Backdoor Defenses Fragmented? A Feature-Level Explanation with Sparse Autoencoderscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Jailbreaking Leaves a Trace: Understanding and Detecting Jailbreak Attacks from Internal Representations of Large Language Models
  2. 2025Do Internal Layers of LLMs Reveal Patterns for Jailbreak Detection?
  3. 2026Circuit Discovery Helps Detect LLM Jailbreaking: A Mechanistic Interpretability Study
  4. 2024Defending against Jailbreak through Early Exit Generation of Large Language Models
  5. 2024Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective
  6. 2026Minimal, Local, Causal Explanations for Jailbreak Success in Large Language Models
  7. 2025JBShield: Defending Large Language Models from Jailbreak Attacks through Activated Concept Analysis and Manipulation
  8. 2026Mechanistic Interpretability of LLM Jailbreaks via Internal Attribution Graphs