mlsanityData to 14 Oct 2026

NeuroBreak: Unveil Internal Jailbreak Mechanisms in Large Language Models

arXiv 2509.03985 ↗First posted 4 Sep 2025cs.CR cs.AIcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Jailbreak attacks bypass the safety alignment of large language models (LLMs) to elicit harmful outputs, yet the vast parameter space makes diagnosing the underlying failure mechanisms extremely challenging. We present NeuroBreak, a visual analytics system that helps experts progressively unpack jailbreak mechanisms from layer-level semantics down to neuron-level behaviors. A layer-wise probing pipeline traces how harmful representations evolve across layers, while a dual-dimensional character--behavior categorization reveals each safety-related neuron's inherent tendency and contextual contribution. These analyses are made interpretable through tailored visualization designs: a task-driven probing projection that reveals safety decision boundaries, a dual-stream semantic evolution flow that traces cross-layer semantic shifts, and a character--behavior chord graph that unifies neuron roles, attribution scores, and collaborative relations in a single view with in-situ causal verification. Quantitative evaluations and case studies show that NeuroBreak uncovers safety failure causes and provides actionable insights for strengthening LLM defenses.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 72 of its 112 references that are in the map. First posted on arXiv 4 Sep 2025.

Builds on

72 of 112 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directions13 Feb
  2. 2024Safety Layers in Aligned Large Language Models: The Key to LLM Security30 Aug
  3. 2024Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons20 Jun
  4. 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden States9 Jun
  5. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications7 Feb
  6. 2024A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity3 Jan
  7. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B31 Oct
  8. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct
  9. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  10. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 62

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  6. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  7. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  8. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  9. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  10. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  11. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  12. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 784
38 more
  1. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  2. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
  3. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
  4. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
  5. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 489
  6. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  7. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 367
  8. 2023Explainability for Large Language Models: A Surveycited by 333
  9. 2019Analyzing the Structure of Attention in a Transformer Language Modelcited by 243
  10. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 230
  11. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 228
  12. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbotscited by 183
  13. 2018Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontierscited by 117
  14. 2024ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teamingcited by 98
  15. 2023Attack Prompt Generation for Red Teaming and Defending Large Language Modelscited by 76
  16. 2022Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Modelscited by 70
  17. 2024Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysiscited by 64
  18. 2017Do Convolutional Neural Networks Learn Class Hierarchy?cited by 61
  19. 2023AttentionViz: A Global View of Transformer Attentioncited by 50
  20. 2020Attention Flows: Analyzing and Comparing Attention Mechanisms in Language Modelscited by 49
  21. 2022Causality-based Neural Network Repaircited by 47
  22. 2017Understanding Hidden Memories of Recurrent Neural Networkscited by 44
  23. 2024A Cross-Language Investigation into Jailbreak Attacks in Large Language Modelscited by 41
  24. 2024Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Modelscited by 37
  25. 2017Towards Better Analysis of Machine Learning Models: A Visual Analytics Perspectivecited by 36
  26. 2024Sharing Matters: Analysing Neurons Across Languages and Tasks in LLMscited by 33
  27. 2023VISIT: Visualizing and Interpreting the Semantic Information Flow of Transformerscited by 32
  28. 2020Analyzing the Noise Robustness of Deep Neural Networkscited by 31
  29. 2023Foundation Models Meet Visualizations: Challenges and Opportunitiescited by 31
  30. 2024What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspectivecited by 29
  31. 2021Dodrio: Exploring Transformer Models with Interactive Visualizationcited by 28
  32. 2023Make Them Spill the Beans! Coercive Knowledge Extraction from (Production) LLMscited by 27
  33. 2023Visual Analytics For Machine Learning: A Data Perspective Surveycited by 25
  34. 2019Explaining Vulnerabilities to Adversarial Machine Learning through Visual Analyticscited by 24
  35. 2024JailbreakLens: Interpreting Jailbreak Mechanism in the Lens of Representation and Circuitcited by 22
  36. 2021On Explaining Your Explanations of BERT: An Empirical Study with Sequence Classificationcited by 21
  37. 2022A Unified Understanding of Deep NLP Models for Text Classificationcited by 18
  38. 2024Transformer Explainer: Learning LLM Transformers with Interactive Visual Explanation and Experimentationcited by 10

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026ORBIT: Training-Free Multi-Attribute Behavioral Steering via Orthogonal Subspace Rotationcited by 4

Similar, not cited

Close in wording, no citation either way

  1. 2025Unraveling LLM Jailbreaks Through Safety Knowledge Neurons
  2. 2026Circuit Discovery Helps Detect LLM Jailbreaking: A Mechanistic Interpretability Study
  3. 2025SoK: a Comprehensive Causality Analysis Framework for Large Language Model Security
  4. 2024Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective
  5. 2026Defending Large Language Models Against Jailbreak Attacks via In-Decoding Safety-Awareness Probing
  6. 2025ConceptGuard: Neuro-Symbolic Safety Guardrails via Sparse Interpretable Jailbreak Concepts
  7. 2025JBShield: Defending Large Language Models from Jailbreak Attacks through Activated Concept Analysis and Manipulation
  8. 2026A Causal Perspective for Enhancing Jailbreak Attack and Defense