mlsanityData to 7 Oct 2026

From Concept-Aligned Tokens to Vulnerable Features: Mechanistic Localization of Jailbreaks

arXiv 2604.23130 ↗First posted 25 Apr 2026cs.CL cs.AIcited by 0

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Jailbreak attacks expose a persistent failure mode in safety-aligned LLMs: models can be pushed into harmful behavior, but the internal representations enabling this shift remain poorly localized. Recent mechanistic safety studies often explain such behavior through broad representational objects, including global refusal directions, activation steering vectors, and refusal-related SAE features. We instead ask whether jailbreak vulnerability can be traced to finer-grained, prompt-conditioned SAE feature subgroups. We introduce a token-driven mechanistic pipeline that decomposes the residual stream of Gemma-2-2B into Sparse Autoencoder (SAE) features and identifies feature subgroups associated with unsafe behavior. Using single-category unsafe examples from BeaverTails to reduce cross-category interference, we extract harmful concepts from adversarial responses and align them with concept-relevant prompt tokens through subspace similarity. We then apply three feature-grouping strategies: cluster-based, hierarchical-linkage, and single-token-driven, to identify SAE feature subgroups across all 26 layers. Finally, we amplify the top features in each subgroup and evaluate the resulting generations with a standardized harmfulness judge. Single-token-driven grouping achieves harmfulness comparable to full cluster-based grouping, showing that individual harmful prompt tokens are sufficient to localize vulnerability-relevant SAE feature subgroups without relying on broader cluster-level aggregation. These subgroups appear across early and mid-to-late layers, with stronger concentration in mid-to-late layers, where targeted steering exposes specific model vulnerabilities. Overall, our results suggest that jailbreak susceptibility can be traced to sparse, token-localized SAE feature subgroups, complementing prior accounts based on broad adversarial, refusal, or steering directions.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 17 of its 24 references that are in the map. First posted on arXiv 25 Apr 2026.

Builds on

17 of 24 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026Sparse Autoencoders are Capable LLM Jailbreak Mitigators12 Feb
  2. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  4. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 13

  1. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  2. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  3. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  4. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  5. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  6. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  7. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
  8. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  9. 2019Superposition of many models into onecited by 64
  10. 2026There Is More to Refusal in Large Language Models than a Single Directioncited by 22
  11. 2025Evaluating Adversarial Robustness of Concept Representations in Sparse Autoencoderscited by 11
  12. 2025Signal in the Noise: Polysemantic Interference Transfers and Predicts Cross-Model Influencecited by 3
1 more
  1. 2025Semantic Convergence: Investigating Shared Representations Across Scaled LLMscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026The Art of the Jailbreak: Formulating Jailbreak Attacks for LLM Security Beyond Binary Scoring
  2. 2025JBShield: Defending Large Language Models from Jailbreak Attacks through Activated Concept Analysis and Manipulation
  3. 2026Circuit Discovery Helps Detect LLM Jailbreaking: A Mechanistic Interpretability Study
  4. 2026Jailbreaking Leaves a Trace: Understanding and Detecting Jailbreak Attacks from Internal Representations of Large Language Models
  5. 2026Causal Front-Door Adjustment for Robust Jailbreak Attacks on LLMs
  6. 2026Understanding and Defending VLM Jailbreaks via Jailbreak-Related Representation Shift
  7. 2024JailPO: A Novel Black-box Jailbreak Framework via Preference Optimization against Aligned LLMs
  8. 2025Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrieval