mlsanityData to 14 Oct 2026

Component and Dimension Sparsity in Transformer Refusal Mechanisms

arXiv 2610.06903 ↗First posted 30 Sep 2026cs.CL cs.AI cs.LGcited by 0

On the map: Mechanistic interpretability → Activation steering methods also fits: Steering and probing for LLM safety

Abstract (arXiv)

Activation steering manipulates large language model behavior by intervening on internal activations, but the mechanistic basis of these interventions remains poorly understood. We decompose refusal steering into component-level interventions across four open-weight models, identifying the sparse subsets of attention and MLP components whose steering suffices to reproduce the full behavioral effect. We find that refusal directions concentrate in sparse component mechanisms comprising 28--48% of upstream components, retaining 88--101% of steering effectiveness. Within these mechanisms, effective steering further concentrates in approximately 50% of residual stream dimensions, retaining 85--98% of the component-mechanism baseline, consistent with a privileged basis structure. Sparsity thus operates at two levels: which components are steered, and which dimensions within those components carry the signal. Together these findings show that refusal is not diffusely encoded across a transformer but assembled by a structured, identifiable mechanism, providing a foundation for mechanistic understanding of how refusal behaviors are represented and steered. To facilitate reproducibility, we release all code and raw experimental results in https://github.com/wang-research-lab/RefusalMechanisms.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability. Builds on 32 of its 44 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

32 of 44 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Steering Llama 2 via Contrastive Activation Addition9 Dec
  2. 2023Steering Language Models With Activation Engineering20 Aug

Other references · 30

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  2. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  3. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  4. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  5. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  6. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  7. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  8. 2022Toy Models of Superpositioncited by 971
  9. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  10. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  11. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  12. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
18 more
  1. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  2. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 455
  3. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 451
  4. 2018Generating Wikipedia by Summarizing Long Sequencescited by 335
  5. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 334
  6. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 323
  7. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 297
  8. 2024How to use and interpret activation patchingcited by 214
  9. 2024Programming Refusal with Conditional Activation Steeringcited by 206
  10. 2023Localizing Model Behavior with Path Patchingcited by 195
  11. 2023Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchillacited by 149
  12. 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independencecited by 115
  13. 2025Taxonomy, Opportunities, and Challenges of Representation Engineering for Large Language Modelscited by 34
  14. 2024Refusal in LLMs is an Affine Functioncited by 29
  15. 2026Language Model Circuits Are Sparse in the Neuron Basiscited by 25
  16. 2025COSMIC: Generalized Refusal Direction Identification in LLM Activationscited by 19
  17. 2025RepIt: Steering Language Models with Concept-Specific Refusal Vectorscited by 12
  18. 2026Spectral Superposition: A Theory of Feature Geometrycited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026On the Steering Dimensionality of Refusal in Language Models
  2. 2026There Is More to Refusal in Large Language Models than a Single Direction
  3. 2026Over-Refusal and Representation Subspaces: A Mechanistic Analysis of Task-Conditioned Refusal in Aligned LLMs
  4. 2026Steering to Say No: Configurable Refusal via Activation Steering in Vision Language Models
  5. 2026What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal
  6. 2026Tool Mediation Alters Refusal Mechanisms in Large Language Models
  7. 2026Surgical Refusal Ablation: Disentangling Safety from Intelligence via Concept-Guided Spectral Cleaning
  8. 2025SOM Directions are Better than One: Multi-Directional Refusal Suppression in Language Models