# There Is More to Refusal in Large Language Models than a Single Direction

arXiv: https://arxiv.org/abs/2602.02132 · HTML: https://mlsanity.com/paper/2602.02132/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Placed in Steering and probing for LLM safety, one of 13 directions in Mechanistic interpretability.
- Builds on 25 of its 41 references that are in the map.
- First posted on arXiv 2 Feb 2026.

## Placed in

- Mechanistic interpretability / [Steering and probing for LLM safety](https://mlsanity.com/mechinterp/steering-and-probing-for-llm-safety/)

## Builds on (most cited first, 25 of 25)

- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [A General Language Assistant as a Laboratory for Alignment](https://mlsanity.com/paper/2112.00861/) (2021)
- [Refusal in Language Models Is Mediated by a Single Direction](https://mlsanity.com/paper/2406.11717/) (2024)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs](https://mlsanity.com/paper/2406.18495/) (2024)
- [Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2](https://mlsanity.com/paper/2408.05147/) (2024)
- [Improving Alignment and Robustness with Circuit Breakers](https://mlsanity.com/paper/2406.04313/) (2024)
- [Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders](https://mlsanity.com/paper/2407.14435/) (2024)
- [Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models](https://mlsanity.com/paper/2411.14257/) (2024)
- [The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence](https://mlsanity.com/paper/2502.17420/) (2025)
- [The Art of Saying No: Contextual Noncompliance in Language Models](https://mlsanity.com/paper/2407.12043/) (2024)
- [Steering Language Model Refusal with Sparse Autoencoders](https://mlsanity.com/paper/2411.11296/) (2024)
- [LLMs Encode Harmfulness and Refusal Separately](https://mlsanity.com/paper/2507.11878/) (2025)
- [Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training](https://mlsanity.com/paper/2407.09121/) (2024)
- [From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Models](https://mlsanity.com/paper/2308.12014/) (2023)
- [Representation Bending for Large Language Model Safety](https://mlsanity.com/paper/2504.01550/) (2025)
- [Understanding Refusal in Language Models with Sparse Autoencoders](https://mlsanity.com/paper/2505.23556/) (2025)
- [Discovering Salient Neurons in Deep NLP Models](https://mlsanity.com/paper/2206.13288/) (2022)
- [On the Transformation of Latent Space in Fine-Tuned NLP Models](https://mlsanity.com/paper/2210.12696/) (2022)
- [Beyond I'm Sorry, I Can't: Dissecting Large Language Model Refusal](https://mlsanity.com/paper/2509.09708/) (2025)
- [Can LLMs facilitate interpretation of pre-trained language models?](https://mlsanity.com/paper/2305.13386/) (2023)
- [Differentiated Directional Intervention A Framework for Evading LLM Safety Alignment](https://mlsanity.com/paper/2511.06852/) (2025)
- [Beyond the Leaderboard: Understanding Performance Disparities in Large Language Models via Model Diffing](https://mlsanity.com/paper/2509.18792/) (2025)

## Built on by (most cited first, 26 of 26)

- [How's it going? Reinforcement learning in language models recruits a functional welfare axis](https://mlsanity.com/paper/2605.30232/) (2026)
- [Activation Steering for Aligned Open-ended Generation without Sacrificing Coherence](https://mlsanity.com/paper/2604.08169/) (2026)
- [A Single Neuron Is Sufficient to Bypass Safety Alignment in Large Language Models](https://mlsanity.com/paper/2605.08513/) (2026)
- [Low-Resource Safety Failures Are Action Failures, Not Representation Failures](https://mlsanity.com/paper/2606.01196/) (2026)
- [The Entanglement Wall: Activation-Space Probes as Risk Detectors, Not Context Adjudicators](https://mlsanity.com/paper/2607.13075/) (2026)
- [Authority Bias in Language Models: Source Deference and User Agreement Are Not Interchangeable](https://mlsanity.com/paper/2609.37616/) (2026)
- [C-DeltaTheta: Circuit-Restricted Weight Arithmetic for Selective Refusal](https://mlsanity.com/paper/2602.04521/) (2026)
- [InsufficiencyBench: Evaluating LLM legal advice on underspecified user queries](https://mlsanity.com/paper/2608.20220/) (2026)
- [Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness](https://mlsanity.com/paper/2609.03887/) (2026)
- [Decoy Direction Optimization: A Post-Hoc Defense Against LLM Abliteration](https://mlsanity.com/paper/2609.16204/) (2026)
- [Behind Harmful Compliance: Behavioral and Mechanistic Divergence Across LLM Jailbreaks](https://mlsanity.com/paper/2604.18510/) (2026)
- [From Concept-Aligned Tokens to Vulnerable Features: Mechanistic Localization of Jailbreaks](https://mlsanity.com/paper/2604.23130/) (2026)
- [Capabilities of Claude Fable 5 on Biomedical Challenge Problems](https://mlsanity.com/paper/2607.10849/) (2026)
- [Breaking Refusal in the First Half: A Mechanistic Study of the Prefill Jailbreak](https://mlsanity.com/paper/2607.14147/) (2026)
- [Minimizing Targeted Activations: Input-Only Suppression of Evaluation-Awareness Latents in Large Language Models](https://mlsanity.com/paper/2607.25907/) (2026)
- [Risky Business: Measuring The Faithfulness-Safety Tension](https://mlsanity.com/paper/2608.03745/) (2026)
- [Fool's Gold: Defensive Deception Against Safety-Removal Attacks on Open-Weight Models](https://mlsanity.com/paper/2608.17202/) (2026)
- [Who Pays More for Safety? Measuring the Disparate Cost of Safety Alignment across Languages](https://mlsanity.com/paper/2608.22490/) (2026)
- [Steering Interference Reflects the Model's Defaults, Not the Behavior Directions](https://mlsanity.com/paper/2609.06951/) (2026)
- [How Fragile Is Safety Alignment at Frontier Scale? A Single-Direction Attack on a 320B MoE](https://mlsanity.com/paper/2609.09793/) (2026)
- [From Trait Vectors to Circuits: Tracing Refusal and Sycophancy Through Language Models](https://mlsanity.com/paper/2609.22224/) (2026)
- ["As a Language Model...": Chat Template Switches LLM Self-Referential Voice and Activation Steering Reproduces It](https://mlsanity.com/paper/2609.25021/) (2026)
- [From Latents to Wires: Surgical Post-Editing on Large Language Models](https://mlsanity.com/paper/2609.32434/) (2026)
- [Tool Mediation Alters Refusal Mechanisms in Large Language Models](https://mlsanity.com/paper/2609.35117/) (2026)
- [Does the Model Use the Feature? Separating Steering from Mechanism in LLMs](https://mlsanity.com/paper/2610.07270/) (2026)
- [Genesis Steering: Better Activation Steering Through Training Data](https://mlsanity.com/paper/2610.19114/) (2026)

## Abstract (arXiv)

Prior work argues that refusal in large language models is mediated by a single direction, enabling steering and abliteration. We show that this account is incomplete: across diverse refusal and non-compliance categories, refusal behaviors correspond to geometrically distinct directions in activation space. Yet activation steering along any refusal-related direction produces nearly identical refusal--over-refusal trade-offs, acting as a shared one-dimensional control knob. Thus, different directions primarily affect not whether the model refuses, but how it refuses. Using sparse autoencoders, we uncover a structured internal representation of refusal: a reusable core of shared refusal latents supplemented by style- and domain-specific latents. Linear interventions collapse this structure into uniform behavioral control, flattening mechanistic differences across refusal types. Our results reconcile the apparent simplicity of refusal steering with the diversity of refusal behaviors, and clarify the limits of linear interpretability for aligned model behavior.