# Understanding Refusal in Language Models with Sparse Autoencoders

arXiv: https://arxiv.org/abs/2505.23556 · HTML: https://mlsanity.com/paper/2505.23556/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 29 May 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Safety alignment and jailbreak attacks](https://mlsanity.com/alignment/safety-alignment-and-jailbreak-attacks/)
- Mechanistic interpretability / [Steering and probing for LLM safety](https://mlsanity.com/mechinterp/steering-and-probing-for-llm-safety/)

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Refusal in Language Models Is Mediated by a Single Direction](https://mlsanity.com/paper/2406.11717/) (2024)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models](https://mlsanity.com/paper/2403.19647/) (2024)
- [Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2](https://mlsanity.com/paper/2408.05147/) (2024)
- [Attribution Patching Outperforms Automated Circuit Discovery](https://mlsanity.com/paper/2310.10348/) (2023)
- [Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic](https://mlsanity.com/paper/2402.11746/) (2024)
- [GASP: Efficient Black-Box Generation of Adversarial Suffixes for Jailbreaking LLMs](https://mlsanity.com/paper/2411.14133/) (2024)

## Built on by (most cited first, 29 of 29)

- [LLMs Encode Harmfulness and Refusal Separately](https://mlsanity.com/paper/2507.11878/) (2025)
- [Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models](https://mlsanity.com/paper/2601.14004/) (2026)
- [There Is More to Refusal in Large Language Models than a Single Direction](https://mlsanity.com/paper/2602.02132/) (2026)
- [Do Sparse Autoencoders Identify Reasoning Features in Language Models?](https://mlsanity.com/paper/2601.05679/) (2026)
- [SAGE: An Agentic Explainer Framework for Interpreting SAE Features in Language Models](https://mlsanity.com/paper/2511.20820/) (2025)
- [Steered LLM Activations are Non-Surjective](https://mlsanity.com/paper/2604.09839/) (2026)
- [Interpretable Safety Alignment via SAE-Constructed Low-Rank Subspace Adaptation](https://mlsanity.com/paper/2512.23260/) (2025)
- [Do Activation Monitors Survive Model Updates? Benchmarking, Predicting, and Repairing Activation-Monitor Staleness](https://mlsanity.com/paper/2606.15980/) (2026)
- [Mitigating Jailbreaks with Intent-Aware LLMs](https://mlsanity.com/paper/2508.12072/) (2025)
- [Large Language Models Generate Harmful Responses Using a Distinct Mechanism, Shared Across Harm Types](https://mlsanity.com/paper/2604.09544/) (2026)
- [Over-Refusal and Representation Subspaces: A Mechanistic Analysis of Task-Conditioned Refusal in Aligned LLMs](https://mlsanity.com/paper/2603.27518/) (2026)
- [OpenSafeIntent: Evaluating Intent-Calibrated Safe Completion Across Dual-Use Prompt Sets](https://mlsanity.com/paper/2607.02047/) (2026)
- [Poison Once, Refuse Forever: Weaponizing Alignment for Injecting Bias in LLMs](https://mlsanity.com/paper/2508.20333/) (2025)
- [Graph-Regularized Sparse Autoencoders for LLM Safety Steering](https://mlsanity.com/paper/2512.06655/) (2025)
- [CRaFT: Circuit-Guided Refusal Feature Selection via Cross-Layer Transcoders](https://mlsanity.com/paper/2604.01604/) (2026)
- [SAE Interventions are Unreliable: Post-Intervention Recovery of Suppressed Behavior](https://mlsanity.com/paper/2606.18322/) (2026)
- [Robust Harmful Features Under Jailbreak Attacks: Mechanistic Evidence from Attention Head Specialization in Large Language Models](https://mlsanity.com/paper/2606.28153/) (2026)
- [C2R: Cross-sample Consistency Regularization Mitigates Feature Splitting and Absorption in Sparse Autoencoders](https://mlsanity.com/paper/2606.30609/) (2026)
- [Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness](https://mlsanity.com/paper/2609.03887/) (2026)
- [Decoy Direction Optimization: A Post-Hoc Defense Against LLM Abliteration](https://mlsanity.com/paper/2609.16204/) (2026)
- [Minimal, Local, Causal Explanations for Jailbreak Success in Large Language Models](https://mlsanity.com/paper/2605.00123/) (2026)
- [Beyond "I cannot fulfill this request": Alleviating Rigid Rejection in LLMs via Label Enhancement](https://mlsanity.com/paper/2605.07883/) (2026)
- [Why Do Aligned LLMs Remain Jailbreakable: Refusal-Escape Directions, Operator-Level Sources, and Safety-Utility Trade-off](https://mlsanity.com/paper/2605.08878/) (2026)
- [ReCal3R: Reliability-Calibrated Learning Rates for Streaming 3D Reconstruction](https://mlsanity.com/paper/2607.05356/) (2026)
- [Do LLMs Know Their Vulnerable Scenarios?](https://mlsanity.com/paper/2607.23496/) (2026)
- [Broken Symmetry in LLM Refusal: Answer Release Is More Local Than Refusal Restoration](https://mlsanity.com/paper/2608.15772/) (2026)
- [The Safety Relay in Roleplay Jailbreaks: A Component-Resolved Causal Analysis of Harm Recognition and Refusal](https://mlsanity.com/paper/2608.30585/) (2026)
- [Don't Judge an LLM Only by Its Activations: Discovering Suppressed Safety Features via Counterfactual Activation Potential](https://mlsanity.com/paper/2610.05541/) (2026)
- [Sparse Autoencoders for Mechanistic Interpretability in NLP: A Critical Survey of Recoverability, Interpretability, and Utility](https://mlsanity.com/paper/2610.20139/) (2026)

## Abstract (arXiv)

Refusal is a key safety behavior in aligned language models, yet the internal mechanisms driving refusals remain opaque. In this work, we conduct a mechanistic study of refusal in instruction-tuned LLMs using sparse autoencoders to identify latent features that causally mediate refusal behaviors. We apply our method to two open-source chat models and intervene on refusal-related features to assess their influence on generation, validating their behavioral impact across multiple harmful datasets. This enables a fine-grained inspection of how refusal manifests at the activation level and addresses key research questions such as investigating upstream-downstream latent relationship and understanding the mechanisms of adversarial jailbreaking techniques. We also establish the usefulness of refusal features in enhancing generalization for linear probes to out-of-distribution adversarial samples in classification tasks. We open source our code in https://github.com/wj210/refusal_sae.