# Refusal in LLMs is an Affine Function

arXiv: https://arxiv.org/abs/2411.09003 · HTML: https://mlsanity.com/paper/2411.09003/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Nov 2024.

## Built on by (most cited first, 29 of 29)

- [Understanding (Un)Reliability of Steering Vectors in Language Models](https://mlsanity.com/paper/2505.22637/) (2025)
- [Refusal Direction is Universal Across Safety-Aligned Languages](https://mlsanity.com/paper/2505.17306/) (2025)
- [Localized Cultural Knowledge is Conserved and Controllable in Large Language Models](https://mlsanity.com/paper/2504.10191/) (2025)
- [COSMIC: Generalized Refusal Direction Identification in LLM Activations](https://mlsanity.com/paper/2506.00085/) (2025)
- [Robustly Improving LLM Fairness in Realistic Settings via Interpretability](https://mlsanity.com/paper/2506.10922/) (2025)
- [RepIt: Steering Language Models with Concept-Specific Refusal Vectors](https://mlsanity.com/paper/2509.13281/) (2025)
- [Jailbreak Attack Initializations as Extractors of Compliance Directions](https://mlsanity.com/paper/2502.09755/) (2025)
- [Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder Decoder Perspective](https://mlsanity.com/paper/2412.12276/) (2024)
- [Towards Understanding and Improving Refusal in Compressed Models via Mechanistic Interpretability](https://mlsanity.com/paper/2504.04215/) (2025)
- [In-Context Representation Hijacking](https://mlsanity.com/paper/2512.03771/) (2025)
- [Inference-Time Toxicity Mitigation in Protein Language Models](https://mlsanity.com/paper/2603.04045/) (2026)
- [Activation Surgery: Jailbreaking White-box LLMs without Touching the Prompt](https://mlsanity.com/paper/2603.14278/) (2026)
- [Locating and Controlling Implicit Personalization in Large Language Models](https://mlsanity.com/paper/2608.11735/) (2026)
- [RefusalGuard: Geometry-Preserving Fine-Tuning for Safety in LLMs](https://mlsanity.com/paper/2605.01913/) (2026)
- [Ablating Safety: Mechanisms for Removing Alignment in Language Models for Security Applications](https://mlsanity.com/paper/2605.17413/) (2026)
- [Concept Concentration for Faithful Representation Intervention](https://mlsanity.com/paper/2505.18672/) (2025)
- [Linear Script Representations in Speech Foundation Models Enable Zero-Shot Transliteration](https://mlsanity.com/paper/2601.02906/) (2026)
- [Less is More: Geometric Unlearning for LLMs with Minimal Data Disclosure](https://mlsanity.com/paper/2605.01735/) (2026)
- [Exemplar Partitioning for Mechanistic Interpretability](https://mlsanity.com/paper/2605.14347/) (2026)
- [Residual Paving: Diagnosing the Routing Bottleneck in Selective Refusal Editing](https://mlsanity.com/paper/2605.20262/) (2026)
- [Decomposing Refusal Steering in Mixture-of-Experts Models](https://mlsanity.com/paper/2606.04160/) (2026)
- [Leveraging Association Context Retrieval in Knowledge Edit- ing to Build White-Box Attacks on LLMs](https://mlsanity.com/paper/2608.17836/) (2026)
- [Controlling Refusal Behavior of LLMs via Stiefel-Constrained Rotation Steering](https://mlsanity.com/paper/2608.30986/) (2026)
- [How Fragile Is Safety Alignment at Frontier Scale? A Single-Direction Attack on a 320B MoE](https://mlsanity.com/paper/2609.09793/) (2026)
- [A Survey on the Linear Representation Hypothesis](https://mlsanity.com/paper/2609.22695/) (2026)
- [Activation Flow: Manufacturing Activations for Steering](https://mlsanity.com/paper/2609.32530/) (2026)
- [Controlling Speaking Rate in Autoregressive TTS via Activation Steering](https://mlsanity.com/paper/2609.33810/) (2026)
- [Capturing In-Context Learning Dynamics with Task Operators](https://mlsanity.com/paper/2610.01054/) (2026)
- [Component and Dimension Sparsity in Transformer Refusal Mechanisms](https://mlsanity.com/paper/2610.06903/) (2026)

## Abstract (arXiv)

We propose affine concept editing (ACE) as an approach for steering language models' behavior by intervening directly in activations. We begin with an affine decomposition of model activation vectors and show that prior methods for steering model behavior correspond to subsets of terms of this decomposition. We then provide a derivation of ACE and use it to control refusal behavior on ten different models, including Llama 3 70B. ACE combines affine subspace projection and activation addition to reliably control the model's refusal responses across prompt types. We evaluate the results using LLM-based scoring on a collection of harmful and harmless prompts. Our experiments demonstrate that ACE consistently achieves more precise control over model behavior than existing methods and generalizes to models where directional ablation via affine subspace projection alone produces incoherent outputs. Code for reproducing our results is available at https://github.com/EleutherAI/steering-llama3 .