Steering Interference Reflects the Model's Defaults, Not the Behavior Directions
On the map: Mechanistic interpretability → Activation steering methods
Abstract (arXiv)
Activation steering promises modular control of language model behavior: a behavior such as politeness corresponds to a direction in a model's activations, and adding that direction while it generates should switch the behavior on and leave everything else alone. It does not. We ask what decides which other behaviors move, and by how much, and find that it is the model rather than the behavior being steered. A steer relaxes the model toward a small set of behaviors it already favors, chiefly refusal, sycophancy, and poeticism, and that set is much the same whatever is steered. Three results across 24 behaviors and ten instruction-tuned models support this, every effect read off the generated text by a language-model judge rather than off a probe. That readout matters: all 24 behaviors are linearly decodable, but only 20 change what the model writes. First, a direction carrying no behavioral content, matched to a real steer only in the size of the vector it adds, moves the same behaviors in the same order as real steers do, while producing none of the behaviors that need a specific direction. Second, most interference runs one way, so it cannot be an overlap between two directions: steering profanity makes the model toxic, while steering toxicity leaves profanity untouched. Third, with a behavior held out entirely, geometry measured on the others explains almost none of the interference it takes part in. The account holds on all ten models, the pull toward defaults strongest below 10B parameters and weakening in each family's largest. Reading a steer as a perturbation whose endpoint the model fixes implies that disentangling behavior directions cannot by itself make steering modular.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability. Builds on 15 of its 18 references that are in the map. First posted on arXiv 7 Sep 2026.
Builds on
15 of 18 references.
Closest ancestors · 4
Same direction, newest first
- 2026Do Personality Traits Interfere? Geometric Limitations of Steering in Large Language Models23 Jan
- 2025In-Distribution Steering: Balancing Control and Coherence in Language Model Generation15 Oct
- 2025Understanding (Un)Reliability of Steering Vectors in Language Models28 May
- 2023Steering Llama 2 via Contrastive Activation Addition9 Dec
Other references · 11
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
- 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 345
- 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independencecited by 115
- 2026The Assistant Axis: Situating and Stabilizing the Default Persona of Language Modelscited by 90
- 2025Multi-Attribute Steering of Language Models via Targeted Interventioncited by 46
- 2026There Is More to Refusal in Large Language Models than a Single Directioncited by 26
- 2025Beyond Linear Steering: Unified Multi-Attribute Control for Language Modelscited by 25
- 2026Analysing the Safety Pitfalls of Steering Vectorscited by 10
- 2026Perfect Detection, Failed Control: The Geometry of Knowing vs. Steering in Language Modelscited by 5
- 2025SteeringSafety: Benchmarking Representation Steering in LLMs Across Safety Perspectivescited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026The Sharp Directions Are Against You: Curvature Analysis of Activation Steering
- 2025What Can We Actually Steer? A Multi-Behavior Study of Activation Control
- 2026Steered LLM Activations are Non-Surjective
- 2026On the Steering Dimensionality of Refusal in Language Models
- 2026Understanding Unreliability of Steering Vectors in Language Models: Geometric Predictors and the Limits of Linear Approximations
- 2026Where Steering Signals Come From: Activation Source Selection in Activation Steering
- 2026Forecasting Side Effects of Activation Steering
- 2026What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal