Does the Model Use the Feature? Separating Steering from Mechanism in LLMs
Not on the map. Why?
Abstract (arXiv)
Internal features in LLMs are often interpreted as mechanisms when they track a concept and their manipulation changes a related behavior. Yet steering can push a feature far outside its natural range, where its effects need not reflect the model's own computation. We examine this inference and propose an empirical contract whose tests evaluate features at values observed on natural inputs. One test copies a feature's value from an input that shows a behavior into a matched input that does not (installation) or the reverse (removal); the other restores the feature after an upstream edit (downstream rescue). Installation measures how far the feature suffices for the behavior; removal and downstream rescue measure how much the model uses it. Applied to three kinds of representations, the two strengths separate sharply. The published unknown-entity latent strongly steers knowledge abstention, yet installing observed values from either published latent into matched prompts transfers only a small fraction of the natural known--unknown abstention contrast. Dense known--unknown directions show opposite asymmetries between installation and removal in Gemma and Llama, and how fully a released subject--verb agreement feature set reproduces and restores the behavior depends on how its values are written into the model. Tracking a concept and steering a behavior therefore do not by themselves show that the model uses a feature, and each conclusion holds only for the intervention tested.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 41 references that are in the map. First posted on arXiv 5 Oct 2026.
Builds on
22 of 41 references.
References · 22
- 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
- 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 345
- 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 300
- 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 243
- 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 218
- 2024How to use and interpret activation patchingcited by 214
- 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 205
- 2023The Hydra Effect: Emergent Self-repair in Language Model Computationscited by 156
- 2021Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Modelscited by 131
- 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independencecited by 115
- 2024RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representationscited by 79
- 2021Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Predictioncited by 59
10 more
- 2022Probing for the Usage of Grammatical Numbercited by 59
- 2026There Is More to Refusal in Large Language Models than a Single Directioncited by 26
- 2024Transformer Circuit Faithfulness Metrics are not Robustcited by 22
- 2020Amnesic Probing: Behavioral Explanation with Amnesic Counterfactualscited by 14
- 2026Steered LLM Activations are Non-Surjectivecited by 8
- 2026Perfect Detection, Failed Control: The Geometry of Knowing vs. Steering in Language Modelscited by 5
- 2026Faithful Bi-Directional Model Steering via Distribution Matching and Distributed Interchange Interventionscited by 4
- 2026Bucketing the Good Apples: A Method for Diagnosing and Improving Causal Abstractioncited by 1
- 2026Position: Mechanistic Interpretability Must Disclose Identification Assumptions for Causal Claimscited by 1
- 2026SAE Interventions are Unreliable: Post-Intervention Recovery of Suppressed Behaviorcited by 1
Built on by
Not cited yet.