mlsanityData to 14 Oct 2026

Disentangling Steering Vectors

arXiv 2609.07037 ↗First posted 7 Sep 2026cs.LGcited by 1

On the map: Mechanistic interpretability → Activation steering methods

Abstract (arXiv)

Activation steering has emerged as a lightweight, inference-time approach to control the behavior of Large Language Models (LLMs). However, traditional steering vectors used to intervene in LLMs' activations, such as those derived from the difference-in-means method, tend to entangle multiple semantic and stylistic concepts into a single composite direction, leading to unpredictable steering effects. Our core objective is to disentangle this composite direction into its constituent concepts. To this end, we propose Steering Vector Dissection, a framework to explicitly isolate individual and semantically consistent features from these composite directions. Specifically, we pair positive and negative activations and take their differences to generate a set of instance-level steering vectors, and train a dedicated Sparse Autoencoder (SAE) directly on them. Quantitative evaluations across two datasets, two models, and two intervention depths show that our method yields a set of semantically consistent basis vectors whose steering effects are mutually distinguishable. Furthermore, we show that this disentanglement enables precise control over model behaviors.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability. Builds on 30 of its 37 references that are in the map. First posted on arXiv 7 Sep 2026.

Builds on

30 of 37 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Improving Instruction-Following in Language Models through Activation Steering15 Oct
  2. 2023Steering Llama 2 via Contrastive Activation Addition9 Dec
  3. 2023Steering Language Models With Activation Engineering20 Aug

Other references · 27

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  2. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  3. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  4. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  5. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  6. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  7. 2022Toy Models of Superpositioncited by 971
  8. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  9. 2024A StrongREJECT for Empty Jailbreakscited by 385
  10. 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 345
  11. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 300
  12. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 297
15 more
  1. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preferencecited by 261
  2. 2022The Alignment Problem from a Deep Learning Perspectivecited by 258
  3. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 218
  4. 2013k-Sparse Autoencoderscited by 215
  5. 2024ReFT: Representation Finetuning for Language Modelscited by 197
  6. 2026Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnetcited by 191
  7. 2025Learning Multi-Level Features with Matryoshka Sparse Autoencoderscited by 143
  8. 2024BatchTopK Sparse Autoencoderscited by 130
  9. 2024Sycophancy in Large Language Models: Causes and Mitigationscited by 101
  10. 2024Don't Listen To Me: Understanding and Exploring Jailbreak Prompts of Large Language Modelscited by 86
  11. 2025Taxonomy, Opportunities, and Challenges of Representation Engineering for Large Language Modelscited by 34
  12. 2025Sparse Autoencoders for Hypothesis Generationcited by 26
  13. 2025What's In My Human Feedback? Learning Interpretable Descriptions of Preference Datacited by 26
  14. 2024Disentangling Dense Embeddings with Sparse Autoencoderscited by 23
  15. 2025Sparse Shift Autoencoders for Identifying Concepts from Large Language Model Activationscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026The Pain Axis: LLMs Represent Self-Directed Harm and Act on Itcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026On the Non-Identifiability of Steering Vectors in Large Language Models
  2. 2025LF-Steering: Latent Feature Activation Steering for Enhancing Semantic Consistency in Large Language Models
  3. 2025Understanding (Un)Reliability of Steering Vectors in Language Models
  4. 2024Activation Scaling for Steering and Interpreting Language Models
  5. 2025Interpretable Steering of Large Language Models with Feature Guided Activation Additions
  6. 2026Signatures of Steerability in Activation Space of Language Models
  7. 2026COLD-Steer: Steering Large Language Models via In-Context One-step Learning Dynamics
  8. 2026Understanding Unreliability of Steering Vectors in Language Models: Geometric Predictors and the Limits of Linear Approximations