mlsanityData to 14 Oct 2026

Analyzing the Generalization and Reliability of Steering Vectors

arXiv 2407.12404 ↗First posted 17 Jul 2024cs.LGcited by 143

On the map: Mechanistic interpretability → Activation steering methods

Abstract (arXiv)

Steering vectors (SVs) have been proposed as an effective approach to adjust language model behaviour at inference time by intervening on intermediate model activations. They have shown promise in terms of improving both capabilities and model alignment. However, the reliability and generalisation properties of this approach are unknown. In this work, we rigorously investigate these properties, and show that steering vectors have substantial limitations both in- and out-of-distribution. In-distribution, steerability is highly variable across different inputs. Depending on the concept, spurious biases can substantially contribute to how effective steering is for each input, presenting a challenge for the widespread use of steering vectors. Out-of-distribution, while steering vectors often generalise well, for several concepts they are brittle to reasonable changes in the prompt, resulting in them failing to generalise well. Overall, our findings show that while steering can work well in the right circumstances, there remain technical difficulties of applying steering vectors to guide models' behaviour at scale. Our code is available at https://github.com/dtch1997/steering-bench

Key facts. Cited by 143 papers in the map (data through 14 Oct 2026). Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability. Builds on 32 of its 41 references that are in the map. First posted on arXiv 17 Jul 2024.

Builds on

32 of 41 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Extending Activation Steering to Broad Skills and Multiple Behaviours9 Mar
  2. 2023Steering Llama 2 via Contrastive Activation Addition9 Dec

Other references · 30

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  7. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  8. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  9. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  10. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  11. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  12. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
18 more
  1. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  2. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  3. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  4. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  5. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  6. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 431
  7. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 396
  8. 2024The Platonic Representation Hypothesiscited by 382
  9. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
  10. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 216
  11. 2023Linear Representations of Sentiment in Large Language Modelscited by 163
  12. 2024Universal Neurons in GPT2 Language Modelscited by 102
  13. 2023Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineeringcited by 94
  14. 2023Eliciting Latent Knowledge from Quirky Language Modelscited by 62
  15. 2023Concept Algebra for (Score-Based) Text-Controlled Generative Modelscited by 59
  16. 2022Interpreting Neural Networks through the Polytope Lenscited by 33
  17. 2024Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Modelscited by 29
  18. 2023Generalization Analogies: A Testbed for Generalizing AI Oversight to Hard-To-Measure Domainscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 44; 2026: 93.

143 papers.

Sort:

By area:

  1. 2024Programming Refusal with Conditional Activation Steeringcited by 206
  2. 2025LLM Social Simulations Are a Promising Research Methodcited by 99
  3. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 95
  4. 2025Do Multilingual LLMs Think In English?cited by 78
  5. 2025Angular Steering: Behavior Control via Rotation in Activation Spacecited by 53
  6. 2025Understanding (Un)Reliability of Steering Vectors in Language Modelscited by 47
  7. 2025A Unified Understanding and Evaluation of Steering Methodscited by 46
  8. 2024Towards Reliable Evaluation of Behavior Steering Interventions in LLMscited by 36
  9. 2025Steering off Course: Reliability Challenges in Steering Language Modelscited by 34
  10. 2025The Hawthorne Effect in Reasoning Models: Evaluating and Steering Test Awarenesscited by 30
  11. 2025To Steer or Not to Steer? Mechanistic Error Reduction with Abstention for Language Modelscited by 29
  12. 2026Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behaviorcited by 26
38 more
  1. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atomscited by 24
  2. 2025Representation Engineering for Large-Language Models: Survey and Research Challengescited by 21
  3. 2025The Rogue Scalpel: Activation Steering Compromises LLM Safetycited by 21
  4. 2025Test-Time Adaptation for LLM Agents via Environment Interactioncited by 20
  5. 2025Steering Language Models with Weight Arithmeticcited by 20
  6. 2024Can sparse autoencoders be used to decompose and interpret steering vectors?cited by 19
  7. 2025The Elicitation Game: Evaluating Capability Elicitation Techniquescited by 18
  8. 2026Valence-Arousal Subspace in LLMs: Circular Emotion Geometry and Multi-Behavioral Controlcited by 16
  9. 2025Spilling the Beans: Teaching LLMs to Self-Report Their Hidden Objectivescited by 15
  10. 2024Comparing Bottom-Up and Top-Down Steering Approaches on In-Context Learning Taskscited by 14
  11. 2026Steering Vector Fields for Context-Aware Inference-Time Control in Large Language Modelscited by 13
  12. 2026Subliminal Learning Is Steering Vector Distillationcited by 13
  13. 2025CorrSteer: Generation-Time LLM Steering via Correlated Sparse Autoencoder Featurescited by 12
  14. 2026The Information Geometry of Softmax: Probing and Steeringcited by 11
  15. 2026Curveball Steering: The Right Direction To Steer Isn't Always Linearcited by 11
  16. 2025Simple Mechanistic Explanations for Out-Of-Context Reasoningcited by 10
  17. 2025Enhancing LLM Steering through Sparse Autoencoder-Based Vector Refinementcited by 10
  18. 2026Analysing the Safety Pitfalls of Steering Vectorscited by 10
  19. 2026Why Steering Works: Toward a Unified View of Language Model Parameter Dynamicscited by 9
  20. 2026Local Linearity of LLMs Enables Activation Steering via Model-Based Linear Optimal Controlcited by 9
  21. 2025Can Interpretation Predict Behavior on Unseen Data?cited by 8
  22. 2025Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoderscited by 8
  23. 2026From Data to Behavior: Predicting Unintended Model Behaviors Before Trainingcited by 8
  24. 2026Causality is Key for Interpretability Claims to Generalisecited by 8
  25. 2026Steered LLM Activations are Non-Surjectivecited by 8
  26. 2025Linear Representation Transferability Hypothesis: Leveraging Small Models to Steer Large Modelscited by 7
  27. 2025Neologism Learning for Controllability and Self-Verbalizationcited by 6
  28. 2025What Can We Actually Steer? A Multi-Behavior Study of Activation Controlcited by 6
  29. 2026Concept Heterogeneity-aware Representation Steeringcited by 6
  30. 2026Steerable but Not Decodable: Function Vectors Operate Beyond the Logit Lenscited by 6
  31. 2026Prompt-Activation Duality: Improving Activation Steering via Attention-Level Interventionscited by 6
  32. 2024SHARP: Unlocking Interactive Hallucination via Stance Transfer in Role-Playing LLMscited by 5
  33. 2025Neurons Speak in Ranges: Breaking Free from Discrete Neuronal Attributioncited by 5
  34. 2025Overcoming Multi-step Complexity in Multimodal Theory-of-Mind Reasoning: A Scalable Bayesian Plannercited by 5
  35. 2025Representation Consistency for Accurate and Coherent LLM Answer Aggregationcited by 5
  36. 2025Activation-Space Personality Steering: Hybrid Layer Selection for Stable Trait Control in LLMscited by 5
  37. 2026Global Evolutionary Steering: Refining Activation Steering Control via Cross-Layer Consistencycited by 5
  38. 2026The Cylindrical Representation Hypothesis for Language Model Steeringcited by 5

50 most cited of 143 All 143 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Understanding Unreliability of Steering Vectors in Language Models: Geometric Predictors and the Limits of Linear Approximations
  2. 2026Towards Steering without Sacrifice: Principled Training of Steering Vectors for Prompt-only Interventions
  3. 2026Signatures of Steerability in Activation Space of Language Models
  4. 2026Genesis Steering: Better Activation Steering Through Training Data
  5. 2026Disentangling Steering Vectors
  6. 2026On the Non-Identifiability of Steering Vectors in Large Language Models
  7. 2026What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal
  8. 2026FineSteer: A Unified Framework for Fine-Grained Inference-Time Steering in Large Language Models