mlsanityData to 14 Oct 2026

Understanding (Un)Reliability of Steering Vectors in Language Models

arXiv 2505.22637 ↗First posted 28 May 2025cs.LGcited by 47

On the map: Mechanistic interpretability → Activation steering methods

Abstract (arXiv)

Steering vectors are a lightweight method to control language model behavior by adding a learned bias to the activations at inference time. Although steering demonstrates promising performance, recent work shows that it can be unreliable or even counterproductive in some cases. This paper studies the influence of prompt types and the geometry of activation differences on steering reliability. First, we find that all seven prompt types used in our experiments produce a net positive steering effect, but exhibit high variance across samples, and often give an effect opposite of the desired one. No prompt type clearly outperforms the others, and yet the steering vectors resulting from the different prompt types often differ directionally (as measured by cosine similarity). Second, we show that higher cosine similarity between training set activation differences predicts more effective steering. Finally, we observe that datasets where positive and negative activations are better separated are more steerable. Our results suggest that vector steering is unreliable when the target behavior is not represented by a coherent direction.

Key facts. Cited by 47 papers in the map (data through 14 Oct 2026). Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability. Builds on 22 of its 30 references that are in the map. First posted on arXiv 28 May 2025.

Builds on

22 of 30 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Towards Reliable Evaluation of Behavior Steering Interventions in LLMs22 Oct
  2. 2024Analyzing the Generalization and Reliability of Steering Vectors17 Jul
  3. 2024Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization28 May
  4. 2023Steering Llama 2 via Contrastive Activation Addition9 Dec
  5. 2022Extracting Latent Steering Vectors from Pretrained Language Models10 May

Other references · 17

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  2. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  3. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  4. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  5. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  6. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  7. 2023In-Context Learning Creates Task Vectorscited by 312
  8. 2023Function Vectors in Large Language Modelscited by 308
  9. 2023Linear Representations of Sentiment in Large Language Modelscited by 163
  10. 2024A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methodscited by 102
  11. 2024Style Vectors for Steering Generative Large Language Modelcited by 56
  12. 2024Representation Surgery: Theory and Practice of Affine Steeringcited by 50
5 more
  1. 2024Steering Without Side Effects: Improving Post-Deployment Control of Language Modelscited by 46
  2. 2024Refusal in LLMs is an Affine Functioncited by 29
  3. 2022NEWTS: A Corpus for News Topic-Focused Summarizationcited by 27
  4. 2023Controllable Text Summarization: Unraveling Challenges, Approaches, and Prospects -- A Surveycited by 18
  5. 2024Comparing Bottom-Up and Top-Down Steering Approaches on In-Context Learning Taskscited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 43.

47 papers.

Sort:

By area:

  1. 2026Valence-Arousal Subspace in LLMs: Circular Emotion Geometry and Multi-Behavioral Controlcited by 16
  2. 2026Steering Vector Fields for Context-Aware Inference-Time Control in Large Language Modelscited by 13
  3. 2026Curveball Steering: The Right Direction To Steer Isn't Always Linearcited by 11
  4. 2026Analysing the Safety Pitfalls of Steering Vectorscited by 10
  5. 2026What Drives Representation Steering? A Mechanistic Case Study on Steering Refusalcited by 7
  6. 2026Steerable but Not Decodable: Function Vectors Operate Beyond the Logit Lenscited by 6
  7. 2026RouteHijack: Routing-Aware Attack on Mixture-of-Experts LLMscited by 6
  8. 2026Prompt-Activation Duality: Improving Activation Steering via Attention-Level Interventionscited by 6
  9. 2026FishBack: Pullback Fisher Geometry for Optimal Activation Steering in Transformerscited by 6
  10. 2026Global Evolutionary Steering: Refining Activation Steering Control via Cross-Layer Consistencycited by 5
  11. 2026The Cylindrical Representation Hypothesis for Language Model Steeringcited by 5
  12. 2026Operationalising the Superficial Alignment Hypothesis via Task Complexitycited by 4
35 more
  1. 2026A Low-Rank Subspace Analysis of LLM Interventionscited by 3
  2. 2025Beyond Multiple Choice: Evaluating Steering Vectors for Summarizationcited by 2
  3. 2026Bifrost: Steering Strategic Trajectories to Bridge Contextual Gaps for Self-Improving Agentscited by 2
  4. 2026Nudging Hidden States: Training-Free Model Steering for Chain-of-Thought Reasoning in Large Audio-Language Modelscited by 2
  5. 2026Over-Refusal and Representation Subspaces: A Mechanistic Analysis of Task-Conditioned Refusal in Aligned LLMscited by 2
  6. 2026Detecting and Controlling Sycophancy with Cascading Linear Featurescited by 2
  7. 2026Conditional Optimal Bridge for Riemannian Activation Steeringcited by 2
  8. 2026Sycophancy Undermines Epistemic Vigilance in Cooperative Vision-Language Taskscited by 2
  9. 2026Forecasting Side Effects of Activation Steeringcited by 2
  10. 2025Confidence is Not Competencecited by 1
  11. 2026When Is Rank-1 Steering Cheap? Geometry, Granularity, and Budgeted Searchcited by 1
  12. 2026Adversarial Robustness of Activation Steering in Large Language Modelscited by 1
  13. 2026Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effectscited by 1
  14. 2026SteerCheck: Attribution Specificity and Alignment Leakage in Activation-Steering Auditscited by 1
  15. 2026Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustnesscited by 1
  16. 2025Logit Reweighting for Topic-Focused Summarizationcited by 0
  17. 2025Navigating through the hidden embedding space: steering LLMs to improve mental health assessmentcited by 0
  18. 2026AntiPaSTO: Self-Supervised Honesty Steering via Anti-Parallel Representationscited by 0
  19. 2026MoRFI: Monotonic Sparse Autoencoder Feature Identificationcited by 0
  20. 2026VSPO: Vector-Steered Policy Optimization for Behavioral Controlcited by 0
  21. 2026Temporal Preference Concepts and their Functions in a Large Language Modelcited by 0
  22. 2026Inverted Detection and Control in Steering Vectorscited by 0
  23. 2026Steering Interference Reflects the Model's Defaults, Not the Behavior Directionscited by 0
  24. 2026Signatures of Steerability in Activation Space of Language Modelscited by 0
  25. 2026Semantic Fibers and Cross-Gram Interference: A Calculus of Safety Drift in Overcomplete Representationscited by 0
  26. 2026From Trait Vectors to Circuits: Tracing Refusal and Sycophancy Through Language Modelscited by 0
  27. 2026Cosine Similarity Is Not Evidence: Measuring the Noise Floor of Interpretability Transfer Under Quantizationcited by 0
  28. 2026Guarded Gradient-Based Activation Steering of Shutdown Responses in Qwen3.5-0.8B: A Minimum-Step Policycited by 0
  29. 2026Write Back the Delta: Revisiting the Same Tokens with Fresh Representationscited by 0
  30. 2026First-Order Steering: Translating Weight Adaptation into Activation Steeringcited by 0
  31. 2026Steering Follows Geometry, Not Labels: Emotion Directions in a Full-Duplex Speech Modelcited by 0
  32. 2026What Survives When Activation Steering Is Used as Memorycited by 0
  33. 2026The Sharp Directions Are Against You: Curvature Analysis of Activation Steeringcited by 0
  34. 2026Genesis Steering: Better Activation Steering Through Training Datacited by 0
  35. 2026Steering Vectors for Competing Ethical Frameworks Are Interchangeable: A Competing-Target Test for Activation Steeringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Understanding Unreliability of Steering Vectors in Language Models: Geometric Predictors and the Limits of Linear Approximations
  2. 2026Does Steering Break Your Model? A Multi-Dimensional Evaluation Suite for LLM Steering Methods
  3. 2026Disentangling Steering Vectors
  4. 2024Activation Scaling for Steering and Interpreting Language Models
  5. 2026Mechanistic Indicators of Steering Effectiveness in Large Language Models
  6. 2026On the Non-Identifiability of Steering Vectors in Large Language Models
  7. 2024Improving Instruction-Following in Language Models through Activation Steering
  8. 2025On the Limitations of Steering in Language Model Alignment