# Understanding (Un)Reliability of Steering Vectors in Language Models

arXiv: https://arxiv.org/abs/2505.22637 · HTML: https://mlsanity.com/paper/2505.22637/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability.
- Builds on 22 of its 30 references that are in the map.
- First posted on arXiv 28 May 2025.

## Placed in

- Mechanistic interpretability / [Activation steering methods](https://mlsanity.com/mechinterp/activation-steering-methods/)

## Builds on (most cited first, 22 of 22)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Refusal in Language Models Is Mediated by a Single Direction](https://mlsanity.com/paper/2406.11717/) (2024)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [In-Context Learning Creates Task Vectors](https://mlsanity.com/paper/2310.15916/) (2023)
- [Function Vectors in Large Language Models](https://mlsanity.com/paper/2310.15213/) (2023)
- [Extracting Latent Steering Vectors from Pretrained Language Models](https://mlsanity.com/paper/2205.05124/) (2022)
- [Linear Representations of Sentiment in Large Language Models](https://mlsanity.com/paper/2310.15154/) (2023)
- [Analyzing the Generalization and Reliability of Steering Vectors](https://mlsanity.com/paper/2407.12404/) (2024)
- [Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization](https://mlsanity.com/paper/2406.00045/) (2024)
- [A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods](https://mlsanity.com/paper/2403.02901/) (2024)
- [Style Vectors for Steering Generative Large Language Model](https://mlsanity.com/paper/2402.01618/) (2024)
- [Representation Surgery: Theory and Practice of Affine Steering](https://mlsanity.com/paper/2402.09631/) (2024)
- [Steering Without Side Effects: Improving Post-Deployment Control of Language Models](https://mlsanity.com/paper/2406.15518/) (2024)
- [Towards Reliable Evaluation of Behavior Steering Interventions in LLMs](https://mlsanity.com/paper/2410.17245/) (2024)
- [Refusal in LLMs is an Affine Function](https://mlsanity.com/paper/2411.09003/) (2024)
- [NEWTS: A Corpus for News Topic-Focused Summarization](https://mlsanity.com/paper/2205.15661/) (2022)
- [Controllable Text Summarization: Unraveling Challenges, Approaches, and Prospects -- A Survey](https://mlsanity.com/paper/2311.09212/) (2023)
- [Comparing Bottom-Up and Top-Down Steering Approaches on In-Context Learning Tasks](https://mlsanity.com/paper/2411.07213/) (2024)

## Built on by (most cited first, 30 of 47)

- [Valence-Arousal Subspace in LLMs: Circular Emotion Geometry and Multi-Behavioral Control](https://mlsanity.com/paper/2604.03147/) (2026)
- [Steering Vector Fields for Context-Aware Inference-Time Control in Large Language Models](https://mlsanity.com/paper/2602.01654/) (2026)
- [Curveball Steering: The Right Direction To Steer Isn't Always Linear](https://mlsanity.com/paper/2603.09313/) (2026)
- [Analysing the Safety Pitfalls of Steering Vectors](https://mlsanity.com/paper/2603.24543/) (2026)
- [What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal](https://mlsanity.com/paper/2604.08524/) (2026)
- [Steerable but Not Decodable: Function Vectors Operate Beyond the Logit Lens](https://mlsanity.com/paper/2604.02608/) (2026)
- [RouteHijack: Routing-Aware Attack on Mixture-of-Experts LLMs](https://mlsanity.com/paper/2605.02946/) (2026)
- [Prompt-Activation Duality: Improving Activation Steering via Attention-Level Interventions](https://mlsanity.com/paper/2605.10664/) (2026)
- [FishBack: Pullback Fisher Geometry for Optimal Activation Steering in Transformers](https://mlsanity.com/paper/2605.17231/) (2026)
- [Global Evolutionary Steering: Refining Activation Steering Control via Cross-Layer Consistency](https://mlsanity.com/paper/2603.12298/) (2026)
- [The Cylindrical Representation Hypothesis for Language Model Steering](https://mlsanity.com/paper/2605.01844/) (2026)
- [Operationalising the Superficial Alignment Hypothesis via Task Complexity](https://mlsanity.com/paper/2602.15829/) (2026)
- [A Low-Rank Subspace Analysis of LLM Interventions](https://mlsanity.com/paper/2606.14388/) (2026)
- [Beyond Multiple Choice: Evaluating Steering Vectors for Summarization](https://mlsanity.com/paper/2505.24859/) (2025)
- [Bifrost: Steering Strategic Trajectories to Bridge Contextual Gaps for Self-Improving Agents](https://mlsanity.com/paper/2602.05810/) (2026)
- [Nudging Hidden States: Training-Free Model Steering for Chain-of-Thought Reasoning in Large Audio-Language Models](https://mlsanity.com/paper/2603.14636/) (2026)
- [Over-Refusal and Representation Subspaces: A Mechanistic Analysis of Task-Conditioned Refusal in Aligned LLMs](https://mlsanity.com/paper/2603.27518/) (2026)
- [Detecting and Controlling Sycophancy with Cascading Linear Features](https://mlsanity.com/paper/2606.26155/) (2026)
- [Conditional Optimal Bridge for Riemannian Activation Steering](https://mlsanity.com/paper/2607.10517/) (2026)
- [Sycophancy Undermines Epistemic Vigilance in Cooperative Vision-Language Tasks](https://mlsanity.com/paper/2607.29585/) (2026)
- [Forecasting Side Effects of Activation Steering](https://mlsanity.com/paper/2608.11227/) (2026)
- [Confidence is Not Competence](https://mlsanity.com/paper/2510.24772/) (2025)
- [When Is Rank-1 Steering Cheap? Geometry, Granularity, and Budgeted Search](https://mlsanity.com/paper/2605.16362/) (2026)
- [Adversarial Robustness of Activation Steering in Large Language Models](https://mlsanity.com/paper/2606.07696/) (2026)
- [Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effects](https://mlsanity.com/paper/2607.24645/) (2026)
- [SteerCheck: Attribution Specificity and Alignment Leakage in Activation-Steering Audits](https://mlsanity.com/paper/2608.24335/) (2026)
- [Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness](https://mlsanity.com/paper/2609.03887/) (2026)
- [Logit Reweighting for Topic-Focused Summarization](https://mlsanity.com/paper/2507.05235/) (2025)
- [Navigating through the hidden embedding space: steering LLMs to improve mental health assessment](https://mlsanity.com/paper/2510.16373/) (2025)
- [AntiPaSTO: Self-Supervised Honesty Steering via Anti-Parallel Representations](https://mlsanity.com/paper/2601.07473/) (2026)

## Abstract (arXiv)

Steering vectors are a lightweight method to control language model behavior by adding a learned bias to the activations at inference time. Although steering demonstrates promising performance, recent work shows that it can be unreliable or even counterproductive in some cases. This paper studies the influence of prompt types and the geometry of activation differences on steering reliability. First, we find that all seven prompt types used in our experiments produce a net positive steering effect, but exhibit high variance across samples, and often give an effect opposite of the desired one. No prompt type clearly outperforms the others, and yet the steering vectors resulting from the different prompt types often differ directionally (as measured by cosine similarity). Second, we show that higher cosine similarity between training set activation differences predicts more effective steering. Finally, we observe that datasets where positive and negative activations are better separated are more steerable. Our results suggest that vector steering is unreliable when the target behavior is not represented by a coherent direction.