mlsanityData to 14 Oct 2026

Predicting Where Steering Vectors Succeed

arXiv 2604.15557 ↗First posted 16 Apr 2026cs.LG cs.CLcited by 4

On the map: Mechanistic interpretability → Activation steering methods

Abstract (arXiv)

Steering vectors work for some concepts and layers but fail for others, and practitioners have no way to predict which setting applies before running an intervention. We introduce the Linear Accessibility Profile (LAP), a per-layer diagnostic that repurposes the logit lens as a predictor of steering vector effectiveness. The key measure, Alin, applies the model's unembedding matrix to intermediate hidden states, requiring no training. Across 24 controlled binary concept families on five models (Pythia-2.8B to Llama-8B), peak Alin predicts steering effectiveness at ρ = +0.86 to +0.91 and layer selection at ρ = +0.63 to +0.92. A three-regime framework explains when difference-of-means steering works, when nonlinear methods are needed, and when no method can work. An entity-steering demo confirms the prediction end-to-end: steering at the LAP-recommended layer redirects completions on Gemma-2-2B and OLMo-2-1B-Instruct, while the middle layer (the standard heuristic) has no effect on either model.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability. Builds on 15 of its 21 references that are in the map. First posted on arXiv 16 Apr 2026.

Builds on

15 of 21 references.

References · 15

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  3. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  4. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  5. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  6. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  7. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  8. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 827
  9. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  10. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  11. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
  12. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
3 more
  1. 2023Jump to Conclusions: Short-Cutting Transformers With Linear Transformationscited by 90
  2. 2026Emotion Concepts and their Function in a Large Language Modelcited by 57
  3. 2024Recurrent Neural Networks Learn to Store and Generate Sequences using Non-Linear Representationscited by 47

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Decodable but Not Corrected by Fixed Residual-Stream Linear Steering: Evidence from Medical LLM Failure Regimescited by 2
  2. 2026Activation Steering Transfer to Agents: One Gain Ratio Does Not Identify Potency and Efficacycited by 0
  3. 2026Adaptive Multi-Value Control in LLMs via Causal Activation Steeringcited by 0
  4. 2026Concept Subspaces Compute Beyond the Logit Lens: A Weights-Only Test for Locating Representations Upstream of Readoutcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Understanding Unreliability of Steering Vectors in Language Models: Geometric Predictors and the Limits of Linear Approximations
  2. 2026What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal
  3. 2025Steering Awareness: Detecting Activation Steering from Within
  4. 2025Understanding (Un)Reliability of Steering Vectors in Language Models
  5. 2026On the Non-Identifiability of Steering Vectors in Large Language Models
  6. 2024Analyzing the Generalization and Reliability of Steering Vectors
  7. 2026Steerable but Not Decodable: Function Vectors Operate Beyond the Logit Lens
  8. 2026When is Your LLM Steerable?