# When Is Rank-1 Steering Cheap? Geometry, Granularity, and Budgeted Search

arXiv: https://arxiv.org/abs/2605.16362 · HTML: https://mlsanity.com/paper/2605.16362/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability.
- Builds on 17 of its 21 references that are in the map.
- First posted on arXiv 9 May 2026.

## Placed in

- Mechanistic interpretability / [Activation steering methods](https://mlsanity.com/mechinterp/activation-steering-methods/)

## Builds on (most cited first, 17 of 17)

- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [Persona Vectors: Monitoring and Controlling Character Traits in Language Models](https://mlsanity.com/paper/2507.21509/) (2025)
- [AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders](https://mlsanity.com/paper/2501.17148/) (2025)
- [Programming Refusal with Conditional Activation Steering](https://mlsanity.com/paper/2409.05907/) (2024)
- [Improving Instruction-Following in Language Models through Activation Steering](https://mlsanity.com/paper/2410.12877/) (2024)
- [Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization](https://mlsanity.com/paper/2406.00045/) (2024)
- [Steering Language Model Refusal with Sparse Autoencoders](https://mlsanity.com/paper/2411.11296/) (2024)
- [Understanding (Un)Reliability of Steering Vectors in Language Models](https://mlsanity.com/paper/2505.22637/) (2025)
- [A Unified Understanding and Evaluation of Steering Methods](https://mlsanity.com/paper/2502.02716/) (2025)
- [HyperSteer: Activation Steering at Scale with Hypernetworks](https://mlsanity.com/paper/2506.03292/) (2025)
- [Improved Representation Steering for Language Models](https://mlsanity.com/paper/2505.20809/) (2025)
- [Steering LLM Reasoning Through Bias-Only Adaptation](https://mlsanity.com/paper/2505.18706/) (2025)
- [Steering Vector Fields for Context-Aware Inference-Time Control in Large Language Models](https://mlsanity.com/paper/2602.01654/) (2026)
- [What Can We Actually Steer? A Multi-Behavior Study of Activation Control](https://mlsanity.com/paper/2511.18284/) (2025)
- [Persona Vectors in Games: Measuring and Steering Strategies via Activation Vectors](https://mlsanity.com/paper/2603.21398/) (2026)

## Built on by (most cited first, 1 of 1)

- [A Causal Model for Locating and Unlocking Sandbagging in Model Organisms](https://mlsanity.com/paper/2608.29461/) (2026)

## Abstract (arXiv)

Activation steering offers a lightweight way to control LLMs without retraining, but its effectiveness varies sharply across concepts. Prior work often reads this variability as evidence that many concepts are not captured by a single steering direction. We argue instead that much of it reflects search difficulty: a useful rank-1 intervention often exists, but finding it can be expensive. We formalize rank-1 steering as a budget-constrained optimization over intervention layer and coefficient. Across concepts and model families, prompt-boundary directional alignment predicts where effective interventions occur, enabling geometry-guided search that reaches high utility with substantially fewer evaluations, reducing the trials needed to recover 95% of best-found utility by 39.8% on average across three model families. To explain why some concepts remain expensive even under better search, we introduce concept granularity, a measure of directional heterogeneity across contrastive contexts. Granularity distinguishes concepts whose difference vectors share a stable global direction from those where prompts agree locally within each input but the utility-maximizing direction rotates systematically across inputs. Higher granularity is associated with slower convergence and lower best-found performance (Pearson r=0.44 with trials-to-95%, r=-0.46 with best-found utility, both p<0.001). We present GRACE, a Granularity- and Representation-Aware Concept Engineering framework that uses activation geometry to diagnose the dominant source of steering difficulty, select the appropriate remedy, and allocate optimization effort efficiently. Our results shift the frame from "when does rank-1 fail?" to "when is rank-1 cheap and stable?", turning activation geometry from a descriptive tool into an actionable prior for LLM control.