Deployable Per-Instance Multi-Layer Activation Steering for Large Language Models
On the map: Mechanistic interpretability → Activation steering methods
Abstract (arXiv)
Activation steering edits the behaviour of a frozen language model by adding a learned vector to its residual stream, and current practice fixes the injection layers globally per task. We argue that the best layers are an instance-level decision, and we make per-instance, multi-layer selection both well understood and deployable. On two open-weight 8B models and six binary persona traits, a per-instance oracle over layer subsets shows that the best layers vary from one input to the next: on most trait-model pairs, no fixed global layer set recovers the per-instance benefit. A greedy rule that ranks layers by single-layer marginal effect recovers nearly all of the oracle's benefit, but both must score candidate layers against the gold answer, so neither can run at deployment; the rule instead becomes the target a prompt-only predictor is trained to reproduce. Our deployable recipe needs no label at inference: a per-instance layer ranker read off the prompt embedding, a classifier that infers the steering direction, and an adaptive gate that scores short steered passes against that inferred direction and steers no more layers than necessary. The recipe recovers most of the oracle's lift (the bulk on the stronger model, a clear majority on the harder one), never drives any trait-model pair below its unsteered alignment baseline on average, and largely avoids the fluency collapse that strong global selection incurs at higher layer counts. A mechanistic account, "direction over magnitude", explains the behavioural flip under a mis-directed global set, the output collapse from steering too many layers, and the ceiling of unsteerable inputs.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability. Builds on 17 of its 27 references that are in the map. First posted on arXiv 9 Aug 2026.
Builds on
17 of 27 references.
Closest ancestors · 7
Same direction, newest first
- 2026Where to Steer: Input-Dependent Layer Selection for Steering Improves LLM Alignment4 Apr
- 2026The Effectiveness of Style Vectors for Steering Large Language Models: A Human Evaluation29 Jan
- 2025Learning to Steer: Input-dependent Steering for Multimodal LLMs18 Aug
- 2025Interpretable Steering of Large Language Models with Feature Guided Activation Additions17 Jan
- 2024Multi-property Steering of Large Language Models with Dynamic Activation Composition25 Jun
- 2023Steering Llama 2 via Contrastive Activation Addition9 Dec
- 2023Steering Language Models With Activation Engineering20 Aug
Other references · 10
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
- 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
- 2024Programming Refusal with Conditional Activation Steeringcited by 206
- 2024Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categoriescited by 77
- 2025Angular Steering: Behavior Control via Rotation in Activation Spacecited by 53
- 2025To Steer or Not to Steer? Mechanistic Error Reduction with Abstention for Language Modelscited by 29
- 2025Beyond Linear Steering: Unified Multi-Attribute Control for Language Modelscited by 25
- 2025Activation-Space Personality Steering: Hybrid Layer Selection for Stable Trait Control in LLMscited by 5
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Replay-Gated Neural Execution: Decoupling Persistent Behavioral Specifications from Neural Realizations in Frozen Language Modelscited by 0
- 2026First-Order Steering: Translating Weight Adaptation into Activation Steeringcited by 0
- 2026Input-Blind Controls Produce Substantial Oracle Headroom for Layer Programs in Multiple-Choice Evaluationcited by 0
- 2026Input-Blind Controls Produce Substantial Oracle Headroom for Layer Programs in Multiple-Choice Evaluation7 Oct 2026
- 2026First-Order Steering: Translating Weight Adaptation into Activation Steering3 Oct 2026
- 2026Replay-Gated Neural Execution: Decoupling Persistent Behavioral Specifications from Neural Realizations in Frozen Language Models5 Sep 2026
Similar, not cited
Close in wording, no citation either way
- 2026Genesis Steering: Better Activation Steering Through Training Data
- 2026Global Evolutionary Steering: Refining Activation Steering Control via Cross-Layer Consistency
- 2026COLD-Steer: Steering Large Language Models via In-Context One-step Learning Dynamics
- 2025Steering When Necessary: Flexible Steering Large Language Models with Backtracking
- 2025Painless Activation Steering: An Automated, Lightweight Approach for Post-Training Large Language Models
- 2026On the Non-Identifiability of Steering Vectors in Large Language Models
- 2026SAE-StatSteer: Statistical Consensus Feature Selection for Optimization-Free Activation Steering of Large Language Models
- 2026Minimally Invasive Steering of Language Models