# Contextual Linear Activation Steering of Language Models

arXiv: https://arxiv.org/abs/2604.24693 · HTML: https://mlsanity.com/paper/2604.24693/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability.
- Builds on 13 of its 40 references that are in the map.
- First posted on arXiv 27 Apr 2026.

## Placed in

- Mechanistic interpretability / [Activation steering methods](https://mlsanity.com/mechinterp/activation-steering-methods/)

## Builds on (most cited first, 13 of 13)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [Persona Vectors: Monitoring and Controlling Character Traits in Language Models](https://mlsanity.com/paper/2507.21509/) (2025)
- [Extending Activation Steering to Broad Skills and Multiple Behaviours](https://mlsanity.com/paper/2403.05767/) (2024)
- [Activation Steering for Chain-of-Thought Compression](https://mlsanity.com/paper/2507.04742/) (2025)
- [Representation Engineering for Large-Language Models: Survey and Research Challenges](https://mlsanity.com/paper/2502.17601/) (2025)
- [Efficient and accurate steering of Large Language Models through attention-guided feature learning](https://mlsanity.com/paper/2602.00333/) (2026)

## Built on by (most cited first, 7 of 7)

- [Prompt-Activation Duality: Improving Activation Steering via Attention-Level Interventions](https://mlsanity.com/paper/2605.10664/) (2026)
- [Does Steering Break Your Model? A Multi-Dimensional Evaluation Suite for LLM Steering Methods](https://mlsanity.com/paper/2610.07722/) (2026)
- [Metacognitive Steering: Learning the Structure of Scientific Judgment](https://mlsanity.com/paper/2609.16245/) (2026)
- [Replay-Gated Neural Execution: Decoupling Persistent Behavioral Specifications from Neural Realizations in Frozen Language Models](https://mlsanity.com/paper/2609.22243/) (2026)
- [MetaSteer: Context-Conditioned, nonlinear Steering via Attention-Projection Adaptation](https://mlsanity.com/paper/2609.38718/) (2026)
- [Hesitation Has a Geometry: Entropy-Trained Hyperbolic Probes for Sparse Activation Steering](https://mlsanity.com/paper/2610.02391/) (2026)
- [Exact Dynamics and Finite-Sample Trajectory Recovery of Linear Recursive Feature Machines](https://mlsanity.com/paper/2610.09196/) (2026)

## Abstract (arXiv)

Linear activation steering is a powerful approach for eliciting the capabilities of large language models and specializing their behavior using limited labeled data. While effective, existing methods often apply a fixed steering strength to all tokens, resulting in inconsistent steering quality across diverse input prompts. In this work, we introduce Contextual Linear Activation Steering (CLAS), a method that dynamically adapts linear activation steering to context-dependent steering strengths. Across eleven steering benchmarks and four model families, it consistently outperforms standard linear activation steering and matches or exceeds the performance of ReFT and LoRA in settings with limited labeled data. We therefore propose CLAS as a scalable, interpretable, and accurate method for specializing and steering large language models.