# Steering Large Language Model Activations in Sparse Spaces

arXiv: https://arxiv.org/abs/2503.00177 · HTML: https://mlsanity.com/paper/2503.00177/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability.
- First posted on arXiv 28 Feb 2025.

## Placed in

- Mechanistic interpretability / [Activation steering methods](https://mlsanity.com/mechinterp/activation-steering-methods/)

## Built on by (most cited first, 30 of 58)

- [Angular Steering: Behavior Control via Rotation in Activation Space](https://mlsanity.com/paper/2510.26243/) (2025)
- [Navigate the Unknown: Enhancing LLM Reasoning with Intrinsic Motivation Guided Exploration](https://mlsanity.com/paper/2505.17621/) (2025)
- [Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models](https://mlsanity.com/paper/2601.14004/) (2026)
- [Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atoms](https://mlsanity.com/paper/2505.20322/) (2025)
- [The Rogue Scalpel: Activation Steering Compromises LLM Safety](https://mlsanity.com/paper/2509.22067/) (2025)
- [Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Models](https://mlsanity.com/paper/2605.11887/) (2026)
- [Robust Hallucination Detection in LLMs via Adaptive Token Selection](https://mlsanity.com/paper/2504.07863/) (2025)
- [Adaptive Multi-Subspace Representation Steering for Attribute Alignment in Large Language Models](https://mlsanity.com/paper/2508.10599/) (2025)
- [Activation Steering with a Feedback Controller](https://mlsanity.com/paper/2510.04309/) (2025)
- [PIXEL: Adaptive Steering Via Position-wise Injection with eXact Estimated Levels under Subspace Calibration](https://mlsanity.com/paper/2510.10205/) (2025)
- [Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety](https://mlsanity.com/paper/2506.05451/) (2025)
- [Enhancing LLM Steering through Sparse Autoencoder-Based Vector Refinement](https://mlsanity.com/paper/2509.23799/) (2025)
- [Impatient Users Confuse AI Agents: High-fidelity Simulations of Human Traits for Testing Agents](https://mlsanity.com/paper/2510.04491/) (2025)
- [SAEdit: Token-level control for continuous image editing via Sparse AutoEncoder](https://mlsanity.com/paper/2510.05081/) (2025)
- [STEER-BENCH: A Benchmark for Evaluating the Steerability of Large Language Models](https://mlsanity.com/paper/2505.20645/) (2025)
- [Mechanistic Indicators of Understanding in Large Language Models](https://mlsanity.com/paper/2507.08017/) (2025)
- [From Weights to Activations: Is Steering the Next Frontier of Adaptation?](https://mlsanity.com/paper/2604.14090/) (2026)
- [EasyEdit2: An Easy-to-use Steering Framework for Editing Large Language Models](https://mlsanity.com/paper/2504.15133/) (2025)
- [Can LLMs Lie? Investigation beyond Hallucination](https://mlsanity.com/paper/2509.03518/) (2025)
- [Resa: Transparent Reasoning Models via SAEs](https://mlsanity.com/paper/2506.09967/) (2025)
- [AI shares emotion with humans across languages and cultures](https://mlsanity.com/paper/2506.13978/) (2025)
- [Sparse Feature Coactivation Reveals Causal Semantic Modules in Large Language Models](https://mlsanity.com/paper/2506.18141/) (2025)
- [RISER: Orchestrating Latent Reasoning Skills for Adaptive Activation Steering](https://mlsanity.com/paper/2601.09269/) (2026)
- [Sparse Autoencoders are Capable LLM Jailbreak Mitigators](https://mlsanity.com/paper/2602.12418/) (2026)
- [The Anatomy of Alignment: Decomposing Preference Optimization by Steering Sparse Features](https://mlsanity.com/paper/2509.12934/) (2025)
- [Compositional Steering of Large Language Models with Steering Tokens](https://mlsanity.com/paper/2601.05062/) (2026)
- [Global Evolutionary Steering: Refining Activation Steering Control via Cross-Layer Consistency](https://mlsanity.com/paper/2603.12298/) (2026)
- [Wiring the 'Why': A Unified Taxonomy and Survey of Abductive Reasoning in LLMs](https://mlsanity.com/paper/2604.08016/) (2026)
- [Perplexity Can Miss SAE Feature Damage Under Quantization](https://mlsanity.com/paper/2606.03002/) (2026)
- [Evaluating Sparse Autoencoders for Monosemantic Representation](https://mlsanity.com/paper/2508.15094/) (2025)

## Abstract (arXiv)

A key challenge in AI alignment is guiding large language models (LLMs) to follow desired behaviors at test time. Activation steering, which modifies internal model activations during inference, offers a potential solution. However, prior work in dense activation spaces struggles with superposition, wherein multiple features become entangled, limiting interpretability and precise control. In contrast, sparse representations provide an untapped opportunity for more interpretable behavior modulation. In this work, we introduce sparse activation steering (SAS), a method that leverages sparse autoencoders (SAEs) to steer LLM behavior in sparse spaces. By isolating behavior-specific features through a contrastive prompt-pairing approach, we define a set of features that can selectively reinforce or suppress behaviors. Experiments on Gemma 2 LLMs show that SAS vectors enable nuanced behavioral modulation and finer-grained control. Furthermore, scaling SAEs improves monosemanticity of SAS vectors, suggesting more reliable and interpretable interventions.