# Activation Flow: Manufacturing Activations for Steering

arXiv: https://arxiv.org/abs/2609.32530 · HTML: https://mlsanity.com/paper/2609.32530/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 51 references that are in the map.
- First posted on arXiv 26 Sep 2026.

## Builds on (most cited first, 30 of 31)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering](https://mlsanity.com/paper/1809.02789/) (2018)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [ReFT: Representation Finetuning for Language Models](https://mlsanity.com/paper/2404.03592/) (2024)
- [AI Sandbagging: Language Models can Strategically Underperform on Evaluations](https://mlsanity.com/paper/2406.07358/) (2024)
- [Verbalizable Representations Form a Global Workspace in Language Models](https://mlsanity.com/paper/2607.15495/) (2026)
- [Stress-Testing Capability Elicitation With Password-Locked Models](https://mlsanity.com/paper/2405.19550/) (2024)
- [Language Models are Injective and Hence Invertible](https://mlsanity.com/paper/2510.15511/) (2025)
- [Refusal in LLMs is an Affine Function](https://mlsanity.com/paper/2411.09003/) (2024)
- [One-shot Optimized Steering Vectors Mediate Safety-relevant Behaviors in LLMs](https://mlsanity.com/paper/2502.18862/) (2025)
- [Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior](https://mlsanity.com/paper/2605.05115/) (2026)
- [Noise Injection Reveals Hidden Capabilities of Sandbagging Language Models](https://mlsanity.com/paper/2412.01784/) (2024)
- [Relational Composition in Neural Networks: A Survey and Call to Action](https://mlsanity.com/paper/2407.14662/) (2024)
- [The Elicitation Game: Evaluating Capability Elicitation Techniques](https://mlsanity.com/paper/2502.02180/) (2025)
- [The Shape of Beliefs: Geometry, Dynamics, and Interventions along Representation Manifolds of Language Models' Posteriors](https://mlsanity.com/paper/2602.02315/) (2026)
- [Small Vectors, Big Effects: A Mechanistic Study of RL-Induced Reasoning via Steering Vectors](https://mlsanity.com/paper/2509.06608/) (2025)
- [On the Non-Identifiability of Steering Vectors in Large Language Models](https://mlsanity.com/paper/2602.06801/) (2026)
- [Steered LLM Activations are Non-Surjective](https://mlsanity.com/paper/2604.09839/) (2026)
- [FishBack: Pullback Fisher Geometry for Optimal Activation Steering in Transformers](https://mlsanity.com/paper/2605.17231/) (2026)
- [The Model Organism Lottery: Model Organism Interpretability Strongly Depends on Training Methodology](https://mlsanity.com/paper/2607.01033/) (2026)
- [Removing Sandbagging in LLMs by Training with Weak Supervision](https://mlsanity.com/paper/2604.22082/) (2026)
- [Mechanistically Eliciting Latent Behaviors in Language Models](https://mlsanity.com/paper/2606.29604/) (2026)
- [Routing Subspaces: Auditing Evaluation-to-Deployment Mismatch in Fine-Tuned Language Models](https://mlsanity.com/paper/2607.20436/) (2026)
- [The information geometry of large language models is shared, learned, and controllable](https://mlsanity.com/paper/2609.11063/) (2026)
- [Reference-Grafting Matches Fine-Tuning at Eliciting Sandbagged Capabilities](https://mlsanity.com/paper/2608.29458/) (2026)

## Abstract (arXiv)

Difference-in-means steering requires activations recorded while a model shows the desired behavior, which a sandbagging model withholds by deliberately underperforming. We introduce Activation Flow (ActFlow), which manufactures these activations from k correct labels without fine-tuning. ActFlow sets target logits that rank each labeled item's correct answer first, and moves the logits toward them by adding one vector x to all k residual streams at one layer. ActFlow is a family of ordinary differential equations for x, one for each rule that maps the required logit change to the velocity of x. The smallest-norm rule lands exactly on the targets, while the others keep only the top singular directions of the Jacobian. We test ActFlow on three instruction-tuned models, each locked by a sandbagging prompt and by a password-locked LoRA. At k=40, ActFlow keeping five singular directions raises the mean held-out ARC-Easy accuracy over the six locked models from 0.05 to 0.85, against 0.88 for rank-16 LoRA fine-tuning, which trains about 3,000 times as many parameters, and 0.92 for the honest models. Furthermore, it scores higher than the smallest-norm rule in 16 of the 18 combinations of locked model and k, and its steering direction is nearly orthogonal to the honest difference-in-means direction. It also unlocks two LoRA locks where the honest direction fails. Surprisingly, with one labeled item, the single-step smallest-norm rule raises the Qwen2.5-7B prompt lock from 0.04 to 0.59.