# Prototype-Based Dynamic Steering for Large Language Models

arXiv: https://arxiv.org/abs/2510.05498 · HTML: https://mlsanity.com/paper/2510.05498/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 25 of its 38 references that are in the map.
- First posted on arXiv 7 Oct 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 25 of 25)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Refusal in Language Models Is Mediated by a Single Direction](https://mlsanity.com/paper/2406.11717/) (2024)
- [Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems](https://mlsanity.com/paper/1705.04146/) (2017)
- [In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering](https://mlsanity.com/paper/2311.06668/) (2023)
- [Programming Refusal with Conditional Activation Steering](https://mlsanity.com/paper/2409.05907/) (2024)
- [Inspecting and Editing Knowledge Representations in Language Models](https://mlsanity.com/paper/2304.00740/) (2023)
- [Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categories](https://mlsanity.com/paper/2406.00034/) (2024)
- [Spectral Editing of Activations for Large Language Model Alignment](https://mlsanity.com/paper/2405.09719/) (2024)
- [Householder Pseudo-Rotation: A Novel Approach to Activation Editing in LLMs with Direction-Magnitude Perspective](https://mlsanity.com/paper/2409.10053/) (2024)
- [Exploring the Personality Traits of LLMs through Latent Features Steering](https://mlsanity.com/paper/2410.10863/) (2024)
- [Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Models](https://mlsanity.com/paper/2505.15634/) (2025)
- [Large Language Models are Contrastive Reasoners](https://mlsanity.com/paper/2403.08211/) (2024)
- [Learning from Contrastive Prompts: Automated Optimization and Adaptation](https://mlsanity.com/paper/2409.15199/) (2024)
- [Focus On This, Not That! Steering LLMs with Adaptive Feature Specification](https://mlsanity.com/paper/2410.22944/) (2024)

## Built on by (most cited first, 6 of 6)

- [Towards Effective Theory of LLMs: A Representation Learning Approach](https://mlsanity.com/paper/2605.09294/) (2026)
- [Steered Generation via Gradient-Based Optimization on Sparse Query Features](https://mlsanity.com/paper/2605.23040/) (2026)
- [Cognitive Fatigue in Autoregressive Transformers: Formalization and Measurement](https://mlsanity.com/paper/2605.30981/) (2026)
- [Metacognitive Steering: Learning the Structure of Scientific Judgment](https://mlsanity.com/paper/2609.16245/) (2026)
- [Concept Subspaces Compute Beyond the Logit Lens: A Weights-Only Test for Locating Representations Upstream of Readout](https://mlsanity.com/paper/2609.39263/) (2026)
- [Offline Guidance, Online Reasoning: Reusing LLM Feedback for Small Language Models](https://mlsanity.com/paper/2609.39346/) (2026)

## Abstract (arXiv)

Despite impressive breadth, LLMs still rely on explicit reasoning instructions or static, one-fits-all steering methods, leaving a gap for adaptive, instruction-free reasoning amplification. We present Prototype-Based Dynamic Steering (PDS), a test-time method that amplifies large language model (LLM) reasoning without adding or altering instructions. We introduce "reasoning prototypes" by clustering activation differences between Chain-of-Thought (CoT) and neutral prompts. At inference, an input's hidden state is projected onto these prototypes to form an instance-specific steering vector. Evaluated on GSM8K, AQuA-RAT, and BIG-Bench tasks, PDS consistently improves accuracy without fine-tuning or prompt engineering. Notably, the gains persist even when CoT is explicitly suppressed to improve cost-efficiency, indicating that the intervention strengthens latent reasoning processes rather than inducing a superficial behavioral shift. These results position dynamic, prototype-guided steering as a lightweight alternative to training-time approaches for enhancing LLM reasoning.