mlsanityData to 14 Oct 2026

Prototype-Based Dynamic Steering for Large Language Models

arXiv 2510.05498 ↗First posted 7 Oct 2025cs.CLcited by 6

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Despite impressive breadth, LLMs still rely on explicit reasoning instructions or static, one-fits-all steering methods, leaving a gap for adaptive, instruction-free reasoning amplification. We present Prototype-Based Dynamic Steering (PDS), a test-time method that amplifies large language model (LLM) reasoning without adding or altering instructions. We introduce "reasoning prototypes" by clustering activation differences between Chain-of-Thought (CoT) and neutral prompts. At inference, an input's hidden state is projected onto these prototypes to form an instance-specific steering vector. Evaluated on GSM8K, AQuA-RAT, and BIG-Bench tasks, PDS consistently improves accuracy without fine-tuning or prompt engineering. Notably, the gains persist even when CoT is explicitly suppressed to improve cost-efficiency, indicating that the intervention strengthens latent reasoning processes rather than inducing a superficial behavioral shift. These results position dynamic, prototype-guided steering as a lightweight alternative to training-time approaches for enhancing LLM reasoning.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 25 of its 38 references that are in the map. First posted on arXiv 7 Oct 2025.

Builds on

25 of 38 references.

References · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  8. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  9. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  10. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  11. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  12. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
13 more
  1. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  2. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  3. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 245
  4. 2024Programming Refusal with Conditional Activation Steeringcited by 206
  5. 2023Inspecting and Editing Knowledge Representations in Language Modelscited by 141
  6. 2024Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categoriescited by 77
  7. 2024Spectral Editing of Activations for Large Language Model Alignmentcited by 56
  8. 2024Householder Pseudo-Rotation: A Novel Approach to Activation Editing in LLMs with Direction-Magnitude Perspectivecited by 23
  9. 2024Exploring the Personality Traits of LLMs through Latent Features Steeringcited by 22
  10. 2025Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Modelscited by 13
  11. 2024Large Language Models are Contrastive Reasonerscited by 11
  12. 2024Learning from Contrastive Prompts: Automated Optimization and Adaptationcited by 7
  13. 2024Focus On This, Not That! Steering LLMs with Adaptive Feature Specificationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Towards Effective Theory of LLMs: A Representation Learning Approachcited by 0
  2. 2026Steered Generation via Gradient-Based Optimization on Sparse Query Featurescited by 0
  3. 2026Cognitive Fatigue in Autoregressive Transformers: Formalization and Measurementcited by 0
  4. 2026Metacognitive Steering: Learning the Structure of Scientific Judgmentcited by 0
  5. 2026Concept Subspaces Compute Beyond the Logit Lens: A Weights-Only Test for Locating Representations Upstream of Readoutcited by 0
  6. 2026Offline Guidance, Online Reasoning: Reusing LLM Feedback for Small Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ATLAS: Verifier-Guided Adaptive Latent Activation Steering for Efficient LLM Reasoning
  2. 2026Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMs
  3. 2025KV Cache Steering for Controlling Frozen LLMs
  4. 2026Can We Break LLMs Out of Self-Loops? Fine-Grained Reasoning Control with Activation Steering
  5. 2025Understanding and Steering the Cognitive Behaviors of Reasoning Models at Test-Time
  6. 2026Nudging Hidden States: Training-Free Model Steering for Chain-of-Thought Reasoning in Large Audio-Language Models
  7. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs
  8. 2025Understanding Reasoning in Thinking Language Models via Steering Vectors