Controllable Mathematical Reasoning via Self-Optimizing Thought Vectors
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
We present a novel approach for controllable mathematical reasoning that leverages self-optimizing thought vectors with entropy minimization. Our method introduces learnable thought vectors that dynamically modulate the internal reasoning process of large language models. Using Gemma-2-9B on GSM8K, we achieve 90.1% accuracy with a controllability score of 0.42, demonstrating that entropy-based rewards effectively guide focused reasoning patterns without requiring external reward annotations. Our analysis reveals distinct thought vector clusters and consistent low-entropy distributions across control conditions, validating our framework for controllable AI reasoning.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 11 of its 11 references that are in the map. First posted on arXiv 25 Oct 2025.
Builds on
11 of 11 references.
Closest ancestors · 1
Same direction, newest first
Other references · 10
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
- 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
- 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
- 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 377
- 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 374
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025PEAR: Phase Entropy Aware Reward for Efficient Reasoning
- 2026Entropy-Gradient Inversion: Moving Toward Internal Mechanism of Large Reasoning Models
- 2026Respecting Self-Uncertainty in On-Policy Self-Distillation for Efficient LLM Reasoning
- 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning
- 2026Shorthand for Thought: Compressing LLM Reasoning via Entropy-Guided Supertokens
- 2024MC-NEST: Enhancing Mathematical Reasoning in Large Language Models leveraging a Monte Carlo Self-Refine Tree
- 2026Artificial Intelligence for Mathematical Reasoning: An Integrated Survey of Language Models, Neuro-symbolic Systems, and Verified Discovery
- 2025A Survey on Mathematical Reasoning and Optimization with Large Language Models