# Efficient Reasoning via Constrained Optimization in Latent Space

arXiv: https://arxiv.org/abs/2609.34181 · HTML: https://mlsanity.com/paper/2609.34181/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 21 of its 47 references that are in the map.
- First posted on arXiv 28 Sep 2026.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 21 of 21)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [Dynamic Early Exit in Reasoning Models](https://mlsanity.com/paper/2504.15895/) (2025)
- [Compressed Chain of Thought: Efficient Reasoning Through Dense Representations](https://mlsanity.com/paper/2412.13171/) (2024)
- [When More is Less: Understanding Chain-of-Thought Length in LLMs](https://mlsanity.com/paper/2502.07266/) (2025)
- [Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification](https://mlsanity.com/paper/2504.05419/) (2025)
- [Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space](https://mlsanity.com/paper/2505.15778/) (2025)
- [Reasoning with Sampling: Your Base Model is Smarter Than You Think](https://mlsanity.com/paper/2510.14901/) (2025)
- [Think Only When You Need with Large Hybrid-Reasoning Models](https://mlsanity.com/paper/2505.14631/) (2025)
- [AdaCtrl: Towards Adaptive and Controllable Reasoning via Difficulty-Aware Budgeting](https://mlsanity.com/paper/2505.18822/) (2025)
- [Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Models](https://mlsanity.com/paper/2505.03469/) (2025)
- [Continuous Chain of Thought Enables Parallel Exploration and Reasoning](https://mlsanity.com/paper/2505.23648/) (2025)
- [Mitigating Overthinking in Large Reasoning Models via Manifold Steering](https://mlsanity.com/paper/2505.22411/) (2025)
- [Not All Tokens Are What You Need In Thinking](https://mlsanity.com/paper/2505.17827/) (2025)
- [EasySteer: A Unified Framework for High-Performance and Extensible LLM Steering](https://mlsanity.com/paper/2509.25175/) (2025)
- [AutoL2S: Auto Long-Short Reasoning for Efficient Large Language Models](https://mlsanity.com/paper/2505.22662/) (2025)
- [ATLAS: Verifier-Guided Adaptive Latent Activation Steering for Efficient LLM Reasoning](https://mlsanity.com/paper/2601.03093/) (2026)
- [CoT Vectors: Transferring and Probing the Reasoning Mechanisms of LLMs](https://mlsanity.com/paper/2510.00579/) (2025)
- [Internalizing LLM Reasoning via Discovery and Replay of Latent Actions](https://mlsanity.com/paper/2602.04925/) (2026)

## Abstract (arXiv)

Large Reasoning Models (LRMs) have shown remarkable reasoning capabilities, yet they still suffer from overthinking, generating redundant reasoning steps which incur substantial token consumption. Existing methods, such as suppressing reflective keywords or forcing shorter reasoning lengths, attempt to mitigate this issue but inevitably truncate necessary steps and induce underthinking, thereby compromising performance. To address this dilemma, we investigate the latent representations and observe that efficient reasoning steps naturally cluster into a concentrated region in latent space, while those deviating from this region tend to produce verbose sequences. To leverage this, we keep reasoning focused within this region via a quadratic program which projects deviating hidden states back into the region. Then we propose a novel training-free framework to achieve efficient reasoning that reduces token generation costs without sacrificing performance. Extensive experiments conducted on four models ranging from 1.5B to 14B, and across six benchmarks in math reasoning, coding, and scientific QA, validate the effectiveness of our method, up to a 12.1% improvement in accuracy while reducing generated tokens by 11.8% to 52.8%. Codes are available at hrefhttps://github.com/hzn18/Opt4Reasoninghttps://github.com/hzn18/Opt4Reasoning.