Partial Reasoning in Language Models: Search and Refinement Guided by Uncertainty
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
The use of Large Language Models (LLMs) for reasoning and planning tasks has drawn increasing attention in Artificial Intelligence research. Despite their remarkable progress, these models still exhibit limitations in multi-step inference scenarios, particularly in mathematical and logical reasoning. We introduce PREGU (Partial Reasoning Guided by Uncertainty). PREGU monitors the entropy of the output distribution during autoregressive generation and halts the process whenever entropy exceeds a defined threshold, signaling uncertainty. From that point, a localized search is performed in the latent space to refine the partial reasoning and select the most coherent answer, using the Soft Reasoning method. Experiments conducted with LLaMA-3-8B, Mistral-7B, and Qwen2-7B across four reasoning benchmarks (GSM8K, GSM-Hard, SVAMP, and StrategyQA) showed performance greater than or similar to Soft Reasoning, indicating that entropy can serve as an effective signal to trigger selective refinement during reasoning.
Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 18 of its 19 references that are in the map. First posted on arXiv 17 Jan 2026.
Builds on
18 of 19 references.
Closest ancestors · 2
Same direction, newest first
- 2025Soft Reasoning: Navigating Solution Spaces in Large Language Models through Controlled Embedding Exploration30 May
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May
Other references · 16
- 2024The Llama 3 Herd of Modelscited by 14,486
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,376
- 2021Training Verifiers to Solve Math Word Problemscited by 9,916
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,502
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,657
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,348
- 2023Mistral 7Bcited by 3,123
- 2024Qwen2 Technical Reportcited by 2,102
- 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 865
- 2023Reasoning with Language Model is Planning with World Modelcited by 851
- 2018A Tutorial on Bayesian Optimizationcited by 739
- 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 738
4 more
- 2022PAL: Program-aided Language Modelscited by 647
- 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matterscited by 221
- 2024Chain-of-Thought Reasoning Without Promptingcited by 209
- 2024Flaming-hot Initiation with Regular Execution Sampling for Large Language Modelscited by 6
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Measuring Reasoning Utility in LLMs via Conditional Entropy Reduction
- 2026Improving reasoning at inference time via uncertainty minimisation
- 2025Think Just Enough: Sequence-Level Entropy as a Confidence Signal for LLM Reasoning
- 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models
- 2025Reasoning Under Uncertainty: Exploring Probabilistic Reasoning Capabilities of LLMs
- 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
- 2025Explore Briefly, Then Decide: Mitigating LLM Overthinking via Cumulative Entropy Regulation
- 2024Does Reasoning Emerge? Examining the Probabilities of Causation in Large Language Models