# Bridging Internal Probability and Self-Consistency for Effective and Efficient LLM Reasoning

arXiv: https://arxiv.org/abs/2502.00511 · HTML: https://mlsanity.com/paper/2502.00511/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 2 of its 56 references that are in the map.
- First posted on arXiv 1 Feb 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 2 of 2)

- [Universal Self-Consistency for Large Language Model Generation](https://mlsanity.com/paper/2311.17311/) (2023)
- [Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models](https://mlsanity.com/paper/2307.10236/) (2023)

## Built on by (most cited first, 18 of 18)

- [Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models](https://mlsanity.com/paper/2503.16419/) (2025)
- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [Efficient Reasoning Models: A Survey](https://mlsanity.com/paper/2504.10903/) (2025)
- [AlphaOne: Reasoning Models Thinking Slow and Fast at Test Time](https://mlsanity.com/paper/2505.24863/) (2025)
- [Slim-SC: Thought Pruning for Efficient Scaling with Self-Consistency](https://mlsanity.com/paper/2509.13990/) (2025)
- [Hidden States as Early Signals: Step-level Trace Evaluation and Pruning for Efficient Test-Time Scaling](https://mlsanity.com/paper/2601.09093/) (2026)
- [Measuring Reasoning Utility in LLMs via Conditional Entropy Reduction](https://mlsanity.com/paper/2508.20395/) (2025)
- [Calibration-Aware Policy Optimization for Reasoning LLMs](https://mlsanity.com/paper/2604.12632/) (2026)
- [Self-Reflective Generation at Test Time](https://mlsanity.com/paper/2510.02919/) (2025)
- [Seer Self-Consistency: Advance Budget Estimation for Adaptive Test-Time Scaling](https://mlsanity.com/paper/2511.09345/) (2025)
- [Reasoning Strategies in Large Language Models: Can They Follow, Prefer, and Optimize?](https://mlsanity.com/paper/2507.11423/) (2025)
- [Enhancing reasoning accuracy in large language models during inference time](https://mlsanity.com/paper/2603.21301/) (2026)
- [Smart Picks in the Dark: Towards Efficient RLVR for Reasoning via Tracing Metacognitive Pivots](https://mlsanity.com/paper/2606.04503/) (2026)
- [ParaTempo: Efficient Parallel Reasoning via Temporal Confidence](https://mlsanity.com/paper/2608.16425/) (2026)
- [Training-Free Object-Background Compositional T2I via Dynamic Spatial Guidance and Multi-Path Pruning](https://mlsanity.com/paper/2604.09850/) (2026)
- [A Self Consistency Based Reranking for Narrative Question Answering](https://mlsanity.com/paper/2606.15741/) (2026)
- [How Hard Does It Think? Analyzing Step-Aware Reasoning Energy in LLM Chain-of-Thought Trajectories](https://mlsanity.com/paper/2607.28674/) (2026)
- [Frustratingly Easy Majority Voting Using Length Conditioning](https://mlsanity.com/paper/2610.19890/) (2026)

## Abstract (arXiv)

Recent advancements in large language models (LLMs) have demonstrated remarkable reasoning capabilities. However, single-shot inference often yields unreliable results for complex reasoning tasks, leading researchers to explore multiple reasoning paths through methods such as perplexity and self-consistency. In this paper, we present the first theoretical error decomposition analysis of these techniques, breaking down their error into estimation error and model error. Our analysis reveals a fundamental trade-off: perplexity methods suffer from substantial model error due to the absence of a proper consistency function, while self-consistency exhibits high estimation error due to a slow error convergence rate. To overcome these limitations, we propose Reasoning-Pruning Perplexity Consistency (RPC). This approach combines Perplexity Consistency, which seamlessly integrates LLM perplexity with self-consistency, and Reasoning Pruning, which eliminates low-probability reasoning paths to effectively prevent the degeneration of estimation error reduction. Theoretical analysis demonstrates that RPC not only accelerates the convergence rate of estimation error to an exponential level but also holds strong potential for further reducing model error. Extensive empirical evaluations on seven benchmark datasets confirm that RPC can significantly improve reasoning performance, sample efficiency, and confidence reliability.