# More Correct Mass, Worse Answers: Why Power Sampling Can Fail and How to Fix It

arXiv: https://arxiv.org/abs/2608.14420 · HTML: https://mlsanity.com/paper/2608.14420/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 27 references that are in the map.
- First posted on arXiv 14 Aug 2026.

## Builds on (most cited first, 25 of 25)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Solving Quantitative Reasoning Problems with Language Models](https://mlsanity.com/paper/2206.14858/) (2022)
- [Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?](https://mlsanity.com/paper/2504.13837/) (2025)
- [BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions](https://mlsanity.com/paper/2406.15877/) (2024)
- [CodeT: Code Generation with Generated Tests](https://mlsanity.com/paper/2207.10397/) (2022)
- [Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models](https://mlsanity.com/paper/2408.00724/) (2024)
- [Universal Self-Consistency for Large Language Model Generation](https://mlsanity.com/paper/2311.17311/) (2023)
- [Confidence Improves Self-Consistency in LLMs](https://mlsanity.com/paper/2502.06233/) (2025)
- [Self-Improvement in Language Models: The Sharpening Mechanism](https://mlsanity.com/paper/2412.01951/) (2024)
- [Reasoning with Sampling: Your Base Model is Smarter Than You Think](https://mlsanity.com/paper/2510.14901/) (2025)
- [Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Sampling](https://mlsanity.com/paper/2408.17017/) (2024)
- [MoreHopQA: More Than Multi-hop Reasoning](https://mlsanity.com/paper/2406.13397/) (2024)
- [Physics: Benchmarking Foundation Models on University-Level Physics Problem Solving](https://mlsanity.com/paper/2503.21821/) (2025)
- [Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening](https://mlsanity.com/paper/2601.21590/) (2026)
- [Leveraging Online Olympiad-Level Math Problems for LLMs Training and Contamination-Resistant Evaluation](https://mlsanity.com/paper/2501.14275/) (2025)
- [Do We Truly Need So Many Samples? Multi-LLM Repeated Sampling Efficiently Scales Test-Time Compute](https://mlsanity.com/paper/2504.00762/) (2025)
- [Power-SMC: Low-Latency Sequence-Level Power Sampling for Training-Free LLM Reasoning](https://mlsanity.com/paper/2602.10273/) (2026)
- [Too Consistent to Detect: A Study of Self-Consistent Errors in LLMs](https://mlsanity.com/paper/2505.17656/) (2025)
- [A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning](https://mlsanity.com/paper/2510.15444/) (2025)
- [Revisiting Self-Consistency from Dynamic Distributional Alignment Perspective on Answer Aggregation](https://mlsanity.com/paper/2502.19830/) (2025)
- [ModeX: Evaluator-Free Best-of-N Selection for Open-Ended Generation](https://mlsanity.com/paper/2601.02535/) (2026)
- [Latent Self-Consistency for Reliable Majority-Set Selection in Short- and Long-Answer Reasoning](https://mlsanity.com/paper/2508.18395/) (2025)
- [Test-time Diverse Reasoning by Riemannian Activation Steering](https://mlsanity.com/paper/2511.08305/) (2025)
- [OptScale: Probabilistic Optimality for Inference-time Scaling](https://mlsanity.com/paper/2506.22376/) (2025)

## Built on by (most cited first, 1 of 1)

- [ReSight-SMC: Two-Stage Power Sampling via Island SMC with Visual Scouts](https://mlsanity.com/paper/2609.34905/) (2026)

## Abstract (arXiv)

Power Sampling sharpens a language model's distribution over complete generation trajectories, offering a verifier-free way to improve reasoning at inference time. It also has the potential to serve as a general-purpose front end for a broad range of downstream sampling methods. However, we uncover a striking paradox: Power Sampling can drive more probability mass toward correct trajectories while degrading the downstream inference it is intended to enhance. Using self-consistency as a representative case, we observe accuracy drops of up to 18.5 percentage points across models and reasoning benchmarks. We trace this paradox to two mismatches. Dose mismatch arises because a fixed exponent induces drastically different amounts of distributional change across problems. Coverage mismatch arises because global sharpening concentrates mass on a narrow set of dominant paths: high pass@k, often interpreted as evidence of preserved diversity, can therefore coexist with the loss of broad reasoning-path support required for downstream aggregation, search, and selection. Guided by this diagnosis, we replace uniform trajectory exponentiation with a deformation-controlled, support-preserving Power target that calibrates sharpening across problems while limiting the suppression of moderate-probability paths. In a same-budget instantiation with weighted self-consistency, the repaired sampler reverses the losses caused by global Power and outperforms standard multi-sample inference across reasoning benchmarks.