# Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening

arXiv: https://arxiv.org/abs/2601.21590 · HTML: https://mlsanity.com/paper/2601.21590/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 36 of its 52 references that are in the map.
- First posted on arXiv 29 Jan 2026.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 30 of 36)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [Accelerating Large Language Model Decoding with Speculative Sampling](https://mlsanity.com/paper/2302.01318/) (2023)
- [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024)
- [Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?](https://mlsanity.com/paper/2504.13837/) (2025)
- [Statistical Rejection Sampling Improves Preference Optimization](https://mlsanity.com/paper/2309.06657/) (2023)
- [Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models](https://mlsanity.com/paper/1610.02424/) (2016)
- [Break the Sequential Dependency of LLM Inference Using Lookahead Decoding](https://mlsanity.com/paper/2402.02057/) (2024)
- [Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models](https://mlsanity.com/paper/2501.09686/) (2025)
- [Large Language Models for Mathematical Reasoning: Progresses and Challenges](https://mlsanity.com/paper/2402.00157/) (2024)
- [Spurious Rewards: Rethinking Training Signals in RLVR](https://mlsanity.com/paper/2506.10947/) (2025)
- [DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Search](https://mlsanity.com/paper/2408.08152/) (2024)
- [Planning with Large Language Models for Code Generation](https://mlsanity.com/paper/2303.05510/) (2023)
- [A Survey of Reinforcement Learning for Large Reasoning Models](https://mlsanity.com/paper/2509.08827/) (2025)
- [Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8B](https://mlsanity.com/paper/2406.07394/) (2024)
- [A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforce](https://mlsanity.com/paper/2504.11343/) (2025)
- [Learning to Discover at Test Time](https://mlsanity.com/paper/2601.16175/) (2026)
- [Is Best-of-N the Best of Them? Coverage, Scaling, and Optimality in Inference-Time Alignment](https://mlsanity.com/paper/2503.21878/) (2025)
- [Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Models](https://mlsanity.com/paper/2412.15287/) (2024)
- [Outcome-based Exploration for LLM Reasoning](https://mlsanity.com/paper/2509.06941/) (2025)
- [Group Preference Optimization: Few-Shot Alignment of Large Language Models](https://mlsanity.com/paper/2310.11523/) (2023)
- [Large Language Models for Planning: A Comprehensive and Systematic Survey](https://mlsanity.com/paper/2505.19683/) (2025)
- [Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs](https://mlsanity.com/paper/2507.02076/) (2025)

## Built on by (most cited first, 30 of 31)

- [Power-SMC: Low-Latency Sequence-Level Power Sampling for Training-Free LLM Reasoning](https://mlsanity.com/paper/2602.10273/) (2026)
- [Rare Event Analysis of Large Language Models](https://mlsanity.com/paper/2602.06791/) (2026)
- [Detecting and Mitigating the Correct-Answer Extinction Window in Test-Time Reinforcement Learning with Majority Voting](https://mlsanity.com/paper/2605.19444/) (2026)
- [The Model Knows, the Decoder Finds: Future Value Guided Particle Power Sampling](https://mlsanity.com/paper/2605.02427/) (2026)
- [Sampling for Quality: Training-Free Reward-Guided LLM Decoding via Sequential Monte Carlo](https://mlsanity.com/paper/2604.16453/) (2026)
- [Power Distribution Bridges Sampling, Self-Reward RL, and Self-Distillation](https://mlsanity.com/paper/2605.04542/) (2026)
- [Self-Consistency via Marginal Sharpening](https://mlsanity.com/paper/2605.28142/) (2026)
- [When are likely answers right? On Sequence Probability and Correctness in LLMs](https://mlsanity.com/paper/2606.27359/) (2026)
- [The Y-Combinator for LLMs: Solving Long-Context Rot with λ-Calculus](https://mlsanity.com/paper/2603.20105/) (2026)
- [Reasoning with Sampling: Cutting at Decision Points](https://mlsanity.com/paper/2605.30327/) (2026)
- [Decoding as Optimisation on the Probability Simplex: From Top-K to Top-P (Nucleus) to Best-of-K Samplers](https://mlsanity.com/paper/2602.18292/) (2026)
- [Surprisal-Guided Selection: Compute-Optimal Test-Time Strategies for Execution-Grounded Code Generation](https://mlsanity.com/paper/2602.07670/) (2026)
- [Sample Where You Struggle: Sharpening Base Model Reasoning via Entropy-Guided Power Sampling](https://mlsanity.com/paper/2606.09926/) (2026)
- [Depth-Entropy Guided Sampling for Training-Free LLM Reasoning](https://mlsanity.com/paper/2607.09693/) (2026)
- [More Correct Mass, Worse Answers: Why Power Sampling Can Fail and How to Fix It](https://mlsanity.com/paper/2608.14420/) (2026)
- [Beyond Distribution Sharpening: The Importance of Task Rewards](https://mlsanity.com/paper/2604.16259/) (2026)
- [Not All Synthetic Data Is Yours to Learn From](https://mlsanity.com/paper/2605.31126/) (2026)
- [Score × Decoder: A Unified View of Unsupervised Inference-Time Scaling for Hallucination Mitigation](https://mlsanity.com/paper/2606.00739/) (2026)
- [CANVAS: Consistency-Aware Navigation via Visual Adaptive Sampling for Long-Context Text-to-SVG Generation](https://mlsanity.com/paper/2608.30689/) (2026)
- [Chopthin-Consensus Power Sampling: A Diversity-Preserving Approach to LLM Decoding](https://mlsanity.com/paper/2609.12243/) (2026)
- [iSDFT: Information-Proximal Self-Distillation for Continual Learning in LLMs](https://mlsanity.com/paper/2609.24646/) (2026)
- [ReSight-SMC: Two-Stage Power Sampling via Island SMC with Visual Scouts](https://mlsanity.com/paper/2609.34905/) (2026)
- [Composable Decoding on the Probability Simplex: Theory and Implementation](https://mlsanity.com/paper/2609.34992/) (2026)
- [Beam Search as Test-Time Self-Distillation via Counterfactual Contexts](https://mlsanity.com/paper/2609.37041/) (2026)
- [Explore Broadly, Reason Sharply: Push Small Models toward the Frontier via Sampling](https://mlsanity.com/paper/2609.38104/) (2026)
- [Does This Action Still Explain the Task? Reverse Scoring for Diffusion Language Model Agents](https://mlsanity.com/paper/2609.38536/) (2026)
- [Finetuning with Sampling: SFT Learns Better Than You Think](https://mlsanity.com/paper/2610.02140/) (2026)
- [Sharpen Without Search: On-Policy Distillation of Sequence-Level Power Distribution](https://mlsanity.com/paper/2610.06804/) (2026)
- [Adaptive Power Sampling for LLM Reasoning](https://mlsanity.com/paper/2610.08563/) (2026)
- [A Critical Look at Power Sampling when Test Time Sharpening LLMs](https://mlsanity.com/paper/2610.13653/) (2026)

## Abstract (arXiv)

Reinforcement learning (RL) post-training is a dominant approach for improving the reasoning performance of large language models (LLMs), yet growing evidence suggests that its gains arise primarily from distribution sharpening rather than the acquisition of new capabilities. Recent work has shown that sampling from the power distribution of LLMs using Markov chain Monte Carlo (MCMC) can recover performance comparable to RL post-training without relying on external rewards; however, the high computational cost of MCMC makes such approaches impractical for widespread adoption. In this work, we propose a theoretically grounded alternative that eliminates the need for iterative MCMC. We derive a token-level decomposition of the autoregressive power distribution, showing that the global power distribution can be approximated by a token-level scaled low-temperature one, where the scaling factor captures future trajectory quality. Leveraging this insight, we introduce a training-free and verifier-free algorithm that sharpens the base model's generative distribution autoregressively. Empirically, we evaluate our method on math, QA, and code tasks, and show that our method matches or surpasses one-shot GRPO without relying on any external rewards, while reducing inference latency by over 10× compared to MCMC-based sampling.