mlsanityData to 14 Oct 2026

Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening

arXiv 2601.21590 ↗First posted 29 Jan 2026cs.LG cs.AIcited by 31

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement learning (RL) post-training is a dominant approach for improving the reasoning performance of large language models (LLMs), yet growing evidence suggests that its gains arise primarily from distribution sharpening rather than the acquisition of new capabilities. Recent work has shown that sampling from the power distribution of LLMs using Markov chain Monte Carlo (MCMC) can recover performance comparable to RL post-training without relying on external rewards; however, the high computational cost of MCMC makes such approaches impractical for widespread adoption. In this work, we propose a theoretically grounded alternative that eliminates the need for iterative MCMC. We derive a token-level decomposition of the autoregressive power distribution, showing that the global power distribution can be approximated by a token-level scaled low-temperature one, where the scaling factor captures future trajectory quality. Leveraging this insight, we introduce a training-free and verifier-free algorithm that sharpens the base model's generative distribution autoregressively. Empirically, we evaluate our method on math, QA, and code tasks, and show that our method matches or surpasses one-shot GRPO without relying on any external rewards, while reducing inference latency by over 10× compared to MCMC-based sampling.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 36 of its 52 references that are in the map. First posted on arXiv 29 Jan 2026.

Builds on

36 of 52 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Differential Smoothing Mitigates Sharpening and Improves LLM Reasoning25 Nov
  2. 2025Can GRPO Help LLMs Transcend Their Pretraining Origin?14 Oct
  3. 2025A Survey of Reinforcement Learning for Large Reasoning Models10 Sep
  4. 2025Outcome-based Exploration for LLM Reasoning8 Sep
  5. 2025Spurious Rewards: Rethinking Training Signals in RLVR12 Jun
  6. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?18 Apr
  7. 2025A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforce15 Apr
  8. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  9. 2023Let's Verify Step by Step31 May

Other references · 27

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  6. 2024Qwen2 Technical Reportcited by 2,120
  7. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
  8. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  9. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  10. 2023Statistical Rejection Sampling Improves Preference Optimizationcited by 323
  11. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 301
  12. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decodingcited by 298
15 more
  1. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 293
  2. 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 284
  3. 2024DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Searchcited by 197
  4. 2023Planning with Large Language Models for Code Generationcited by 182
  5. 2024Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8Bcited by 158
  6. 2026Learning to Discover at Test Timecited by 115
  7. 2025Is Best-of-N the Best of Them? Coverage, Scaling, and Optimality in Inference-Time Alignmentcited by 97
  8. 2024Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Modelscited by 89
  9. 2023Group Preference Optimization: Few-Shot Alignment of Large Language Modelscited by 63
  10. 2025Large Language Models for Planning: A Comprehensive and Systematic Surveycited by 40
  11. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMscited by 31
  12. 2020Contextual Temperature for Language Modelingcited by 27
  13. 2024Mixture of Attentions For Speculative Decodingcited by 17
  14. 2025Bourbaki: Self-Generated and Goal-Conditioned MDPs for Theorem Provingcited by 12
  15. 2016The Jackknife Estimation Methodcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 31.

31 papers.

Sort:

By area:

  1. 2026Power-SMC: Low-Latency Sequence-Level Power Sampling for Training-Free LLM Reasoningcited by 21
  2. 2026Rare Event Analysis of Large Language Modelscited by 8
  3. 2026Detecting and Mitigating the Correct-Answer Extinction Window in Test-Time Reinforcement Learning with Majority Votingcited by 7
  4. 2026The Model Knows, the Decoder Finds: Future Value Guided Particle Power Samplingcited by 6
  5. 2026Sampling for Quality: Training-Free Reward-Guided LLM Decoding via Sequential Monte Carlocited by 5
  6. 2026Power Distribution Bridges Sampling, Self-Reward RL, and Self-Distillationcited by 5
  7. 2026Self-Consistency via Marginal Sharpeningcited by 4
  8. 2026When are likely answers right? On Sequence Probability and Correctness in LLMscited by 4
  9. 2026The Y-Combinator for LLMs: Solving Long-Context Rot with λ-Calculuscited by 3
  10. 2026Reasoning with Sampling: Cutting at Decision Pointscited by 3
  11. 2026Decoding as Optimisation on the Probability Simplex: From Top-K to Top-P (Nucleus) to Best-of-K Samplerscited by 2
  12. 2026Surprisal-Guided Selection: Compute-Optimal Test-Time Strategies for Execution-Grounded Code Generationcited by 1
19 more
  1. 2026Sample Where You Struggle: Sharpening Base Model Reasoning via Entropy-Guided Power Samplingcited by 1
  2. 2026Depth-Entropy Guided Sampling for Training-Free LLM Reasoningcited by 1
  3. 2026More Correct Mass, Worse Answers: Why Power Sampling Can Fail and How to Fix Itcited by 1
  4. 2026Beyond Distribution Sharpening: The Importance of Task Rewardscited by 0
  5. 2026Not All Synthetic Data Is Yours to Learn Fromcited by 0
  6. 2026Score × Decoder: A Unified View of Unsupervised Inference-Time Scaling for Hallucination Mitigationcited by 0
  7. 2026CANVAS: Consistency-Aware Navigation via Visual Adaptive Sampling for Long-Context Text-to-SVG Generationcited by 0
  8. 2026Chopthin-Consensus Power Sampling: A Diversity-Preserving Approach to LLM Decodingcited by 0
  9. 2026iSDFT: Information-Proximal Self-Distillation for Continual Learning in LLMscited by 0
  10. 2026ReSight-SMC: Two-Stage Power Sampling via Island SMC with Visual Scoutscited by 0
  11. 2026Composable Decoding on the Probability Simplex: Theory and Implementationcited by 0
  12. 2026Beam Search as Test-Time Self-Distillation via Counterfactual Contextscited by 0
  13. 2026Explore Broadly, Reason Sharply: Push Small Models toward the Frontier via Samplingcited by 0
  14. 2026Does This Action Still Explain the Task? Reverse Scoring for Diffusion Language Model Agentscited by 0
  15. 2026Finetuning with Sampling: SFT Learns Better Than You Thinkcited by 0
  16. 2026Sharpen Without Search: On-Policy Distillation of Sequence-Level Power Distributioncited by 0
  17. 2026Adaptive Power Sampling for LLM Reasoningcited by 0
  18. 2026A Critical Look at Power Sampling when Test Time Sharpening LLMscited by 0
  19. 2026Decoupled Marginal Sharpening for Training-Free Inference-Time Scalingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Sampling via Decision-Flow: Training-Free Extraction of Improved Latent Reasoning Paths in Large Language Models
  2. 2025Reasoning with Sampling: Your Base Model is Smarter Than You Think
  3. 2025From Data-Centric to Sample-Centric: Enhancing LLM Reasoning via Progressive Optimization
  4. 2026IsoCompute Playbook: Optimally Scaling Sampling Compute for LLM RL
  5. 2025Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimization
  6. 2025DUMP: Automated Distribution-Level Curriculum Learning for RL-based LLM Post-training
  7. 2025T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling
  8. 2026PowerFlow: Unlocking the Dual Nature of LLMs via Principled Distribution Matching