mlsanityData to 7 Oct 2026

Reasoning with Sampling: Your Base Model is Smarter Than You Think

arXiv 2510.14901 ↗First posted 16 Oct 2025cs.LG cs.AI cs.CLcited by 84

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Frontier reasoning models have exhibited incredible capabilities across a wide array of disciplines, driven by posttraining large language models (LLMs) with reinforcement learning (RL). However, despite the widespread success of this paradigm, much of the literature has been devoted to disentangling truly novel behaviors that emerge during RL but are not present in the base models. In our work, we approach this question from a different angle, instead asking whether comparable reasoning capabilites can be elicited from base models at inference time by pure sampling, without any additional training. Inspired by Markov chain Monte Carlo (MCMC) techniques for sampling from sharpened distributions, we propose a simple iterative sampling algorithm leveraging the base models' own likelihoods. Over different base models, we show that our algorithm offers substantial boosts in reasoning that nearly match and even outperform those from RL on a wide variety of single-shot tasks, including MATH500, HumanEval, and GPQA. Moreover, our sampler avoids the collapse in diversity over multiple samples that is characteristic of RL-posttraining. Crucially, our method does not require training, curated datasets, or a verifier, suggesting broad applicability beyond easily verifiable domains.

Key facts. Cited by 84 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 32 of its 42 references that are in the map. First posted on arXiv 16 Oct 2025.

Builds on

32 of 42 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Outcome-based Exploration for LLM Reasoning8 Sep
  2. 2025Spurious Rewards: Rethinking Training Signals in RLVR12 Jun
  3. 2025Rewarding the Unlikely: Lifting GRPO Beyond Distribution Sharpening3 Jun
  4. 2025Maximizing Confidence Alone Improves Reasoning28 May
  5. 2025Learning to Reason without External Rewards26 May
  6. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?18 Apr
  7. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  8. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild24 Mar
  9. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  10. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Training22 Nov
  11. 2023Let's Verify Step by Step31 May

Other references · 21

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  5. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  6. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 865
  7. 2024Phi-4 Technical Reportcited by 675
  8. 2023Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMCcited by 216
  9. 2025Test-time Alignment of Diffusion Models without Reward Over-optimizationcited by 89
  10. 2024Probabilistic Inference in Language Models via Twisted Sequential Monte Carlocited by 85
  11. 2025Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Expertscited by 85
  12. 2025Proteina: Scaling Flow-based Protein Structure Generative Modelscited by 63
9 more
  1. 2025Inference-time Scaling of Diffusion Models through Classical Searchcited by 48
  2. 2024Inference-Time Policy Steering through Human Interactionscited by 35
  3. 2024Diffusion Models as Constrained Samplers for Optimization with Unknown Constraintscited by 33
  4. 2020Contextual Temperature for Language Modelingcited by 25
  5. 2024QUEST: Quality-Aware Metropolis-Hastings Sampling for Machine Translationcited by 16
  6. 2025Blink of an eye: a simple theory for feature localization in generative modelscited by 10
  7. 2025Temporal Score Rescaling for Temperature Sampling in Diffusion and Flow Modelscited by 7
  8. 2022On free energy barriers in Gaussian priors and failure of cold start MCMC for high-dimensional unimodal distributionscited by 6
  9. 2025ReGuidance: A Simple Diffusion Wrapper for Boosting Sample Quality on Hard Inverse Problemscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 77.

84 papers.

Sort:

By area:

  1. 2026Self-Refining Video Samplingcited by 14
  2. 2026TSRBench: A Comprehensive Multi-task Multi-modal Time Series Reasoning Benchmark for Generalist Modelscited by 14
  3. 2025Certified Self-Consistency: Statistical Guarantees and Test-Time Training for Reliable Reasoning in LLMscited by 12
  4. 2025From Efficiency to Adaptivity: A Deeper Look at Adaptive Reasoning in Large Language Modelscited by 11
  5. 2025Semantic Soft Bootstrapping: Long Context Reasoning in LLMs without Reinforcement Learningcited by 9
  6. 2026Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learningcited by 8
  7. 2026Rare Event Analysis of Large Language Modelscited by 6
  8. 2026ThinkRouter: Efficient Reasoning via Routing Thinking between Latent and Discrete Spacescited by 6
  9. 2026Uniform-Correct Policy Optimization: Breaking RLVR's Indifference to Diversitycited by 6
  10. 2026Thinking by Subtraction: Confidence-Driven Contrastive Decoding for LLM Reasoningcited by 5
  11. 2026Locally Confident, Globally Stuck: The Quality-Exploration Dilemma in Diffusion Language Modelscited by 5
  12. 2026Beyond Mode Collapse: Distribution Matching for Diverse Reasoningcited by 5
38 more
  1. 2026When Sharpening Becomes Collapse: Sampling Bias and Semantic Coupling in RL with Verifiable Rewardscited by 4
  2. 2026On the Power of (Approximate) Reward Models for Inference-Time Scalingcited by 4
  3. 2026Monitorability as a Free Gift: How RLVR Spontaneously Aligns Reasoningcited by 4
  4. 2026Operationalising the Superficial Alignment Hypothesis via Task Complexitycited by 4
  5. 2026Efficient Reasoning with Balanced Thinkingcited by 4
  6. 2026Sampling for Quality: Training-Free Reward-Guided LLM Decoding via Sequential Monte Carlocited by 4
  7. 2025Tracking large chemical reaction networks and rare events by neural networkscited by 3
  8. 2026ETS: Energy-Guided Test-Time Scaling for Training-Free RL Alignmentcited by 3
  9. 2026Effective Reasoning Chains Reduce Intrinsic Dimensionalitycited by 3
  10. 2026Position: agentic AI orchestration should be Bayes-consistentcited by 3
  11. 2026The Model Knows, the Decoder Finds: Future Value Guided Particle Power Samplingcited by 3
  12. 2026Drift is a Sampling Error: SNR-Aware Power Distributions for Long-Horizon Robotic Planningcited by 3
  13. 2026Self-Consistency via Marginal Sharpeningcited by 3
  14. 2026When are likely answers right? On Sequence Probability and Correctness in LLMscited by 3
  15. 2025A Model Can Help Itself: Reward-Free Self-Training for LLM Reasoningcited by 2
  16. 2026Linear Dynamics in the RLVR Training of Large Language Modelscited by 2
  17. 2026Simulated Annealing Enhances Theory-of-Mind Reasoning in Autoregressive Language Modelscited by 2
  18. 2026Decoding as Optimisation on the Probability Simplex: From Top-K to Top-P (Nucleus) to Best-of-K Samplerscited by 2
  19. 2026When Does RL Help Medical VLMs? Disentangling Vision, SFT, and RL Gainscited by 2
  20. 2026Believe Your Model: Distribution-Guided Confidence Calibrationcited by 2
  21. 2026Reject, Resample, Repeat: Understanding Parallel Reasoning in Language Model Inferencecited by 2
  22. 2026PowerFlow: Unlocking the Dual Nature of LLMs via Principled Distribution Matchingcited by 2
  23. 2026Rethinking RL for LLM Reasoning: It's Sparse Policy Selection, Not Capability Learningcited by 2
  24. 2026How You Begin is How You Reason: Driving Exploration in RLVR via Prefix-Tuned Priorscited by 2
  25. 2026Sampling More, Getting Less: Calibration is the Diversity Bottleneck in LLMscited by 2
  26. 2026PAEC: Position-Aware Entropy Calibration for LLM Reasoning in RLVRcited by 2
  27. 2026Neglected Free Lunch from Post-training: Progress Advantage for LLM Agentscited by 2
  28. 2026NeuReasoner: Theory-grounded Mapping of Reasoning Elicitation Boundariescited by 2
  29. 2026TSRouter: Dynamic Modality-Model Selection for Time Series Reasoningcited by 2
  30. 2026Understanding Reasoning from Pretraining to Post-Trainingcited by 2
  31. 2026TailSFT: Filtered Fine-Tuning Improves Post-Training Performancecited by 2
  32. 2025Beyond Token-level Supervision: Unlocking the Potential of Decoding-based Regression via Reinforcement Learningcited by 1
  33. 2026In-Context Source and Channel Codingcited by 1
  34. 2026Self-Improvement as Coherence Optimization: A Theoretical Accountcited by 1
  35. 2026On the Generalization Gap in LLM Planning: Tests and Verifier-Reward RLcited by 1
  36. 2026The Geometric Reasoner: Manifold-Informed Latent Foresight Search for Long-Context Reasoningcited by 1
  37. 2026Inference-Aware Meta-Alignment of LLMs via Non-Linear GRPOcited by 1
  38. 2026Post-Training with Policy Gradients: Optimality and the Base Model Barriercited by 1

50 most cited of 84 All 84 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Reasoning with Sampling: Cutting at Decision Points
  2. 2026Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening
  3. 2026Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data
  4. 2025T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling
  5. 2026Dynamics-Predictive Sampling for Active RL Finetuning of Large Reasoning Models
  6. 2025On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models
  7. 2025Dissecting Long-Chain-of-Thought Reasoning Models: An Empirical Study
  8. 2026Small RL Controller, Large Language Model: RL-Guided Adaptive Sampling for Test-Time Scaling