mlsanityData to 14 Oct 2026

Step-by-Step Reasoning for Math Problems via Twisted Sequential Monte Carlo

arXiv 2410.01920 ↗First posted 2 Oct 2024cs.LGcited by 22

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Augmenting the multi-step reasoning abilities of Large Language Models (LLMs) has been a persistent challenge. Recently, verification has shown promise in improving solution consistency by evaluating generated outputs. However, current verification approaches suffer from sampling inefficiencies, requiring a large number of samples to achieve satisfactory performance. Additionally, training an effective verifier often depends on extensive process supervision, which is costly to acquire. In this paper, we address these limitations by introducing a novel verification method based on Twisted Sequential Monte Carlo (TSMC). TSMC sequentially refines its sampling effort to focus exploration on promising candidates, resulting in more efficient generation of high-quality solutions. We apply TSMC to LLMs by estimating the expected future rewards at partial solutions. This approach results in a more straightforward training target that eliminates the need for step-wise human annotations. We empirically demonstrate the advantages of our method across multiple math benchmarks, and also validate our theoretical analysis of both our approach and existing verification methods.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 33 of its 71 references that are in the map. First posted on arXiv 2 Oct 2024.

Builds on

33 of 71 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models5 Feb
  2. 2023Llemma: An Open Language Model For Mathematics16 Oct
  3. 2022NumGLUE: A Suite of Fundamental yet Challenging Mathematical Reasoning Tasks12 Apr

Other references · 30

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  6. 2023Let's Verify Step by Stepcited by 3,789
  7. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  8. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  9. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  10. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  11. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  12. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
18 more
  1. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 441
  2. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 401
  3. 2016Solving General Arithmetic Word Problemscited by 360
  4. 2023Challenges and Applications of Large Language Modelscited by 254
  5. 2022Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problemcited by 236
  6. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 233
  7. 2024AlphaMath Almost Zero: Process Supervision without Processcited by 162
  8. 2021Monte Carlo Tree Search: A Review of Recent Modifications and Applicationscited by 161
  9. 2023Controlled Decoding from Language Modelscited by 135
  10. 2023M3IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuningcited by 126
  11. 2024Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervisioncited by 96
  12. 2024Probabilistic Inference in Language Models via Twisted Sequential Monte Carlocited by 90
  13. 2021Generate & Rank: A Multi-task Framework for Math Word Problemscited by 82
  14. 2023Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programscited by 80
  15. 2019Elements of Sequential Monte Carlocited by 64
  16. 2023Making Large Language Models Better Reasoners with Alignmentcited by 63
  17. 2024Multi-step Problem Solving Through a Verifier: An Empirical Analysis on Model-induced Process Supervisioncited by 52
  18. 2023Towards End-to-End Embodied Decision Making via Multi-modal Large Language Model: Explorations with GPT4-Vision and Beyondcited by 51

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 15.

22 papers.

Sort:

By area:

  1. 2025A Survey on Large Language Models for Mathematical Reasoningcited by 52
  2. 2025Rollout Roulette: A Probabilistic Inference Approach to Inference-Time Scaling of LLMs using Particle-Based Monte Carlo Methodscited by 20
  3. 2025Latent Thinking Optimization: Your Latent Reasoning Language Model Secretly Encodes Reward Signals in Its Latent Thoughtscited by 19
  4. 2025Inference-Time Scaling of Discrete Diffusion Models via Importance Weighting and Optimal Proposal Designcited by 14
  5. 2025Self-Steering Language Modelscited by 13
  6. 2026Understanding Reasoning in LLMs through Strategic Information Allocation under Uncertaintycited by 9
  7. 2026On the Power of (Approximate) Reward Models for Inference-Time Scalingcited by 7
  8. 2026The Model Knows, the Decoder Finds: Future Value Guided Particle Power Samplingcited by 6
  9. 2026Segmental Advantage Estimation: Enhancing PPO for Long-Context LLM Trainingcited by 5
  10. 2025Mitigating Premature Exploitation in Particle-based Monte Carlo for Inference-Time Scalingcited by 4
  11. 2026Online Reasoning Calibration: Test-Time Training Enables Generalizable Conformal LLM Reasoningcited by 4
  12. 2026Reject, Resample, Repeat: Understanding Parallel Reasoning in Language Model Inferencecited by 3
10 more
  1. 2026Finding the Cracks: Improving LLMs Reasoning with Paraphrastic Probing and Consistency Verificationcited by 2
  2. 2026SMCEvolve: Principled Scientific Discovery via Sequential Monte Carlo Evolutioncited by 2
  3. 2026Diffusion Alignment Beyond KL: Variance Minimisation as Effective Policy Optimisercited by 1
  4. 2026One-Token Verification for Reasoning Correctness Estimationcited by 1
  5. 2026Decoupling Exploration and Policy Optimization: Uncertainty Guided Tree Search for Hard Explorationcited by 1
  6. 2026LatticeBridge: Rare-Event Sequential Inference for Faithful Structured Sequence Synthesiscited by 1
  7. 2026FluxLite: Inference-Time Proposal Control for Discrete Diffusion Modelscited by 1
  8. 2025Bradley-Terry Policy Optimization for Generative Preference Modelingcited by 0
  9. 2026Explore Broadly, Reason Sharply: Push Small Models toward the Frontier via Samplingcited by 0
  10. 2026Decoupled Marginal Sharpening for Training-Free Inference-Time Scalingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MC-NEST: Enhancing Mathematical Reasoning in Large Language Models leveraging a Monte Carlo Self-Refine Tree
  2. 2025Hard2Verify: A Step-Level Verification Benchmark for Open-Ended Frontier Math
  3. 2025Step-KTO: Optimizing Mathematical Reasoning through Stepwise Binary Feedback
  4. 2026FABSVer: Faster Training and Better Self-Verification for LLM Mathematical Reasoning
  5. 2025Safe: Enhancing Mathematical Reasoning in Large Language Models via Retrospective Step-aware Formal Verification
  6. 2024Multi-Step Reasoning with Large Language Models, a Survey
  7. 2025FormalMATH: Benchmarking Formal Mathematical Reasoning of Large Language Models
  8. 2026LEAP: Supercharging LLMs for Formal Mathematics with Agentic Frameworks