mlsanityData to 14 Oct 2026

A Survey on Feedback-based Multi-step Reasoning for Large Language Models on Mathematics

arXiv 2502.14333 ↗First posted 20 Feb 2025cs.CL cs.AIcited by 11

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Recent progress in large language models (LLM) found chain-of-thought prompting strategies to improve the reasoning ability of LLMs by encouraging problem solving through multiple steps. Therefore, subsequent research aimed to integrate the multi-step reasoning process into the LLM itself through process rewards as feedback and achieved improvements over prompting strategies. Due to the cost of step-level annotation, some turn to outcome rewards as feedback. Aside from these training-based approaches, training-free techniques leverage frozen LLMs or external tools for feedback at each step to enhance the reasoning process. With the abundance of work in mathematics due to its logical nature, we present a survey of strategies utilizing feedback at the step and outcome levels to enhance multi-step math reasoning for LLMs. As multi-step reasoning emerges a crucial component in scaling LLMs, we hope to establish its foundation for easier understanding and empower further research.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 80 of its 125 references that are in the map. First posted on arXiv 20 Feb 2025.

Builds on

80 of 125 references.

Closest ancestors · 18

Same direction, newest first

  1. 2024Ensembling Large Language Models with Process Reward-Guided Tree Search for Better Complex Reasoning20 Dec
  2. 2024FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI7 Nov
  3. 2024Enhancing Mathematical Reasoning in LLMs by Stepwise Correction16 Oct
  4. 2024Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models10 Oct
  5. 2024Token-Supervised Value Models for Enhancing Mathematical Problem-Solving Capabilities of Large Language Models12 Jul
  6. 2024Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs26 Jun
  7. 2024LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback20 Jun
  8. 2024OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI18 Jun
  9. 2024MindStar: Enhancing Math Reasoning in Pre-trained LLMs at Inference Time25 May
  10. 2024MathDivide: Improved mathematical reasoning by large language models12 May
  11. 2024Evaluating Mathematical Reasoning Beyond Accuracy8 Apr
  12. 2024Don't Trust: Verify -- Grounding LLM Quantitative Reasoning with Autoformalization26 Mar
6 more
  1. 2024MathScale: Scaling Instruction Tuning for Mathematical Reasoning5 Mar
  2. 2024Stepwise Self-Consistent Mathematical Reasoning with Large Language Models24 Feb
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models5 Feb
  4. 2024MARIO: MAth Reasoning with code Interpreter Output -- A Reproducible Pipeline16 Jan
  5. 2023Learning From Mistakes Makes LLM Better Reasoner31 Oct
  6. 2023No Train Still Gain. Unleash Mathematical Reasoning of Large Language Models with Monte Carlo Tree Search Guided by Energy Function1 Sep

Other references · 62

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2023Let's Verify Step by Stepcited by 3,789
  6. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  7. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  8. 2016Concrete Problems in AI Safetycited by 1,642
  9. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  10. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  11. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  12. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
38 more
  1. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 662
  2. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  3. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 499
  4. 2025Humanity's Last Examcited by 491
  5. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
  6. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 441
  7. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoningcited by 415
  8. 2016Solving General Arithmetic Word Problemscited by 360
  9. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 344
  10. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 318
  11. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Trainingcited by 295
  12. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoningcited by 292
  13. 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoningcited by 279
  14. 2024From r to Q^*: Your Language Model is Secretly a Q-Functioncited by 242
  15. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 233
  16. 2024V-STaR: Training Verifiers for Self-Taught Reasonerscited by 217
  17. 2023ProofNet: Autoformalizing and Formally Proving Undergraduate-Level Mathematicscited by 192
  18. 2024Free Process Rewards without Process Labelscited by 165
  19. 2024AlphaMath Almost Zero: Process Supervision without Processcited by 162
  20. 2023SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoningcited by 159
  21. 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 143
  22. 2023Can Large Language Models Really Improve by Self-critiquing Their Own Plans?cited by 108
  23. 2025Reward-Guided Speculative Decoding for Efficient LLM Reasoningcited by 106
  24. 2024GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinementscited by 103
  25. 2023Let's reward step by step: Step-Level reward model as the Navigators for Reasoningcited by 102
  26. 2025PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Modelscited by 99
  27. 2024On Designing Effective RL Reward at Training Time for LLM Reasoningcited by 87
  28. 2024Process Reward Model with Q-Value Rankingscited by 80
  29. 2024TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Samplingcited by 61
  30. 2024Enhancing LLM Reasoning with Reward-guided Tree Searchcited by 58
  31. 2024Offline Reinforcement Learning for LLM Multi-Step Reasoningcited by 54
  32. 2024Multi-step Problem Solving Through a Verifier: An Empirical Analysis on Model-induced Process Supervisioncited by 52
  33. 2024MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Datacited by 52
  34. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspectivecited by 50
  35. 2024Learning Planning-based Reasoning by Trajectories Collection and Process Reward Synthesizingcited by 49
  36. 2024Entropy-Regularized Process Reward Modelcited by 38
  37. 2023GRACE: Discriminator-Guided Chain-of-Thought Reasoningcited by 35
  38. 2023Mathematical Language Models: A Surveycited by 24

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 2.

11 papers.

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  2. 2025SIM-CoT: Supervised Implicit Chain-of-Thoughtcited by 70
  3. 2025Does Reasoning Introduce Bias? A Study of Social Bias Evaluation and Mitigation in LLM Reasoningcited by 21
  4. 2025A Survey of Scaling in Large Language Model Reasoningcited by 10
  5. 2026SeLaR: Selective Latent Reasoning in Large Language Modelscited by 7
  6. 2025SSR: Speculative Parallel Scaling Reasoning in Test-timecited by 4
  7. 2025Mathematics Isn't Culture-Free: Probing Cultural Gaps via Entity and Scenario Perturbationscited by 3
  8. 2025Unveiling Instruction-Specific Neurons & Experts: An Analytical Framework for LLM's Instruction-Following Capabilitiescited by 2
  9. 2025PiERN: Token-Level Routing for Integrating High-Precision Computation and Reasoningcited by 0
  10. 2025Iterative LLM-Based Generation and Refinement of Distracting Conditions in Math Word Problemscited by 0
  11. 2026Gradually Compacting Large Language Models for Reasoning Like a Boiling Frogcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Step-KTO: Optimizing Mathematical Reasoning through Stepwise Binary Feedback
  2. 2025A Survey on Large Language Models for Mathematical Reasoning
  3. 2025Advancing Multi-Step Mathematical Reasoning in Large Language Models through Multi-Layered Self-Reflection with Auto-Prompting
  4. 2024Step Guided Reasoning: Improving Mathematical Reasoning using Guidance Generation and Step Reasoning
  5. 2023Learning Multi-Step Reasoning by Solving Arithmetic Tasks
  6. 2024Large Language Models for Mathematical Reasoning: Progresses and Challenges
  7. 2024Teaching-Inspired Integrated Prompting Framework: A Novel Approach for Enhancing Reasoning in Large Language Models
  8. 2025Thinking Machines: Mathematical Reasoning in the Age of LLMs