mlsanityData to 14 Oct 2026

A Survey on Large Language Models for Mathematical Reasoning

arXiv 2506.08446 ↗First posted 10 Jun 2025cs.AI cs.CLcited by 52

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Mathematical reasoning has long represented one of the most fundamental and challenging frontiers in artificial intelligence research. In recent years, large language models (LLMs) have achieved significant advances in this area. This survey examines the development of mathematical reasoning abilities in LLMs through two high-level cognitive phases: comprehension, where models gain mathematical understanding via diverse pretraining strategies, and answer generation, which has progressed from direct prediction to step-by-step Chain-of-Thought (CoT) reasoning. We review methods for enhancing mathematical reasoning, ranging from training-free prompting to fine-tuning approaches such as supervised fine-tuning and reinforcement learning, and discuss recent work on extended CoT and "test-time scaling". Despite notable progress, fundamental challenges remain in terms of capacity, efficiency, and generalization. To address these issues, we highlight promising research directions, including advanced pretraining and knowledge augmentation techniques, formal reasoning frameworks, and meta-generalization through principled learning paradigms. This survey tries to provide some insights for researchers interested in enhancing reasoning capabilities of LLMs and for those seeking to apply these techniques to other domains.

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 193 of its 246 references that are in the map. First posted on arXiv 10 Jun 2025.

Builds on

193 of 246 references.

Closest ancestors · 39

Same direction, newest first

  1. 2025LIMO: Less is More for Reasoning5 Feb
  2. 2025Step-KTO: Optimizing Mathematical Reasoning through Stepwise Binary Feedback18 Jan
  3. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking8 Jan
  4. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges16 Dec
  5. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models7 Oct
  6. 2024LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoning3 Oct
  7. 2024Step-by-Step Reasoning for Math Problems via Twisted Sequential Monte Carlo2 Oct
  8. 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data2 Oct
  9. 2024Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problems29 Aug
  10. 2024Advancing Process Verification for Large Language Models via Tree-Based Preference Learning29 Jun
  11. 2024Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs26 Jun
  12. 2024LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback20 Jun
27 more
  1. 2024JiuZhang3.0: Efficiently Improving Mathematical Reasoning by Training Small Data Synthesis Models23 May
  2. 2024Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solving20 May
  3. 2024Iterative Reasoning Preference Optimization30 Apr
  4. 2024Achieving >97% on GSM8K: Deeply Understanding the Problems Makes LLMs Better Solvers for Math Word Problems23 Apr
  5. 2024Common 7B Language Models Already Possess Strong Math Capabilities7 Mar
  6. 2024MathScale: Scaling Instruction Tuning for Mathematical Reasoning5 Mar
  7. 2024Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning4 Mar
  8. 2024GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers29 Feb
  9. 2024MATHSENSEI: A Tool-Augmented Large Language Model for Mathematical Reasoning27 Feb
  10. 2024OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset15 Feb
  11. 2024InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoning9 Feb
  12. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models5 Feb
  13. 2024Large Language Models for Mathematical Reasoning: Progresses and Challenges31 Jan
  14. 2024Augmenting Math Word Problems via Iterative Question Composing17 Jan
  15. 2023TinyGSM: achieving >80% on GSM8k with small language models14 Dec
  16. 2023Learning From Mistakes Makes LLM Better Reasoner31 Oct
  17. 2023Llemma: An Open Language Model For Mathematics16 Oct
  18. 2023OpenWebMath: An Open Dataset of High-Quality Mathematical Web Text10 Oct
  19. 2023MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning9 Oct
  20. 2023Forward-Backward Reasoning in Large Language Models for Mathematical Verification15 Aug
  21. 2023Evaluating and Improving Tool-Augmented Computation-Intensive Math Reasoning4 Jun
  22. 2023Learning Multi-Step Reasoning by Solving Arithmetic Tasks2 Jun
  23. 2023MathPrompter: Mathematical Reasoning using Large Language Models4 Mar
  24. 2022A Survey of Deep Learning for Mathematical Reasoning20 Dec
  25. 2022Teaching Algorithmic Reasoning via In-context Learning15 Nov
  26. 2022Language Models are Multilingual Chain-of-Thought Reasoners6 Oct
  27. 2022Solving Quantitative Reasoning Problems with Language Models29 Jun

Other references · 154

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  7. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  9. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  10. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  11. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  12. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
38 more
  1. 2017Deep reinforcement learning from human preferencescited by 3,912
  2. 2023Let's Verify Step by Stepcited by 3,789
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  4. 2021Program Synthesis with Large Language Modelscited by 3,637
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  7. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  8. 2020Learning to summarize from human feedbackcited by 2,588
  9. 2023A Survey of Large Language Modelscited by 2,576
  10. 2022Emergent Abilities of Large Language Modelscited by 2,512
  11. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  12. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  13. 2024Qwen2 Technical Reportcited by 2,120
  14. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  15. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  16. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  17. 2025s1: Simple test-time scalingcited by 1,341
  18. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  19. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  20. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  21. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  22. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  23. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  24. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
  25. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  26. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 886
  27. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  28. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  29. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  30. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 688
  31. 2022Large Language Models Can Self-Improvecited by 664
  32. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 662
  33. 2022PAL: Program-aided Language Modelscited by 652
  34. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  35. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
  36. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 526
  37. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 511
  38. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 487

50 most cited of 154 All 154 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 15; 2026: 37.

52 papers.

Sort:

By area:

  1. 2025Winning Gold at IMO 2025 with a Model-Agnostic Verification-and-Refinement Pipelinecited by 82
  2. 2025VCRL: Variance-based Curriculum Reinforcement Learning for Large Language Modelscited by 36
  3. 2025Knapsack RL: Unlocking Exploration of LLMs via Optimizing Budget Allocationcited by 22
  4. 2026AgentArk: Distilling Multi-Agent Intelligence into a Single LLM Agentcited by 13
  5. 2025Supervised Reinforcement Learning: From Expert Trajectories to Step-wise Reasoningcited by 11
  6. 2026Towards Natural Personalization: Evaluating Long-Horizon Preference Following in Personalized User-LLM Interactionscited by 10
  7. 2026Token-Level LLM Collaboration via FusionRoutecited by 9
  8. 2026Uncertainty-Aware Clarification in LLM Agents with Information Gaincited by 8
  9. 2025Thinking Longer, Not Always Smarter: Evaluating LLM Capabilities in Hierarchical Legal Reasoningcited by 7
  10. 2025RSPO: Risk-Seeking Policy Optimization for Pass@k and Max@k Metrics in Large Language Modelscited by 5
  11. 2025Teaching Language Models to Reason with Toolscited by 5
  12. 2026LawThinker: A Deep Research Legal Agent in Dynamic Environmentscited by 4
38 more
  1. 2026Learning from Your Own Mistakes: Constructing Learnable Micro-Reflective Trajectories for Self-Distillationcited by 4
  2. 2026SP-Mind: An Autonomous Reasoning Agent for Spatial Proteomics Analysiscited by 4
  3. 2026FormalRx: Rectify and eXamine Semantic Failures in Autoformalizationcited by 4
  4. 2025AgentCoMa: A Compositional Benchmark Mixing Commonsense and Mathematical Reasoning in Real-World Scenarioscited by 3
  5. 2025PISA-Bench: The PISA Index as a Multilingual and Multimodal Metric for the Evaluation of Vision-Language Modelscited by 3
  6. 2025"To Survive, I Must Defect": Jailbreaking LLMs via the Game-Theory Scenarioscited by 3
  7. 2026Self-Supervised Bootstrapping of Action-Predictive Embodied Reasoningcited by 3
  8. 2026Agentic Game Development as a Verifiable Trajectory Data Engine for Scaling World Modelscited by 3
  9. 2026MMR-GRPO: Accelerating GRPO-Style Training through Diversity-Aware Reward Reweightingcited by 2
  10. 2026Adaptive Robust Estimator for Multi-Agent Reinforcement Learningcited by 2
  11. 2026Neuron-Anchored Rule Extraction for Large Language Models via Contrastive Hierarchical Ablationcited by 2
  12. 2026RAG over Thinking Traces Can Improve Reasoning Taskscited by 2
  13. 2026Single-Rollout Hidden-State Dynamics for Training-Free RLVR Data Selectioncited by 2
  14. 2026Harnessing Routing Foresight for Micro-step-level MoE load balancing in RL Post-trainingcited by 2
  15. 2026Efficient Test-Time Optimization for Multi-Agent Proof Autoformalizationcited by 2
  16. 2025Proof2Hybrid: Automatic Mathematical Benchmark Synthesis for Proof-Centric Problemscited by 1
  17. 2025Beyond Token-level Supervision: Unlocking the Potential of Decoding-based Regression via Reinforcement Learningcited by 1
  18. 2025Constructive Circuit Amplification: Improving Math Reasoning in LLMs via Targeted Sub-Network Updatescited by 1
  19. 2026TARo: Token-level Adaptive Routing for LLM Test-time Alignmentcited by 1
  20. 2026SABER-Math: Automated Benchmark for Information Retrieval Evaluation in Mathematicscited by 1
  21. 2026Hi-TTRL: Regulating Consensus with Hints for Test-Time Reinforcement Learningcited by 1
  22. 2026MedCalc-R1: Knowledge-Guided Reward Framework for Medical Mathematical Reasoningcited by 1
  23. 2025PiERN: Token-Level Routing for Integrating High-Precision Computation and Reasoningcited by 0
  24. 2025SMRC: Aligning Large Language Models with Student Reasoning for Mathematical Error Correctioncited by 0
  25. 2026Can Large Language Models Solve Engineering Equations? A Systematic Comparison of Direct Prediction and Solver-Assisted Approachescited by 0
  26. 2026From Implicit to Explicit: Token-Efficient Logical Supervision for Mathematical Reasoning in LLMscited by 0
  27. 2026From Meta-Thought to Execution: Cognitively Aligned Post-Training for Generalizable and Reliable LLM Reasoningcited by 0
  28. 2026SSL: Sweet Spot Learning for Differentiated Guidance in Agentic Optimizationcited by 0
  29. 2026Reliable Use of Lemmas via Eligibility Reasoning and Section-Aware Reinforcement Learningcited by 0
  30. 2026Large Language Models for Math Education in Low-Resource Languages: A Study in Sinhala and Tamilcited by 0
  31. 2026RouteGoT: Node-Adaptive Routing for Cost-Efficient Graph of Thoughts Reasoningcited by 0
  32. 2026Off-Policy Value-Based Reinforcement Learning for Large Language Modelscited by 0
  33. 2026A Data-Efficient Path to Multilingual LLMs: Language Expansion via Post-training PARAMDelta Integration into Upcycled MoEcited by 0
  34. 2026ReverseMath: Answer Inversion for Scalable and Verifiable Mathematical Problem Generationcited by 0
  35. 2026EAPO: Entropy-Driven Adaptive Positive-Negative Sample Weighting for Policy Optimization in Open-Ended QAcited by 0
  36. 2026CA-BED: Conversation-Aware Bayesian Experimental Designcited by 0
  37. 2026Benchmarking Large Language Models on Floating-Point Error Classificationcited by 0
  38. 2026Evaluating SageMath-Augmented LLM Agents for Computational and Experimental Mathematicscited by 0

50 most cited of 52 All 52 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
  2. 2024LLM Reasoning Engine: Specialized Training for Enhanced Mathematical Reasoning
  3. 2024Benchmarking Large Language Models for Math Reasoning Tasks
  4. 2025A Survey on Mathematical Reasoning and Optimization with Large Language Models
  5. 2025Large Language Models and Mathematical Reasoning Failures
  6. 2024From Next-Token to Mathematics: The Learning Dynamics of Mathematical Reasoning in Language Models
  7. 2024Step Guided Reasoning: Improving Mathematical Reasoning using Guidance Generation and Step Reasoning
  8. 2025Evaluating Mathematical Reasoning Across Large Language Models: A Fine-Grained Approach