mlsanityData to 7 Oct 2026

Scaling Relationship on Learning Mathematical Reasoning with Large Language Models

arXiv 2308.01825 ↗First posted 3 Aug 2023cs.CLcited by 417

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Mathematical reasoning is a challenging task for large language models (LLMs), while the scaling relationship of it with respect to LLM capacity is under-explored. In this paper, we investigate how the pre-training loss, supervised data amount, and augmented data amount influence the reasoning performances of a supervised LLM. We find that pre-training loss is a better indicator of the model's performance than the model's parameter count. We apply supervised fine-tuning (SFT) with different amounts of supervised data and empirically find a log-linear relation between data amount and model performance, and we find better models improve less with enlarged supervised datasets. To augment more data samples for improving model performances without any human effort, we propose to apply Rejection sampling Fine-Tuning (RFT). RFT uses supervised models to generate and collect correct reasoning paths as augmented fine-tuning datasets. We find with augmented samples containing more distinct reasoning paths, RFT improves mathematical reasoning performance more for LLMs. We also find RFT brings more improvement for less performant LLMs. Furthermore, we combine rejection samples from multiple models which push LLaMA-7B to an accuracy of 49.3% on GSM8K which outperforms the supervised fine-tuning (SFT) accuracy of 35.9% significantly.

Key facts. Cited by 417 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 41 of its 54 references that are in the map. First posted on arXiv 3 Aug 2023.

Builds on

41 of 54 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023How well do Large Language Models perform in Arithmetic tasks?16 Mar
  2. 2022Solving Quantitative Reasoning Problems with Language Models29 Jun
  3. 2022Learning Math Reasoning from Self-Sampled Correct and Partially-Correct Solutions28 May
  4. 2022Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction19 Mar

Other references · 37

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  10. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  11. 2023Let's Verify Step by Stepcited by 3,598
  12. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
25 more
  1. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  2. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  3. 2022Emergent Abilities of Large Language Modelscited by 2,483
  4. 2023PaLM 2 Technical Reportcited by 1,154
  5. 2017Hindsight Experience Replaycited by 1,096
  6. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  7. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  8. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  9. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  10. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  11. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  12. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 672
  13. 2022Large Language Models Can Self-Improvecited by 656
  14. 2020Scaling Laws for Autoregressive Generative Modelingcited by 510
  15. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 454
  16. 2019Analysing Mathematical Reasoning Abilities of Neural Modelscited by 331
  17. 2023Preference Ranking Optimization for Human Alignmentcited by 283
  18. 2021Scaling Laws for Transfercited by 265
  19. 2023Specializing Smaller Language Models towards Multi-Step Reasoningcited by 249
  20. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 231
  21. 2022Distilling Reasoning Capabilities into Smaller Language Modelscited by 155
  22. 2022Broken Neural Scaling Lawscited by 102
  23. 2023Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performancecited by 102
  24. 2023The Wisdom of Hindsight Makes Language Models Better Instruction Followerscited by 59
  25. 2021MWPToolkit: An Open-Source Framework for Deep Learning-Based Math Word Problem Solverscited by 39

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 27; 2024: 119; 2025: 158; 2026: 113.

417 papers.

Sort:

By area:

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2024Qwen2.5 Technical Reportcited by 4,713
  3. 2023Qwen Technical Reportcited by 3,273
  4. 2024Qwen2 Technical Reportcited by 2,076
  5. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  6. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  7. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  8. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  9. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  10. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  11. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  12. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 345
38 more
  1. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 338
  2. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 325
  3. 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 323
  4. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 308
  5. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Trainingcited by 291
  6. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?cited by 281
  7. 2023ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solvingcited by 276
  8. 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoningcited by 267
  9. 2024Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMscited by 265
  10. 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 255
  11. 2025SAM 3D: 3Dfy Anything in Imagescited by 230
  12. 2023How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Compositioncited by 222
  13. 2024V-STaR: Training Verifiers for Self-Taught Reasonerscited by 214
  14. 2024NVILA: Efficient Frontier Visual Language Modelscited by 206
  15. 2024Many-Shot In-Context Learningcited by 203
  16. 2024Agent Q: Advanced Reasoning and Learning for Autonomous AI Agentscited by 201
  17. 2024Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferencescited by 176
  18. 2024VinePPO: Refining Credit Assignment in RL Training of LLMscited by 168
  19. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 166
  20. 2025Agentic Reinforced Policy Optimizationcited by 166
  21. 2025An Illusion of Progress? Assessing the Current State of Web Agentscited by 165
  22. 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoningcited by 160
  23. 2024Teaching Large Language Models to Reason with Reinforcement Learningcited by 158
  24. 2024Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8Bcited by 156
  25. 2023MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoningcited by 155
  26. 2024BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrievalcited by 153
  27. 2024OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Datasetcited by 143
  28. 2025A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforcecited by 141
  29. 2024AgentGym: Evolving Large Language Model-based Agents across Diverse Environmentscited by 120
  30. 2024InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoningcited by 117
  31. 2025Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thoughtcited by 108
  32. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  33. 2024A Survey on Large Language Model-Based Game Agentscited by 106
  34. 2024Best Practices and Lessons Learned on Synthetic Datacited by 104
  35. 2025Reinforcement Learning for Long-Horizon Interactive LLM Agentscited by 103
  36. 2024RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Foldcited by 99
  37. 2023Learning From Mistakes Makes LLM Better Reasonercited by 97
  38. 2024Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervisioncited by 95

50 most cited of 417 All 417 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Climbing the Ladder of Reasoning: What LLMs Can-and Still Can't-Solve after SFT?
  2. 2026Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
  3. 2025A Survey on Large Language Models for Mathematical Reasoning
  4. 2024Benchmarking Large Language Models for Math Reasoning Tasks
  5. 2026Improving Mathematical Reasoning Capabilities in Large Language Models via Reasoning Process Error Classification
  6. 2025Advancing Mathematical Reasoning in Language Models: The Impact of Problem-Solving Data, Data Synthesis Methods, and Training Stages
  7. 2025More Data or Better Data? A Critical Analysis of Data Selection and Synthesis for Mathematical Reasoning
  8. 2025DeepDistill: Enhancing LLM Reasoning Capabilities via Large-Scale Difficulty-Graded Data Training