mlsanityData to 7 Oct 2026

Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problems

arXiv 2408.16293 ↗First posted 29 Aug 2024cs.CL cs.AI cs.LGcited by 49

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Language models have demonstrated remarkable performance in solving reasoning tasks; however, even the strongest models still occasionally make reasoning mistakes. Recently, there has been active research aimed at improving reasoning accuracy, particularly by using pretrained language models to "self-correct" their mistakes via multi-round prompting. In this paper, we follow this line of work but focus on understanding the usefulness of incorporating "error-correction" data directly into the pretraining stage. This data consists of erroneous solution steps immediately followed by their corrections. Using a synthetic math dataset, we show promising results: this type of pretrain data can help language models achieve higher reasoning accuracy directly (i.e., through simple auto-regression, without multi-round prompting) compared to pretraining on the same amount of error-free data. We also delve into many details, such as (1) how this approach differs from beam search, (2) how such data can be prepared, (3) whether masking is needed on the erroneous tokens, (4) the amount of error required, (5) whether such data can be deferred to the fine-tuning stage, and many others.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 20 of its 31 references that are in the map. First posted on arXiv 29 Aug 2024.

Builds on

20 of 31 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process29 Jul
  2. 2023TinyGSM: achieving >80% on GSM8k with small language models14 Dec

Other references · 18

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  4. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  5. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  6. 2022In-context Learning and Induction Headscited by 930
  7. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  8. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 304
  9. 2023Physics of Language Models: Part 3.1, Knowledge Storage and Extractioncited by 264
  10. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  11. 2023What Algorithms can Transformers Learn? A Study in Length Generalizationcited by 198
  12. 2023Physics of Language Models: Part 3.2, Knowledge Manipulationcited by 142
6 more
  1. 2024Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Lawscited by 137
  2. 2024Small Language Models Need Strong Verifiers to Self-Correct Reasoningcited by 85
  3. 2023Physics of Language Models: Part 1, Learning Hierarchical Language Structurescited by 83
  4. 2022Generating Natural Language Proofs with Verifier-Guided Searchcited by 52
  5. 2023Errors are Useful Prompts: Instruction Guided Task Programming with Verifier-Assisted Iterative Promptingcited by 45
  6. 2024From Instructions to Constraints: Language Model Alignment with Automatic Constraint Verificationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 33; 2026: 8.

49 papers.

Sort:

By area:

  1. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 405
  2. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 373
  3. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
  4. 2024O1 Replication Journey: A Strategic Progress Report -- Part 1cited by 140
  5. 2025Process Reward Models That Thinkcited by 118
  6. 2025Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thoughtcited by 108
  7. 2025Self-Training Elicits Concise Reasoning in Large Language Modelscited by 97
  8. 2024MALT: Improving Reasoning with Multi-Agent LLM Trainingcited by 71
  9. 2025A Survey on Large Language Models for Mathematical Reasoningcited by 52
  10. 2025AgentRefine: Enhancing Agent Generalization through Refinement Tuningcited by 51
  11. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspectivecited by 49
  12. 2024GameArena: Evaluating LLM Reasoning through Live Computer Gamescited by 32
37 more
  1. 2025Physics of Language Models: Part 4.1, Architecture Design and the Magic of Canon Layerscited by 32
  2. 2025Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Propertiescited by 28
  3. 2025Step Back to Leap Forward: Self-Backtracking for Boosting Reasoning of Language Modelscited by 27
  4. 2026MAS-Orchestra: Understanding and Improving Multi-Agent Reasoning Through Holistic Orchestration and Controlled Benchmarkscited by 26
  5. 2025PALADIN: Self-Correcting Language Model Agents to Cure Tool-Failure Casescited by 21
  6. 2025Boosting LLM Reasoning via Spontaneous Self-Correctioncited by 18
  7. 2025OneFlow: Concurrent Mixed-Modal and Interleaved Generation with Edit Flowscited by 17
  8. 2025GSM-Infinite: How Do Your LLMs Behave over Infinitely Increasing Context Length and Reasoning Complexity?cited by 16
  9. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 14
  10. 2025LongPerceptualThoughts: Distilling System-2 Reasoning for System-1 Perceptioncited by 14
  11. 2025Advancing Mathematical Reasoning in Language Models: The Impact of Problem-Solving Data, Data Synthesis Methods, and Training Stagescited by 13
  12. 2024Chasing Progress, Not Perfection: Revisiting Strategies for End-to-End LLM Plan Generationcited by 11
  13. 2025RARE: Retrieval-Augmented Reasoning Modelingcited by 11
  14. 2025Learning to Reason from Feedback at Test-Timecited by 10
  15. 2025Modeling Student Learning with 3.8 Million Program Tracescited by 10
  16. 2025Behavior Injection: Preparing Language Models for Reinforcement Learningcited by 8
  17. 2025MedReflect: Teaching Medical LLMs to Self-Improve via Reflective Correctioncited by 8
  18. 2026ZipServ: Fast and Memory-Efficient LLM Inference with Hardware-Aware Lossless Compressioncited by 8
  19. 2025Analyzing the Effects of Supervised Fine-Tuning on Model Knowledge from Token and Parameter Levelscited by 7
  20. 2025When Do LLMs Admit Their Mistakes? Understanding The Role Of Model Belief In Retractioncited by 6
  21. 2025Think Before You Accept: Semantic Reflective Verification for Faster Speculative Decodingcited by 6
  22. 2024Learning to Better Search with Language Models via Guided Reinforced Self-Trainingcited by 5
  23. 2025Controlling Large Language Model with Latent Actionscited by 5
  24. 2025Tailored Primitive Initialization is the Secret Key to Reinforcement Learningcited by 4
  25. 2024COrAL: Order-Agnostic Language Modeling for Efficient Iterative Refinementcited by 3
  26. 2025Your Language Model May Think Too Rigidly: Achieving Reasoning Consistency with Symmetry-Enhanced Trainingcited by 3
  27. 2025EVALUESTEER: Measuring Reward Model Steerability Towards Values and Preferencescited by 3
  28. 2025CASCADE Your Datasets for Cross-Mode Knowledge Retrieval of Language Modelscited by 2
  29. 2025Beyond Theorem Proving: Formulation, Framework and Benchmark for Formal Problem-Solvingcited by 2
  30. 2025RetrySQL: text-to-SQL training with retry data for self-correcting query generationcited by 2
  31. 2025Teaching Transformers to Solve Combinatorial Problems through Efficient Trial & Errorcited by 2
  32. 2026STAR-S: Improving Safety Alignment through Self-Taught Reasoning on Safety Rulescited by 2
  33. 2026A Relative-Budget Theory for Reinforcement Learning with Verifiable Rewards in Large Language Model Reasoningcited by 2
  34. 2026Understanding Curriculum Learning in Large Language Models via Cross-Difficulty Optimization Dynamicscited by 0
  35. 2026LogicTrack: Auditing Reasoning Trajectories of Large Language Models with Formal Logic Solverscited by 0
  36. 2026Decoupling Token Roles in Autoregressive Pretrainingcited by 0
  37. 2026Correct Answers, Invalid Traces: What Verifiable Grade-School Math Reveals About Chain-of-Thought Tracescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Evaluating Mathematical Reasoning of Large Language Models: A Focus on Error Identification and Correction
  2. 2024Exposing the Achilles' Heel: Evaluating LLMs Ability to Handle Mistakes in Mathematical Reasoning
  3. 2025MathRobust-LV: Evaluation of Large Language Models' Robustness to Linguistic Variations in Mathematical Reasoning
  4. 2025Error Classification of Large Language Models on Math Word Problems: A Dynamically Adaptive Framework
  5. 2022Learning Math Reasoning from Self-Sampled Correct and Partially-Correct Solutions
  6. 2025LEMMA: Learning from Errors for MatheMatical Advancement in LLMs
  7. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoning
  8. 2025MathDebugger: Detecting and Diagnosing Errors in Synthetic Mathematical Data