mlsanityData to 7 Oct 2026

A Survey of Deep Learning for Mathematical Reasoning

arXiv 2212.10535 ↗First posted 20 Dec 2022cs.AI cs.CL cs.CVcited by 124

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Mathematical reasoning is a fundamental aspect of human intelligence and is applicable in various fields, including science, engineering, finance, and everyday life. The development of artificial intelligence (AI) systems capable of solving math problems and proving theorems has garnered significant interest in the fields of machine learning and natural language processing. For example, mathematics serves as a testbed for aspects of reasoning that are challenging for powerful deep learning models, driving new algorithmic and modeling advances. On the other hand, recent advances in large-scale neural language models have opened up new benchmarks and opportunities to use deep learning for mathematical reasoning. In this survey paper, we review the key tasks, datasets, and methods at the intersection of mathematical reasoning and deep learning over the past decade. We also evaluate existing benchmarks and methods, and discuss future research directions in this domain.

Key facts. Cited by 124 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 155 of its 234 references that are in the map. First posted on arXiv 20 Dec 2022.

Builds on

155 of 234 references.

Closest ancestors · 5

Same direction, newest first

  1. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning29 Sep
  2. 2022Solving Quantitative Reasoning Problems with Language Models29 Jun
  3. 2022NumGLUE: A Suite of Fundamental yet Challenging Mathematical Reasoning Tasks12 Apr
  4. 2022Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction19 Mar
  5. 2020Generative Language Modeling for Automated Theorem Proving7 Sep

Other references · 150

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  8. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  9. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  10. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  11. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  12. 2020Measuring Massive Multitask Language Understandingcited by 7,768
38 more
  1. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  7. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  8. 2021Program Synthesis with Large Language Modelscited by 3,509
  9. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  10. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  11. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  12. 2018CBAM: Convolutional Block Attention Modulecited by 2,463
  13. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  14. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  15. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  16. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  17. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  18. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  19. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  20. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  21. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  22. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  23. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  24. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  25. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  26. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  27. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  28. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  29. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  30. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  31. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  32. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  33. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  34. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  35. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  36. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  37. 2022PAL: Program-aided Language Modelscited by 635
  38. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554

50 most cited of 150 All 150 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 46; 2024: 37; 2025: 31; 2026: 10.

124 papers.

Sort:

By area:

  1. 2023A Survey of Large Language Modelscited by 2,537
  2. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  3. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  4. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
  5. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  6. 2023LeanDojo: Theorem Proving with Retrieval-Augmented Language Modelscited by 414
  7. 2023Llemma: An Open Language Model For Mathematicscited by 385
  8. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 352
  9. 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 283
  10. 2023Mathematical Capabilities of ChatGPTcited by 263
  11. 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 261
  12. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 246
38 more
  1. 2023TheoremQA: A Theorem-driven Question Answering datasetcited by 189
  2. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  3. 2023GeomVerse: A Systematic Evaluation of Large Models for Geometric Reasoningcited by 107
  4. 2023Don't Trust ChatGPT when Your Question is not in English: A Study of Multilingual Abilities and Types of LLMscited by 103
  5. 2024Large Language Models Meet NLP: A Surveycited by 102
  6. 2023Let's reward step by step: Step-Level reward model as the Navigators for Reasoningcited by 96
  7. 2023CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Modelscited by 90
  8. 2023Answering Questions by Meta-Reasoning over Multiple Chains of Thoughtcited by 87
  9. 2023Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Taskscited by 84
  10. 2023Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agentscited by 72
  11. 2024Evaluating Mathematical Reasoning Beyond Accuracycited by 69
  12. 2023Auto-Regressive Next-Token Predictors are Universal Learnerscited by 65
  13. 2024LLM With Tools: A Surveycited by 60
  14. 2023A Multi-Modal Neural Geometric Solver with Textual Clauses Parsed from Diagramcited by 55
  15. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challengescited by 55
  16. 2024Jailbreak Large Vision-Language Models Through Multi-Modal Linkagecited by 53
  17. 2023Autonomous GIS: the next-generation AI-powered GIScited by 52
  18. 2025SMART: Self-Aware Agent for Tool Overuse Mitigationcited by 52
  19. 2025A Survey on Large Language Models for Mathematical Reasoningcited by 52
  20. 2023Evaluating and Improving Tool-Augmented Computation-Intensive Math Reasoningcited by 49
  21. 2024JiuZhang3.0: Efficiently Improving Mathematical Reasoning by Training Small Data Synthesis Modelscited by 44
  22. 2023SNIP: Bridging Mathematical Symbolic and Numeric Realms with Unified Pre-trainingcited by 41
  23. 2024Number Cookbook: Number Understanding of Language Models and How to Improve Itcited by 41
  24. 2024Advancing Tool-Augmented Large Language Models: Integrating Insights from Errors in Inference Treescited by 38
  25. 2024ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detectioncited by 37
  26. 2024miniCTX: Neural Theorem Proving with (Long-)Contextscited by 35
  27. 2024MATHSENSEI: A Tool-Augmented Large Language Model for Mathematical Reasoningcited by 34
  28. 2024AlphaVerus: Bootstrapping Formally Verified Code Generation through Self-Improving Translation and Treefinementcited by 33
  29. 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoningcited by 33
  30. 2024Weak-to-Strong Reasoningcited by 32
  31. 2023Toolink: Linking Toolkit Creation and Using through Chain-of-Solving on Open-Source Modelcited by 29
  32. 2025Prover Agent: An Agent-Based Framework for Formal Mathematical Proofscited by 28
  33. 2023Plan, Verify and Switch: Integrated Reasoning with Diverse X-of-Thoughtscited by 27
  34. 2024Are Large Language Models Good Statisticians?cited by 26
  35. 2024What Are the Odds? Language Models Are Capable of Probabilistic Reasoningcited by 25
  36. 2023Mathematical Language Models: A Surveycited by 24
  37. 2024AutoPRM: Automating Procedural Supervision for Multi-Step Reasoning via Controllable Question Decompositioncited by 24
  38. 2023LANS: A Layout-Aware Neural Solver for Plane Geometry Problemcited by 23

50 most cited of 124 All 124 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Artificial Intelligence for Mathematical Reasoning: An Integrated Survey of Language Models, Neuro-symbolic Systems, and Verified Discovery
  2. 2026Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
  3. 2025Evaluating Mathematical Reasoning Across Large Language Models: A Fine-Grained Approach
  4. 2025Thinking Machines: Mathematical Reasoning in the Age of LLMs
  5. 2024Formal Mathematical Reasoning: A New Frontier in AI
  6. 2024Benchmarking Large Language Models for Math Reasoning Tasks
  7. 2025Mathematical Computation and Reasoning Errors by Large Language Models
  8. 2024Mathematics and Machine Creativity: A Survey on Bridging Mathematics with AI