mlsanityData to 14 Oct 2026

Analysing Mathematical Reasoning Abilities of Neural Models

arXiv 1904.01557 ↗First posted 2 Apr 2019cs.LG stat.MLcited by 335

Not on the map. Why?

Cited by 93 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Mathematical reasoning---a core ability within human intelligence---presents some unique challenges as a domain: we do not come to understand and solve mathematical problems primarily on the back of experience and evidence, but on the basis of inferring, learning, and exploiting laws, axioms, and symbol manipulation rules. In this paper, we present a new challenge for the evaluation (and eventually the design) of neural architectures and similar system, developing a task suite of mathematics problems involving sequential questions and answers in a free-form textual input/output format. The structured nature of the mathematics domain, covering arithmetic, algebra, probability and calculus, enables the construction of training and test splits designed to clearly illuminate the capabilities and failure-modes of different architectures, as well as evaluate their ability to compose and relate knowledge and learned processes. Having described the data generation process and its potential future expansions, we conduct a comprehensive analysis of models from two broad classes of the most powerful sequence-to-sequence architectures and find notable differences in their ability to resolve mathematical problems and generalize their knowledge.

Key facts. Cited by 335 papers in the map (data through 14 Oct 2026). First posted on arXiv 2 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 50; 2022: 43; 2023: 61; 2024: 80; 2025: 63; 2026: 38.

335 papers.

Sort:

By area:

  1. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  2. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  3. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  4. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  5. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  6. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  7. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  8. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  9. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
  10. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 629
  11. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 427
  12. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
38 more
  1. 2021MiniF2F: a cross-system benchmark for formal Olympiad-level mathematicscited by 391
  2. 2023The Impact of Positional Encoding on Length Generalization in Transformerscited by 327
  3. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 226
  4. 2022HyperTree Proof Search for Neural Theorem Provingcited by 197
  5. 2021Neural Symbolic Regression that Scalescited by 192
  6. 2023TheoremQA: A Theorem-driven Question Answering datasetcited by 192
  7. 2022Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Modelscited by 191
  8. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoningcited by 173
  9. 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoningcited by 160
  10. 2023Orca 2: Teaching Small Language Models How to Reasoncited by 159
  11. 2022A Review on Language Models as Knowledge Basescited by 157
  12. 2023MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoningcited by 156
  13. 2022Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphscited by 144
  14. 2023SGD learning on neural networks: leap complexity and saddle-to-saddle dynamicscited by 143
  15. 2022Impact of Pretraining Term Frequencies on Few-Shot Reasoningcited by 142
  16. 2021Investigating the Limitations of Transformers with Simple Arithmetic Taskscited by 139
  17. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 135
  18. 2022MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derenderingcited by 133
  19. 2023How well do Large Language Models perform in Arithmetic tasks?cited by 130
  20. 2024ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoningcited by 128
  21. 2022A Survey of Deep Learning for Mathematical Reasoningcited by 124
  22. 2021The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformerscited by 117
  23. 2024Tokenization counts: the impact of tokenization on arithmetic in frontier LLMscited by 115
  24. 2022NumGLUE: A Suite of Fundamental yet Challenging Mathematical Reasoning Taskscited by 112
  25. 2022Teaching Algorithmic Reasoning via In-context Learningcited by 112
  26. 2024Best Practices and Lessons Learned on Synthetic Datacited by 106
  27. 2024MathBench: Evaluating the Theory and Application Proficiency of LLMs with a Hierarchical Mathematics Benchmarkcited by 105
  28. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 98
  29. 2022Language Models are General-Purpose Interfacescited by 97
  30. 2024DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solvingcited by 95
  31. 2022On the Paradox of Learning to Reason from Datacited by 93
  32. 2025Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoningcited by 91
  33. 2021Going Beyond Linear Transformers with Recurrent Fast Weight Programmerscited by 89
  34. 2021Size-Invariant Graph Representations for Graph Classification Extrapolationscited by 84
  35. 2021Representing Numbers in NLP: a Survey and a Visioncited by 83
  36. 2021NaturalProofs: Mathematical Theorem Proving in Natural Languagecited by 81
  37. 2021OOD-GNN: Out-of-Distribution Generalized Graph Neural Networkcited by 78
  38. 2024Transformers Can Do Arithmetic with the Right Embeddingscited by 78

50 most cited of 335 All 335 (JSON)