mlsanityData to 14 Oct 2026

Can neural networks do arithmetic? A survey on the elementary numerical skills of state-of-the-art deep learning models

arXiv 2303.07735 ↗First posted 14 Mar 2023cs.AI cs.LGcited by 18

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Creating learning models that can exhibit sophisticated reasoning skills is one of the greatest challenges in deep learning research, and mathematics is rapidly becoming one of the target domains for assessing scientific progress in this direction. In the past few years there has been an explosion of neural network architectures, data sets, and benchmarks specifically designed to tackle mathematical problems, reporting notable success in disparate fields such as automated theorem proving, numerical integration, and discovery of new conjectures or matrix multiplication algorithms. However, despite these impressive achievements it is still unclear whether deep learning models possess an elementary understanding of quantities and symbolic numbers. In this survey we critically examine the recent literature, concluding that even state-of-the-art architectures often fall short when probed with relatively simple tasks designed to test basic numerical and arithmetic knowledge.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 61 of its 120 references that are in the map. First posted on arXiv 14 Mar 2023.

Builds on

61 of 120 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023MathPrompter: Mathematical Reasoning using Large Language Models4 Mar
  2. 2023Do Deep Neural Networks Capture Compositionality in Arithmetic Reasoning?15 Feb
  3. 2023Mathematics, word problems, common sense, and artificial intelligence23 Jan
  4. 2022A Survey of Deep Learning for Mathematical Reasoning20 Dec
  5. 2022Teaching Algorithmic Reasoning via In-context Learning15 Nov
  6. 2022Logical Tasks for Measuring Extrapolation and Rule Comprehension14 Nov
  7. 2022Transformers discover an elementary calculation system exploiting local attention and grid-like problem representation6 Jul
  8. 2022Solving Quantitative Reasoning Problems with Language Models29 Jun
  9. 2022NumGLUE: A Suite of Fundamental yet Challenging Mathematical Reasoning Tasks12 Apr
  10. 2020Generative Language Modeling for Automated Theorem Proving7 Sep

Other references · 51

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  8. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  9. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  10. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 903
  11. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  12. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
38 more
  1. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 662
  2. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  3. 2020Scaling Laws for Autoregressive Generative Modelingcited by 522
  4. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 506
  5. 2019Analysing Mathematical Reasoning Abilities of Neural Modelscited by 335
  6. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 318
  7. 2019Deep Learning for Symbolic Mathematicscited by 270
  8. 2023Mathematical Capabilities of ChatGPTcited by 268
  9. 2018Deep Learning: A Critical Appraisalcited by 217
  10. 2022Exploring Length Generalization in Large Language Modelscited by 206
  11. 2022TALM: Tool Augmented Language Modelscited by 197
  12. 2021PonderNet: Learning to Pondercited by 168
  13. 2019Do NLP Models Know Numbers? Probing Numeracy in Embeddingscited by 159
  14. 2022MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoningcited by 156
  15. 2021Investigating the Limitations of Transformers with Simple Arithmetic Taskscited by 139
  16. 2020Injecting Numerical Reasoning Skills into Language Modelscited by 133
  17. 2015Neural GPUs Learn Algorithmscited by 121
  18. 2020Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Modelscited by 110
  19. 2021A Neural Network Solves, Explains, and Generates University Math Problems by Program Synthesis and Few-Shot Learning at Human Levelcited by 96
  20. 2018Bias and Generalization in Deep Generative Models: An Empirical Studycited by 83
  21. 2021Representing Numbers in NLP: a Survey and a Visioncited by 83
  22. 2018Neural Arithmetic Logic Unitscited by 70
  23. 2019NumNet: Machine Reading Comprehension with Numerical Reasoningcited by 59
  24. 2018Translating a Math Word Problem to an Expression Treecited by 57
  25. 2020Learning to Prove Theorems by Learning to Generate Theoremscited by 43
  26. 2020Neural Arithmetic Unitscited by 28
  27. 2021Symbolic Brittleness in Sequence Models: on Systematic Generalization in Symbolic Mathematicscited by 27
  28. 2021Symbolic Behaviour in Artificial Intelligencecited by 25
  29. 2019Mathematical Reasoning in Latent Spacecited by 17
  30. 2019Learning Numeral Embeddingscited by 15
  31. 2021NumGPT: Improving Numeracy Ability of Generative Pre-trained Modelscited by 15
  32. 2022"John is 50 years old, can his son be 65?" Evaluating NLP Models' Understanding of Feasibilitycited by 15
  33. 2021Investigating Numeracy Learning Ability of a Text-to-Text Transfer Modelcited by 13
  34. 2021Towards Tractable Mathematical Reasoning: Challenges, Strategies, and Opportunities for Solving Math Word Problemscited by 12
  35. 2022Discrete and continuous representations and processing in deep learning: Looking forwardcited by 12
  36. 2021NT5?! Training T5 to Perform Numerical Reasoningcited by 11
  37. 2021Neural Sequence-to-grid Module for Learning Symbolic Rulescited by 5
  38. 2021Learning Mathematical Properties of Integerscited by 4

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 7; 2025: 5; 2026: 3.

18 papers.

Sort:

By area:

  1. 2024Transformers Can Do Arithmetic with the Right Embeddingscited by 78
  2. 2023Positional Description Matters for Transformers Arithmeticcited by 56
  3. 2024Large Language Models in Drug Discovery and Development: From Disease Mechanisms to Clinical Trialscited by 24
  4. 2024Stepwise Self-Consistent Mathematical Reasoning with Large Language Modelscited by 8
  5. 2024Visual Enumeration Remains Challenging for Multimodal Generative AIcited by 5
  6. 2024Interleaving Text and Number Embeddings to Solve Mathemathics Problemscited by 5
  7. 2024Assessing the Emergent Symbolic Reasoning Abilities of Llama Large Language Modelscited by 3
  8. 2025Sequential Enumeration in Large Language Modelscited by 3
  9. 2024A Neural Rewriting System to Solve Algorithmic Problemscited by 2
  10. 2023A Hybrid System for Systematic Generalization in Simple Arithmetic Problemscited by 1
  11. 2025Large Language Models Don't Make Sense of Word Problems. A Scoping Review from a Mathematics Education Perspectivecited by 1
  12. 2026AI-rithmeticcited by 1
6 more
  1. 2023Calabi-Yau Four/Five/Six-folds as mathbbPnw Hypersurfaces: Machine Learning, Approximation, and Generationcited by 0
  2. 2025A Synthetic Pseudo-Autoencoder Invites Examination of Tacit Assumptions in Neural Network Designcited by 0
  3. 2025Learning neuro-symbolic convergent term rewriting systemscited by 0
  4. 2025Iterative In-Context Learning to Enhance LLMs Abstract Reasoning: The Case-Study of Algebraic Taskscited by 0
  5. 2026SenseMath: Do LLMs Have Number Sense? Evaluating Shortcut Use, Judgment, and Generationcited by 0
  6. 2026BRo-JEPA: Learning Modular Transformations in Latent Spacecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Artificial Intelligence for Mathematical Reasoning: An Integrated Survey of Language Models, Neuro-symbolic Systems, and Verified Discovery
  2. 2024Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristics
  3. 2022Teaching Neural Module Networks to Do Arithmetic
  4. 2024Mathematics and Machine Creativity: A Survey on Bridging Mathematics with AI
  5. 2025Integrating Arithmetic Learning Improves Mathematical Reasoning in Smaller Models
  6. 2025Modular Arithmetic: Language Models Solve Math Digit by Digit
  7. 2024Unraveling Arithmetic in Large Language Models: The Role of Algebraic Structures
  8. 2024Language Models Do Hard Arithmetic Tasks Easily and Hardly Do Easy Arithmetic Tasks