Mathematics, word problems, common sense, and artificial intelligence
On the map: LLM reasoning → Mathematical reasoning
Abstract (arXiv)
The paper discusses the capacities and limitations of current artificial intelligence (AI) technology to solve word problems that combine elementary knowledge with commonsense reasoning. No existing AI systems can solve these reliably. We review three approaches that have been developed, using AI natural language technology: outputting the answer directly, outputting a computer program that solves the problem, and outputting a formalized representation that can be input to an automated theorem verifier. We review some benchmarks that have been developed to evaluate these systems and some experimental studies. We discuss the limitations of the existing technology at solving these kinds of problems. We argue that it is not clear whether these kinds of limitations will be important in developing AI technology for pure mathematical research, but that they will be important in applications of mathematics, and may well be important in developing programs capable of reading and understanding mathematical content written by humans.
Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 9 of its 35 references that are in the map. First posted on arXiv 23 Jan 2023.
Builds on
9 of 35 references.
Closest ancestors · 1
Same direction, newest first
Other references · 8
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
- 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
- 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
- 2022Autoformalization with Large Language Modelscited by 252
- 2021A Neural Network Solves, Explains, and Generates University Math Problems by Program Synthesis and Few-Shot Learning at Human Levelcited by 96
- 2022Limits of an AI program for solving college math problemscited by 1
Built on by
Citing papers per year: 2023: 6; 2024: 3; 2025: 2; 2026: 1.
12 papers.
- 2023Mathematical Capabilities of ChatGPTcited by 263
- 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 175
- 2023Benchmarks for Automated Commonsense Reasoning: A Surveycited by 40
- 2024Easy Problems That LLMs Get Wrongcited by 29
- 2023Can neural networks do arithmetic? A survey on the elementary numerical skills of state-of-the-art deep learning modelscited by 18
- 2024MALMM: Multi-Agent Large Language Models for Zero-Shot Robotics Manipulationcited by 18
- 2023An Interdisciplinary Outlook on Large Language Models for Scientific Researchcited by 15
- 2023Taqyim: Evaluating Arabic NLP Tasks Using ChatGPT Modelscited by 10
- 2024AI and Generative AI for Research Discovery and Summarizationcited by 4
- 2025Conflict-Aware Fusion: Mitigating Logic Inertia in Large Language Models via Structured Cognitive Priorscited by 4
- 2026From Solvers to Research: Large Language Model-Driven Formal Mathematics at the Research Frontiercited by 1
- 2025Learning neuro-symbolic convergent term rewriting systemscited by 0
- 2026From Solvers to Research: Large Language Model-Driven Formal Mathematics at the Research Frontier8 Jul 2026
- 2025Conflict-Aware Fusion: Mitigating Logic Inertia in Large Language Models via Structured Cognitive Priors6 Dec 2025
- 2025Learning neuro-symbolic convergent term rewriting systems25 Jul 2025
- 2024MALMM: Multi-Agent Large Language Models for Zero-Shot Robotics Manipulation26 Nov 2024
- 2024Easy Problems That LLMs Get Wrong30 May 2024
- 2024AI and Generative AI for Research Discovery and Summarization8 Jan 2024
- 2023An Interdisciplinary Outlook on Large Language Models for Scientific Research3 Nov 2023
- 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve24 Sep 2023
- 2023Taqyim: Evaluating Arabic NLP Tasks Using ChatGPT Models28 Jun 2023
- 2023Can neural networks do arithmetic? A survey on the elementary numerical skills of state-of-the-art deep learning models14 Mar 2023
- 2023Benchmarks for Automated Commonsense Reasoning: A Survey9 Feb 2023
- 2023Mathematical Capabilities of ChatGPT31 Jan 2023
Similar, not cited
Close in wording, no citation either way
- 2026Artificial Intelligence for Mathematical Reasoning: An Integrated Survey of Language Models, Neuro-symbolic Systems, and Verified Discovery
- 2024Mathematics and Machine Creativity: A Survey on Bridging Mathematics with AI
- 2024Formal Mathematical Reasoning: A New Frontier in AI
- 2024Large Language Models for Mathematical Reasoning: Progresses and Challenges
- 2024Assessing the Creativity of LLMs in Proposing Novel Solutions to Mathematical Problems
- 2022Why are NLP Models Fumbling at Elementary Math? A Survey of Deep Learning based Word Problem Solvers
- 2025Large Language Models and Mathematical Reasoning Failures
- 2025Thinking Machines: Mathematical Reasoning in the Age of LLMs