# Why are NLP Models Fumbling at Elementary Math? A Survey of Deep Learning based Word Problem Solvers

arXiv: https://arxiv.org/abs/2205.15683 · HTML: https://mlsanity.com/paper/2205.15683/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 81 references that are in the map.
- First posted on arXiv 31 May 2022.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Are NLP Models really able to Solve Simple Math Word Problems?](https://mlsanity.com/paper/2103.07191/) (2021)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms](https://mlsanity.com/paper/1905.13319/) (2019)
- [Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems](https://mlsanity.com/paper/1705.04146/) (2017)
- [Solving General Arithmetic Word Problems](https://mlsanity.com/paper/1608.01413/) (2016)
- [Keystroke Dynamics as Part of Lifelogging](https://mlsanity.com/paper/2010.10144/) (2020)
- [A Diverse Corpus for Evaluating and Developing English Math Word Problem Solvers](https://mlsanity.com/paper/2106.15772/) (2021)
- [GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning](https://mlsanity.com/paper/2105.14517/) (2021)
- [Generate & Rank: A Multi-task Framework for Math Word Problems](https://mlsanity.com/paper/2109.03034/) (2021)
- [Semantically-Aligned Equation Generation for Solving and Reasoning Math Word Problems](https://mlsanity.com/paper/1811.00720/) (2018)
- [MWPToolkit: An Open-Source Framework for Deep Learning-Based Math Word Problem Solvers](https://mlsanity.com/paper/2109.00799/) (2021)
- [Neural-Symbolic Solver for Math Word Problems with Auxiliary Tasks](https://mlsanity.com/paper/2107.01431/) (2021)
- [Dynamic Knowledge Distillation for Pre-trained Language Models](https://mlsanity.com/paper/2109.11295/) (2021)
- [Mapping to Declarative Knowledge for Word Problem Solving](https://mlsanity.com/paper/1712.09391/) (2017)
- [Semantically-Aligned Universal Tree-Structured Solver for Math Word Problems](https://mlsanity.com/paper/2010.06823/) (2020)
- [The Gap of Semantic Parsing: A Survey on Automatic Math Word Problem Solvers](https://mlsanity.com/paper/1808.07290/) (2018)
- [Graph-to-Tree Neural Networks for Learning Structured Input-Output Translation with Applications to Semantic Parsing and Math Word Problem](https://mlsanity.com/paper/2004.13781/) (2020)
- [Seeking Patterns, Not just Memorizing Procedures: Contrastive Learning for Solving Math Word Problems](https://mlsanity.com/paper/2110.08464/) (2021)
- [Annotating Derivations: A New Evaluation Strategy and Dataset for Algebra Word Problems](https://mlsanity.com/paper/1609.07197/) (2016)
- [Unit Dependency Graph and its Application to Arithmetic Word Problem Solving](https://mlsanity.com/paper/1612.00969/) (2016)
- [Learning by Fixing: Solving Math Word Problems with Weak Supervision](https://mlsanity.com/paper/2012.10582/) (2020)

## Built on by (most cited first, 9 of 9)

- [Learning by Analogy: Diverse Questions Generation in Math Word Problem](https://mlsanity.com/paper/2306.09064/) (2023)
- [Interpretable Math Word Problem Solution Generation Via Step-by-step Planning](https://mlsanity.com/paper/2306.00784/) (2023)
- [Language Models are Symbolic Learners in Arithmetic](https://mlsanity.com/paper/2410.15580/) (2024)
- [LUNA: Language Understanding with Number Augmentations on Transformers via Number Plugins and Pre-training](https://mlsanity.com/paper/2212.02691/) (2022)
- [Math Word Problem Solving by Generating Linguistic Variants of Problem Statements](https://mlsanity.com/paper/2306.13899/) (2023)
- [Towards Robust Numerical Question Answering: Diagnosing Numerical Capabilities of NLP Systems](https://mlsanity.com/paper/2211.07455/) (2022)
- [JiuZhang 2.0: A Unified Chinese Pre-trained Language Model for Multi-task Mathematical Problem Solving](https://mlsanity.com/paper/2306.11027/) (2023)
- [Artificial Intelligence for Mathematical Reasoning: An Integrated Survey of Language Models, Neuro-symbolic Systems, and Verified Discovery](https://mlsanity.com/paper/2606.08728/) (2026)
- [Techniques to Improve Neural Math Word Problem Solvers](https://mlsanity.com/paper/2302.03145/) (2023)

## Abstract (arXiv)

From the latter half of the last decade, there has been a growing interest in developing algorithms for automatically solving mathematical word problems (MWP). It is a challenging and unique task that demands blending surface level text pattern recognition with mathematical reasoning. In spite of extensive research, we are still miles away from building robust representations of elementary math word problems and effective solutions for the general task. In this paper, we critically examine the various models that have been developed for solving word problems, their pros and cons and the challenges ahead. In the last two years, a lot of deep learning models have recorded competing results on benchmark datasets, making a critical and conceptual analysis of literature highly useful at this juncture. We take a step back and analyse why, in spite of this abundance in scholarly interest, the predominantly used experiment and dataset designs continue to be a stumbling block. From the vantage point of having analyzed the literature closely, we also endeavour to provide a road-map for future math word problem research.