mlsanityData to 7 Oct 2026

MWP-BERT: Numeracy-Augmented Pre-training for Math Word Problem Solving

arXiv 2107.13435 ↗First posted 28 Jul 2021cs.AIcited by 32

Not on the map. Why?

Abstract (arXiv)

Math word problem (MWP) solving faces a dilemma in number representation learning. In order to avoid the number representation issue and reduce the search space of feasible solutions, existing works striving for MWP solving usually replace real numbers with symbolic placeholders to focus on logic reasoning. However, different from common symbolic reasoning tasks like program synthesis and knowledge graph reasoning, MWP solving has extra requirements in numerical reasoning. In other words, instead of the number value itself, it is the reusable numerical property that matters more in numerical reasoning. Therefore, we argue that injecting numerical properties into symbolic placeholders with contextualized representation learning schema can provide a way out of the dilemma in the number representation issue here. In this work, we introduce this idea to the popular pre-training language model (PLM) techniques and build MWP-BERT, an effective contextual number representation PLM. We demonstrate the effectiveness of our MWP-BERT on MWP solving and several MWP-specific understanding tasks on both English and Chinese benchmarks.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 23 of its 52 references that are in the map. First posted on arXiv 28 Jul 2021.

Builds on

23 of 52 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  6. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  7. 2020Revisiting Pre-Trained Models for Chinese Natural Language Processingcited by 262
  8. 2019Do NLP Models Know Numbers? Probing Numeracy in Embeddingscited by 157
  9. 2020Injecting Numerical Reasoning Skills into Language Modelscited by 133
  10. 2021Representing Numbers in NLP: a Survey and a Visioncited by 83
  11. 2020Do Language Embeddings Capture Scales?cited by 63
  12. 2020Ape210K: A Large-Scale and Template-Rich Dataset of Math Word Problemscited by 60
11 more
  1. 2018Translating a Math Word Problem to an Expression Treecited by 57
  2. 2019Giving BERT a Calculator: Finding Operations and Arguments with Reading Comprehensioncited by 51
  3. 2018Semantically-Aligned Equation Generation for Solving and Reasoning Math Word Problemscited by 49
  4. 2021Neural-Symbolic Solver for Math Word Problems with Auxiliary Taskscited by 38
  5. 2020Semantically-Aligned Universal Tree-Structured Solver for Math Word Problemscited by 35
  6. 2020Learning by Fixing: Solving Math Word Problems with Weak Supervisioncited by 25
  7. 2021Investigating Math Word Problems using Pretrained Multilingual Language Modelscited by 21
  8. 2019Mathematical Reasoning in Latent Spacecited by 17
  9. 2020SMART: A Situation Model for Algebra Story Problems via Attributed Grammarcited by 14
  10. 2019Text2Math: End-to-end Parsing Text into Math Expressionscited by 13
  11. 2019Quantity Tagger: A Latent-Variable Sequence Labeling Approach to Solving Addition-Subtraction Word Problemscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 3; 2023: 12; 2024: 8; 2025: 6; 2026: 2.

32 papers.

Sort:

By area:

  1. 2022A Survey of Deep Learning for Mathematical Reasoningcited by 124
  2. 2021A Neural Network Solves, Explains, and Generates University Math Problems by Program Synthesis and Few-Shot Learning at Human Levelcited by 96
  3. 2023A Multi-Modal Neural Geometric Solver with Textual Clauses Parsed from Diagramcited by 55
  4. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challengescited by 55
  5. 2023SNIP: Bridging Mathematical Symbolic and Numeric Realms with Unified Pre-trainingcited by 41
  6. 2023Let GPT be a Math Tutor: Teaching Math Word Problem Solvers with Customized Exercise Generationcited by 27
  7. 2024Evaluating LLMs' Mathematical Reasoning in Financial Document Question Answeringcited by 25
  8. 2023Mathematical Language Models: A Surveycited by 24
  9. 2023Learning by Analogy: Diverse Questions Generation in Math Word Problemcited by 18
  10. 2023MinT: Boosting Generalization in Mathematical Reasoning via Multi-View Fine-Tuningcited by 16
  11. 2023Tree-Based Representation and Generation of Natural and Mathematical Languagecited by 12
  12. 2023Learning Multi-Step Reasoning by Solving Arithmetic Taskscited by 12
20 more
  1. 2025A Survey on Mathematical Reasoning and Optimization with Large Language Modelscited by 12
  2. 2024FineMath: A Fine-Grained Mathematical Evaluation Benchmark for Chinese Large Language Modelscited by 10
  3. 2025Leveraging Large Language Models for Bengali Math Word Problem Solving with Chain of Thought Reasoningcited by 8
  4. 2022Generalizing Math Word Problem Solvers via Solution Diversificationcited by 7
  5. 2023Non-Autoregressive Math Word Problem Solver with Unified Tree Structurecited by 7
  6. 2023Solving Math Word Problem with Problem Type Classificationcited by 6
  7. 2022Analogical Math Word Problems Solving with Enhanced Problem-Solution Associationcited by 4
  8. 2023FERMAT: An Alternative to Accuracy for Numerical Reasoningcited by 3
  9. 2024Exploring the Limits of Fine-grained LLM-based Physics Inference via Premise Removal Interventionscited by 3
  10. 2025Empowering Bengali Education with AI: Solving Bengali Math Word Problems through Transformer Modelscited by 3
  11. 2024From Large to Tiny: Distilling and Refining Mathematical Expertise for Math Word Problems with Weakly Supervisioncited by 2
  12. 2026Transparent AI for Mathematics: Transformer-Based Large Language Models for Mathematical Entity Relationship Extraction with XAIcited by 2
  13. 2024FGeo-TP: A Language Model-Enhanced Solver for Geometry Problemscited by 1
  14. 2025Foundation of Intelligence: Review of Math Word Problems from Human Cognition Perspectivecited by 1
  15. 2026Artificial Intelligence for Mathematical Reasoning: An Integrated Survey of Language Models, Neuro-symbolic Systems, and Verified Discoverycited by 1
  16. 2023Comprehensive Solution Program Centric Pretraining for Table-and-Text Hybrid Numerical Reasoningcited by 0
  17. 2024Data Augmentation with In-Context Learning and Comparative Evaluation in Math Word Problem Solvingcited by 0
  18. 2024A Theoretical Review on Solving Algebra Problemscited by 0
  19. 2025A Diversity-Enhanced Knowledge Distillation Model for Practical Math Word Problem Solvingcited by 0
  20. 2025Efficient Mathematical Reasoning Models via Dynamic Pruning and Knowledge Distillationcited by 0