mlsanityData to 7 Oct 2026

Training Verifiers to Solve Math Word Problems

arXiv 2110.14168 ↗First posted 27 Oct 2021cs.LG cs.CLcited by 9,743

Not on the map. Why?

Cited by 3,585 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

State-of-the-art language models can match human performance on many tasks, but they still struggle to robustly perform multi-step mathematical reasoning. To diagnose the failures of current models and support research, we introduce GSM8K, a dataset of 8.5K high quality linguistically diverse grade school math word problems. We find that even the largest transformer models fail to achieve high test performance, despite the conceptual simplicity of this problem distribution. To increase performance, we propose training verifiers to judge the correctness of model completions. At test time, we generate many candidate solutions and select the one ranked highest by the verifier. We demonstrate that verification significantly improves performance on GSM8K, and we provide strong empirical evidence that verification scales more effectively with increased data than a finetuning baseline.

Key facts. Cited by 9,743 papers in the map (data through 7 Oct 2026). Builds on 20 of its 31 references that are in the map. First posted on arXiv 27 Oct 2021.

Builds on

20 of 31 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  6. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  7. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  8. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  9. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  10. 2016Solving General Arithmetic Word Problemscited by 353
  11. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
  12. 2019Deep Learning for Symbolic Mathematicscited by 266
8 more
  1. 2020Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learningcited by 216
  2. 2021Generate & Rank: A Multi-task Framework for Math Word Problemscited by 82
  3. 2020Ape210K: A Large-Scale and Template-Rich Dataset of Math Word Problemscited by 60
  4. 2021MathBERT: A Pre-Trained Model for Mathematical Formula Understandingcited by 58
  5. 2018Semantically-Aligned Equation Generation for Solving and Reasoning Math Word Problemscited by 49
  6. 2021MathBERT: A Pre-trained Language Model for General NLP Tasks in Mathematics Educationcited by 37
  7. 2020Graph-to-Tree Neural Networks for Learning Structured Input-Output Translation with Applications to Semantic Parsing and Math Word Problemcited by 31
  8. 2020Collaborative Storytelling with Large-scale Neural Language Modelscited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 128; 2023: 601; 2024: 1928; 2025: 3408; 2026: 3674.

9,743 papers.

Sort:

By area:

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2025Qwen3 Technical Reportcited by 8,195
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2024Qwen2.5 Technical Reportcited by 4,713
  11. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  12. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
38 more
  1. 2023Let's Verify Step by Stepcited by 3,598
  2. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  3. 2023Qwen Technical Reportcited by 3,273
  4. 2023Mistral 7Bcited by 3,065
  5. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  6. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  7. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  8. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  9. 2023A Survey of Large Language Modelscited by 2,537
  10. 2022Emergent Abilities of Large Language Modelscited by 2,483
  11. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  12. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  13. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  14. 2024Qwen2 Technical Reportcited by 2,076
  15. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  16. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  17. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  18. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  19. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  20. 2024Mixtral of Expertscited by 1,785
  21. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  22. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  23. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  24. 2025Gemma 3 Technical Reportcited by 1,686
  25. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  26. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  27. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  28. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  29. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  30. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  31. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  32. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  33. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  34. 2023PaLM 2 Technical Reportcited by 1,154
  35. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  36. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  37. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  38. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116

50 most cited of 9,743 All 9,743 (JSON)