mlsanityData to 7 Oct 2026

Measuring Mathematical Problem Solving With the MATH Dataset

arXiv 2103.03874 ↗First posted 5 Mar 2021cs.LG cs.AI cs.CLcited by 5,203

Not on the map. Why?

Cited by 2,282 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Many intellectual endeavors require mathematical problem solving, but this skill remains beyond the capabilities of computers. To measure this ability in machine learning models, we introduce MATH, a new dataset of 12,500 challenging competition mathematics problems. Each problem in MATH has a full step-by-step solution which can be used to teach models to generate answer derivations and explanations. To facilitate future research and increase accuracy on MATH, we also contribute a large auxiliary pretraining dataset which helps teach models the fundamentals of mathematics. Even though we are able to increase accuracy on MATH, our results show that accuracy remains relatively low, even with enormous Transformer models. Moreover, we find that simply increasing budgets and model parameter counts will be impractical for achieving strong mathematical reasoning if scaling trends continue. While scaling Transformers is automatically solving most other text-based tasks, scaling is not currently solving MATH. To have more traction on mathematical problem solving we will likely need new algorithmic advancements from the broader research community.

Key facts. Cited by 5,203 papers in the map (data through 7 Oct 2026). Builds on 28 of its 71 references that are in the map. First posted on arXiv 5 Mar 2021.

Builds on

28 of 71 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  8. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  9. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  10. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  11. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  12. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
16 more
  1. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  2. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  3. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  4. 2020Scaling Laws for Autoregressive Generative Modelingcited by 510
  5. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
  6. 2020Generative Language Modeling for Automated Theorem Provingcited by 379
  7. 2019Analysing Mathematical Reasoning Abilities of Neural Modelscited by 331
  8. 2019Deep Learning for Symbolic Mathematicscited by 266
  9. 2021Scaling Laws for Transfercited by 265
  10. 2020Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learningcited by 216
  11. 2018GamePad: A Learning Environment for Theorem Provingcited by 48
  12. 2021LIME: Learning Inductive Bias for Primitives of Mathematical Reasoningcited by 46
  13. 2019Improving Graph Neural Network Representations of Logical Formulae with Subgraph Poolingcited by 20
  14. 2019HOList: An Environment for Machine Learning of Higher-Order Theorem Provingcited by 19
  15. 2020IsarStep: a Benchmark for High-level Mathematical Reasoningcited by 7
  16. 2020MathZero, The Classification Problem, and Set-Theoretic Type Theorycited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 46; 2023: 221; 2024: 768; 2025: 2066; 2026: 2086.

5,203 papers.

Sort:

By area:

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2023Let's Verify Step by Stepcited by 3,598
  7. 2023Qwen Technical Reportcited by 3,273
  8. 2023Mistral 7Bcited by 3,065
  9. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  10. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  11. 2024Qwen2 Technical Reportcited by 2,076
  12. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
38 more
  1. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  2. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  3. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  4. 2024Mixtral of Expertscited by 1,785
  5. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  6. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  7. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  8. 2025Gemma 3 Technical Reportcited by 1,686
  9. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  10. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  11. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  12. 2025s1: Simple test-time scalingcited by 1,311
  13. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  14. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  15. 2023PaLM 2 Technical Reportcited by 1,154
  16. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  17. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  18. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  19. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  20. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  21. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  22. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  23. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  24. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
  25. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  26. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  27. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  28. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
  29. 2021Measuring Coding Challenge Competence With APPScited by 879
  30. 2025Large Language Diffusion Modelscited by 872
  31. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  32. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  33. 2022Galactica: A Large Language Model for Sciencecited by 736
  34. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  35. 2024DeepSeek-V3 Technical Reportcited by 724
  36. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  37. 2024Yi: Open Foundation Models by 01.AIcited by 706
  38. 2025Qwen2.5-Omni Technical Reportcited by 704

50 most cited of 5,203 All 5,203 (JSON)