mlsanityData to 7 Oct 2026

MathBench: Evaluating the Theory and Application Proficiency of LLMs with a Hierarchical Mathematics Benchmark

arXiv 2405.12209 ↗First posted 20 May 2024cs.CLcited by 103

Not on the map. Why?

Cited by 45 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent advancements in large language models (LLMs) have showcased significant improvements in mathematics. However, traditional math benchmarks like GSM8k offer a unidimensional perspective, falling short in providing a holistic assessment of the LLMs' math capabilities. To address this gap, we introduce MathBench, a new benchmark that rigorously assesses the mathematical capabilities of large language models. MathBench spans a wide range of mathematical disciplines, offering a detailed evaluation of both theoretical understanding and practical problem-solving skills. The benchmark progresses through five distinct stages, from basic arithmetic to college mathematics, and is structured to evaluate models at various depths of knowledge. Each stage includes theoretical questions and application problems, allowing us to measure a model's mathematical proficiency and its ability to apply concepts in practical scenarios. MathBench aims to enhance the evaluation of LLMs' mathematical abilities, providing a nuanced view of their knowledge understanding levels and problem solving skills in a bilingual context. The project is released at https://github.com/open-compass/MathBench .

Key facts. Cited by 103 papers in the map (data through 7 Oct 2026). Builds on 20 of its 36 references that are in the map. First posted on arXiv 20 May 2024.

Builds on

20 of 36 references.

References · 20

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2020Scaling Laws for Neural Language Modelscited by 6,411
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  9. 2023Qwen Technical Reportcited by 3,273
  10. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  11. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
  12. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
8 more
  1. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  2. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  3. 2016Solving General Arithmetic Word Problemscited by 353
  4. 2019Analysing Mathematical Reasoning Abilities of Neural Modelscited by 331
  5. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
  6. 2023How well do Large Language Models perform in Arithmetic tasks?cited by 130
  7. 2022NumGLUE: A Suite of Fundamental yet Challenging Mathematical Reasoning Taskscited by 110
  8. 2016Annotating Derivations: A New Evaluation Strategy and Dataset for Algebra Word Problemscited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 24; 2025: 60; 2026: 19.

103 papers.

Sort:

By area:

  1. 2024Token-Budget-Aware LLM Reasoningcited by 242
  2. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 118
  3. 2024Are Your LLMs Capable of Stable Reasoning?cited by 65
  4. 2025Efficient Inference for Large Reasoning Models: A Surveycited by 58
  5. 2025MixLLM: Dynamic Routing in Mixed Large Language Modelscited by 56
  6. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challengescited by 55
  7. 2024Is Your Model Really A Good Math Reasoner? Evaluating Mathematical Reasoning with Checklistcited by 48
  8. 2024LLMs for Mathematical Modeling: Towards Bridging the Gap between Natural and Mathematical Languagescited by 46
  9. 2025UGPhysics: A Comprehensive Benchmark for Undergraduate Physics Reasoning with Large Language Modelscited by 45
  10. 2025Towards Greater Leverage: Scaling Laws for Efficient Mixture-of-Experts Language Modelscited by 44
  11. 2024ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detectioncited by 37
  12. 2025The Avengers: A Simple Recipe for Uniting Smaller Language Models to Challenge Proprietary Giantscited by 37
38 more
  1. 2026LLMRouterBench: A Massive Benchmark and Unified Framework for LLM Routingcited by 35
  2. 2024RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewardscited by 33
  3. 2024HARDMath: A Benchmark Dataset for Challenging Problems in Applied Mathematicscited by 32
  4. 2025MV-MATH: Evaluating Multimodal Math Reasoning in Multi-Visual Contextscited by 32
  5. 2025PHYSICS: Benchmarking Foundation Models on University-Level Physics Problem Solvingcited by 31
  6. 2025SCP-116K: A High-Quality Problem-Solution Dataset and a Generalized Pipeline for Automated Extraction in the Higher Education Science Domaincited by 30
  7. 2025Benchmarking Reasoning Robustness in Large Language Modelscited by 30
  8. 2025WSM: Decay-Free Learning Rate Schedule via Checkpoint Merging for LLM Pre-trainingcited by 21
  9. 2025Do We Truly Need So Many Samples? Multi-LLM Repeated Sampling Efficiently Scales Test-Time Computecited by 20
  10. 2025BoostStep: Boosting mathematical capability of Large Language Models via improved single-step reasoningcited by 19
  11. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  12. 2025SOTOPIA-Omega: Dynamic Strategy Injection Learning and Social Instruction Following Evaluation for Social Agentscited by 16
  13. 2025SciHorizon: Benchmarking AI-for-Science Readiness from Scientific Data to Large Language Modelscited by 15
  14. 2025Exposing Numeracy Gaps: A Benchmark to Evaluate Fundamental Numerical Abilities in Large Language Modelscited by 14
  15. 2025Potemkin Understanding in Large Language Modelscited by 14
  16. 2024Exposing the Achilles' Heel: Evaluating LLMs Ability to Handle Mistakes in Mathematical Reasoningcited by 13
  17. 2024Do LLMs Understand Ambiguity in Text? A Case Study in Open-world Question Answeringcited by 13
  18. 2025UGMathBench: A Diverse and Dynamic Benchmark for Undergraduate-Level Mathematical Reasoning with Large Language Modelscited by 13
  19. 2025DeepSeek vs. ChatGPT vs. Claude: A Comparative Study for Scientific Computing and Scientific Machine Learning Taskscited by 12
  20. 2024Flow-DPO: Improving LLM Mathematical Reasoning through Online Multi-Agent Learningcited by 11
  21. 2025CipherBank: Exploring the Boundary of LLM Reasoning Capabilities through Cryptography Challengescited by 11
  22. 2025ReCode: Updating Code API Knowledge with Reinforcement Learningcited by 11
  23. 2025MathFlow: Enhancing the Perceptual Flow of MLLMs for Visual Mathematical Problemscited by 10
  24. 2025Benchmarking Multimodal Mathematical Reasoning with Explicit Visual Dependencycited by 10
  25. 2025What Makes a Good Curriculum? Disentangling the Effects of Data Ordering on LLM Mathematical Reasoningcited by 10
  26. 2024Can Large Language Models Always Solve Easy Problems if They Can Solve Harder Ones?cited by 9
  27. 2024MoDEM: Mixture of Domain Expert Modelscited by 9
  28. 2025GuidedBench: Measuring and Mitigating the Evaluation Discrepancies of In-the-wild LLM Jailbreak Methodscited by 9
  29. 2025Process or Result? Manipulated Ending Tokens Can Mislead Reasoning LLMs to Ignore the Correct Reasoning Stepscited by 9
  30. 2024Understanding Depth and Height Perception in Large Visual-Language Modelscited by 8
  31. 2026Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrowcited by 8
  32. 2026Riemann-Bench: A Benchmark for Moonshot Mathematicscited by 8
  33. 2025Evaluation of LLMs for mathematical problem solvingcited by 7
  34. 2025ORGEval: Graph-Theoretic Evaluation of LLMs in Optimization Modelingcited by 7
  35. 2026MergeMix: Optimizing Mid-Training Data Mixtures via Learnable Model Mergingcited by 7
  36. 2024SIaM: Self-Improving Code-Assisted Mathematical Reasoning of Large Language Modelscited by 6
  37. 2024Report Cards: Qualitative Evaluation of Language Models Using Natural Language Summariescited by 6
  38. 2024Give me a hint: Can LLMs take a hint to solve math problems?cited by 6

50 most cited of 103 All 103 (JSON)