mlsanityData to 14 Oct 2026

Beyond Benchmarks: MathArena as an Evaluation Platform for Mathematics with LLMs

arXiv 2605.00674 ↗First posted 1 May 2026cs.CLcited by 137

Not on the map. Why?

Cited by 60 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) are becoming increasingly capable mathematical collaborators, but static benchmarks are no longer sufficient for evaluating progress: they are often narrow in scope, quickly saturated, and rarely updated. This makes it hard to compare models reliably and track progress over time. Instead, we need evaluation platforms: continuously maintained systems that run, aggregate, and analyze evaluations across many benchmarks to give a comprehensive picture of model performance within a broad domain. In this work, we build on the original MathArena benchmark by substantially broadening its scope from final-answer olympiad problems to a continuously maintained evaluation platform for mathematical reasoning with LLMs. MathArena now covers a much wider range of tasks, including proof-based competitions, research-level arXiv problems, and formal proof generation in Lean. Additionally, we maintain a clear evaluation protocol for all models and regularly design new benchmarks as model capabilities improve to ensure that MathArena remains challenging. Notably, the strongest model, GPT-5.5, now reaches 98% on the 2026 USA Math Olympiad and 74% on research-level questions, showing that frontier models can now comfortably solve extremely challenging mathematical problems. This highlights the importance of continuously maintained evaluation platforms like MathArena to track the rapid progress of LLMs in mathematical reasoning.

Key facts. Cited by 137 papers in the map (data through 14 Oct 2026). Builds on 57 of its 105 references that are in the map. First posted on arXiv 1 May 2026.

Builds on

57 of 105 references.

References · 57

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  3. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  4. 2024OpenAI o1 System Cardcited by 1,244
  5. 2025OpenAI GPT-5 System Cardcited by 989
  6. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 815
  7. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 587
  8. 2025Humanity's Last Examcited by 491
  9. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 467
  10. 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 326
  11. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 297
  12. 2024Self-Preference Bias in LLM-as-a-Judgecited by 258
38 more
  1. 2024FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AIcited by 243
  2. 2023ProofNet: Autoformalizing and Formally Proving Undergraduate-Level Mathematicscited by 192
  3. 2025Phi-4-reasoning Technical Reportcited by 151
  4. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 145
  5. 2025Measuring what Matters: Construct Validity in Large Language Model Benchmarkscited by 119
  6. 2025DeepSeekMath-V2: Towards Self-Verifiable Mathematical Reasoningcited by 94
  7. 2025Winning Gold at IMO 2025 with a Model-Agnostic Verification-and-Refinement Pipelinecited by 82
  8. 2025Mathematical exploration and discovery at scalecited by 77
  9. 2025Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiadcited by 74
  10. 2026DeepSeek-OCR 2: Visual Causal Flowcited by 74
  11. 2023Evaluating Language Models for Mathematics through Interactionscited by 70
  12. 2025Aristotle: IMO-level Automated Theorem Provingcited by 58
  13. 2026Numina-Lean-Agent: An Open and General Agentic Reasoning System for Formal Mathematicscited by 50
  14. 2025Seed-Prover 1.5: Mastering Undergraduate-Level Theorem Proving via Learning from Experiencecited by 44
  15. 2025A Rosetta Stone for AI Benchmarkscited by 36
  16. 2025Recursive Self-Aggregation Unlocks Deep Thinking in Large Language Modelscited by 35
  17. 2026First Proofcited by 32
  18. 2025Ax-Prover: A Deep Reasoning Agentic Framework for Theorem Proving in Mathematics and Quantum Physicscited by 31
  19. 2025The Open Proof Corpus: A Large-Scale Study of LLM-Generated Mathematical Proofscited by 30
  20. 2025BrokenMath: A Benchmark for Sycophancy in Theorem Proving with LLMscited by 27
  21. 2025FATE: A Formal Benchmark Series for Frontier Algebra of Multiple Difficulty Levelscited by 27
  22. 2025Brains vs. Bytes: Evaluating LLM Proficiency in Olympiad Mathematicscited by 24
  23. 2025Hard2Verify: A Step-Level Verification Benchmark for Open-Ended Frontier Mathcited by 22
  24. 2025Mathematical Proof as a Litmus Test: Revealing Failure Modes of Advanced Large Reasoning Modelscited by 20
  25. 2025LeanExplore: A search engine for Lean 4 declarationscited by 18
  26. 2025Reliable Fine-Grained Evaluation of Natural Language Math Proofscited by 18
  27. 2025FLAWS: A Benchmark for Error Identification and Localization in Scientific Paperscited by 14
  28. 2025IMProofBench: Benchmarking AI on Research-Level Mathematical Proof Generationcited by 13
  29. 2026Self-Preference Bias in Rubric-Based Evaluation of Large Language Modelscited by 11
  30. 2026LemmaBench: A Live, Research-Level Benchmark to Evaluate LLM Capabilities in Mathematicscited by 10
  31. 2026Measuring Progress in Reasoning Toward Mathematical Discovery with Automatic Verificationcited by 9
  32. 2025ReliableMath: Benchmark of Reliable Mathematical Reasoning on Large Language Modelscited by 8
  33. 2026Project Aletheia: Verifier-Guided Distillation of Backtracking for Small Language Modelscited by 8
  34. 2026SorryDB: Can AI Provers Complete Real-World Lean Theorems?cited by 8
  35. 2025Solving a Research Problem in Mathematical Statistics with AI Assistancecited by 6
  36. 2026TaoBench: Do Automated Theorem Prover LLMs Generalize Beyond MathLib?cited by 6
  37. 2024Lattice-Valued Bottleneck Dualitycited by 5
  38. 2026Interactive Benchmarkscited by 5

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 137.

137 papers.

Sort:

By area:

  1. 2026Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 42
  2. 2026When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoningcited by 30
  3. 2026Demystifying On-Policy Distillation: Roles, Pathologies, and Regulationscited by 19
  4. 2026Privileged, but Biased: How PI-Conditioned Teachers Break Self-Distillationcited by 18
  5. 2026OGLS-SD: On-Policy Self-Distillation with Outcome-Guided Logit Steering for LLM Reasoningcited by 16
  6. 2026On-Policy Delta Distillationcited by 15
  7. 2026Anti-Self-Distillation for Reasoning RL via Pointwise Mutual Informationcited by 14
  8. 2026Your Teacher Can't Help You Here: Combating Supervision Fidelity Decay in On-Policy Distillationcited by 13
  9. 2026Cornerstones or Stumbling Blocks? Deciphering the Rock Tokens in On-Policy Distillationcited by 11
  10. 2026Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillationcited by 11
  11. 2026Nanbeige4.2-3B: Unlocking Agentic Capabilities in a Compact Modelcited by 11
  12. 2026Quantized Reasoning Models Think They Need to Think Longer, but They Do Notcited by 10
38 more
  1. 2026Breaking Entropy Bounds: Accelerating RL Training via MTP with Rejection Samplingcited by 9
  2. 2026OmniOPD: Logit-Free On-Policy Distillation via Speculative Verificationcited by 7
  3. 2026ReNIO: Reweighting Negative Trajectory Importance for LLM On-Policy Distillationcited by 7
  4. 2026Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibilitycited by 6
  5. 2026DAMP: Decay-Aware Mixed-Precision Recurrent-State Quantizationcited by 6
  6. 2026Diagnosing and Mitigating Thinking Collapse in On-Policy Self-Distillationcited by 5
  7. 2026DIVE: Unlocking Self-Improvement in Frozen Language Models Through Diversity-Driven Skill Evolutioncited by 5
  8. 2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligencecited by 5
  9. 2026ZAYA1-8B Technical Reportcited by 4
  10. 2026Mix-Quant: Quantized Prefilling, Precise Decoding for Agentic LLMscited by 4
  11. 2026Intern-S2-Preview: Scientific Agentic Foundation Modelcited by 4
  12. 2026You Only Need Minimal RLVR Training: Extrapolating LLMs via Rank-1 Trajectoriescited by 3
  13. 2026A Local Perturbation Theory for Cross-Domain Interference and Recovery in Multi-Domain RLcited by 3
  14. 2026Optimizing the Cost-Quality Tradeoff of Agentic Theorem Provers in Leancited by 3
  15. 2026Breaking the Solver Bottleneck: Training Task Generators at the Learnable Frontiercited by 3
  16. 2026TCPO: Turn-Level Credit Policy Optimizationcited by 3
  17. 2026CITE: Anytime-Valid Statistical Inference in LLM Self-Consistencycited by 2
  18. 2026Teaching Thinking Models to Reason with Tools: A Full-Pipeline Recipe for Tool-Integrated Reasoningcited by 2
  19. 2026Share More, Search Less: Collaborative Parallel Thinking for Efficient Test-Time Scalingcited by 2
  20. 2026ResearchMath-14K: Scaling Research-Level Mathematics via Agentscited by 2
  21. 2026Rubric-Guided Process Reward for Stepwise Model Routingcited by 2
  22. 2026Small RL Controller, Large Language Model: RL-Guided Adaptive Sampling for Test-Time Scalingcited by 2
  23. 2026β-OPSD: Deriving with Policy Optimization, Training with Self-Distillationcited by 2
  24. 2026LongCat Sparse Attention: Taming the Lightning via Streaming-aware Hierarchical Cross-Layer Indexingcited by 2
  25. 2026D3-MOPD: Dynamic Domain ScheDuling for Efficient Multi-Teacher Distillationcited by 2
  26. 2026TTPO: Test-Time Policy Optimizationcited by 2
  27. 2026DASC: Decay-Aware State Compression for Hybrid Linear-Attention Servingcited by 2
  28. 2026Unlocking Lossless Speedups in LLMs via Discrete Diffusioncited by 2
  29. 2026Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillationcited by 2
  30. 2026Negative Self-Distillation: Learning to Reason by Avoiding Flawscited by 2
  31. 20261% of Tokens Can Be Enough: On Gradient Estimation in On-Policy Distillationcited by 2
  32. 2026Prefix-Safe Bayesian Belief Tracking for LLM Reasoning Reliability:Separating Calibration from Rankingcited by 1
  33. 2026Off-Policy Learning to Reason Works Because It Is More Pessimistic Than You Thinkcited by 1
  34. 2026FAPO: Fully Automated Prompt Optimization of Multi-Step LLM Pipelinescited by 1
  35. 2026Serving Agentic Workflows with a Physical-Plan Compiler and Adaptive Runtimecited by 1
  36. 2026Cost of Reasoning in non-English Languages: A Case Study on Japanesecited by 1
  37. 2026Route, Communicate, and Reason: Gated Routing and Adaptive Depth for Efficient Multi-Agent Reasoningcited by 1
  38. 2026Back from the Future: Key-Value Cache Management by Counter-Causal Surprisecited by 1

50 most cited of 137 All 137 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025EEFSUVA: A New Mathematical Olympiad Benchmark
  2. 2025AMO-Bench: Large Language Models Still Struggle in High School Math Competitions
  3. 2025RealMath: A Continuous Benchmark for Evaluating Language Models on Research-Level Mathematics
  4. 2025SKYLENAGE Technical Report: Mathematical Reasoning and Contest-Innovation Benchmarks for Multi-Level Math Evaluation
  5. 2024MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Data
  6. 2024Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models
  7. 2025Evaluation of LLMs for mathematical problem solving
  8. 2025Evaluating the Reasoning Abilities of LLMs on Underrepresented Mathematics Competition Problems