EternalMath: A Living Benchmark of Frontier Mathematics that Evolves with Human Discovery
On the map: LLM reasoning → Mathematical reasoning
Abstract (arXiv)
Current evaluations of mathematical reasoning in large language models (LLMs) are dominated by static benchmarks, either derived from competition-style problems or curated through costly expert effort, resulting in limited coverage of research-level mathematics and rapid performance saturation. We propose a fully automated, theorem-grounded pipeline for evaluating frontier mathematical reasoning, which directly transforms recent peer-reviewed mathematical literature into executable and verifiable reasoning tasks. The pipeline identifies constructive or quantitative results, instantiates them into parameterized problem templates, and generates deterministic solutions through execution-based verification, enabling scalable, reproducible, and continuously updatable evaluation without reliance on large-scale expert authoring. By design, this approach supports temporal extensibility, intrinsic correctness checking, and domain-specific customization across mathematical subfields. Applying this pipeline yields EternalMath, an evolving evaluation suite derived from contemporary research papers. Experiments with state-of-the-art LLMs reveal substantial performance gaps, indicating that mathematical reasoning at the research frontier remains far from saturated and underscoring the need for evaluation methodologies that evolve in step with human mathematical discovery.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 12 of its 23 references that are in the map. First posted on arXiv 4 Jan 2026.
Builds on
12 of 23 references.
Closest ancestors · 4
Same direction, newest first
- 2025MathSmith: Towards Extremely Hard Mathematical Reasoning by Forging Synthetic Problems with a Reinforced Policy7 Aug
- 2025VAR-MATH: Probing True Mathematical Reasoning in LLMS via Symbolic Multi-Instance Benchmarks17 Jul
- 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitions29 May
- 2024MathScale: Scaling Instruction Tuning for Mathematical Reasoning5 Mar
Other references · 8
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
- 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
- 2021MiniF2F: a cross-system benchmark for formal Olympiad-level mathematicscited by 380
- 2023Evaluating the Performance of Large Language Models on GAOKAO Benchmarkcited by 182
- 2023ProofNet: Autoformalizing and Formally Proving Undergraduate-Level Mathematicscited by 180
- 2024PutnamBench: Evaluating Neural Theorem-Provers on the Putnam Mathematical Competitioncited by 153
- 2024Lean Workbook: A large-scale Lean problem set formalized from natural language math problemscited by 115
- 2024Are Your LLMs Capable of Stable Reasoning?cited by 65
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026LemmaBench: A Live, Research-Level Benchmark to Evaluate LLM Capabilities in Mathematicscited by 7
- 2026Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMscited by 4
- 2026Evaluating SageMath-Augmented LLM Agents for Computational and Experimental Mathematicscited by 0
- 2026Evaluating SageMath-Augmented LLM Agents for Computational and Experimental Mathematics7 Jul 2026
- 2026Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs9 May 2026
- 2026LemmaBench: A Live, Research-Level Benchmark to Evaluate LLM Capabilities in Mathematics27 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2025RealMath: A Continuous Benchmark for Evaluating Language Models on Research-Level Mathematics
- 2026Measuring Progress in Reasoning Toward Mathematical Discovery with Automatic Verification
- 2025Evaluating Frontier LLMs on PhD-Level Mathematical Reasoning: A Benchmark on a Textbook in Theoretical Computer Science about Randomized Algorithms
- 2026LiveMathematicianBench: A Live Benchmark for Research-Level Mathematical Reasoning with Proof Sketches
- 2026AdvancedMathBench: A Benchmark Suite for Advanced Mathematical Proof Generation and Verification
- 2024FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI
- 2025Putnam-AXIOM: A Functional and Static Benchmark for Measuring Higher Level Mathematical Reasoning in LLMs
- 2025FormalMATH: Benchmarking Formal Mathematical Reasoning of Large Language Models