mlsanityData to 7 Oct 2026

MA-ProofBench: A Two-Tiered Evaluation of LLMs for Theorem Proving in Mathematical Analysis

arXiv 2606.13782 ↗First posted 11 Jun 2026cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have made notable progress in automated theorem proving, yet existing formal benchmarks remain limited in both mathematical coverage and difficulty. Most are concentrated in areas that are easier to formalize, such as algebra and elementary number theory, and provide limited coverage of subfields that require deeper reasoning, including mathematical analysis. To address this gap, we introduce MA-ProofBench, to the best of our knowledge, the first formal theorem-proving benchmark dedicated to Mathematical Analysis. The benchmark contains 200 formalized theorems covering 6 core topics and 27 subcategories, including measure and integration theory, complex analysis, and functional analysis. The problems are divided into two difficulty levels, an undergraduate level (Level I, 100 problems) and a Ph.D. qualifying level (Level II, 100 problems), to evaluate how well LLMs perform formal reasoning at different mathematical depths. Each problem is constructed through a human-led, LLM-assisted formalization pipeline followed by independent expert review, ensuring that the formal statements remain faithful to the original mathematics. We evaluate a range of recent general-purpose reasoning models and formal theorem provers on MA-ProofBench. However, most models perform poorly: even the best-performing model, GPT-5.5, achieves only 16% Pass@8 on Level I and 5% on Level II, while most models stay close to 0% on Level II. Further analysis identifies Mathlib hallucinations and incomplete proofs as the two dominant failure modes, while an evaluation on the natural-language version of the benchmark exposes a clear gap between informal and formal reasoning. MA-ProofBench is intended to serve as a reliable reference for tracking progress in formal mathematical reasoning in advanced domains.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 31 of its 42 references that are in the map. First posted on arXiv 11 Jun 2026.

Builds on

31 of 42 references.

References · 31

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  6. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 502
  7. 2021MiniF2F: a cross-system benchmark for formal Olympiad-level mathematicscited by 380
  8. 2020Generative Language Modeling for Automated Theorem Provingcited by 379
  9. 2025DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decompositioncited by 270
  10. 2025Kimina-Prover Preview: Towards Large Formal Reasoning Models with Reinforcement Learningcited by 173
  11. 2024PutnamBench: Evaluating Neural Theorem-Provers on the Putnam Mathematical Competitioncited by 153
  12. 2025Goedel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data Synthesis and Self-Correctioncited by 140
19 more
  1. 2019The Lean mathematical librarycited by 127
  2. 2025Seed-Prover: Deep and Broad Reasoning for Automated Theorem Provingcited by 96
  3. 2025Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 85
  4. 2025BFS-Prover: Scalable Best-First Tree Search for LLM-based Automatic Theorem Provingcited by 82
  5. 2025Towards Robust Mathematical Reasoningcited by 67
  6. 2025Aristotle: IMO-level Automated Theorem Provingcited by 47
  7. 2025FormalMATH: Benchmarking Formal Mathematical Reasoning of Large Language Modelscited by 45
  8. 2026Numina-Lean-Agent: An Open and General Agentic Reasoning System for Formal Mathematicscited by 43
  9. 2025AMO-Bench: Large Language Models Still Struggle in High School Math Competitionscited by 36
  10. 2025Seed-Prover 1.5: Mastering Undergraduate-Level Theorem Proving via Learning from Experiencecited by 36
  11. 2024InternLM2.5-StepProver: Advancing Automated Theorem Proving via Critic-Guided Searchcited by 34
  12. 2024HunyuanProver: A Scalable Data Synthesis Framework and Guided Tree Search for Automated Theorem Provingcited by 32
  13. 2025CombiBench: Benchmarking LLM Capability for Combinatorial Mathematicscited by 28
  14. 2025Ax-Prover: A Deep Reasoning Agentic Framework for Theorem Proving in Mathematics and Quantum Physicscited by 27
  15. 2025FATE: A Formal Benchmark Series for Frontier Algebra of Multiple Difficulty Levelscited by 23
  16. 2025Kimina Lean Server: A High-Performance Lean Server for Large-Scale Verificationcited by 16
  17. 2025Reliable Fine-Grained Evaluation of Natural Language Math Proofscited by 14
  18. 2025miniF2F-Lean Revisited: Reviewing Limitations and Charting a Path Forwardcited by 13
  19. 2025LeanCat: A Benchmark Suite for Formal Category Theory in Lean (Part I: 1-Categories)cited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026TheoremBench: Evaluating LLMs on Theorem Proving in Formal Mathematics
  2. 2025MSC-180: A Benchmark for Automated Formal Theorem Proving from Mathematical Subject Classification
  3. 2026AdvancedMathBench: A Benchmark Suite for Advanced Mathematical Proof Generation and Verification
  4. 2026MathAdv: What Theorem Provers Know, Reason, Formalize, and Generalize
  5. 2025ProofCompass: Enhancing Specialized Provers with LLM Guidance
  6. 2023math-PVS: A Large Language Model Framework to Map Scientific Publications to PVS Theories
  7. 2025Hilbert: Recursively Building Formal Proofs with Informal Reasoning
  8. 2025HERMES: Towards Efficient and Verifiable Mathematical Reasoning in LLMs