mlsanityData to 14 Oct 2026

Lean-GAP: A Dataset of Formalized Graduate Algebra Problems

arXiv 2606.02588 ↗First posted 20 May 2026cs.LO cs.AI cs.PLcited by 0

Not on the map. Why?

Abstract (arXiv)

We present Lean-GAP (Lean-Graduate Agebra Problems), 430 formalized graduate-level algebra problems from the textbook Abstract Algebra by Dummit and Foote. We develop a scalable pipeline consisting of PDF-to-LaTeX preprocessing, autoformalization into Lean 4, and verification of informal-formal correspondence. While the preprocessing and autoformalization stages can be largely automated, we find that verification remains the most subtle and labor-intensive component, requiring careful human oversight. Our contributions include (i) the construction of a structured dataset of formalized exercises, (ii) a systematic methodology for formalizing textbook mathematics, and (iii) an analysis of recurring challenges in the formalization process. We also compare the performance of different autoformalization models and highlight key bottlenecks in translating informal statements into formal language.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 29 references that are in the map. First posted on arXiv 20 May 2026.

Builds on

15 of 29 references.

References · 15

  1. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  2. 2021MiniF2F: a cross-system benchmark for formal Olympiad-level mathematicscited by 391
  3. 2024FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AIcited by 243
  4. 2024DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Datacited by 234
  5. 2024PutnamBench: Evaluating Neural Theorem-Provers on the Putnam Mathematical Competitioncited by 165
  6. 2024Lean Workbook: A large-scale Lean problem set formalized from natural language math problemscited by 122
  7. 2024Herald: A Natural Language Annotated Lean 4 Datasetcited by 59
  8. 2025FormalMATH: Benchmarking Formal Mathematical Reasoning of Large Language Modelscited by 48
  9. 2024Autoformalize Mathematical Statements by Symbolic Equivalence and Semantic Consistencycited by 41
  10. 2024Reliable Evaluation and Benchmarks for Statement Autoformalizationcited by 32
  11. 2024Autoformalizing Euclidean Geometrycited by 27
  12. 2025FATE: A Formal Benchmark Series for Frontier Algebra of Multiple Difficulty Levelscited by 27
3 more
  1. 2026M2F: Automated Formalization of Mathematical Literature at Scalecited by 15
  2. 2025ASSESS: A Semantic and Structural Evaluation Framework for Statement Similaritycited by 10
  3. 2025Beyond Theorem Proving: Formulation, Framework and Benchmark for Formal Problem-Solvingcited by 2

Built on by

Not cited yet.