Lean-GAP: A Dataset of Formalized Graduate Algebra Problems
Not on the map. Why?
Abstract (arXiv)
We present Lean-GAP (Lean-Graduate Agebra Problems), 430 formalized graduate-level algebra problems from the textbook Abstract Algebra by Dummit and Foote. We develop a scalable pipeline consisting of PDF-to-LaTeX preprocessing, autoformalization into Lean 4, and verification of informal-formal correspondence. While the preprocessing and autoformalization stages can be largely automated, we find that verification remains the most subtle and labor-intensive component, requiring careful human oversight. Our contributions include (i) the construction of a structured dataset of formalized exercises, (ii) a systematic methodology for formalizing textbook mathematics, and (iii) an analysis of recurring challenges in the formalization process. We also compare the performance of different autoformalization models and highlight key bottlenecks in translating informal statements into formal language.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 29 references that are in the map. First posted on arXiv 20 May 2026.
Builds on
15 of 29 references.
References · 15
- 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
- 2021MiniF2F: a cross-system benchmark for formal Olympiad-level mathematicscited by 391
- 2024FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AIcited by 243
- 2024DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Datacited by 234
- 2024PutnamBench: Evaluating Neural Theorem-Provers on the Putnam Mathematical Competitioncited by 165
- 2024Lean Workbook: A large-scale Lean problem set formalized from natural language math problemscited by 122
- 2024Herald: A Natural Language Annotated Lean 4 Datasetcited by 59
- 2025FormalMATH: Benchmarking Formal Mathematical Reasoning of Large Language Modelscited by 48
- 2024Autoformalize Mathematical Statements by Symbolic Equivalence and Semantic Consistencycited by 41
- 2024Reliable Evaluation and Benchmarks for Statement Autoformalizationcited by 32
- 2024Autoformalizing Euclidean Geometrycited by 27
- 2025FATE: A Formal Benchmark Series for Frontier Algebra of Multiple Difficulty Levelscited by 27
Built on by
Not cited yet.