Ineq-Comp: Benchmarking Human-Intuitive Compositional Reasoning in Automated Theorem Proving on Inequalities
On the map: LLM reasoning → Mathematical reasoning
Abstract (arXiv)
LLM-based formal proof assistants (e.g., in Lean) hold great promise for automating mathematical discovery. But beyond syntactic correctness, do these systems truly understand mathematical structure as humans do? We investigate this question in context of mathematical inequalities -- specifically the prover's ability to recognize that the given problem simplifies by applying a known inequality such as AM/GM. Specifically, we are interested in their ability to do this in a compositional setting where multiple inequalities must be applied as part of a solution. We introduce Ineq-Comp, a benchmark built from elementary inequalities through systematic transformations, including variable duplication, algebraic rewriting, and multi-step composition. Although these problems remain easy for humans, we find that most provers -- including Goedel, STP, and Kimina-7B -- struggle significantly. DeepSeek-Prover-V2-7B shows relative robustness, but still suffers a 20% performance drop (pass@32). Even for DeepSeek-Prover-V2-671B model, the gap between compositional variants and seed problems exists, implying that simply scaling up the model size alone does not fully solve the compositional weakness. Strikingly, performance remains poor for all models even when formal proofs of the constituent parts are provided in context, revealing that the source of weakness is indeed in compositional reasoning. Our results expose a persisting gap between the generalization behavior of current AI provers and human mathematical intuition. All data and evaluation code can be found at https://github.com/haoyuzhao123/LeanIneqComp.
Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 35 references that are in the map. First posted on arXiv 19 May 2025.
Builds on
17 of 35 references.
Closest ancestors · 7
Same direction, newest first
- 2025CombiBench: Benchmarking LLM Capability for Combinatorial Mathematics6 May
- 2025FormalMATH: Benchmarking Formal Mathematical Reasoning of Large Language Models5 May
- 2025Kimina-Prover Preview: Towards Large Formal Reasoning Models with Reinforcement Learning15 Apr
- 2025Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad27 Mar
- 2025STP: Self-play LLM Theorem Provers with Iterative Conjecturing and Proving31 Jan
- 2024HunyuanProver: A Scalable Data Synthesis Framework and Guided Tree Search for Automated Theorem Proving30 Dec
- 2020Generative Language Modeling for Automated Theorem Proving7 Sep
Other references · 10
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2024Qwen2.5-Coder Technical Reportcited by 1,203
- 2021MiniF2F: a cross-system benchmark for formal Olympiad-level mathematicscited by 391
- 2024DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Searchcited by 197
- 2023ProofNet: Autoformalizing and Formally Proving Undergraduate-Level Mathematicscited by 192
- 2024PutnamBench: Evaluating Neural Theorem-Provers on the Putnam Mathematical Competitioncited by 165
- 2025Goedel-Prover: A Frontier Model for Open-Source Automated Theorem Provingcited by 133
- 2025BFS-Prover: Scalable Best-First Tree Search for LLM-based Automatic Theorem Provingcited by 84
- 2024InternLM2.5-StepProver: Advancing Automated Theorem Proving via Critic-Guided Searchcited by 34
- 2025Task Generalization With AutoRegressive Compositional Structure: Can Learning From D Tasks Generalize to DT Tasks?cited by 11
Built on by
Citing papers per year: 2025: 4; 2026: 4.
8 papers.
- 2025RL Grokking Recipe: How Does RL Unlock and Transfer New Algorithms in LLMs?cited by 23
- 2025Solving Inequality Proofs with Large Language Modelscited by 21
- 2025EvolProver: Advancing Automated Theorem Proving by Evolving Formalized Problems via Symmetry and Difficultycited by 7
- 2026FormalProofBench: Can Models Write Graduate Level Math Proofs That Are Formally Verified?cited by 5
- 2025AgentCoMa: A Compositional Benchmark Mixing Commonsense and Mathematical Reasoning in Real-World Scenarioscited by 3
- 2026ConvexBench: Can LLMs Recognize Convex Functions?cited by 3
- 2026FormalEvolve: Neuro-Symbolic Evolutionary Search for Diverse Autoformalizationcited by 1
- 2026What are the Right Symmetries for Formal Theorem Proving?cited by 1
- 2026What are the Right Symmetries for Formal Theorem Proving?21 May 2026
- 2026FormalProofBench: Can Models Write Graduate Level Math Proofs That Are Formally Verified?27 Mar 2026
- 2026FormalEvolve: Neuro-Symbolic Evolutionary Search for Diverse Autoformalization20 Mar 2026
- 2026ConvexBench: Can LLMs Recognize Convex Functions?1 Feb 2026
- 2025EvolProver: Advancing Automated Theorem Proving by Evolving Formalized Problems via Symmetry and Difficulty1 Oct 2025
- 2025RL Grokking Recipe: How Does RL Unlock and Transfer New Algorithms in LLMs?25 Sep 2025
- 2025AgentCoMa: A Compositional Benchmark Mixing Commonsense and Mathematical Reasoning in Real-World Scenarios27 Aug 2025
- 2025Solving Inequality Proofs with Large Language Models9 Jun 2025
Similar, not cited
Close in wording, no citation either way
- 2026IneqAgent: Agentic Inequality Proving in Lean with Symbolic Computation
- 2026From LLM-Generated Conjectures to Lean Formalizations: Automated Polynomial Inequality Proving via Sum-of-Squares Certificates
- 2025IndiMathBench: Autoformalizing Mathematical Reasoning Problems with a Human Touch
- 2026AdvancedMathBench: A Benchmark Suite for Advanced Mathematical Proof Generation and Verification
- 2026MA-ProofBench: A Two-Tiered Evaluation of LLMs for Theorem Proving in Mathematical Analysis
- 2026Construction-Verification: A Benchmark for Applied Mathematics in Lean 4
- 2026MathAdv: What Theorem Provers Know, Reason, Formalize, and Generalize
- 2024MathGAP: Out-of-Distribution Evaluation on Problems with Arbitrarily Complex Proofs