mlsanityData to 14 Oct 2026

Ineq-Comp: Benchmarking Human-Intuitive Compositional Reasoning in Automated Theorem Proving on Inequalities

arXiv 2505.12680 ↗First posted 19 May 2025cs.AI cs.CL cs.LGcited by 8

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

LLM-based formal proof assistants (e.g., in Lean) hold great promise for automating mathematical discovery. But beyond syntactic correctness, do these systems truly understand mathematical structure as humans do? We investigate this question in context of mathematical inequalities -- specifically the prover's ability to recognize that the given problem simplifies by applying a known inequality such as AM/GM. Specifically, we are interested in their ability to do this in a compositional setting where multiple inequalities must be applied as part of a solution. We introduce Ineq-Comp, a benchmark built from elementary inequalities through systematic transformations, including variable duplication, algebraic rewriting, and multi-step composition. Although these problems remain easy for humans, we find that most provers -- including Goedel, STP, and Kimina-7B -- struggle significantly. DeepSeek-Prover-V2-7B shows relative robustness, but still suffers a 20% performance drop (pass@32). Even for DeepSeek-Prover-V2-671B model, the gap between compositional variants and seed problems exists, implying that simply scaling up the model size alone does not fully solve the compositional weakness. Strikingly, performance remains poor for all models even when formal proofs of the constituent parts are provided in context, revealing that the source of weakness is indeed in compositional reasoning. Our results expose a persisting gap between the generalization behavior of current AI provers and human mathematical intuition. All data and evaluation code can be found at https://github.com/haoyuzhao123/LeanIneqComp.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 35 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

17 of 35 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025CombiBench: Benchmarking LLM Capability for Combinatorial Mathematics6 May
  2. 2025FormalMATH: Benchmarking Formal Mathematical Reasoning of Large Language Models5 May
  3. 2025Kimina-Prover Preview: Towards Large Formal Reasoning Models with Reinforcement Learning15 Apr
  4. 2025Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad27 Mar
  5. 2025STP: Self-play LLM Theorem Provers with Iterative Conjecturing and Proving31 Jan
  6. 2024HunyuanProver: A Scalable Data Synthesis Framework and Guided Tree Search for Automated Theorem Proving30 Dec
  7. 2020Generative Language Modeling for Automated Theorem Proving7 Sep

Other references · 10

  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  2. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  3. 2021MiniF2F: a cross-system benchmark for formal Olympiad-level mathematicscited by 391
  4. 2024DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Searchcited by 197
  5. 2023ProofNet: Autoformalizing and Formally Proving Undergraduate-Level Mathematicscited by 192
  6. 2024PutnamBench: Evaluating Neural Theorem-Provers on the Putnam Mathematical Competitioncited by 165
  7. 2025Goedel-Prover: A Frontier Model for Open-Source Automated Theorem Provingcited by 133
  8. 2025BFS-Prover: Scalable Best-First Tree Search for LLM-based Automatic Theorem Provingcited by 84
  9. 2024InternLM2.5-StepProver: Advancing Automated Theorem Proving via Critic-Guided Searchcited by 34
  10. 2025Task Generalization With AutoRegressive Compositional Structure: Can Learning From D Tasks Generalize to DT Tasks?cited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 4.

8 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026IneqAgent: Agentic Inequality Proving in Lean with Symbolic Computation
  2. 2026From LLM-Generated Conjectures to Lean Formalizations: Automated Polynomial Inequality Proving via Sum-of-Squares Certificates
  3. 2025IndiMathBench: Autoformalizing Mathematical Reasoning Problems with a Human Touch
  4. 2026AdvancedMathBench: A Benchmark Suite for Advanced Mathematical Proof Generation and Verification
  5. 2026MA-ProofBench: A Two-Tiered Evaluation of LLMs for Theorem Proving in Mathematical Analysis
  6. 2026Construction-Verification: A Benchmark for Applied Mathematics in Lean 4
  7. 2026MathAdv: What Theorem Provers Know, Reason, Formalize, and Generalize
  8. 2024MathGAP: Out-of-Distribution Evaluation on Problems with Arbitrarily Complex Proofs