mlsanityData to 7 Oct 2026

Clarifying Before Reasoning: A Coq Prover with Structural Context

arXiv 2507.02541 ↗First posted 3 Jul 2025cs.AIcited by 1

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

In this work, we investigate whether improving task clarity can enhance reasoning ability of large language models, focusing on theorem proving in Coq. We introduce a concept-level metric to evaluate task clarity and show that adding structured semantic context to the standard input used by modern LLMs, leads to a 1.85× improvement in clarity score (44.5%~rightarrow~82.3%). Using the general-purpose model textttDeepSeek-V3, our approach leads to a 2.1× improvement in proof success (21.8%~rightarrow~45.8%) and outperforms the previous state-of-the-art textttGraph2Tac (33.2%). We evaluate this on 1,386 theorems randomly sampled from 15 standard Coq packages, following the same evaluation protocol as textttGraph2Tac. Furthermore, fine-tuning smaller models on our structured data can achieve even higher performance (48.6%). Our method uses selective concept unfolding to enrich task descriptions, and employs a Planner--Executor architecture. These findings highlight the value of structured task representations in bridging the gap between understanding and reasoning.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 19 of its 39 references that are in the map. First posted on arXiv 3 Jul 2025.

Builds on

19 of 39 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024DeepSeek-V3 Technical Report27 Dec

Other references · 18

  1. 2023Llemma: An Open Language Model For Mathematicscited by 385
  2. 2024DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Searchcited by 189
  3. 2025Kimina-Prover Preview: Towards Large Formal Reasoning Models with Reinforcement Learningcited by 173
  4. 2022Formal Mathematics Statement Curriculum Learningcited by 141
  5. 2021Proof Artifact Co-training for Theorem Proving with Language Modelscited by 130
  6. 2025Goedel-Prover: A Frontier Model for Open-Source Automated Theorem Provingcited by 126
  7. 2023LEGO-Prover: Neural Theorem Proving with Growing Librariescited by 108
  8. 2024Lean-STaR: Learning to Interleave Thinking and Provingcited by 64
  9. 2024TheoremLlama: Transforming General-Purpose LLMs into Lean4 Expertscited by 58
  10. 2023Magnushammer: A Transformer-Based Approach to Premise Selectioncited by 50
  11. 2024InternLM2.5-StepProver: Advancing Automated Theorem Proving via Critic-Guided Searchcited by 34
  12. 2025APOLLO: Automated LLM and Lean Collaboration for Advanced Formal Reasoningcited by 34
6 more
  1. 2023LLMSTEP: LLM proofstep suggestions in Leancited by 32
  2. 2025MA-LoT: Model-Collaboration Lean-based Long Chain-of-Thought Reasoning enhances Formal Theorem Provingcited by 24
  3. 2024Cobblestone: A Divide-and-Conquer Approach for Automating Formal Verificationcited by 8
  4. 2023Succinct Representations for Conceptscited by 2
  5. 2025Automated Discovery of Tactic Libraries for Interactive Theorem Provingcited by 2
  6. 2024Enhancing Formal Theorem Proving: A Comprehensive Dataset for Training AI Models on Coq Codecited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2024Exploring the Role of Reasoning Structures for Constructing Proofs in Multi-Step Natural Language Reasoning with Large Language Models
  2. 2026LogicSkills: A Structured Benchmark for Formal Reasoning in Large Language Models
  3. 2025Can Large Language Models Learn Formal Logic? A Data-Driven Training and Evaluation Framework
  4. 2023An In-Context Learning Agent for Formal Theorem-Proving
  5. 2024Rango: Adaptive Retrieval-Augmented Proving for Automated Software Verification
  6. 2025What Defines Good Reasoning in LLMs? Dissecting Reasoning Steps with Multi-Aspect Evaluation
  7. 2025ProofCompass: Enhancing Specialized Provers with LLM Guidance
  8. 2026TCSAlgBench: Benchmarking Automated Proving for Research-Level Theoretical Computer Science