CAPRI: Contract-Aware Proof Repair for Isabelle
Not on the map. Why?
Abstract (arXiv)
We address the use of large language models (LLMs) to help discover Isabelle proofs. An Isabelle build establishes that the submitted theory is accepted, but not that an LLM changed only what the developer authorised. We present CAPRI, a contract-aware repair workflow in which Isabelle checks the proof and an independent checker enforces a machine-readable edit contract. Prompts, proposals, candidate repositories, diagnostics, verdicts, and hashes are retained for audit. We evaluate five workflows on twelve failed proofs from four developments, with three replicates per task and condition, giving 180 runs and 138 valid repairs. Of 144 terminal candidates accepted by Isabelle, six had modified protected text; all arose in iterative workflows that could edit a complete theory. A proof-body-only interface produced 29/36 valid repairs and no contract violations, compared with 31/36 for the corresponding full-theory workflow. One-shot repair produced 22/36, while a later prospectively frozen iterative workflow produced 32/36; these figures compare complete workflows rather than individual mechanisms. A separate post hoc OpenRouter campaign found no improvement in the designated Luna comparisons. A Sol configuration with matched demonstrations produced 33/36 repairs, compared with 29/36 in the frozen OpenAI Responses condition, but the difference was not statistically significant in a one-sided exact McNemar test (p=0.0625).
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 25 references that are in the map. First posted on arXiv 13 Aug 2026.
Builds on
16 of 25 references.
References · 16
- 2023Baldur: Whole-Proof Generation and Repair with Large Language Modelscited by 119
- 2022Thor: Wielding Hammers to Integrate Language Models and Automated Theorem Proverscited by 98
- 2023An In-Context Learning Agent for Formal Theorem-Provingcited by 82
- 2025APOLLO: Automated LLM and Lean Collaboration for Advanced Formal Reasoningcited by 35
- 2026A Minimal Agent for Automated Theorem Provingcited by 18
- 2026Solver-Aided Verification of Policy Compliance in Tool-Augmented LLM Agentscited by 16
- 2019Automated Patch Assessment for Program Repair at Scalecited by 14
- 2020Proof Repair across Type Equivalencescited by 8
- 2026Learning to Repair Lean Proofs from Compiler Feedbackcited by 7
- 2023Specification, Validation and Verification of Social, Legal, Ethical, Empathetic and Cultural Requirements for Autonomous Agentscited by 6
- 2019Unifying Semantic Foundations for Automated Verification Tools in Isabelle/UTPcited by 4
- 2026Event-B Agent: Towards LLM Agent for Formal Model Synthesis and Repaircited by 3
4 more
- 2026Vibe Coding an LLM-powered Theorem Provercited by 1
- 2026Towards Real-World Industrial-Scale Verification: LLM-Driven Theorem Proving on seL4cited by 1
- 2026TLA-Prover: Verifiable TLA+ Specification Synthesis via Preference-Optimized Low-Rank Adaptationcited by 1
- 2026IsabeLLM: Automated Theorem Proving Applied to Formally Verifying Consensuscited by 1
Built on by
Not cited yet.