mlsanityData to 7 Oct 2026

Viva La Vida: Verification and Accumulation Failures in Multi-Agent Proof Search

arXiv 2610.04829 ↗First posted 4 Oct 2026cs.CL cs.AI cs.MAcited by 0

On the map: LLM agents and tool use → Software engineering coding agents also fits: General LLM agent frameworks and benchmarks

Abstract (arXiv)

When an agentic prover works on an open problem, there is no proof assistant to fall back on: its verifier and lemma library are ultimately language models judging model outputs. We instrumented such a system end to end and analyzed 51,754 traced observations across three full runs (186 hours, 5,694). We find three connected failure modes. First, the three-model verifier requires unanimity and treats parse or API failure as non-approval; in 10 of 12 verification events, one member returned no parseable output or an API error, making acceptance arithmetically impossible without surfacing an error. Second, when the ensemble did function, one verifier approved 3 attempts that GPT rejected, each claiming to resolve the open problem; a single-verifier design would therefore have announced a solution three times. Third, because nothing could be approved, every review was a refutation, yet the lemma extractor mines reviews as well as proofs: 24 of 93 lemmas (26%) were extracted from rejected arguments with their refutational context removed. Taken together, these findings show that without external verification, supervision is itself a critical trust boundary: systems must distinguish abstention from rejection, preserve useful disagreement, and preserve the provenance and polarity of information before it becomes future context.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. First posted on arXiv 4 Oct 2026.

Builds on

No reference list yet.

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Why Agentic Theorem Prover Works: A Statistical Provability Theory of Mathematical Reasoning Models
  2. 2026Verification and Self-Improvement in Agentic AI: Foundations and Limits
  3. 2026VeriHarness: Scaling Agentic Verification for Long-Horizon Tasks
  4. 2026ProofCouncil: An LLM Agent for Solving Open Mathematical Problems
  5. 2026Agentic Systems as Boosting Weak Reasoning Models
  6. 2026Cogentic: Multi-Agent Orchestration for Automated Proof Discovery
  7. 2026Automated Lemma Discovery in Agentic Program Verification
  8. 2026Efficient Test-Time Optimization for Multi-Agent Proof Autoformalization