mlsanityData to 14 Oct 2026

Theoria: Rewrite-Acceptability Verification over Informal Reasoning States

arXiv 2607.01223 ↗First posted 1 Jul 2026cs.AI cs.CL cs.LGcited by 0

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

When should an AI system's answer be trusted? Formal proof assistants offer certainty but cannot reach most of the problem distribution; scalar LLM judges offer coverage but produce opaque scores that cannot be audited after the fact and are subject to the same coherence issues as any LLM. We present Theoria, a verification architecture that closes this gap. A candidate solution is rewritten into a sequence of typed state transitions, each licensed by an explicit justification, whether that be a citation, computation, or problem-given fact, and every transition is independently auditable. The foundational invariant is completeness of change: every difference between consecutive proof states must be accounted for, so hidden premises surface as unlicensed mutations rather than passing silently. On HLE-Verified Gold (185 text-only expert problems), Theoria certifies 105 at 91.4% strict precision (Wilson 95% CI [84.5%, 95.4%]). Every certification produces a human readable proof trace in which each step can be independently challenged. Holistic LLM judges achieve comparable precision at matched coverage but fail on different problems (Jaccard 0.14-0.36), making the approaches complementary. On 95 adversarial poisoned proofs across 15 domains, structured judges catch 94.7% versus 83.2% for holistic judging (p= 0.0017). The overall 11.5 pp gap concentrates in hidden premises (90.6% vs. 62.5%, a 28 pp difference) and fabricated citations (100% vs. 90%), the error classes where the formal analysis predicts an advantage; performance is identical on arithmetic and theorem-misapplication errors, where no advantage is predicted. On GPQA Diamond (n= 65), certified precision is 97.1% (Wilson CI [85.1%, 99.5%]).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 29 references that are in the map. First posted on arXiv 1 Jul 2026.

Builds on

17 of 29 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026Advancing Mathematics Research with AI-Driven Formal Proof Search21 May
  2. 2025Step-Wise Formal Verification for LLM-Based Mathematical Problem Solving27 May

Other references · 15

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2023Let's Verify Step by Stepcited by 3,789
  3. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  4. 2024A Survey on LLM-as-a-Judgecited by 1,423
  5. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  6. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
  7. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 344
  8. 2024DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Datacited by 234
  9. 2024Free Process Rewards without Process Labelscited by 165
  10. 2023Deductive Verification of Chain-of-Thought Reasoningcited by 124
  11. 2024Prover-Verifier Games improve legibility of LLM outputscited by 85
  12. 2025Aristotle: IMO-level Automated Theorem Provingcited by 58
3 more
  1. 2026HLE-Verified: A Systematic Verification and Structured Revision of Humanity's Last Examcited by 12
  2. 2025Graph of Verification: Structured Verification of LLM Reasoning with Directed Acyclic Graphscited by 8
  3. 2026Verify Before You Commit: Towards Faithful Reasoning in LLM Agents via Self-Auditingcited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Hilbert: Recursively Building Formal Proofs with Informal Reasoning
  2. 2026On Reasoning-Centric LLM-based Automated Theorem Proving
  3. 2025Typed Chain-of-Thought: A Curry-Howard Framework for Verifying LLM Reasoning
  4. 2026Stress-Testing the Reasoning Competence of LLMs With Proofs Under Minimal Formalism
  5. 2026Mask-Proof: An LLM-based Automated Data Curation Pipeline on Mathematical Proofs
  6. 2026Magenta: Closing the Loop Between Mathematical Reasoning and Lean Verification
  7. 2025From Informal to Formal -- Incorporating and Evaluating LLMs on Natural Language Requirements to Verifiable Formal Proofs
  8. 2026Evidence-Grounded Verified Agentic Reasoning: A Path Toward Eliminating LLM Hallucination in Empirical Inference via Tool-Attested Kernel Proofs