# Avoiding Obfuscation with Prover-Estimator Debate

arXiv: https://arxiv.org/abs/2506.13609 · HTML: https://mlsanity.com/paper/2506.13609/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 31 references that are in the map.
- First posted on arXiv 16 Jun 2025.

## Builds on (most cited first, 16 of 16)

- [Introduction to Online Convex Optimization](https://mlsanity.com/paper/1909.05207/) (2019)
- [Scalable agent alignment via reward modeling: a research direction](https://mlsanity.com/paper/1811.07871/) (2018)
- [AI safety via debate](https://mlsanity.com/paper/1805.00899/) (2018)
- [Debating with More Persuasive LLMs Leads to More Truthful Answers](https://mlsanity.com/paper/2402.06782/) (2024)
- [Supervising strong learners by amplifying weak experts](https://mlsanity.com/paper/1810.08575/) (2018)
- [On scalable oversight with weak LLMs judging strong LLMs](https://mlsanity.com/paper/2407.04622/) (2024)
- [Prover-Verifier Games improve legibility of LLM outputs](https://mlsanity.com/paper/2407.13692/) (2024)
- [Outcome Indistinguishability](https://mlsanity.com/paper/2011.13426/) (2020)
- [Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?](https://mlsanity.com/paper/2502.15657/) (2025)
- [Scalable AI Safety via Doubly-Efficient Debate](https://mlsanity.com/paper/2311.14125/) (2023)
- [Debate Helps Supervise Unreliable Experts](https://mlsanity.com/paper/2311.08702/) (2023)
- [Learning to Give Checkable Answers with Prover-Verifier Games](https://mlsanity.com/paper/2108.12099/) (2021)
- [An alignment safety case sketch based on debate](https://mlsanity.com/paper/2505.03989/) (2025)
- [Models That Prove Their Own Correctness](https://mlsanity.com/paper/2405.15722/) (2024)
- [Neural Interactive Proofs](https://mlsanity.com/paper/2412.08897/) (2024)
- [Playing Large Games with Oracles and AI Debate](https://mlsanity.com/paper/2312.04792/) (2023)

## Built on by (most cited first, 14 of 14)

- [Core Safety Values for Provably Corrigible Agents](https://mlsanity.com/paper/2507.20964/) (2025)
- [Emergent Alignment via Competition](https://mlsanity.com/paper/2509.15090/) (2025)
- [AI Debaters are More Persuasive when Arguing in Alignment with Their Own Beliefs](https://mlsanity.com/paper/2510.13912/) (2025)
- [When and Why Does Multi-Agent Debate Fail and Does It Really Underperform?](https://mlsanity.com/paper/2510.20963/) (2025)
- [Debate is efficient with your time](https://mlsanity.com/paper/2602.08630/) (2026)
- [Knowledge Divergence and the Value of Debate for Scalable Oversight](https://mlsanity.com/paper/2603.05293/) (2026)
- [Self-Improvement as Coherence Optimization: A Theoretical Account](https://mlsanity.com/paper/2601.13566/) (2026)
- [Debate Helps Weak Judges Reward Stronger Models](https://mlsanity.com/paper/2605.27483/) (2026)
- [Collaborative Disagreement Resolution for Scalable Oversight](https://mlsanity.com/paper/2607.01251/) (2026)
- [TRACE: An Operational Reasoning Schema for Auditable Agentic Commitments](https://mlsanity.com/paper/2607.12480/) (2026)
- [When Honesty is Not Enough in AI Debate](https://mlsanity.com/paper/2609.29189/) (2026)
- [Can AI Oversight Be Zero Knowledge?](https://mlsanity.com/paper/2610.01995/) (2026)
- [PyINE: A Framework for Scalable Elicitation and Oversight via Code Execution](https://mlsanity.com/paper/2610.04737/) (2026)
- [L3M: A Verifiable Harness and Runtime for Self-Extending and Self-Improving Agents](https://mlsanity.com/paper/2610.19591/) (2026)

## Abstract (arXiv)

Training powerful AI systems to exhibit desired behaviors hinges on the ability to provide accurate human supervision on increasingly complex tasks. A promising approach to this problem is to amplify human judgement by leveraging the power of two competing AIs in a debate about the correct solution to a given problem. Prior theoretical work has provided a complexity-theoretic formalization of AI debate, and posed the problem of designing protocols for AI debate that guarantee the correctness of human judgements for as complex a class of problems as possible. Recursive debates, in which debaters decompose a complex problem into simpler subproblems, hold promise for growing the class of problems that can be accurately judged in a debate. However, existing protocols for recursive debate run into the obfuscated arguments problem: a dishonest debater can use a computationally efficient strategy that forces an honest opponent to solve a computationally intractable problem to win. We mitigate this problem with a new recursive debate protocol that, under certain stability assumptions, ensures that an honest debater can win with a strategy requiring computational efficiency comparable to their opponent.