# Debate Helps Supervise Unreliable Experts

arXiv: https://arxiv.org/abs/2311.08702 · HTML: https://mlsanity.com/paper/2311.08702/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 38 references that are in the map.
- First posted on arXiv 15 Nov 2023.

## Builds on (most cited first, 18 of 18)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate](https://mlsanity.com/paper/2308.07201/) (2023)
- [Scalable agent alignment via reward modeling: a research direction](https://mlsanity.com/paper/1811.07871/) (2018)
- [AI safety via debate](https://mlsanity.com/paper/1805.00899/) (2018)
- [Self-critiquing models for assisting human evaluators](https://mlsanity.com/paper/2206.05802/) (2022)
- [Measuring Progress on Scalable Oversight for Large Language Models](https://mlsanity.com/paper/2211.03540/) (2022)
- [A Categorical Archive of ChatGPT Failures](https://mlsanity.com/paper/2302.03494/) (2023)
- [PRD: Peer Rank and Discussion Improve Large Language Model based Evaluations](https://mlsanity.com/paper/2307.02762/) (2023)
- [Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate](https://mlsanity.com/paper/2305.13160/) (2023)
- [Learning to Give Checkable Answers with Prover-Verifier Games](https://mlsanity.com/paper/2108.12099/) (2021)
- [Two-Turn Debate Doesn't Help Humans Answer Hard Reading Comprehension Questions](https://mlsanity.com/paper/2210.10860/) (2022)

## Built on by (most cited first, 30 of 43)

- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Debating with More Persuasive LLMs Leads to More Truthful Answers](https://mlsanity.com/paper/2402.06782/) (2024)
- [AI Control: Improving Safety Despite Intentional Subversion](https://mlsanity.com/paper/2312.06942/) (2023)
- [Language Models Learn to Mislead Humans via RLHF](https://mlsanity.com/paper/2409.12822/) (2024)
- [On scalable oversight with weak LLMs judging strong LLMs](https://mlsanity.com/paper/2407.04622/) (2024)
- [Prover-Verifier Games improve legibility of LLM outputs](https://mlsanity.com/paper/2407.13692/) (2024)
- [How Susceptible are LLMs to Influence in Prompts?](https://mlsanity.com/paper/2408.11865/) (2024)
- [ACC-Collab: An Actor-Critic Approach to Multi-Agent LLM Collaboration](https://mlsanity.com/paper/2411.00053/) (2024)
- [Scaling Laws For Scalable Oversight](https://mlsanity.com/paper/2504.18530/) (2025)
- [Avoiding Obfuscation with Prover-Estimator Debate](https://mlsanity.com/paper/2506.13609/) (2025)
- [Must Read: A Comprehensive Survey of Computational Persuasion](https://mlsanity.com/paper/2505.07775/) (2025)
- [When Persuasion Overrides Truth in Multi-Agent LLM Debates: Introducing a Confidence-Weighted Persuasion Override Rate (CW-POR)](https://mlsanity.com/paper/2504.00374/) (2025)
- [A Survey of Scaling in Large Language Model Reasoning](https://mlsanity.com/paper/2504.02181/) (2025)
- [Control Tax: The Price of Keeping AI in Check](https://mlsanity.com/paper/2506.05296/) (2025)
- [The Singapore Consensus on Global AI Safety Research Priorities](https://mlsanity.com/paper/2506.20702/) (2025)
- [Verdict: A Library for Scaling Judge-Time Compute](https://mlsanity.com/paper/2502.18018/) (2025)
- [Helpful Agent Meets Deceptive Judge: Understanding Vulnerabilities in Agentic Workflows](https://mlsanity.com/paper/2506.03332/) (2025)
- [Iterative Label Refinement Matters More than Preference Optimization under Weak Supervision](https://mlsanity.com/paper/2501.07886/) (2025)
- [Scalable Oversight for Superhuman AI via Recursive Self-Critiquing](https://mlsanity.com/paper/2502.04675/) (2025)
- [RADAR: A Risk-Aware Dynamic Multi-Agent Framework for LLM Safety Evaluation via Role-Specialized Collaboration](https://mlsanity.com/paper/2509.25271/) (2025)
- [AI Debaters are More Persuasive when Arguing in Alignment with Their Own Beliefs](https://mlsanity.com/paper/2510.13912/) (2025)
- [Trust but Verify: Prover-Verifier Deliberation for Selective LLM Prediction](https://mlsanity.com/paper/2605.25133/) (2026)
- [Training Deliberative Monitors for Black-Box Scheming Detection](https://mlsanity.com/paper/2605.29601/) (2026)
- [Debate Training Reduces Reward Hacking in RLAIF](https://mlsanity.com/paper/2608.17776/) (2026)
- [FindTheFlaws: Annotated Errors for Detecting Flawed Reasoning and Scalable Oversight Research](https://mlsanity.com/paper/2503.22989/) (2025)
- [Societal Impacts Research Requires Benchmarks for Creative Composition Tasks](https://mlsanity.com/paper/2504.06549/) (2025)
- [Out of Control -- Why Alignment Needs Formal Control Theory (and an Alignment Control Stack)](https://mlsanity.com/paper/2506.17846/) (2025)
- [Human-AI Complementarity: A Goal for Amplified Oversight](https://mlsanity.com/paper/2510.26518/) (2025)
- [Inclusion-of-Thoughts: Mitigating Preference Instability via Purifying the Decision Space](https://mlsanity.com/paper/2604.04944/) (2026)
- [Tell Me Why: Incentivizing Explanations](https://mlsanity.com/paper/2502.13410/) (2025)

## Abstract (arXiv)

As AI systems are used to answer more difficult questions and potentially help create new knowledge, judging the truthfulness of their outputs becomes more difficult and more important. How can we supervise unreliable experts, which have access to the truth but may not accurately report it, to give answers that are systematically true and don't just superficially seem true, when the supervisor can't tell the difference between the two on their own? In this work, we show that debate between two unreliable experts can help a non-expert judge more reliably identify the truth. We collect a dataset of human-written debates on hard reading comprehension questions where the judge has not read the source passage, only ever seeing expert arguments and short quotes selectively revealed by 'expert' debaters who have access to the passage. In our debates, one expert argues for the correct answer, and the other for an incorrect answer. Comparing debate to a baseline we call consultancy, where a single expert argues for only one answer which is correct half of the time, we find that debate performs significantly better, with 84% judge accuracy compared to consultancy's 74%. Debates are also more efficient, being 68% of the length of consultancies. By comparing human to AI debaters, we find evidence that with more skilled (in this case, human) debaters, the performance of debate goes up but the performance of consultancy goes down. Our error analysis also supports this trend, with 46% of errors in human debate attributable to mistakes by the honest debater (which should go away with increased skill); whereas 52% of errors in human consultancy are due to debaters obfuscating the relevant evidence from the judge (which should become worse with increased skill). Overall, these results show that debate is a promising approach for supervising increasingly capable but potentially unreliable AI systems.