# Two-Turn Debate Doesn't Help Humans Answer Hard Reading Comprehension Questions

arXiv: https://arxiv.org/abs/2210.10860 · HTML: https://mlsanity.com/paper/2210.10860/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 9 references that are in the map.
- First posted on arXiv 19 Oct 2022.

## Builds on (most cited first, 7 of 7)

- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [AI safety via debate](https://mlsanity.com/paper/1805.00899/) (2018)
- [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020)
- [QuALITY: Question Answering with Long Input Texts, Yes!](https://mlsanity.com/paper/2112.08608/) (2021)
- [On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection](https://mlsanity.com/paper/1811.07901/) (2018)
- [Quantifying Interpretability and Trust in Machine Learning Systems](https://mlsanity.com/paper/1901.08558/) (2019)
- [Single-Turn Debate Does Not Help Humans Answer Hard Reading-Comprehension Questions](https://mlsanity.com/paper/2204.05212/) (2022)

## Built on by (most cited first, 17 of 17)

- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [Measuring Progress on Scalable Oversight for Large Language Models](https://mlsanity.com/paper/2211.03540/) (2022)
- [Language Models Learn to Mislead Humans via RLHF](https://mlsanity.com/paper/2409.12822/) (2024)
- [On scalable oversight with weak LLMs judging strong LLMs](https://mlsanity.com/paper/2407.04622/) (2024)
- [Prover-Verifier Games improve legibility of LLM outputs](https://mlsanity.com/paper/2407.13692/) (2024)
- [Debate Helps Supervise Unreliable Experts](https://mlsanity.com/paper/2311.08702/) (2023)
- [How Susceptible are LLMs to Influence in Prompts?](https://mlsanity.com/paper/2408.11865/) (2024)
- [Training Language Models to Win Debates with Self-Play Improves Judge Accuracy](https://mlsanity.com/paper/2409.16636/) (2024)
- [An alignment safety case sketch based on debate](https://mlsanity.com/paper/2505.03989/) (2025)
- [Learning Task Decomposition to Assist Humans in Competitive Programming](https://mlsanity.com/paper/2406.04604/) (2024)
- [AI Debaters are More Persuasive when Arguing in Alignment with Their Own Beliefs](https://mlsanity.com/paper/2510.13912/) (2025)
- [Trust but Verify: Prover-Verifier Deliberation for Selective LLM Prediction](https://mlsanity.com/paper/2605.25133/) (2026)
- [Debate Training Reduces Reward Hacking in RLAIF](https://mlsanity.com/paper/2608.17776/) (2026)
- [Human-AI Complementarity: A Goal for Amplified Oversight](https://mlsanity.com/paper/2510.26518/) (2025)
- [Reasoning Court: Combining Reasoning, Action, and Judgment for Multi-Hop Reasoning](https://mlsanity.com/paper/2504.09781/) (2025)
- [Debate Helps Weak Judges Reward Stronger Models](https://mlsanity.com/paper/2605.27483/) (2026)
- [Collaborative Disagreement Resolution for Scalable Oversight](https://mlsanity.com/paper/2607.01251/) (2026)

## Abstract (arXiv)

The use of language-model-based question-answering systems to aid humans in completing difficult tasks is limited, in part, by the unreliability of the text these systems generate. Using hard multiple-choice reading comprehension questions as a testbed, we assess whether presenting humans with arguments for two competing answer options, where one is correct and the other is incorrect, allows human judges to perform more accurately, even when one of the arguments is unreliable and deceptive. If this is helpful, we may be able to increase our justified trust in language-model-based systems by asking them to produce these arguments where needed. Previous research has shown that just a single turn of arguments in this format is not helpful to humans. However, as debate settings are characterized by a back-and-forth dialogue, we follow up on previous results to test whether adding a second round of counter-arguments is helpful to humans. We find that, regardless of whether they have access to arguments or not, humans perform similarly on our task. These findings suggest that, in the case of answering reading comprehension questions, debate is not a helpful format.