mlsanityData to 7 Oct 2026

Debating with More Persuasive LLMs Leads to More Truthful Answers

arXiv 2402.06782 ↗First posted 9 Feb 2024cs.AI cs.CLcited by 255

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Common methods for aligning large language models (LLMs) with desired behaviour heavily rely on human-labelled data. However, as models grow increasingly sophisticated, they will surpass human expertise, and the role of human evaluation will evolve into non-experts overseeing experts. In anticipation of this, we ask: can weaker models assess the correctness of stronger models? We investigate this question in an analogous setting, where stronger models (experts) possess the necessary information to answer questions and weaker models (non-experts) lack this information. The method we evaluate is debate, where two LLM experts each argue for a different answer, and a non-expert selects the answer. We find that debate consistently helps both non-expert models and humans answer questions, achieving 76% and 88% accuracy respectively (naive baselines obtain 48% and 60%). Furthermore, optimising expert debaters for persuasiveness in an unsupervised manner improves non-expert ability to identify the truth in debates. Our results provide encouraging empirical evidence for the viability of aligning models with debate in the absence of ground truth.

Key facts. Cited by 255 papers in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 25 of its 40 references that are in the map. First posted on arXiv 9 Feb 2024.

Builds on

25 of 40 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision14 Dec
  2. 2022Constitutional AI: Harmlessness from AI Feedback15 Dec

Other references · 23

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  8. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  9. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  10. 2016Concrete Problems in AI Safetycited by 1,582
  11. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  12. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
11 more
  1. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  2. 2018AI safety via debatecited by 384
  3. 2022Self-critiquing models for assisting human evaluatorscited by 372
  4. 2021True Few-Shot Learning with Language Modelscited by 343
  5. 2022Teaching language models to support answers with verified quotescited by 290
  6. 2022Measuring Progress on Scalable Oversight for Large Language Modelscited by 212
  7. 2023AI Control: Improving Safety Despite Intentional Subversioncited by 207
  8. 2018Supervising strong learners by amplifying weak expertscited by 139
  9. 2023Question Decomposition Improves the Faithfulness of Model-Generated Reasoningcited by 107
  10. 2023Debate Helps Supervise Unreliable Expertscited by 39
  11. 2023AQuA: A Benchmarking Tool for Label Quality Assessmentcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 51; 2025: 119; 2026: 85.

255 papers.

Sort:

By area:

  1. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 451
  2. 2025Accelerating scientific discovery with Co-Scientistcited by 353
  3. 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 294
  4. 2024Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judgecited by 249
  5. 2024Scaling Large Language Model-based Multi-Agent Collaborationcited by 235
  6. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  7. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 141
  8. 2024LLM Critics Help Catch LLM Bugscited by 138
  9. 2024SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinementcited by 124
  10. 2024Language Models Learn to Mislead Humans via RLHFcited by 118
  11. 2025Multi-Agent Design: Optimizing Agents with Better Prompts and Topologiescited by 106
  12. 2024On scalable oversight with weak LLMs judging strong LLMscited by 94
38 more
  1. 2024Embodied LLM Agents Learn to Cooperate in Organized Teamscited by 81
  2. 2025MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learningcited by 80
  3. 2024Prover-Verifier Games improve legibility of LLM outputscited by 79
  4. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 74
  5. 2024Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Samplingcited by 63
  6. 2025Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debatecited by 63
  7. 2024Agents' Room: Narrative Generation through Multi-step Collaborationcited by 57
  8. 2024MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debatecited by 54
  9. 2024Sabotage Evaluations for Frontier Modelscited by 45
  10. 2025Scaling Laws of Synthetic Data for Language Modelscited by 44
  11. 2025Stop Overvaluing Multi-Agent Debate -- We Must Rethink Evaluation and Embrace Model Heterogeneitycited by 43
  12. 2024Auto-Arena: Automating LLM Evaluations with Agent Peer Battles and Committee Discussionscited by 40
  13. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  14. 2025Teaching Language Models to Critique via Reinforcement Learningcited by 38
  15. 2024LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraintscited by 34
  16. 2025Optimizing Model Selection for Compound AI Systemscited by 34
  17. 2025Investigating Non-Transitivity in LLM-as-a-Judgecited by 33
  18. 2024How Susceptible are LLMs to Influence in Prompts?cited by 32
  19. 2024Adversaries Can Misuse Combinations of Safe Modelscited by 31
  20. 2024Multi-Agent Collaboration via Cross-Team Orchestrationcited by 30
  21. 2024Conformity in Large Language Modelscited by 30
  22. 2025The Truth Becomes Clearer Through Debate! Multi-Agent Systems with Large Language Models Unmask Fake Newscited by 29
  23. 2024A Survey of Useful LLM Evaluationcited by 28
  24. 2024Adaptive Deployment of Untrusted LLMs Reduces Distributed Threatscited by 27
  25. 2025Can LLM Agents Really Debate? A Controlled Study of Multi-Agent Debate in Logical Reasoningcited by 27
  26. 2024Reference-Guided Verdict: LLMs-as-Judges in Automatic Evaluation of Free-Form QAcited by 25
  27. 2025Beyond Majority Voting: LLM Aggregation by Leveraging Higher-Order Informationcited by 25
  28. 2025Accumulating Context Changes the Beliefs of Language Modelscited by 25
  29. 2024ACC-Collab: An Actor-Critic Approach to Multi-Agent LLM Collaborationcited by 22
  30. 2025QualityFlow: An Agentic Workflow for Program Synthesis Controlled by LLM Quality Checkscited by 22
  31. 2025Multiple LLM Agents Debate for Equitable Cultural Alignmentcited by 20
  32. 2025Multi-Agent Debate for LLM Judges with Adaptive Stability Detectioncited by 20
  33. 2024Evaluating the Performance of Large Language Models via Debatescited by 19
  34. 2024Teaching Models to Balance Resisting and Accepting Persuasioncited by 19
  35. 2025Scaling Laws For Scalable Oversightcited by 18
  36. 2025Enhancing Multi-Agent Debate System Performance via Confidence Expressioncited by 18
  37. 2024Plurals: A System for Guiding LLMs Via Simulated Social Ensemblescited by 17
  38. 2024Training Language Models to Win Debates with Self-Play Improves Judge Accuracycited by 16

50 most cited of 255 All 255 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Evaluating the Capabilities of LLMs for Persuasive Dialogue
  2. 2025When Two LLMs Debate, Both Think They'll Win
  3. 2023Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs
  4. 2026Argument Quality Assessment with Large Language Models: A Pairwise Bradley-Terry Approach
  5. 2025Single LLM Debate, MoLaCE: Mixture of Latent Concept Experts Against Confirmation Bias
  6. 2026Measuring Collapse and Correction in Homogeneous-Panel LLM Debate
  7. 2024Can LLMs Beat Humans in Debating? A Dynamic Multi-agent Framework for Competitive Debate
  8. 2024DebateQA: Evaluating Question Answering on Debatable Knowledge