When Helping Hurts and How to Fix It: Multi-Agent Debate for Data Cleaning
On the map: LLM agents and tool use → Social simulation and multi-agent debate
Abstract (arXiv)
When does multi-agent debate help data cleaning, and when does it hurt? Across three benchmarks, four model families, and over 6,000 task-condition pairs, we find debate's effect reverses sign: it degrades generation across all four models (-1.6 to -15.5pp) through critique-induced confusion (CIC), hallucinated Critic feedback that the Generator accepts uncritically, yet improves error detection (+27.4pp F1, d=1.0). We derive a debate benefit condition: debate helps when the probability of rescuing a wrong output (Critic verification odds weighted by fixability) exceeds the probability of destroying a correct one. A factorial experiment proves adversarial separation is essential: self-verification with identical tools fails, while a separate Critic with code-execution grounding and evidence-gated generation produces the first debate configuration to significantly exceed single-agent on a generative task (+5.3pp, p<0.05). The condition correctly predicts all nine task types and generalizes with zero false positives across 19 published comparisons in seven domains.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 41 of its 55 references that are in the map. First posted on arXiv 1 Jun 2026.
Builds on
41 of 55 references.
Closest ancestors · 6
Same direction, newest first
- 2025Can LLM Agents Really Debate? A Controlled Study of Multi-Agent Debate in Logical Reasoning11 Nov
- 2025When Identity Skews Debate: Anonymization for Bias-Reduced Multi-Agent Reasoning8 Oct
- 2025Revisiting Multi-Agent Debate as Test-Time Scaling: A Systematic Study of Conditional Effectiveness29 May
- 2025Stop Overvaluing Multi-Agent Debate -- We Must Rethink Evaluation and Embrace Model Heterogeneity12 Feb
- 2023Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View3 Oct
- 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate30 May
Other references · 35
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
- 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
23 more
- 2018Scalable agent alignment via reward modeling: a research directioncited by 438
- 2018AI safety via debatecited by 384
- 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 329
- 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 297
- 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 255
- 2023Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaborationcited by 246
- 2022Measuring Progress on Scalable Oversight for Large Language Modelscited by 212
- 2023Simple synthetic data reduces sycophancy in large language modelscited by 174
- 2022Can Foundation Models Wrangle Your Data?cited by 173
- 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 157
- 2024Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?cited by 153
- 2020Deep Entity Matching with Pre-Trained Language Modelscited by 143
- 2021Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspectivecited by 116
- 2017HoloClean: Holistic Data Repairs with Probabilistic Inferencecited by 79
- 2025SWE-Debate: Competitive Multi-Agent Debate for Software Issue Resolutioncited by 52
- 2020Valentine: Evaluating Matching Techniques for Dataset Discoverycited by 34
- 2019HoloDetect: Few-Shot Learning for Error Detectioncited by 31
- 2024AutoDCWorkflow: LLM-based Data Cleaning Workflow Auto-Generation and Benchmarkcited by 23
- 2022Single-Turn Debate Does Not Help Humans Answer Hard Reading-Comprehension Questionscited by 22
- 2023Using ChatGPT for Entity Matchingcited by 22
- 2024CleanAgent: Automating Data Standardization with LLM-based Agentscited by 22
- 2023Jellyfish: A Large Language Model for Data Preprocessingcited by 20
- 2025MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmarkcited by 13
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Black-Box Auditing of Epistemic Reliability in Multi-Agent Debate Distillation
- 2025Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debate
- 2026The Cost of Consensus: Isolated Self-Correction Prevails Over Unguided Homogeneous Multi-Agent Debate
- 2026Beyond Symmetric Agents: Cognitive Diversity and Multi-Agent Debate in Small Language Models
- 2025Stay Focused: Problem Drift in Multi-Agent Debate
- 2026The Confident Liar: Diagnosing Multi-Agent Debate with Log-Probabilities and LLM-as-Judge
- 2026The Reasoning Trap: An Information-Theoretic Bound on Closed-System Multi-Step LLM Reasoning
- 2026Demystifying Multi-Agent Debate: The Role of Confidence and Diversity