# Learning to Break: Knowledge-Enhanced Reasoning in Multi-Agent Debate System

arXiv: https://arxiv.org/abs/2312.04854 · HTML: https://mlsanity.com/paper/2312.04854/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 27 of its 59 references that are in the map.
- First posted on arXiv 8 Dec 2023.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 27 of 27)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [Atlas: Few-shot Learning with Retrieval Augmented Language Models](https://mlsanity.com/paper/2208.03299/) (2022)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate](https://mlsanity.com/paper/2308.07201/) (2023)
- [In-Context Retrieval-Augmented Language Models](https://mlsanity.com/paper/2302.00083/) (2023)
- [Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration](https://mlsanity.com/paper/2307.05300/) (2023)
- [Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedback](https://mlsanity.com/paper/2305.10142/) (2023)
- [Rethinking with Retrieval: Faithful Large Language Model Inference](https://mlsanity.com/paper/2301.00303/) (2022)
- [War and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Wars](https://mlsanity.com/paper/2311.17227/) (2023)
- [Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs](https://mlsanity.com/paper/2311.17371/) (2023)
- [ProAgent: Building Proactive Cooperative Agents with Large Language Models](https://mlsanity.com/paper/2308.11339/) (2023)
- [Towards Reasoning in Large Language Models via Multi-Agent Peer Review Collaboration](https://mlsanity.com/paper/2311.08152/) (2023)
- [Multi-Agent Consensus Seeking via Large Language Models](https://mlsanity.com/paper/2310.20151/) (2023)
- [Epidemic Modeling with Generative Agents](https://mlsanity.com/paper/2307.04986/) (2023)
- [Summarization is (Almost) Dead](https://mlsanity.com/paper/2309.09558/) (2023)
- [Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applications](https://mlsanity.com/paper/2311.05876/) (2023)
- [ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up?](https://mlsanity.com/paper/2311.16989/) (2023)

## Built on by (most cited first, 26 of 26)

- [Scaling Large Language Model-based Multi-Agent Collaboration](https://mlsanity.com/paper/2406.07155/) (2024)
- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligence](https://mlsanity.com/paper/2407.07061/) (2024)
- [Do as We Do, Not as You Think: the Conformity of Large Language Models](https://mlsanity.com/paper/2501.13381/) (2025)
- [Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworks](https://mlsanity.com/paper/2407.21059/) (2024)
- [Free-MAD: Consensus-Free Multi-Agent Debate](https://mlsanity.com/paper/2509.11035/) (2025)
- [Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievals](https://mlsanity.com/paper/2502.16101/) (2025)
- [A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challenges](https://mlsanity.com/paper/2403.10249/) (2024)
- [ToMAP: Training Opponent-Aware LLM Persuaders with Theory of Mind](https://mlsanity.com/paper/2505.22961/) (2025)
- [Debate as Optimization: Adaptive Conformal Prediction and Diverse Retrieval for Event Extraction](https://mlsanity.com/paper/2406.12197/) (2024)
- [Amplified Vulnerabilities: Structured Jailbreak Attacks on LLM-based Multi-Agent Debate](https://mlsanity.com/paper/2504.16489/) (2025)
- [Multi-Agent Debate Strategies to Enhance Requirements Engineering with Large Language Models](https://mlsanity.com/paper/2507.05981/) (2025)
- [Privacy-Enhancing Paradigms within Federated Multi-Agent Systems](https://mlsanity.com/paper/2503.08175/) (2025)
- [Benefits and Limitations of Communication in Multi-Agent Reasoning](https://mlsanity.com/paper/2510.13903/) (2025)
- [Cross-Task Experiential Learning on LLM-based Multi-Agent Collaboration](https://mlsanity.com/paper/2505.23187/) (2025)
- [Courtroom-Style Multi-Agent Debate with Progressive RAG and Role-Switching for Controversial Claim Verification](https://mlsanity.com/paper/2603.28488/) (2026)
- [EquiMem: Calibrating Shared Memory in Multi-Agent Debate via Game-Theoretic Equilibrium](https://mlsanity.com/paper/2605.09278/) (2026)
- [Finding Common Ground: Using Large Language Models to Detect Agreement in Multi-Agent Decision Conferences](https://mlsanity.com/paper/2507.08440/) (2025)
- [AgriEval: A Comprehensive Chinese Agricultural Benchmark for Large Language Models](https://mlsanity.com/paper/2507.21773/) (2025)
- [The Effect of Belief Boxes and Open-mindedness on Persuasion](https://mlsanity.com/paper/2512.06573/) (2025)
- [Tool-MAD: A Multi-Agent Debate Framework for Fact Verification with Diverse Tool Augmentation and Adaptive Retrieval](https://mlsanity.com/paper/2601.04742/) (2026)
- [STAMPsy: Towards SpatioTemporal-Aware Mixed-Type Dialogues for Psychological Counseling](https://mlsanity.com/paper/2412.16674/) (2024)
- [Belief-Calibrated Multi-Agent Consensus Seeking for Complex NLP Tasks](https://mlsanity.com/paper/2510.06307/) (2025)
- [Unleashing Diverse Thinking Modes in LLMs through Multi-Agent Collaboration](https://mlsanity.com/paper/2510.16645/) (2025)
- [AgenticSimLaw: A Juvenile Courtroom Multi-Agent Debate Simulation for Explainable High-Stakes Tabular Decision Making](https://mlsanity.com/paper/2601.21936/) (2026)
- [ReFeR: Improving Evaluation and Reasoning through Hierarchy of Models](https://mlsanity.com/paper/2407.12877/) (2024)

## Abstract (arXiv)

Multi-agent debate system (MAD) imitating the process of human discussion in pursuit of truth, aims to align the correct cognition of different agents for the optimal solution. It is challenging to make various agents perform right and highly consistent cognition due to their limited and different knowledge backgrounds (i.e., cognitive islands), which hinders the search for the optimal solution. To address the challenge, we propose a novel underlineMulti-underlineAgent underlineDebate with underlineKnowledge-underlineEnhanced framework (MADKE) to promote the system to find the solution. First, we involve a shared retrieval knowledge pool in the debate process to solve the problem of limited and different knowledge backgrounds. Then, we propose an adaptive knowledge selection method to guarantee the accuracy and personalization of knowledge. This method allows agents to choose whether to use external knowledge in each conversation round according to their own needs. Our experimental results on six datasets show that our method achieves state-of-the-art results compared to existing single-agent and multi-agent methods. Further analysis reveals that the introduction of retrieval knowledge can help the agent to break cognitive islands in the debate process and effectively improve the consistency and correctness of the model. Moreover, MADKE using Qwen1.5-72B-Chat surpasses GPT-4 by +1.26% on average in six datasets, which validates that our method can help open-source LLMs achieve or even surpass the performance of GPT-4. Our code is available at urlhttps://github.com/FutureForMe/MADKE.