mlsanityData to 7 Oct 2026

Improving Multi-Agent Debate with Sparse Communication Topology

arXiv 2406.11776 ↗First posted 17 Jun 2024cs.CLcited by 103

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Multi-agent debate has proven effective in improving large language models quality for reasoning and factuality tasks. While various role-playing strategies in multi-agent debates have been explored, in terms of the communication among agents, existing approaches adopt a brute force algorithm -- each agent can communicate with all other agents. In this paper, we systematically investigate the effect of communication connectivity in multi-agent systems. Our experiments on GPT and Mistral models reveal that multi-agent debates leveraging sparse communication topology can achieve comparable or superior performance while significantly reducing computational costs. Furthermore, we extend the multi-agent debate framework to multimodal reasoning and alignment labeling tasks, showcasing its broad applicability and effectiveness. Our findings underscore the importance of communication connectivity on enhancing the efficiency and effectiveness of the "society of minds" approach.

Key facts. Cited by 103 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 30 of its 50 references that are in the map. First posted on arXiv 17 Jun 2024.

Builds on

30 of 50 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?28 Feb
  2. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs22 Sep
  3. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate30 May

Other references · 27

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  7. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  9. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  10. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  11. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  12. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
15 more
  1. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  2. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  3. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  4. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  5. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  6. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  7. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  8. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
  9. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  10. 2022Generating Sequences by Learning to Self-Correctcited by 271
  11. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Modelscited by 260
  12. 2024Direct Language Model Alignment from Online AI Feedbackcited by 257
  13. 2024The Impact of Reasoning Step Length on Large Language Modelscited by 152
  14. 2023Large Language Models are Visual Reasoning Coordinatorscited by 58
  15. 2024Question Aware Vision Transformer for Multimodal Reasoningcited by 32

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 46; 2026: 50.

103 papers.

Sort:

By area:

  1. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  2. 2025Multi-Agent Design: Optimizing Agents with Better Prompts and Topologiescited by 106
  3. 2025MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learningcited by 80
  4. 2025Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?cited by 74
  5. 2024GroupDebate: Enhancing the Efficiency of Multi-Agent Debate Using Group Discussioncited by 64
  6. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  7. 2025A Survey on the Optimization of Large Language Model-based Agentscited by 45
  8. 2025Stop Overvaluing Multi-Agent Debate -- We Must Rethink Evaluation and Embrace Model Heterogeneitycited by 43
  9. 2024Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent Systemcited by 40
  10. 2025Understanding the Information Propagation Effects of Communication Topologies in LLM-based Multi-Agent Systemscited by 39
  11. 2025BlindGuard: Safeguarding LLM-based Multi-Agent Systems under Unknown Attackscited by 37
  12. 2025Free-MAD: Consensus-Free Multi-Agent Debatecited by 27
38 more
  1. 2025Can LLM Agents Really Debate? A Controlled Study of Multi-Agent Debate in Logical Reasoningcited by 27
  2. 2024ACC-Collab: An Actor-Critic Approach to Multi-Agent LLM Collaborationcited by 22
  3. 2025S2-MAD: Breaking the Token Barrier to Enhance Multi-Agent Debate Efficiencycited by 21
  4. 2025Do We Truly Need So Many Samples? Multi-LLM Repeated Sampling Efficiently Scales Test-Time Computecited by 20
  5. 2025Enhancing Multi-Agent Debate System Performance via Confidence Expressioncited by 18
  6. 2025Literature Review Of Multi-Agent Debate For Problem-Solvingcited by 15
  7. 2025MALLM: Multi-Agent Large Language Models Frameworkcited by 15
  8. 2025Graph-Augmented Large Language Model Agents: Current Progress and Future Prospectscited by 14
  9. 2025Cost-Effective Communication: An Auction-based Method for Language Agent Interactioncited by 14
  10. 2025LLM as a Broken Telephone: Iterative Generation Distorts Informationcited by 12
  11. 2025Debate-to-Detect: Reformulating Misinformation Detection as a Real-World Debate with Large Language Modelscited by 12
  12. 2025Enhancing LLM Performance Through Debate: An Empirical Study on Multi-Agent Debate for Coding Taskscited by 11
  13. 2025How to Train a Leader: Hierarchical Reasoning in Multi-Agent LLMscited by 11
  14. 2025United Minds or Isolated Agents? Exploring Coordination of LLMs under Cognitive Load Theorycited by 10
  15. 2025Harnessing Consistency for Robust Test-Time LLM Ensemblecited by 10
  16. 2026The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMscited by 10
  17. 2024A Multi-LLM Debiasing Frameworkcited by 9
  18. 2025Amplified Vulnerabilities: Structured Jailbreak Attacks on LLM-based Multi-Agent Debatecited by 8
  19. 2025AgentDNS: A Root Domain Naming System for LLM Agentscited by 8
  20. 2025Multi-Agent Debate Strategies to Enhance Requirements Engineering with Large Language Modelscited by 8
  21. 2026Multi-Agent Debate with Memory Maskingcited by 8
  22. 2025Tiered Agentic Oversight: A Hierarchical Multi-Agent System for Healthcare Safetycited by 7
  23. 2025CortexDebate: Debating Sparsely and Equally for Multi-Agent Debatecited by 6
  24. 2024Multi-LLM Text Summarizationcited by 5
  25. 2025AlphaApollo: A System for Deep Agentic Reasoningcited by 5
  26. 2025From Evidence to Verdict: An Agent-Based Forensic Framework for AI-Generated Image Detectioncited by 5
  27. 2026DynaDebate: Breaking Homogeneity in Multi-Agent Debate with Dynamic Path Generationcited by 5
  28. 2026Dynamic Role Assignment for Multi-Agent Debatecited by 5
  29. 2026SYMPHONY: Synergistic Multi-agent Planning with Heterogeneous Language Model Assemblycited by 5
  30. 2024Gradual Vigilance and Interval Communication: Enhancing Value Alignment in Multi-Agent Debatescited by 4
  31. 2025Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectivescited by 4
  32. 2025MUPA: Towards Multi-Path Agentic Reasoning for Grounded Video Question Answeringcited by 4
  33. 2025Beyond Detection: Exploring Evidence-based Multi-Agent Debate for Misinformation Intervention and Persuasioncited by 4
  34. 2026ValueFlow: Measuring the Propagation of Value Perturbations in Multi-Agent LLM Systemscited by 4
  35. 2026Hear Both Sides: Efficient Multi-Agent Debate via Diversity-Aware Message Retentioncited by 4
  36. 2026Delayed Verification Destabilizes Multi-Agent LLM Belief: Instability Thresholds and Optimal Corrector Placementcited by 4
  37. 2025Sharing is Caring: Efficient LM Post-Training with Collective RL Experience Sharingcited by 3
  38. 2025The Effect of Belief Boxes and Open-mindedness on Persuasioncited by 3

50 most cited of 103 All 103 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025When and Why Does Multi-Agent Debate Fail and Does It Really Underperform?
  2. 2025OPTAGENT: Optimizing Multi-Agent LLM Interactions Through Verbal Reinforcement Learning for Enhanced Reasoning
  3. 2026Dynamic Trust-Aware Sparse Communication Topology for LLM-Based Multi-Agent Consensus
  4. 2025AgentsNet: Coordination and Collaborative Reasoning in Multi-Agent LLMs
  5. 2025Rethinking Multi-Agent Intelligence Through the Lens of Small-World Networks
  6. 2026CRAFT: Grounded Multi-Agent Coordination Under Partial Information
  7. 2026Mixture of Debaters: Learn to Debate at Architectural Level in Multi-Agent Reasoning
  8. 2026SVR-MAD: A Bayesian-Inspired Framework for Posterior-Guided Multi-Agent Debate