mlsanityData to 14 Oct 2026

Enhancing LLM Performance Through Debate: An Empirical Study on Multi-Agent Debate for Coding Tasks

arXiv 2503.12029 ↗First posted 15 Mar 2025cs.SEcited by 11

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

Large Language Models (LLMs) have advanced autonomous agents' planning and decision-making, yet they struggle with complex tasks requiring diverse expertise and multi-step reasoning. Multi-Agent Debate (MAD) systems, introduced in NLP research, address this gap by enabling structured debates among LLM-based agents to refine solutions iteratively. MAD promotes divergent thinking through role-specific agents, dynamic interactions, and structured decision-making. Recognizing parallels between Software Engineering (SE) and collaborative human problem-solving, this study investigates MAD's effectiveness on four coding tasks in SE. We adapt a MAD framework from NLP, analyze agent interactions to assess consensus-building and iterative refinement, and propose two MAD variants that enhance agent debate for coding tasks by addressing the observed weaknesses. Our findings show that structured debate and collaboration improve problem-solving and yield strong performance in some cases, highlighting the collaborative debate synergy between LLM agents for coding tasks in SE while identifying areas for future exploration.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 46 of its 90 references that are in the map. First posted on arXiv 15 Mar 2025.

Builds on

46 of 90 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Voting or Consensus? Decision-Making in Multi-Agent Debate26 Feb
  2. 2023Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs29 Nov
  3. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate30 May

Other references · 43

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  5. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  6. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  7. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
  8. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  9. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  10. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  11. 2023ChatDev: Communicative Agents for Software Developmentcited by 919
  12. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 654
31 more
  1. 2018Universal Sentence Encodercited by 489
  2. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 399
  3. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 300
  4. 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 266
  5. 2021CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Taskscited by 200
  6. 2024Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?cited by 159
  7. 2024LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Aheadcited by 155
  8. 2022Few-shot training LLMs for project-specific code-summarizationcited by 129
  9. 2024Improving Multi-Agent Debate with Sparse Communication Topologycited by 107
  10. 2018Improving Automatic Source Code Summarization via Deep Reinforcement Learningcited by 100
  11. 2023Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debatecited by 97
  12. 2021Leveraging Automated Unit Tests for Unsupervised Code Translationcited by 82
  13. 2023Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization)cited by 72
  14. 2023Towards an Understanding of Large Language Models in Software Engineering Taskscited by 68
  15. 2021AVATAR: A Parallel Corpus for Java-Python Program Translationcited by 59
  16. 2024Exploring and Unleashing the Power of Large Language Models in Automated Code Translationcited by 57
  17. 2024MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language Modelscited by 43
  18. 2024Can LLMs Replace Manual Annotation of Software Engineering Artifacts?cited by 42
  19. 2023On the Evaluation of Neural Code Translation: Taxonomy and Benchmarkcited by 23
  20. 2023AI-assisted Code Authoring at Scale: Fine-tuning, deploying, and mixed methods evaluationcited by 21
  21. 2023Evaluating Code Summarization Techniques: A New Metric and an Empirical Characterizationcited by 21
  22. 2024The Current Challenges of Software Engineering in the Era of Large Language Modelscited by 21
  23. 2022An Extractive-and-Abstractive Framework for Source Code Summarizationcited by 18
  24. 2023EditSum: A Retrieve-and-Edit Framework for Source Code Summarizationcited by 18
  25. 2024Unraveling the Potential of Large Language Models in Code Translation: How Far Are We?cited by 17
  26. 2023Developer-Intent Driven Code Comment Generationcited by 16
  27. 2024InterTrans: Leveraging Transitive Intermediate Translations to Enhance LLM-based Code Translationcited by 16
  28. 2022Semantic Similarity Metrics for Evaluating Source Code Summarizationcited by 14
  29. 2024Calibration of Large Language Models on Code Summarizationcited by 12
  30. 2024Mitigating Data Imbalance for Software Vulnerability Assessment: Does Data Augmentation Help?cited by 6
  31. 2024Context Conquers Parameters: Outperforming Proprietary LLM in Commit Message Generationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 5.

11 papers.

  1. 2025Free-MAD: Consensus-Free Multi-Agent Debatecited by 29
  2. 2025Can LLM Agents Really Debate? A Controlled Study of Multi-Agent Debate in Logical Reasoningcited by 29
  3. 2025Revisiting Multi-Agent Debate as Test-Time Scaling: A Systematic Study of Conditional Effectivenesscited by 20
  4. 2025Byzantine-Robust Decentralized Coordination of LLM Agentscited by 14
  5. 2025The Effect of Belief Boxes and Open-mindedness on Persuasioncited by 3
  6. 2026Multi-Agent Debate Strategies: Survey, Taxonomy, and Challengescited by 3
  7. 2026Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Modelcited by 3
  8. 2025CRAwDAD: Causal Reasoning Augmentation with Dual-Agent Debatecited by 1
  9. 2026What Do Agents Communicate? Characterizing Information Exchange in Multi-Agent Systemscited by 1
  10. 2026Understanding Conversational Patterns in Multi-agent Programming: A Case Study on Fibonacci Game Developmentcited by 1
  11. 2026Fair Agents: Balancing Multistakeholder Alignment in Multi-Agent Personalization Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026How AI Coders Discuss, Disagree, and Reach Consensus: Challenges and Opportunities for LLM-Based Qualitative Coding
  2. 2025Multi-Agent Debate Strategies to Enhance Requirements Engineering with Large Language Models
  3. 2025Temperature and Persona Shape LLM Agent Consensus With Minimal Accuracy Gains in Qualitative Coding
  4. 2025Debate Only When Necessary: Adaptive Multiagent Collaboration for Efficient LLM Reasoning
  5. 2026Enhancing LLM Problem Solving via Tutor-Student Multi-Agent Interaction
  6. 2026From Diagnosis to Redesign: Using Quantitative Ethnography to Improve Multi-Agent LLM Reasoning
  7. 2025MARS: toward more efficient multi-agent collaboration for LLM reasoning
  8. 2025Literature Review Of Multi-Agent Debate For Problem-Solving