# Enhancing LLM Performance Through Debate: An Empirical Study on Multi-Agent Debate for Coding Tasks

arXiv: https://arxiv.org/abs/2503.12029 · HTML: https://mlsanity.com/paper/2503.12029/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 46 of its 90 references that are in the map.
- First posted on arXiv 15 Mar 2025.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 30 of 46)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019)
- [Universal Sentence Encoder](https://mlsanity.com/paper/1803.11175/) (2018)
- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution](https://mlsanity.com/paper/2401.03065/) (2024)
- [Debating with More Persuasive LLMs Leads to More Truthful Answers](https://mlsanity.com/paper/2402.06782/) (2024)
- [CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Tasks](https://mlsanity.com/paper/2105.12655/) (2021)
- [Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?](https://mlsanity.com/paper/2402.18272/) (2024)
- [LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead](https://mlsanity.com/paper/2404.04834/) (2024)
- [Few-shot training LLMs for project-specific code-summarization](https://mlsanity.com/paper/2207.04237/) (2022)
- [Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs](https://mlsanity.com/paper/2311.17371/) (2023)
- [Improving Multi-Agent Debate with Sparse Communication Topology](https://mlsanity.com/paper/2406.11776/) (2024)
- [Improving Automatic Source Code Summarization via Deep Reinforcement Learning](https://mlsanity.com/paper/1811.07234/) (2018)
- [Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate](https://mlsanity.com/paper/2305.11595/) (2023)
- [Leveraging Automated Unit Tests for Unsupervised Code Translation](https://mlsanity.com/paper/2110.06773/) (2021)
- [Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization)](https://mlsanity.com/paper/2304.06815/) (2023)
- [Towards an Understanding of Large Language Models in Software Engineering Tasks](https://mlsanity.com/paper/2308.11396/) (2023)
- [AVATAR: A Parallel Corpus for Java-Python Program Translation](https://mlsanity.com/paper/2108.11590/) (2021)
- [Voting or Consensus? Decision-Making in Multi-Agent Debate](https://mlsanity.com/paper/2502.19130/) (2025)

## Built on by (most cited first, 11 of 11)

- [Free-MAD: Consensus-Free Multi-Agent Debate](https://mlsanity.com/paper/2509.11035/) (2025)
- [Can LLM Agents Really Debate? A Controlled Study of Multi-Agent Debate in Logical Reasoning](https://mlsanity.com/paper/2511.07784/) (2025)
- [Revisiting Multi-Agent Debate as Test-Time Scaling: A Systematic Study of Conditional Effectiveness](https://mlsanity.com/paper/2505.22960/) (2025)
- [Byzantine-Robust Decentralized Coordination of LLM Agents](https://mlsanity.com/paper/2507.14928/) (2025)
- [The Effect of Belief Boxes and Open-mindedness on Persuasion](https://mlsanity.com/paper/2512.06573/) (2025)
- [Multi-Agent Debate Strategies: Survey, Taxonomy, and Challenges](https://mlsanity.com/paper/2607.26212/) (2026)
- [Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Model](https://mlsanity.com/paper/2608.13867/) (2026)
- [CRAwDAD: Causal Reasoning Augmentation with Dual-Agent Debate](https://mlsanity.com/paper/2511.22854/) (2025)
- [What Do Agents Communicate? Characterizing Information Exchange in Multi-Agent Systems](https://mlsanity.com/paper/2605.20548/) (2026)
- [Understanding Conversational Patterns in Multi-agent Programming: A Case Study on Fibonacci Game Development](https://mlsanity.com/paper/2605.24138/) (2026)
- [Fair Agents: Balancing Multistakeholder Alignment in Multi-Agent Personalization Systems](https://mlsanity.com/paper/2605.02379/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have advanced autonomous agents' planning and decision-making, yet they struggle with complex tasks requiring diverse expertise and multi-step reasoning. Multi-Agent Debate (MAD) systems, introduced in NLP research, address this gap by enabling structured debates among LLM-based agents to refine solutions iteratively. MAD promotes divergent thinking through role-specific agents, dynamic interactions, and structured decision-making. Recognizing parallels between Software Engineering (SE) and collaborative human problem-solving, this study investigates MAD's effectiveness on four coding tasks in SE. We adapt a MAD framework from NLP, analyze agent interactions to assess consensus-building and iterative refinement, and propose two MAD variants that enhance agent debate for coding tasks by addressing the observed weaknesses. Our findings show that structured debate and collaboration improve problem-solving and yield strong performance in some cases, highlighting the collaborative debate synergy between LLM agents for coding tasks in SE while identifying areas for future exploration.