mlsanityData to 7 Oct 2026

Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?

arXiv 2402.18272 ↗First posted 28 Feb 2024cs.CL cs.AIcited by 153

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Recent progress in LLMs discussion suggests that multi-agent discussion improves the reasoning abilities of LLMs. In this work, we reevaluate this claim through systematic experiments, where we propose a novel group discussion framework to enrich the set of discussion mechanisms. Interestingly, our results show that a single-agent LLM with strong prompts can achieve almost the same performance as the best existing discussion approach on a wide range of reasoning tasks and backbone LLMs. We observe that the multi-agent discussion performs better than a single agent only when there is no demonstration in the prompt. Further study reveals the common interaction mechanisms of LLMs during the discussion.

Key facts. Cited by 153 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 25 of its 44 references that are in the map. First posted on arXiv 28 Feb 2024.

Builds on

25 of 44 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs22 Sep
  2. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate30 May
  3. 2023Teaching Large Language Models to Self-Debug11 Apr

Other references · 22

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  9. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  10. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  11. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  12. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
10 more
  1. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  2. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
  3. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 387
  4. 2023Deductive Verification of Chain-of-Thought Reasoningcited by 122
  5. 2022Successive Prompting for Decomposing Complex Questionscited by 94
  6. 2022Mind's Eye: Grounded Language Model Reasoning through Simulationcited by 83
  7. 2023RCOT: Detecting and Rectifying Factual Inconsistency in Reasoning by Reversing Chain-of-Thoughtcited by 47
  8. 2023Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinementcited by 43
  9. 2023Re-Reading Improves Reasoning in Large Language Modelscited by 37
  10. 2022LogicSolver: Towards Interpretable Math Word Problem Solving with Logical Prompt-enhanced Learningcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 26; 2025: 80; 2026: 47.

153 papers.

Sort:

By area:

  1. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  2. 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 420
  3. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 357
  4. 2024Scaling Large Language Model-based Multi-Agent Collaborationcited by 235
  5. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  6. 2024The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Surveycited by 164
  7. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  8. 2025Multi-Agent Design: Optimizing Agents with Better Prompts and Topologiescited by 106
  9. 2024Improving Multi-Agent Debate with Sparse Communication Topologycited by 103
  10. 2024From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Futurecited by 85
  11. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 74
  12. 2025Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?cited by 74
38 more
  1. 2024Reinforcement Learning Enhanced LLMs: A Surveycited by 69
  2. 2024Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent Systemcited by 63
  3. 2025Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifierscited by 58
  4. 2024Unlocking the Capabilities of Thought: A Reasoning Boundary Framework to Quantify and Optimize Chain-of-Thoughtcited by 56
  5. 2025Voting or Consensus? Decision-Making in Multi-Agent Debatecited by 55
  6. 2025OdysseyBench: Evaluating LLM Agents on Long-Horizon Complex Office Application Workflowscited by 54
  7. 2025Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?cited by 52
  8. 2025Stop Overvaluing Multi-Agent Debate -- We Must Rethink Evaluation and Embrace Model Heterogeneitycited by 43
  9. 2025SiriuS: Self-improving Multi-agent Systems via Bootstrapped Reasoningcited by 41
  10. 2025Hogwild! Inference: Parallel LLM Generation via Concurrent Attentioncited by 32
  11. 2025OSC: Cognitive Orchestration through Dynamic Knowledge Alignment in Multi-Agent LLM Collaborationcited by 29
  12. 2024Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligencecited by 27
  13. 2025LLMs Can Generate a Better Answer by Aggregating Their Own Responsescited by 27
  14. 2025MAS-ZERO: Designing Multi-Agent Systems with Zero Supervisioncited by 26
  15. 2025OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior Simulationcited by 26
  16. 2024Multi-Agent Large Language Models for Conversational Task-Solvingcited by 24
  17. 2024ACC-Collab: An Actor-Critic Approach to Multi-Agent LLM Collaborationcited by 22
  18. 2024Automate Strategy Finding with LLM in Quant Investmentcited by 21
  19. 2024Collective Critics for Creative Story Generationcited by 21
  20. 2025Do We Truly Need So Many Samples? Multi-LLM Repeated Sampling Efficiently Scales Test-Time Computecited by 20
  21. 2024Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challengescited by 18
  22. 2025Revisiting Multi-Agent Debate as Test-Time Scaling: A Systematic Study of Conditional Effectivenesscited by 18
  23. 2024Counterfactual Debating with Preset Stances for Hallucination Elimination of LLMscited by 17
  24. 2025Talk Structurally, Act Hierarchically: A Collaborative Framework for LLM Multi-Agent Systemscited by 17
  25. 2026Lying with Truths: Open-Channel Multi-Agent Collusion for Belief Manipulation via Generative Montagecited by 17
  26. 2026Silo-Bench: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM Systemscited by 16
  27. 2024Interactive Speculative Planning: Enhance Agent Efficiency through Co-design of System and User Interfacecited by 15
  28. 2025CompileAgent: Automated Real-World Repo-Level Compilation with Tool-Integrated LLM-based Agent Systemcited by 15
  29. 2025Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compressioncited by 15
  30. 2025MALLM: Multi-Agent Large Language Models Frameworkcited by 15
  31. 2025SHIELDA: Structured Handling of Exceptions in LLM-Driven Agentic Workflowscited by 12
  32. 2025The Hidden Strength of Disagreement: Unraveling the Consensus-Diversity Tradeoff in Adaptive Multi-Agent Systemscited by 11
  33. 2025Enhancing LLM Performance Through Debate: An Empirical Study on Multi-Agent Debate for Coding Taskscited by 11
  34. 2025Faithful, Unfaithful or Ambiguous? Multi-Agent Debate with Initial Stance for Summary Evaluationcited by 10
  35. 2025LLMs Working in Harmony: A Survey on the Technological Aspects of Building Effective LLM-Based Multi Agent Systemscited by 10
  36. 2025United Minds or Isolated Agents? Exploring Coordination of LLMs under Cognitive Load Theorycited by 10
  37. 2025POLO: Preference-Guided Multi-Turn Reinforcement Learning for Lead Optimizationcited by 10
  38. 2026The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMscited by 10

50 most cited of 153 All 153 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Can LLM Agents Really Debate? A Controlled Study of Multi-Agent Debate in Logical Reasoning
  2. 2023Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs
  3. 2023LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination Abilities in Large Language Models
  4. 2025Unlocking the Power of Multi-Agent LLM for Reasoning: From Lazy Agents to Deliberation
  5. 2025MAR:Multi-Agent Reflexion Improves Reasoning Abilities in LLMs
  6. 2024Enhancing LLM Reasoning with Multi-Path Collaborative Reactive and Reflection agents
  7. 2026The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modes
  8. 2025MARS: toward more efficient multi-agent collaboration for LLM reasoning