mlsanityData to 14 Oct 2026

CoMM: Collaborative Multi-Agent, Multi-Reasoning-Path Prompting for Complex Problem Solving

arXiv 2404.17729 ↗First posted 26 Apr 2024cs.CLcited by 31

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Large Language Models (LLMs) have shown great ability in solving traditional natural language tasks and elementary reasoning tasks with appropriate prompting techniques. However, their ability is still limited in solving complicated science problems. In this work, we aim to push the upper bound of the reasoning capability of LLMs by proposing a collaborative multi-agent, multi-reasoning-path (CoMM) prompting framework. Specifically, we prompt LLMs to play different roles in a problem-solving team, and encourage different role-play agents to collaboratively solve the target task. In particular, we discover that applying different reasoning paths for different roles is an effective strategy to implement few-shot prompting approaches in the multi-agent scenarios. Empirical results demonstrate the effectiveness of the proposed methods on two college-level science problems over competitive baselines. Our further analysis shows the necessity of prompting LLMs to play different roles or experts independently. We release the code at: https://github.com/amazon-science/comm-prompt

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 44 of its 52 references that are in the map. First posted on arXiv 26 Apr 2024.

Builds on

44 of 52 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate30 May
  2. 2023Reasoning with Language Model is Planning with World Model24 May

Other references · 42

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  8. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  9. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  10. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  11. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  12. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
30 more
  1. 2023Mistral 7Bcited by 3,164
  2. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  3. 2022Emergent Abilities of Large Language Modelscited by 2,512
  4. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  5. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  6. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
  7. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  8. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  9. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
  10. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  11. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  12. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  13. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 742
  14. 2022Large Language Models Can Self-Improvecited by 664
  15. 2022PAL: Program-aided Language Modelscited by 652
  16. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
  17. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 431
  18. 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 307
  19. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 284
  20. 2023Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaborationcited by 251
  21. 2023Large Language Models Understand and Can be Enhanced by Emotional Stimulicited by 146
  22. 2023ExpertPrompting: Instructing Large Language Models to be Distinguished Expertscited by 142
  23. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 135
  24. 2023MathChat: Converse to Tackle Challenging Math Problems with LLM Agentscited by 68
  25. 2023PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documentscited by 61
  26. 2024Uncertainty Quantification for In-Context Learning of Large Language Modelscited by 45
  27. 2023Let's Do a Thought Experiment: Using Counterfactuals to Improve Moral Reasoningcited by 20
  28. 2022Few-Shot Character Understanding in Movies as an Assessment to Meta-Learning of Theory-of-Mindcited by 19
  29. 2024Fine-Grained Modeling of Narrative Context: A Coherence Perspective via Retrospective Questionscited by 11
  30. 2024Identifying Factual Inconsistencies in Summaries: Grounding LLM Inference via Task Taxonomycited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 19; 2026: 10.

31 papers.

Sort:

By area:

  1. 2024Scaling Large Language Model-based Multi-Agent Collaborationcited by 241
  2. 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 149
  3. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 111
  4. 2025A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Modelscited by 87
  5. 2025SiriuS: Self-improving Multi-agent Systems via Bootstrapped Reasoningcited by 44
  6. 2025Stop Overvaluing Multi-Agent Debate -- We Must Rethink Evaluation and Embrace Model Heterogeneitycited by 44
  7. 2025Talk Structurally, Act Hierarchically: A Collaborative Framework for LLM Multi-Agent Systemscited by 17
  8. 2025Learning from Peers in Reasoning Modelscited by 16
  9. 2024Enhancing LLM Reasoning with Multi-Path Collaborative Reactive and Reflection agentscited by 15
  10. 2025Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoningcited by 11
  11. 2025Synergizing Logical Reasoning, Knowledge Management and Collaboration in Multi-Agent LLM Systemcited by 10
  12. 2026The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMscited by 10
19 more
  1. 2025Aurelia: Test-time Reasoning Distillation in Audio-Visual LLMscited by 9
  2. 2025Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaborationcited by 7
  3. 2025IR-Agent: Expert-Inspired LLM Agents for Structure Elucidation from Infrared Spectracited by 5
  4. 2026EvoMAS: Evolutionary Generation of Multi-Agent Systemscited by 5
  5. 2026Evaluating Theory of Mind and Internal Beliefs in LLM-Based Multi-Agent Systemscited by 5
  6. 2025When and Why Does Multi-Agent Debate Fail and Does It Really Underperform?cited by 4
  7. 2025Trace-of-Thought Prompting: Investigating Prompt-Based Knowledge Distillation Through Question Decompositioncited by 3
  8. 2026Breaking the Martingale Curse: Multi-Agent Debate via Asymmetric Cognitive Potential Energycited by 3
  9. 2025Swarm Intelligence Enhanced Reasoning: A Density-Driven Framework for LLM-Based Multi-Agent Optimizationcited by 2
  10. 2025Collaborative QA using Interacting LLMs. Impact of Network Structure, Node Capability and Distributed Datacited by 2
  11. 2026Communicating about Space: Language-Mediated Spatial Integration Across Partial Viewscited by 2
  12. 2025RevoNAD: Reflective Evolutionary Exploration for Neural Architecture Designcited by 1
  13. 2026Weak-Link Optimization for Multi-Agent Reasoning and Collaborationcited by 1
  14. 2025A Flexible Multi-Agent LLM-Human Framework for Fast Human Validated Tool Buildingcited by 0
  15. 2025One Tool Is Enough: Reinforcement Learning for Repository-Level LLM Agentscited by 0
  16. 2026Process-Supervised Multi-Agent Reinforcement Learning for Reliable Clinical Reasoningcited by 0
  17. 2026DART: A DAG-Based Reputation and Incentive Framework via Blockchain-Enabled Governance for Trustworthy LLM Multi-Agent Collaborationcited by 0
  18. 2026SEPAL: Separated Expert Pairs with Answer-Level Fusion for Reliable LLM Collaborationcited by 0
  19. 2026Right Answers, Wrong States: Hidden Information Failures in Multi-Agent Collaborationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024LLM Harmony: Multi-Agent Communication for Problem Solving
  2. 2024Expanding Search Space with Diverse Prompting Agents: An Efficient Sampling Approach for LLM Mathematical Reasoning
  3. 2024Navigating Complexity: Orchestrated Problem Solving with Multi-Agent LLMs
  4. 2025Harnessing Multi-Agent LLMs for Complex Engineering Problem-Solving: A Framework for Senior Design Projects
  5. 2025Multi-LLM Collaborative Search for Complex Problem Solving
  6. 2023StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solving
  7. 2025Understanding LLM Scientific Reasoning through Promptings and Model's Explanation on the Answers
  8. 2026Enhancing LLM Problem Solving via Tutor-Student Multi-Agent Interaction