mlsanityData to 14 Oct 2026

LLM Harmony: Multi-Agent Communication for Problem Solving

arXiv 2401.01312 ↗First posted 2 Jan 2024cs.MAcited by 35

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Large Language Models (LLMs) have revolutionized Natural Language Processing but exhibit limitations, particularly in autonomously addressing novel challenges such as reasoning and problem-solving. Traditional techniques like chain-of-thought prompting necessitate explicit human guidance. This paper introduces a novel multi-agent communication framework, inspired by the CAMEL model, to enhance LLMs' autonomous problem-solving capabilities. The framework employs multiple LLM agents, each with a distinct persona, engaged in role-playing communication, offering a nuanced and adaptable approach to diverse problem scenarios. Extensive experimentation demonstrates the framework's superior performance and adaptability, providing valuable insights into the collaborative potential of multiple agents in overcoming the limitations of individual models.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 42 of its 59 references that are in the map. First posted on arXiv 2 Jan 2024.

Builds on

42 of 59 references.

References · 42

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  9. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  10. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  11. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  12. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
30 more
  1. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  2. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  3. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  4. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  5. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
  6. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  7. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  8. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 662
  9. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  10. 2022Self-critiquing models for assisting human evaluatorscited by 380
  11. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  12. 2016Solving General Arithmetic Word Problemscited by 360
  13. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 356
  14. 2020Evaluation of Text Generation: A Surveycited by 277
  15. 2017Why We Need New Evaluation Metrics for NLGcited by 233
  16. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
  17. 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 182
  18. 2020Open Problems in Cooperative AIcited by 168
  19. 2023LM vs LM: Detecting Factual Errors via Cross Examinationcited by 156
  20. 2023Human-like Summarization Evaluation with ChatGPTcited by 126
  21. 2021The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generationcited by 97
  22. 2023Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Trackercited by 95
  23. 2020Multi-agent Communication meets Natural Language: Synergies between Functional and Structural Language Learningcited by 66
  24. 2023Large Language Models are Diverse Role-Players for Summarization Evaluationcited by 62
  25. 2021What Would Jiminy Cricket Do? Towards Agents That Behave Morallycited by 57
  26. 2018Semantically-Aligned Equation Generation for Solving and Reasoning Math Word Problemscited by 49
  27. 2019Emergent Linguistic Phenomena in Multi-Agent Communication Gamescited by 36
  28. 2017Emergent Translation in Multi-Agent Communicationcited by 22
  29. 2022I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragonscited by 22
  30. 2023Beyond Segmentation: Road Network Generation with Multi-Modal LLMscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 11; 2025: 13; 2026: 11.

35 papers.

Sort:

By area:

  1. 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systemscited by 141
  2. 2024Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Modelscited by 105
  3. 2024NetSafe: Exploring the Topological Safety of Multi-agent Networkscited by 29
  4. 2024Social Science Meets LLMs: How Reliable Are Large Language Models in Social Simulations?cited by 26
  5. 2024ACC-Collab: An Actor-Critic Approach to Multi-Agent LLM Collaborationcited by 24
  6. 2025EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safetycited by 21
  7. 2025Talk Structurally, Act Hierarchically: A Collaborative Framework for LLM Multi-Agent Systemscited by 17
  8. 2024Enhancing LLM Reasoning with Multi-Path Collaborative Reactive and Reflection agentscited by 15
  9. 2025A Survey of Scaling in Large Language Model Reasoningcited by 10
  10. 2026LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agentscited by 10
  11. 2024LLM Based Multi-Agent Generation of Semi-structured Documents from Semantic Templates in the Public Administration Domaincited by 9
  12. 2025Large Language Model Based Multi-Agent System Augmented Complex Event Processing Pipeline for Internet of Multimedia Thingscited by 8
23 more
  1. 2024Multi-Agent Planning Using Visual Language Modelscited by 7
  2. 2025"X of Information'' Continuum: A Survey on AI-Driven Multi-dimensional Metrics for Next-Generation Networked Systemscited by 7
  3. 2026Belief in Authority: Impact of Authority in Multi-Agent Evaluation Frameworkcited by 6
  4. 2025Town Hall Debate Prompting: Enhancing Logical Reasoning in LLMs through Multi-Persona Interactioncited by 5
  5. 2024Navigating Complexity: Orchestrated Problem Solving with Multi-Agent LLMscited by 4
  6. 2025RADAR: A Risk-Aware Dynamic Multi-Agent Framework for LLM Safety Evaluation via Role-Specialized Collaborationcited by 4
  7. 2024Optimal Decision Making Through Scenario Simulations Using Large Language Modelscited by 3
  8. 2024EMPOWER: Embodied Multi-role Open-vocabulary Planning with Online Grounding and Executioncited by 3
  9. 2025BELLE: A Bi-Level Multi-Agent Reasoning Framework for Multi-Hop Question Answeringcited by 3
  10. 2025AgentCoMa: A Compositional Benchmark Mixing Commonsense and Mathematical Reasoning in Real-World Scenarioscited by 3
  11. 2026CommCP: Efficient Multi-Agent Coordination via LLM-Based Communication with Conformal Predictioncited by 3
  12. 2026MHSafeEval: Role-Aware Interaction-Level Evaluation of Mental Health Safety in Large Language Modelscited by 3
  13. 2025Adaptive Multi-Agent Response Refinement in Conversational Systemscited by 2
  14. 2026Multi-Agent Coordination Adaptation via Structure-Guided Orchestrationcited by 2
  15. 2026Misinformation Propagation in Benign Multi-Agent Systemscited by 2
  16. 2025SOCIA-nabla: Textual Gradient Meets Multi-Agent Orchestration for Automated Simulator Generationcited by 1
  17. 2025Evaluating LLMs and Prompting Strategies for Automated Hardware Diagnosis from Textual User-Reportscited by 1
  18. 2026MA-CoNav: A Master-Slave Multi-Agent Framework with Hierarchical Collaboration and Dual-Level Reflection for Long-Horizon Embodied VLNcited by 1
  19. 2025SOCIA-Nabla: Textual Gradient Meets Multi-Agent Orchestration for Automated Simulator Generationcited by 0
  20. 2026Predictive Batch Scheduling: Accelerating Language Model Training Through Loss-Aware Sample Prioritizationcited by 0
  21. 2026SOM: Structured Opponent Modeling for LLM-based Agents via Structural Causal Modelcited by 0
  22. 2026Automated Mediator for Human Negotiation: Pre-Mediation via a Structured LLM Pipelinecited by 0
  23. 2026Swarm Telepathy: Prefill-Free KV-Cache Transfer Across Familiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Literature Review Of Multi-Agent Debate For Problem-Solving
  2. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges
  3. 2024Synergistic Simulations: Multi-Agent Problem Solving with Large Language Models
  4. 2025Grounding Natural Language for Multi-agent Decision-Making with Multi-agentic LLMs
  5. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects
  6. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society
  7. 2025LLMs Working in Harmony: A Survey on the Technological Aspects of Building Effective LLM-Based Multi Agent Systems
  8. 2023LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination Abilities in Large Language Models