# LLM Harmony: Multi-Agent Communication for Problem Solving

arXiv: https://arxiv.org/abs/2401.01312 · HTML: https://mlsanity.com/paper/2401.01312/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 42 of its 59 references that are in the map.
- First posted on arXiv 2 Jan 2024.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 30 of 42)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [Are NLP Models really able to Solve Simple Math Word Problems?](https://mlsanity.com/paper/2103.07191/) (2021)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms](https://mlsanity.com/paper/1905.13319/) (2019)
- [Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems](https://mlsanity.com/paper/1705.04146/) (2017)
- [Self-critiquing models for assisting human evaluators](https://mlsanity.com/paper/2206.05802/) (2022)
- [Explain Yourself! Leveraging Language Models for Commonsense Reasoning](https://mlsanity.com/paper/1906.02361/) (2019)
- [Solving General Arithmetic Word Problems](https://mlsanity.com/paper/1608.01413/) (2016)
- [Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models](https://mlsanity.com/paper/2304.09842/) (2023)
- [Evaluation of Text Generation: A Survey](https://mlsanity.com/paper/2006.14799/) (2020)
- [Why We Need New Evaluation Metrics for NLG](https://mlsanity.com/paper/1707.06875/) (2017)
- [Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs](https://mlsanity.com/paper/2210.13312/) (2022)
- [Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark](https://mlsanity.com/paper/2304.03279/) (2023)
- [Open Problems in Cooperative AI](https://mlsanity.com/paper/2012.08630/) (2020)

## Built on by (most cited first, 30 of 35)

- [Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2410.02506/) (2024)
- [Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models](https://mlsanity.com/paper/2402.04614/) (2024)
- [NetSafe: Exploring the Topological Safety of Multi-agent Networks](https://mlsanity.com/paper/2410.15686/) (2024)
- [Social Science Meets LLMs: How Reliable Are Large Language Models in Social Simulations?](https://mlsanity.com/paper/2410.23426/) (2024)
- [ACC-Collab: An Actor-Critic Approach to Multi-Agent LLM Collaboration](https://mlsanity.com/paper/2411.00053/) (2024)
- [EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safety](https://mlsanity.com/paper/2504.09689/) (2025)
- [Talk Structurally, Act Hierarchically: A Collaborative Framework for LLM Multi-Agent Systems](https://mlsanity.com/paper/2502.11098/) (2025)
- [Enhancing LLM Reasoning with Multi-Path Collaborative Reactive and Reflection agents](https://mlsanity.com/paper/2501.00430/) (2024)
- [A Survey of Scaling in Large Language Model Reasoning](https://mlsanity.com/paper/2504.02181/) (2025)
- [LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agents](https://mlsanity.com/paper/2602.01053/) (2026)
- [LLM Based Multi-Agent Generation of Semi-structured Documents from Semantic Templates in the Public Administration Domain](https://mlsanity.com/paper/2402.14871/) (2024)
- [Large Language Model Based Multi-Agent System Augmented Complex Event Processing Pipeline for Internet of Multimedia Things](https://mlsanity.com/paper/2501.00906/) (2025)
- [Multi-Agent Planning Using Visual Language Models](https://mlsanity.com/paper/2408.05478/) (2024)
- ["X of Information'' Continuum: A Survey on AI-Driven Multi-dimensional Metrics for Next-Generation Networked Systems](https://mlsanity.com/paper/2507.19657/) (2025)
- [Belief in Authority: Impact of Authority in Multi-Agent Evaluation Framework](https://mlsanity.com/paper/2601.04790/) (2026)
- [Town Hall Debate Prompting: Enhancing Logical Reasoning in LLMs through Multi-Persona Interaction](https://mlsanity.com/paper/2502.15725/) (2025)
- [Navigating Complexity: Orchestrated Problem Solving with Multi-Agent LLMs](https://mlsanity.com/paper/2402.16713/) (2024)
- [RADAR: A Risk-Aware Dynamic Multi-Agent Framework for LLM Safety Evaluation via Role-Specialized Collaboration](https://mlsanity.com/paper/2509.25271/) (2025)
- [Optimal Decision Making Through Scenario Simulations Using Large Language Models](https://mlsanity.com/paper/2407.06486/) (2024)
- [EMPOWER: Embodied Multi-role Open-vocabulary Planning with Online Grounding and Execution](https://mlsanity.com/paper/2408.17379/) (2024)
- [BELLE: A Bi-Level Multi-Agent Reasoning Framework for Multi-Hop Question Answering](https://mlsanity.com/paper/2505.11811/) (2025)
- [AgentCoMa: A Compositional Benchmark Mixing Commonsense and Mathematical Reasoning in Real-World Scenarios](https://mlsanity.com/paper/2508.19988/) (2025)
- [CommCP: Efficient Multi-Agent Coordination via LLM-Based Communication with Conformal Prediction](https://mlsanity.com/paper/2602.06038/) (2026)
- [MHSafeEval: Role-Aware Interaction-Level Evaluation of Mental Health Safety in Large Language Models](https://mlsanity.com/paper/2604.17730/) (2026)
- [Adaptive Multi-Agent Response Refinement in Conversational Systems](https://mlsanity.com/paper/2511.08319/) (2025)
- [Multi-Agent Coordination Adaptation via Structure-Guided Orchestration](https://mlsanity.com/paper/2605.25746/) (2026)
- [Misinformation Propagation in Benign Multi-Agent Systems](https://mlsanity.com/paper/2606.16710/) (2026)
- [SOCIA-nabla: Textual Gradient Meets Multi-Agent Orchestration for Automated Simulator Generation](https://mlsanity.com/paper/2505.12006/) (2025)
- [Evaluating LLMs and Prompting Strategies for Automated Hardware Diagnosis from Textual User-Reports](https://mlsanity.com/paper/2507.00742/) (2025)
- [MA-CoNav: A Master-Slave Multi-Agent Framework with Hierarchical Collaboration and Dual-Level Reflection for Long-Horizon Embodied VLN](https://mlsanity.com/paper/2603.03024/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have revolutionized Natural Language Processing but exhibit limitations, particularly in autonomously addressing novel challenges such as reasoning and problem-solving. Traditional techniques like chain-of-thought prompting necessitate explicit human guidance. This paper introduces a novel multi-agent communication framework, inspired by the CAMEL model, to enhance LLMs' autonomous problem-solving capabilities. The framework employs multiple LLM agents, each with a distinct persona, engaged in role-playing communication, offering a nuanced and adaptable approach to diverse problem scenarios. Extensive experimentation demonstrates the framework's superior performance and adaptability, providing valuable insights into the collaborative potential of multiple agents in overcoming the limitations of individual models.