mlsanityData to 7 Oct 2026

Large Language Model based Multi-Agents: A Survey of Progress and Challenges

arXiv 2402.01680 ↗First posted 21 Jan 2024cs.CL cs.AI cs.MAcited by 752

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Large Language Models (LLMs) have achieved remarkable success across a wide array of tasks. Due to the impressive planning and reasoning abilities of LLMs, they have been used as autonomous agents to do many tasks automatically. Recently, based on the development of using one LLM as a single planning or decision-making agent, LLM-based multi-agent systems have achieved considerable progress in complex problem-solving and world simulation. To provide the community with an overview of this dynamic field, we present this survey to offer an in-depth discussion on the essential aspects of multi-agent systems based on LLMs, as well as the challenges. Our goal is for readers to gain substantial insights on the following questions: What domains and environments do LLM-based multi-agents simulate? How are these agents profiled and how do they communicate? What mechanisms contribute to the growth of agents' capacities? For those interested in delving into this field of study, we also summarize the commonly used datasets or benchmarks for them to have convenient access. To keep researchers updated on the latest studies, we maintain an open-source GitHub repository, dedicated to outlining the research on LLM-based multi-agent systems.

Key facts. Cited by 752 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 65 of its 87 references that are in the map. First posted on arXiv 21 Jan 2024.

Builds on

65 of 87 references.

Closest ancestors · 18

Same direction, newest first

  1. 2023Multi-Agent Consensus Seeking via Large Language Models31 Oct
  2. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents18 Oct
  3. 2023Theory of Mind for Multi-Agent Collaboration via Large Language Models16 Oct
  4. 2023OpenAgents: An Open Platform for Language Agents in the Wild16 Oct
  5. 2023Lyfe Agents: Generative agents for low-cost real-time social interactions3 Oct
  6. 2023A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration3 Oct
  7. 2023AutoAgents: A Framework for Automatic Agent Generation29 Sep
  8. 2023Agents: An Open-source Framework for Autonomous Language Agents14 Sep
  9. 2023The Rise and Potential of Large Language Model Based Agents: A Survey14 Sep
  10. 2023Cognitive Architectures for Language Agents5 Sep
  11. 2023A Survey on Large Language Model based Autonomous Agents22 Aug
  12. 2023ProAgent: Building Proactive Cooperative Agents with Large Language Models22 Aug
6 more
  1. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors21 Aug
  2. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate14 Aug
  3. 2023TPTU: Large Language Model-based AI Agents for Task Planning and Tool Usage7 Aug
  4. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug
  5. 2023The SocialAI School: Insights from Developmental Psychology Towards Artificial Socio-Cultural Agents15 Jul
  6. 2023Building Cooperative Embodied Agents Modularly with Large Language Models5 Jul

Other references · 47

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  6. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  8. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  9. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  10. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  11. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  12. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 574
35 more
  1. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
  2. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 506
  3. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 311
  4. 2022Social Simulacra: Creating Populated Prototypes for Social Computing Systemscited by 300
  5. 2023Self-collaboration Code Generation via ChatGPTcited by 255
  6. 2023Can Large Language Models Transform Computational Social Science?cited by 245
  7. 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 238
  8. 2023S3: Social-network Simulation System with Large Language Model-Empowered Agentscited by 228
  9. 2023Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology Viewcited by 221
  10. 2023Playing repeated games with Large Language Modelscited by 213
  11. 2023RoCo: Dialectic Multi-Robot Collaboration with Large Language Modelscited by 171
  12. 2023EconAgent: Large Language Model-Empowered Agents for Simulating Macroeconomic Activitiescited by 154
  13. 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 148
  14. 2023War and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Warscited by 143
  15. 2023AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systemscited by 116
  16. 2023Scalable Multi-Robot Collaboration with Large Language Models: Centralized or Decentralized Systems?cited by 104
  17. 2023Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplationcited by 102
  18. 2023Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debatecited by 94
  19. 2023Can Large Language Models Serve as Rational Players in Game Theory? A Systematic Analysiscited by 91
  20. 2023On Generative Agents in Recommendationcited by 88
  21. 2023MindAgent: Emergent Gaming Interactioncited by 87
  22. 2023MetaAgents: Large Language Model Based Agents for Decision-Making on Teamingcited by 85
  23. 2023Understanding the Benefits and Challenges of Using Large Language Model-based Conversational Agents for Mental Well-being Supportcited by 75
  24. 2023Collaborating with language models for embodied reasoningcited by 61
  25. 2023MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability, Rationality and Collaborationcited by 56
  26. 2023Epidemic Modeling with Generative Agentscited by 49
  27. 2023AvalonBench: Evaluating LLMs Playing the Game of Avaloncited by 45
  28. 2023Welfare Diplomacy: Benchmarking Language Model Cooperationcited by 43
  29. 2021Putting Humans in the Natural Language Processing Loop: A Surveycited by 40
  30. 2023Self-Adaptive Large Language Model (LLM)-Based Multiagent Systemscited by 31
  31. 2023Let GPT be a Math Tutor: Teaching Math Word Problem Solvers with Customized Exercise Generationcited by 27
  32. 2023Quantifying the Impact of Large Language Models on Collective Opinion Dynamicscited by 22
  33. 2023Simulating Public Administration Crisis: A Novel Generative Agent-Based Simulation System to Lower Technology Barriers in Social Science Researchcited by 21
  34. 2023Generative Agent-Based Modeling: Unveiling Social System Dynamics through Coupling Mechanistic Models with Generative Artificial Intelligencecited by 19
  35. 2023Are you in a Masquerade? Exploring the Behavior and Impact of Large Language Model Driven Social Bots in Online Social Networkscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 121; 2025: 334; 2026: 297.

752 papers.

Sort:

By area:

  1. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  2. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  3. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 354
  4. 2024Data Interpreter: An LLM Agent For Data Sciencecited by 222
  5. 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 221
  6. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 201
  7. 2024A Survey on Efficient Inference for Large Language Modelscited by 197
  8. 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agentscited by 196
  9. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  10. 2024Large Language Model-Based Agents for Software Engineering: A Surveycited by 173
  11. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  12. 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 146
38 more
  1. 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systemscited by 129
  2. 2025OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automationcited by 129
  3. 2024Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systemscited by 126
  4. 2025Towards a Science of Scaling Agent Systemscited by 125
  5. 2026SoK: Agentic Skills -- Beyond Tool Use in LLM Agentscited by 120
  6. 2025WebShaper: Agentically Data Synthesizing via Information-Seeking Formalizationcited by 118
  7. 2024Empowering Biomedical Discovery with AI Agentscited by 117
  8. 2024Emergent social conventions and collective bias in LLM populationscited by 115
  9. 2025G-Memory: Tracing Hierarchical Memory for Multi-Agent Systemscited by 112
  10. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  11. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  12. 2025A survey of agent interoperability protocols: Model Context Protocol (MCP), Agent Communication Protocol (ACP), Agent-to-Agent Protocol (A2A), and Agent Network Protocol (ANP)cited by 98
  13. 2024LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Modelscited by 97
  14. 2025Recursive Language Modelscited by 94
  15. 2024LLM Hallucinations in Practical Code Generation: Phenomena, Mechanism, and Mitigationcited by 92
  16. 2025Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solvingcited by 90
  17. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 86
  18. 2025Harnessing Multiple Large Language Models: A Survey on LLM Ensemblecited by 86
  19. 2025AgentNet: Decentralized Evolutionary Coordination for LLM-based Multi-Agent Systemscited by 86
  20. 2024AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoMLcited by 84
  21. 2025A Survey of AI Agent Protocolscited by 77
  22. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 76
  23. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 74
  24. 2025Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?cited by 74
  25. 2025Interactive Debugging and Steering of Multi-Agent AI Systemscited by 68
  26. 2024LLM-based Multi-Agent Reinforcement Learning: Current and Future Directionscited by 67
  27. 2025Latent Collaboration in Multi-Agent Systemscited by 67
  28. 2024Agent Planning with World Knowledge Modelcited by 66
  29. 2024SpeechVerse: A Large-scale Generalizable Audio Language Modelcited by 62
  30. 2024CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databasescited by 62
  31. 2024Agents' Room: Narrative Generation through Multi-step Collaborationcited by 57
  32. 2025FlowReasoner: Reinforcing Query-Level Meta-Agentscited by 56
  33. 2025Collaborative Memory: Multi-User Memory Sharing in LLM Agents with Dynamic Access Controlcited by 56
  34. 2024Determinants of LLM-assisted Decision-Makingcited by 55
  35. 2024Game-theoretic LLM: Agent Workflow for Negotiation Gamescited by 55
  36. 2025Cache-to-Cache: Direct Semantic Communication Between Large Language Modelscited by 54
  37. 2024VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledgecited by 50
  38. 2024(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Textscited by 49

50 most cited of 752 All 752 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024A Survey on LLM-based Multi-Agent System: Recent Advances and New Frontiers in Application
  2. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects
  3. 2024Understanding the planning of LLM agents: A survey
  4. 2025Literature Review Of Multi-Agent Debate For Problem-Solving
  5. 2024Practical Considerations for Agentic LLM Systems
  6. 2025A Survey on the Optimization of Large Language Model-based Agents
  7. 2025LLM-Based Human-Agent Collaboration and Interaction Systems: A Survey
  8. 2024LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead