mlsanityData to 7 Oct 2026

A Survey on LLM-based Multi-Agent System: Recent Advances and New Frontiers in Application

arXiv 2412.17481 ↗First posted 23 Dec 2024cs.CL cs.MAcited by 54

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

LLM-based Multi-Agent Systems ( LLM-MAS ) have become a research hotspot since the rise of large language models (LLMs). However, with the continuous influx of new related works, the existing reviews struggle to capture them comprehensively. This paper presents a comprehensive survey of these studies. We first discuss the definition of LLM-MAS, a framework encompassing much of previous work. We provide an overview of the various applications of LLM-MAS in (i) solving complex tasks, (ii) simulating specific scenarios, and (iii) evaluating generative agents. Building on previous studies, we also highlight several challenges and propose future directions for research in this field.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 52 of its 82 references that are in the map. First posted on arXiv 23 Dec 2024.

Builds on

52 of 82 references.

Closest ancestors · 18

Same direction, newest first

  1. 2024Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System10 Oct
  2. 2024MuMA-ToM: Multi-modal Multi-Agent Theory of Mind22 Aug
  3. 2024Very Large-Scale Multi-Agent Simulation in AgentScope25 Jul
  4. 2024Synergistic Multi-Agent Framework with Trajectory Learning for Knowledge-Intensive Tasks13 Jul
  5. 2024Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligence9 Jul
  6. 2024Autonomous Agents for Collaborative Task under Information Asymmetry21 Jun
  7. 2024Multi-Agent Collaboration via Cross-Team Orchestration13 Jun
  8. 2024Scaling Large Language Model-based Multi-Agent Collaboration11 Jun
  9. 2024CoEvol: Constructing Better Responses for Instruction Finetuning through Multi-Agent Cooperation11 Jun
  10. 2024AgentScope: A Flexible yet Robust Multi-Agent Platform21 Feb
  11. 2024LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration18 Feb
  12. 2024Small LLMs Are Weak Tool Learners: A Multi-LLM Agent14 Jan
6 more
  1. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives19 Dec
  2. 2023AutoAgents: A Framework for Automatic Agent Generation29 Sep
  3. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate14 Aug
  4. 2023BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents11 Aug
  5. 2023AgentBench: Evaluating LLMs as Agents7 Aug
  6. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug

Other references · 34

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  5. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  6. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  7. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 407
  8. 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentationcited by 339
  9. 2022Social Simulacra: Creating Populated Prototypes for Social Computing Systemscited by 300
  10. 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 238
  11. 2023S3: Social-network Simulation System with Large Language Model-Empowered Agentscited by 228
  12. 2024Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agentscited by 210
22 more
  1. 2023User Behavior Simulation with Large Language Model based Agentscited by 136
  2. 2024Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agentscited by 120
  3. 2023AgentSims: An Open-Source Sandbox for Large Language Model Evaluationcited by 86
  4. 2023MetaAgents: Large Language Model Based Agents for Decision-Making on Teamingcited by 85
  5. 2023Put Your Money Where Your Mouth Is: Evaluating Strategic Planning and Execution of LLM Agents in an Auction Arenacited by 78
  6. 2023CompeteAI: Understanding the Competition Dynamics in Large Language Model-based Agentscited by 65
  7. 2023LLM-Based Agent Society Investigation: Collaboration and Confrontation in Avalon Gameplaycited by 62
  8. 2023MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability, Rationality and Collaborationcited by 56
  9. 2024Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agentcited by 52
  10. 2024MACM: Utilizing a Multi-Agent System for Condition Mining in Solving Complex Mathematical Problemscited by 41
  11. 2023Urban Generative Intelligence (UGI): A Foundational Platform for Agents in Embodied City Environmentcited by 37
  12. 2024AgentCourt: Simulating Court with Adversarial Evolvable Lawyer Agentscited by 36
  13. 2023Skill Reinforcement Learning and Planning for Open-World Long-Horizon Taskscited by 35
  14. 2024CodeAgent: Autonomous Communicative Agents for Code Reviewcited by 33
  15. 2024DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Servingcited by 30
  16. 2023Wireless Multi-Agent Generative AI: From Connected Intelligence to Collective Intelligencecited by 25
  17. 2024AdaSociety: An Adaptive Environment with Social Structures for Multi-Agent Decision-Makingcited by 19
  18. 2024Debate, Deliberate, Decide (D3): A Cost-Aware Adversarial Framework for Reliable and Interpretable LLM Evaluationcited by 14
  19. 2024An Electoral Approach to Diversify LLM-based Multi-Agent Collective Decision-Makingcited by 14
  20. 2023Computational Agent-based Models in Opinion Dynamics: A Survey on Social Simulations and Empirical Studiescited by 9
  21. 2024MAO: A Framework for Process Model Generation with Multi-Agent Orchestrationcited by 8
  22. 2024PEAR: A Robust and Flexible Automation Framework for Ptychography Enabled by Multiple Large Language Model Agentscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 25; 2026: 29.

54 papers.

Sort:

By area:

  1. 2025TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systemscited by 58
  2. 2025Agentic AI: A Comprehensive Survey of Architectures, Applications, and Future Directionscited by 46
  3. 2025A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectivescited by 27
  4. 2025OptimAI: Optimization from Natural Language Using LLM-Powered AI Agentscited by 22
  5. 2026From Static Templates to Dynamic Runtime Graphs: A Survey of Workflow Optimization for LLM Agentscited by 22
  6. 2026Seeing the Whole Elephant: A Benchmark for Failure Attribution in LLM-based Multi-Agent Systemscited by 22
  7. 2025SABER: Small Actions, Big Errors -- Safeguarding Mutating Steps in LLM Agentscited by 21
  8. 2025Multi-Agent Deep Research: Training Multi-Agent Systems with M-GRPOcited by 17
  9. 2025LLMs Can't Handle Peer Pressure: Crumbling under Multi-Agent Social Interactionscited by 14
  10. 2025SHIELDA: Structured Handling of Exceptions in LLM-Driven Agentic Workflowscited by 12
  11. 2026The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestrationcited by 11
  12. 2025BAMAS: Structuring Budget-Aware Multi-Agent Systemscited by 5
38 more
  1. 2026MonoScale: Scaling Multi-Agent System with Monotonic Improvementcited by 5
  2. 2025Learning to Be A Doctor: Searching for Effective Medical Agent Architecturescited by 4
  3. 2025Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectivescited by 4
  4. 2025Adaptive GPU Resource Allocation for Multi-Agent Collaborative Reasoning in Serverless Environmentscited by 4
  5. 2026Reinforcement Learning for LLM-based Multi-Agent Systems through Orchestration Tracescited by 4
  6. 2026MAS-PromptBench: When Does Prompt Optimization Improve Multi-Agent LLM Systems?cited by 4
  7. 2025Designing LLM-based Multi-Agent Systems for Software Engineering Tasks: Quality Attributes, Design Patterns and Rationalecited by 3
  8. 2025MARINE: Theoretical Optimization and Design for Multi-Agent Recursive IN-context Enhancementcited by 3
  9. 2026MOC: Multi-Order Communication in LLM-based Multi-Agent Systemscited by 3
  10. 2025Agentic AutoSurvey: Let LLMs Survey LLMscited by 2
  11. 2025Static Sandboxes Are Inadequate: Modeling Societal Complexity Requires Open-Ended Co-Evolution in LLM-Based Multi-Agent Simulationscited by 2
  12. 2025Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agentscited by 2
  13. 2025AgentODRL: A Large Language Model-based Multi-agent System for ODRL Generationcited by 2
  14. 2025Augmented Runtime Collaboration for Self-Organizing Multi-Agent Systems: A Hybrid Bi-Criteria Routing Approachcited by 2
  15. 2026OpenClaw-Skill: Collective Skill Tree Search for Agentic Large Language Modelscited by 2
  16. 2025Neo-Grounded Theory: A Methodological Innovation Integrating High-Dimensional Vector Clustering and Multi-Agent Collaboration for Qualitative Researchcited by 1
  17. 2026ClinicalReTrial: Clinical Trial Redesign with Self-Evolving Agentscited by 1
  18. 2026Many Hands Make Light Work: An LLM-based Multi-Agent System for Detecting Malicious PyPI Packagescited by 1
  19. 2026VLM-DEWM: Dynamic External World Model for Verifiable and Resilient Vision-Language Planning in Manufacturingcited by 1
  20. 2026Unified-MAS: Universally Generating Domain-Specific Nodes for Empowering Automatic Multi-Agent Systemscited by 1
  21. 2026Can LLM Agents Sustain Long-Horizon Organizational Dynamics?cited by 1
  22. 2026Blockchain Empowered Trustworthy Agent Networks: Foundations, Taxonomy, and Future Directionscited by 1
  23. 2026Position: Multi-Agent Systems Should Prioritize Concurrency Controlcited by 1
  24. 2025Perceiving, Reasoning, Adapting: A Dual-Layer Framework for VLM-Guided Precision Robotic Manipulationcited by 0
  25. 2025From Unstructured Communication to Intelligent RAG: Multi-Agent Automation for Supply Chain Knowledge Basescited by 0
  26. 2025Tractable Asymmetric Verification for Large Language Models via Deterministic Replicabilitycited by 0
  27. 2025MetaboT: An LLM-based Multi-Agent Frameworkfor Interactive Analysis of Mass SpectrometryMetabolomics Knowledge Graphscited by 0
  28. 2025IMAGINE: Integrating Multi-Agent System into One Model for Complex Reasoning and Planningcited by 0
  29. 2026Exploring Implicit Perspectives on Autism in Large Language Models Through Multi-Agent Simulationscited by 0
  30. 2026Agentic AI-based Coverage Closure for Formal Verificationcited by 0
  31. 2026Optimizing Multi-Agent Weather Captioning via Text Gradient Descent: A Training-Free Approach with Consensus-Aware Gradient Fusioncited by 0
  32. 2026Agent Manufacturing: Foundation-Model Agents as First-Class Industrial Entitiescited by 0
  33. 2026Multi-Agent Systems are Mixtures of Experts: Who Becomes an Influencer?cited by 0
  34. 2026When Cloud Agents Meet Device Agents: Lessons from Hybrid Multi-Agent Systemscited by 0
  35. 2026Toward Human-Centered Multi-Agent Systems: Integrating Cognition, Culture, Values, and Cooperation in AI Agentscited by 0
  36. 2026AdaSTORM: Scaling LLM Reasoning on Dynamic Graphs via Adaptive Spatio-Temporal Multi-Agent Collaborationcited by 0
  37. 2026Clarus: Coordinating Autonomous Research Agents toward Web-Scale Scientific Collaborationcited by 0
  38. 2026When Do Multi-Agent Systems Help? An Information Bottleneck Perspectivecited by 0

50 most cited of 54 All 54 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges
  2. 2026LLM-Based Multi-Agent Systems for Code Generation: A Multi-Vocal Literature Review
  3. 2023A Survey on Large Language Model based Autonomous Agents
  4. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects
  5. 2025LLM-Based Human-Agent Collaboration and Interaction Systems: A Survey
  6. 2025Contemporary Agent Technology: LLM-Driven Advancements vs Classic Multi-Agent Systems
  7. 2024LLM-based Multi-Agent Reinforcement Learning: Current and Future Directions
  8. 2025LLMs Working in Harmony: A Survey on the Technological Aspects of Building Effective LLM-Based Multi Agent Systems