# A Survey on LLM-based Multi-Agent System: Recent Advances and New Frontiers in Application

arXiv: https://arxiv.org/abs/2412.17481 · HTML: https://mlsanity.com/paper/2412.17481/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 52 of its 82 references that are in the map.
- First posted on arXiv 23 Dec 2024.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 30 of 52)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate](https://mlsanity.com/paper/2308.07201/) (2023)
- [AppAgent: Multimodal Agents as Smartphone Users](https://mlsanity.com/paper/2312.13771/) (2023)
- [MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation](https://mlsanity.com/paper/2310.03302/) (2023)
- [Social Simulacra: Creating Populated Prototypes for Social Computing Systems](https://mlsanity.com/paper/2208.04024/) (2022)
- [Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives](https://mlsanity.com/paper/2312.11970/) (2023)
- [Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf](https://mlsanity.com/paper/2309.04658/) (2023)
- [Scaling Large Language Model-based Multi-Agent Collaboration](https://mlsanity.com/paper/2406.07155/) (2024)
- [S3: Social-network Simulation System with Large Language Model-Empowered Agents](https://mlsanity.com/paper/2307.14984/) (2023)
- [Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents](https://mlsanity.com/paper/2405.02957/) (2024)
- [AutoAgents: A Framework for Automatic Agent Generation](https://mlsanity.com/paper/2309.17288/) (2023)
- [User Behavior Simulation with Large Language Model based Agents](https://mlsanity.com/paper/2306.02552/) (2023)
- [Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents](https://mlsanity.com/paper/2404.16698/) (2024)
- [AgentScope: A Flexible yet Robust Multi-Agent Platform](https://mlsanity.com/paper/2402.14034/) (2024)
- [Small LLMs Are Weak Tool Learners: A Multi-LLM Agent](https://mlsanity.com/paper/2401.07324/) (2024)
- [AgentSims: An Open-Source Sandbox for Large Language Model Evaluation](https://mlsanity.com/paper/2308.04026/) (2023)
- [Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligence](https://mlsanity.com/paper/2407.07061/) (2024)
- [MetaAgents: Large Language Model Based Agents for Decision-Making on Teaming](https://mlsanity.com/paper/2310.06500/) (2023)
- [BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents](https://mlsanity.com/paper/2308.05960/) (2023)
- [Put Your Money Where Your Mouth Is: Evaluating Strategic Planning and Execution of LLM Agents in an Auction Arena](https://mlsanity.com/paper/2310.05746/) (2023)
- [CompeteAI: Understanding the Competition Dynamics in Large Language Model-based Agents](https://mlsanity.com/paper/2310.17512/) (2023)
- [LLM-Based Agent Society Investigation: Collaboration and Confrontation in Avalon Gameplay](https://mlsanity.com/paper/2310.14985/) (2023)
- [MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability, Rationality and Collaboration](https://mlsanity.com/paper/2311.08562/) (2023)

## Built on by (most cited first, 30 of 55)

- [TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systems](https://mlsanity.com/paper/2506.04133/) (2025)
- [Agentic AI: A Comprehensive Survey of Architectures, Applications, and Future Directions](https://mlsanity.com/paper/2510.25445/) (2025)
- [A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectives](https://mlsanity.com/paper/2503.13415/) (2025)
- [Seeing the Whole Elephant: A Benchmark for Failure Attribution in LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2604.22708/) (2026)
- [From Static Templates to Dynamic Runtime Graphs: A Survey of Workflow Optimization for LLM Agents](https://mlsanity.com/paper/2603.22386/) (2026)
- [OptimAI: Optimization from Natural Language Using LLM-Powered AI Agents](https://mlsanity.com/paper/2504.16918/) (2025)
- [SABER: Small Actions, Big Errors -- Safeguarding Mutating Steps in LLM Agents](https://mlsanity.com/paper/2512.07850/) (2025)
- [Multi-Agent Deep Research: Training Multi-Agent Systems with M-GRPO](https://mlsanity.com/paper/2511.13288/) (2025)
- [LLMs Can't Handle Peer Pressure: Crumbling under Multi-Agent Social Interactions](https://mlsanity.com/paper/2508.18321/) (2025)
- [The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestration](https://mlsanity.com/paper/2603.22862/) (2026)
- [SHIELDA: Structured Handling of Exceptions in LLM-Driven Agentic Workflows](https://mlsanity.com/paper/2508.07935/) (2025)
- [BAMAS: Structuring Budget-Aware Multi-Agent Systems](https://mlsanity.com/paper/2511.21572/) (2025)
- [Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectives](https://mlsanity.com/paper/2506.09656/) (2025)
- [MonoScale: Scaling Multi-Agent System with Monotonic Improvement](https://mlsanity.com/paper/2601.23219/) (2026)
- [Reinforcement Learning for LLM-based Multi-Agent Systems through Orchestration Traces](https://mlsanity.com/paper/2605.02801/) (2026)
- [MAS-PromptBench: When Does Prompt Optimization Improve Multi-Agent LLM Systems?](https://mlsanity.com/paper/2606.23664/) (2026)
- [Learning to Be A Doctor: Searching for Effective Medical Agent Architectures](https://mlsanity.com/paper/2504.11301/) (2025)
- [Agentic AutoSurvey: Let LLMs Survey LLMs](https://mlsanity.com/paper/2509.18661/) (2025)
- [Adaptive GPU Resource Allocation for Multi-Agent Collaborative Reasoning in Serverless Environments](https://mlsanity.com/paper/2512.22149/) (2025)
- [Unified-MAS: Universally Generating Domain-Specific Nodes for Empowering Automatic Multi-Agent Systems](https://mlsanity.com/paper/2603.21475/) (2026)
- [MOC: Multi-Order Communication in LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2606.02359/) (2026)
- [Designing LLM-based Multi-Agent Systems for Software Engineering Tasks: Quality Attributes, Design Patterns and Rationale](https://mlsanity.com/paper/2511.08475/) (2025)
- [MARINE: Theoretical Optimization and Design for Multi-Agent Recursive IN-context Enhancement](https://mlsanity.com/paper/2512.07898/) (2025)
- [Static Sandboxes Are Inadequate: Modeling Societal Complexity Requires Open-Ended Co-Evolution in LLM-Based Multi-Agent Simulations](https://mlsanity.com/paper/2510.13982/) (2025)
- [Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents](https://mlsanity.com/paper/2510.17491/) (2025)
- [AgentODRL: A Large Language Model-based Multi-agent System for ODRL Generation](https://mlsanity.com/paper/2512.00602/) (2025)
- [Augmented Runtime Collaboration for Self-Organizing Multi-Agent Systems: A Hybrid Bi-Criteria Routing Approach](https://mlsanity.com/paper/2512.00740/) (2025)
- [OpenClaw-Skill: Collective Skill Tree Search for Agentic Large Language Models](https://mlsanity.com/paper/2606.16774/) (2026)
- [Neo-Grounded Theory: A Methodological Innovation Integrating High-Dimensional Vector Clustering and Multi-Agent Collaboration for Qualitative Research](https://mlsanity.com/paper/2509.25244/) (2025)
- [ClinicalReTrial: Clinical Trial Redesign with Self-Evolving Agents](https://mlsanity.com/paper/2601.00290/) (2026)

## Abstract (arXiv)

LLM-based Multi-Agent Systems ( LLM-MAS ) have become a research hotspot since the rise of large language models (LLMs). However, with the continuous influx of new related works, the existing reviews struggle to capture them comprehensively. This paper presents a comprehensive survey of these studies. We first discuss the definition of LLM-MAS, a framework encompassing much of previous work. We provide an overview of the various applications of LLM-MAS in (i) solving complex tasks, (ii) simulating specific scenarios, and (iii) evaluating generative agents. Building on previous studies, we also highlight several challenges and propose future directions for research in this field.