LLM-Enabled Multi-Agent Systems: Empirical Evaluation and Insights into Emerging Design Patterns & Paradigms
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
This paper formalises the literature on emerging design patterns and paradigms for Large Language Model (LLM)-enabled multi-agent systems (MAS), evaluating their practical utility across various domains. We define key architectural components, including agent orchestration, communication mechanisms, and control-flow strategies, and demonstrate how these enable rapid development of modular, domain-adaptive solutions. Three real-world case studies are tested in controlled, containerised pilots in telecommunications security, national heritage asset management, and utilities customer service automation. Initial empirical results show that, for these case studies, prototypes were delivered within two weeks and pilot-ready solutions within one month, suggesting reduced development overhead compared to conventional approaches and improved user accessibility. However, findings also reinforce limitations documented in the literature, including variability in LLM behaviour that leads to challenges in transitioning from prototype to production maturity. We conclude by outlining critical research directions for improving reliability, scalability, and governance in MAS architectures and the further work needed to mature MAS design patterns to mitigate the inherent challenges.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 34 of its 112 references that are in the map. First posted on arXiv 6 Jan 2026.
Builds on
34 of 112 references.
Closest ancestors · 4
Same direction, newest first
- 2025A Taxonomy of Hierarchical Multi-Agent Systems: Design Patterns, Coordination Mechanisms, and Industrial Applications18 Aug
- 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Review28 Apr
- 2024On the Brittle Foundations of ReAct Prompting for Agentic Large Language Models22 May
- 2024A Survey on Context-Aware Multi-Agent Systems: Techniques, Challenges and Future Directions3 Feb
Other references · 30
- 2017Attention Is All You Needcited by 59,932
- 2023GPT-4 Technical Reportcited by 18,132
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
- 2023A Comprehensive Overview of Large Language Modelscited by 639
- 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 257
- 2024Rethinking Interpretability in the Era of Large Language Modelscited by 110
- 2023A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directionscited by 101
- 2023A Confederacy of Models: a Comprehensive Evaluation of LLMs on Creative Writingcited by 98
18 more
- 2024LLM-Based Test-Driven Interactive Code Generation: User Study and Empirical Evaluationcited by 46
- 2024Interpreting and Improving Large Language Models in Arithmetic Calculationcited by 43
- 2024Computing Power and the Governance of Artificial Intelligencecited by 38
- 2021Neural Natural Language Processing for Unstructured Data in Electronic Health Records: a Reviewcited by 35
- 2025Emergent Abilities in Large Language Models: A Surveycited by 34
- 2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directionscited by 30
- 2024Navigating LLM Ethics: Advancements, Challenges, and Future Directionscited by 28
- 2023Effective Distillation of Table-based Reasoning Ability from LLMscited by 23
- 2024Beyond Direct Diagnosis: LLM-based Multi-Specialist Agent Consultation for Automatic Diagnosiscited by 23
- 2025Beyond DNS: Unlocking the Internet of AI Agents via the NANDA Index and Verified AgentFactscited by 20
- 2025Risk Analysis Techniques for Governed LLM-based Multi-Agent Systemscited by 16
- 2025Grid-Agent: An LLM-Powered Multi-Agent System for Power Grid Controlcited by 15
- 2024"I'm categorizing LLM as a productivity tool": Examining ethics of LLM use in HCI research practicescited by 14
- 2024A Study on the Implementation Method of an Agent-Based Advanced RAG System Using Graphcited by 8
- 2025How Well Do LLMs Predict Prerequisite Skills? Zero-Shot Comparison to Expert-Defined Conceptscited by 7
- 2024Leveraging Multi-AI Agents for Cross-Domain Knowledge Discoverycited by 4
- 2015Organization of Multi-Agent Systems: An Overviewcited by 3
- 2025Unstructured Data Analysis using LLMs: A Comprehensive Benchmarkcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025LLMs Working in Harmony: A Survey on the Technological Aspects of Building Effective LLM-Based Multi Agent Systems
- 2025Beyond Self-Talk: A Communication-Centric Survey of LLM-Based Multi-Agent Systems
- 2025Large Language Models Miss the Multi-Agent Mark
- 2026Scaling LLM-Driven Multi-Agent Systems: Design Principles and Architectural Scalability Analysis
- 2025Contemporary Agent Technology: LLM-Driven Advancements vs Classic Multi-Agent Systems
- 2024Practical Considerations for Agentic LLM Systems
- 2024LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead
- 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges