# LLM-Enabled Multi-Agent Systems: Empirical Evaluation and Insights into Emerging Design Patterns & Paradigms

arXiv: https://arxiv.org/abs/2601.03328 · HTML: https://mlsanity.com/paper/2601.03328/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 34 of its 112 references that are in the map.
- First posted on arXiv 6 Jan 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directions](https://mlsanity.com/paper/2503.23278/) (2025)
- [From LLM Reasoning to Autonomous AI Agents: A Comprehensive Review](https://mlsanity.com/paper/2504.19678/) (2025)
- [Rethinking Interpretability in the Era of Large Language Models](https://mlsanity.com/paper/2402.01761/) (2024)
- [A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions](https://mlsanity.com/paper/2310.14724/) (2023)
- [A Confederacy of Models: a Comprehensive Evaluation of LLMs on Creative Writing](https://mlsanity.com/paper/2310.08433/) (2023)
- [LLM-Based Test-Driven Interactive Code Generation: User Study and Empirical Evaluation](https://mlsanity.com/paper/2404.10100/) (2024)
- [Interpreting and Improving Large Language Models in Arithmetic Calculation](https://mlsanity.com/paper/2409.01659/) (2024)
- [Computing Power and the Governance of Artificial Intelligence](https://mlsanity.com/paper/2402.08797/) (2024)
- [Emergent Abilities in Large Language Models: A Survey](https://mlsanity.com/paper/2503.05788/) (2025)
- [Neural Natural Language Processing for Unstructured Data in Electronic Health Records: a Review](https://mlsanity.com/paper/2107.02975/) (2021)
- [A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directions](https://mlsanity.com/paper/2507.18910/) (2025)
- [Navigating LLM Ethics: Advancements, Challenges, and Future Directions](https://mlsanity.com/paper/2406.18841/) (2024)
- [A Survey on Context-Aware Multi-Agent Systems: Techniques, Challenges and Future Directions](https://mlsanity.com/paper/2402.01968/) (2024)
- [Effective Distillation of Table-based Reasoning Ability from LLMs](https://mlsanity.com/paper/2309.13182/) (2023)
- [Beyond Direct Diagnosis: LLM-based Multi-Specialist Agent Consultation for Automatic Diagnosis](https://mlsanity.com/paper/2401.16107/) (2024)
- [Beyond DNS: Unlocking the Internet of AI Agents via the NANDA Index and Verified AgentFacts](https://mlsanity.com/paper/2507.14263/) (2025)
- [Risk Analysis Techniques for Governed LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2508.05687/) (2025)
- [On the Brittle Foundations of ReAct Prompting for Agentic Large Language Models](https://mlsanity.com/paper/2405.13966/) (2024)
- [Grid-Agent: An LLM-Powered Multi-Agent System for Power Grid Control](https://mlsanity.com/paper/2508.05702/) (2025)
- ["I'm categorizing LLM as a productivity tool": Examining ethics of LLM use in HCI research practices](https://mlsanity.com/paper/2403.19876/) (2024)
- [A Study on the Implementation Method of an Agent-Based Advanced RAG System Using Graph](https://mlsanity.com/paper/2407.19994/) (2024)
- [A Taxonomy of Hierarchical Multi-Agent Systems: Design Patterns, Coordination Mechanisms, and Industrial Applications](https://mlsanity.com/paper/2508.12683/) (2025)

## Abstract (arXiv)

This paper formalises the literature on emerging design patterns and paradigms for Large Language Model (LLM)-enabled multi-agent systems (MAS), evaluating their practical utility across various domains. We define key architectural components, including agent orchestration, communication mechanisms, and control-flow strategies, and demonstrate how these enable rapid development of modular, domain-adaptive solutions. Three real-world case studies are tested in controlled, containerised pilots in telecommunications security, national heritage asset management, and utilities customer service automation. Initial empirical results show that, for these case studies, prototypes were delivered within two weeks and pilot-ready solutions within one month, suggesting reduced development overhead compared to conventional approaches and improved user accessibility. However, findings also reinforce limitations documented in the literature, including variability in LLM behaviour that leads to challenges in transitioning from prototype to production maturity. We conclude by outlining critical research directions for improving reliability, scalability, and governance in MAS architectures and the further work needed to mature MAS design patterns to mitigate the inherent challenges.