# A Large-Scale Study on the Development and Issues of Multi-Agent AI Systems

arXiv: https://arxiv.org/abs/2601.07136 · HTML: https://mlsanity.com/paper/2601.07136/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 7 of its 24 references that are in the map.
- First posted on arXiv 12 Jan 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 7 of 7)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Large Language Model based Multi-Agents: A Survey of Progress and Challenges](https://mlsanity.com/paper/2402.01680/) (2024)
- [Analyzing the Evolution and Maintenance of ML Models on Hugging Face](https://mlsanity.com/paper/2311.13380/) (2023)
- [Smart Contract Vulnerability Detection Technique: A Survey](https://mlsanity.com/paper/2209.05872/) (2022)

## Built on by (most cited first, 6 of 6)

- [Agent Skills: A Data-Driven Analysis of Claude Skills for Extending Large Language Model Functionality](https://mlsanity.com/paper/2602.08004/) (2026)
- [Understanding Bugs in Modern Agentic Frameworks: A Study of Symptoms, Root Causes, and Triggering Conditions](https://mlsanity.com/paper/2604.08906/) (2026)
- [ADK Arena: Evaluating Agent Development Kits via LLM-as-a-Developer](https://mlsanity.com/paper/2606.05548/) (2026)
- [Engineering Reliable Commit Gates for Agentic AI: Cost-Aware Verification Portfolios under Common-Mode Data Failures](https://mlsanity.com/paper/2609.10969/) (2026)
- [When Intelligence Becomes Agency: A Theory of Governed, Proactive Agency for Symbiotic AI Systems](https://mlsanity.com/paper/2609.07741/) (2026)
- [Understanding Issues, Causes and Solutions in Open-Source LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2610.00905/) (2026)

## Abstract (arXiv)

The rapid emergence of multi-agent AI systems (MAS), including LangChain, CrewAI, and AutoGen, has shaped how large language model (LLM) applications are developed and orchestrated. However, little is known about how these systems evolve and are maintained in practice. This paper presents the first large-scale empirical study of open-source MAS, analyzing over 42K unique commits and over 4.7K resolved issues across eight leading systems. Our analysis identifies three distinct development profiles: sustained, steady, and burst-driven. These profiles reflect substantial variation in ecosystem maturity. Perfective commits constitute 40.8% of all changes, suggesting that feature enhancement is prioritized over corrective maintenance (27.4%) and adaptive updates (24.3%). Data about issues shows that the most frequent concerns involve bugs (22%), infrastructure (14%), and agent coordination challenges (10%). Issue reporting also increased sharply across all frameworks starting in 2023. Median resolution times range from under one day to about two weeks, with distributions skewed toward fast responses but a minority of issues requiring extended attention. These results highlight both the momentum and the fragility of the current ecosystem, emphasizing the need for improved testing infrastructure, documentation quality, and maintenance practices to ensure long-term reliability and sustainability.