# Understanding Issues, Causes and Solutions in Open-Source LLM-based Multi-Agent Systems

arXiv: https://arxiv.org/abs/2610.00905 · HTML: https://mlsanity.com/paper/2610.00905/

## Key facts

- No citing papers in the map yet (data through 7 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 1 Oct 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 15 of 15)

- [Large Language Model based Multi-Agents: A Survey of Progress and Challenges](https://mlsanity.com/paper/2402.01680/) (2024)
- [Large Language Models for Software Engineering: Survey and Open Problems](https://mlsanity.com/paper/2310.03533/) (2023)
- [Large Language Model-Based Agents for Software Engineering: A Survey](https://mlsanity.com/paper/2409.02977/) (2024)
- [Software Testing with Large Language Models: Survey, Landscape, and Vision](https://mlsanity.com/paper/2307.07221/) (2023)
- [AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodology](https://mlsanity.com/paper/2406.11912/) (2024)
- [AutoGen Studio: A No-Code Developer Tool for Building and Debugging Multi-Agent Systems](https://mlsanity.com/paper/2408.15247/) (2024)
- [Exploring the Problems, their Causes and Solutions of AI Pair Programming: A Study on GitHub and Stack Overflow](https://mlsanity.com/paper/2311.01020/) (2023)
- [An LLM-as-Judge Metric for Bridging the Gap with Human Evaluation in SE Tasks](https://mlsanity.com/paper/2505.20854/) (2025)
- [Large Language Models in Thematic Analysis: Prompt Engineering, Evaluation, and Guidelines for Qualitative Software Engineering Research](https://mlsanity.com/paper/2510.18456/) (2025)
- [A Large-Scale Study on the Development and Issues of Multi-Agent AI Systems](https://mlsanity.com/paper/2601.07136/) (2026)
- [How do Software Engineering Researchers Use GitHub? An Empirical Study of Artifacts & Impact](https://mlsanity.com/paper/2310.01566/) (2023)
- [An Empirical Study of ChatGPT-Related Projects and Their Issues on GitHub](https://mlsanity.com/paper/2403.17437/) (2024)
- [Designing LLM-based Multi-Agent Systems for Software Engineering Tasks: Quality Attributes, Design Patterns and Rationale](https://mlsanity.com/paper/2511.08475/) (2025)
- [On the Nature of Issues in Five Open Source Microservices Systems: An Empirical Study](https://mlsanity.com/paper/2104.12192/) (2021)
- [What Challenges Do Developers Face in AI Agent Systems? An Empirical Study on Stack Overflow & GitHub Issues](https://mlsanity.com/paper/2510.25423/) (2025)

## Abstract (arXiv)

With the advancement of LLM-based multi-agent systems (MAS), an increasing number of opensource projects are adopting multi-agent architectures as the foundation of their core functionality. Although research and practice on MAS have attracted considerable attention, limited studies have explored the challenges faced by practitioners of open-source LLM-based MAS, the causes of these challenges, and potential solutions. To address this gap,we conducted an empirical study to understand the issues that practitioners encounter when developing and using open-source LLM-based MAS, the possible causes of these issues, and potential solutions. We collected 22,848 closed issues from 21 open-source LLM-basedMASand applied a mixed automated and manual filtering approach to reduce the dataset to 944 issues related to LLM-based MAS.We then analyzed these issues to understand the frequent issues encountered by practitioners, their underlying causes, and potential solutions. Our study results show that (1) Orchestration & Execution Issue is the most common issue faced by practitioners, (2) Workflow Problem, Tool Integration Problem, and Memory Problem are identified as the most frequent causes of the issues, and (3) Optimize Workflow is the predominant solution to the issues. Based on the study results, we derive empirically grounded implications for practitioners and researchers aimed at improving orchestration, tool integration, and memory mechanisms in LLM-based MAS.