# LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead

arXiv: https://arxiv.org/abs/2404.04834 · HTML: https://mlsanity.com/paper/2404.04834/

## Key facts

- Cited by 155 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 54 of its 181 references that are in the map.
- First posted on arXiv 7 Apr 2024.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 30 of 54)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning](https://mlsanity.com/paper/2205.05638/) (2022)
- [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://mlsanity.com/paper/2310.03714/) (2023)
- [Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents](https://mlsanity.com/paper/2302.01560/) (2023)
- [Explaining Neural Scaling Laws](https://mlsanity.com/paper/2102.06701/) (2021)
- [A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration](https://mlsanity.com/paper/2310.02170/) (2023)
- [Self-collaboration Code Generation via ChatGPT](https://mlsanity.com/paper/2304.07590/) (2023)
- [Is Self-Repair a Silver Bullet for Code Generation?](https://mlsanity.com/paper/2306.09896/) (2023)
- [More Agents Is All You Need](https://mlsanity.com/paper/2402.05120/) (2024)
- [MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution](https://mlsanity.com/paper/2403.17927/) (2024)
- [Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects](https://mlsanity.com/paper/2401.03428/) (2024)
- [Autonomous Agents Modelling Other Agents: A Comprehensive Survey and Open Problems](https://mlsanity.com/paper/1709.08071/) (2017)
- [Scalable Multi-Robot Collaboration with Large Language Models: Centralized or Decentralized Systems?](https://mlsanity.com/paper/2309.15943/) (2023)
- [PentestGPT: An LLM-empowered Automatic Penetration Testing Tool](https://mlsanity.com/paper/2308.06782/) (2023)
- [MetaAgents: Large Language Model Based Agents for Decision-Making on Teaming](https://mlsanity.com/paper/2310.06500/) (2023)
- [Self-Organized Agents: A LLM Multi-Agent Framework toward Ultra Large-Scale Code Generation and Optimization](https://mlsanity.com/paper/2404.02183/) (2024)
- [Benchmark Self-Evolving: A Multi-Agent Framework for Dynamic LLM Evaluation](https://mlsanity.com/paper/2402.11443/) (2024)
- [Fairness Testing: A Comprehensive Survey and Analysis of Trends](https://mlsanity.com/paper/2207.10223/) (2022)

## Built on by (most cited first, 30 of 155)

- [Large Language Model-Based Agents for Software Engineering: A Survey](https://mlsanity.com/paper/2409.02977/) (2024)
- [SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarios](https://mlsanity.com/paper/2512.18470/) (2025)
- [On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub](https://mlsanity.com/paper/2509.14745/) (2025)
- [Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI](https://mlsanity.com/paper/2505.19443/) (2025)
- [Internet of Agents: Fundamentals, Applications, and Challenges](https://mlsanity.com/paper/2505.07176/) (2025)
- [Speed at the Cost of Quality: How Cursor AI Increases Short-Term Velocity and Long-Term Complexity in Open-Source Projects](https://mlsanity.com/paper/2511.04427/) (2025)
- [Retrieval-Augmented Code Generation: A Survey with Focus on Repository-Level Approaches](https://mlsanity.com/paper/2510.04905/) (2025)
- [Aegis: Automated Error Generation and Attribution for Multi-Agent Systems](https://mlsanity.com/paper/2509.14295/) (2025)
- [The Coming Crisis of Multi-Agent Misalignment: AI Alignment Must Be a Dynamic and Social Process](https://mlsanity.com/paper/2506.01080/) (2025)
- [Let the Trial Begin: A Mock-Court Approach to Vulnerability Detection using LLM-Based Agents](https://mlsanity.com/paper/2505.10961/) (2025)
- [Exploring Autonomous Agents: A Closer Look at Why They Fail When Completing Tasks](https://mlsanity.com/paper/2508.13143/) (2025)
- [Intrinsic Memory Agents: Heterogeneous Multi-Agent LLM Systems through Structured Contextual Memory](https://mlsanity.com/paper/2508.08997/) (2025)
- [From CVE Entries to Verifiable Exploits: An Automated Multi-Agent Framework for Reproducing CVEs](https://mlsanity.com/paper/2509.01835/) (2025)
- [Process-Level Trajectory Evaluation for Environment Configuration in Software Engineering Agents](https://mlsanity.com/paper/2510.25694/) (2025)
- [Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems](https://mlsanity.com/paper/2602.15198/) (2026)
- [GNNs as Predictors of Agentic Workflow Performances](https://mlsanity.com/paper/2503.11301/) (2025)
- [Defining and Detecting the Defects of the Large Language Model-based Autonomous Agents](https://mlsanity.com/paper/2412.18371/) (2024)
- [A Survey on Inference Engines for Large Language Models: Perspectives on Optimization and Efficiency](https://mlsanity.com/paper/2505.01658/) (2025)
- [Towards Transparent and Incentive-Compatible Collaboration in Decentralized LLM Multi-Agent Systems: A Blockchain-Driven Approach](https://mlsanity.com/paper/2509.16736/) (2025)
- [The Trust Paradox in LLM-Based Multi-Agent Systems: When Collaboration Becomes a Security Vulnerability](https://mlsanity.com/paper/2510.18563/) (2025)
- [Stop Wasting Your Tokens: Towards Efficient Runtime Multi-Agent Systems](https://mlsanity.com/paper/2510.26585/) (2025)
- [Enhancing LLM Performance Through Debate: An Empirical Study on Multi-Agent Debate for Coding Tasks](https://mlsanity.com/paper/2503.12029/) (2025)
- [Traceability and Accountability in Role-Specialized Multi-Agent LLM Pipelines](https://mlsanity.com/paper/2510.07614/) (2025)
- [Generative AI and Empirical Software Engineering: A Paradigm Shift](https://mlsanity.com/paper/2502.08108/) (2025)
- [TAI3: Testing Agent Integrity in Interpreting User Intent](https://mlsanity.com/paper/2506.07524/) (2025)
- [Multi-agent systems for chemical engineering: A review and perspective](https://mlsanity.com/paper/2508.07880/) (2025)
- [Practical challenges of control monitoring in frontier AI deployments](https://mlsanity.com/paper/2512.22154/) (2025)
- [Agentic Business Process Management: A Research Manifesto](https://mlsanity.com/paper/2603.18916/) (2026)
- [Facilitating Trustworthy Human-Agent Collaboration in LLM-based Multi-Agent System oriented Software Engineering](https://mlsanity.com/paper/2505.04251/) (2025)
- [Dive into the Agent Matrix: A Realistic Evaluation of Self-Replication Risk in LLM Agents](https://mlsanity.com/paper/2509.25302/) (2025)

## Abstract (arXiv)

Integrating Large Language Models (LLMs) into autonomous agents marks a significant shift in the research landscape by offering cognitive abilities that are competitive with human planning and reasoning. This paper explores the transformative potential of integrating Large Language Models into Multi-Agent (LMA) systems for addressing complex challenges in software engineering (SE). By leveraging the collaborative and specialized abilities of multiple agents, LMA systems enable autonomous problem-solving, improve robustness, and provide scalable solutions for managing the complexity of real-world software projects. In this paper, we conduct a systematic review of recent primary studies to map the current landscape of LMA applications across various stages of the software development lifecycle (SDLC). To illustrate current capabilities and limitations, we perform two case studies to demonstrate the effectiveness of state-of-the-art LMA frameworks. Additionally, we identify critical research gaps and propose a comprehensive research agenda focused on enhancing individual agent capabilities and optimizing agent synergy. Our work outlines a forward-looking vision for developing fully autonomous, scalable, and trustworthy LMA systems, laying the foundation for the evolution of Software Engineering 2.0.