mlsanityData to 7 Oct 2026

LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead

arXiv 2404.04834 ↗First posted 7 Apr 2024cs.SEcited by 155

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Integrating Large Language Models (LLMs) into autonomous agents marks a significant shift in the research landscape by offering cognitive abilities that are competitive with human planning and reasoning. This paper explores the transformative potential of integrating Large Language Models into Multi-Agent (LMA) systems for addressing complex challenges in software engineering (SE). By leveraging the collaborative and specialized abilities of multiple agents, LMA systems enable autonomous problem-solving, improve robustness, and provide scalable solutions for managing the complexity of real-world software projects. In this paper, we conduct a systematic review of recent primary studies to map the current landscape of LMA applications across various stages of the software development lifecycle (SDLC). To illustrate current capabilities and limitations, we perform two case studies to demonstrate the effectiveness of state-of-the-art LMA frameworks. Additionally, we identify critical research gaps and propose a comprehensive research agenda focused on enhancing individual agent capabilities and optimizing agent synergy. Our work outlines a forward-looking vision for developing fully autonomous, scalable, and trustworthy LMA systems, laying the foundation for the evolution of Software Engineering 2.0.

Key facts. Cited by 155 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 54 of its 181 references that are in the map. First posted on arXiv 7 Apr 2024.

Builds on

54 of 181 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Elicitron: An LLM Agent-Based Simulation Framework for Design Requirements Elicitation4 Apr
  2. 2024Self-Organized Agents: A LLM Multi-Agent Framework toward Ultra Large-Scale Code Generation and Optimization2 Apr
  3. 2024MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution26 Mar
  4. 2024SOEN-101: Code Generation by Emulating Software Process Models Using Large Language Model Agents23 Mar
  5. 2023Static Code Analysis in the AI Era: An In-depth Exploration of the Concept, Function, and Potential of Intelligent Code Analysis Agents13 Oct
  6. 2023ChatDev: Communicative Agents for Software Development16 Jul
  7. 2023Self-collaboration Code Generation via ChatGPT15 Apr

Other references · 47

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  5. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  6. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  7. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  8. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  9. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  10. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  11. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  12. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
35 more
  1. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  2. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  3. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
  4. 2021Explaining Neural Scaling Lawscited by 353
  5. 2023A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaborationcited by 281
  6. 2023Is Self-Repair a Silver Bullet for Code Generation?cited by 191
  7. 2024More Agents Is All You Needcited by 174
  8. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospectscited by 130
  9. 2017Autonomous Agents Modelling Other Agents: A Comprehensive Survey and Open Problemscited by 126
  10. 2023Scalable Multi-Robot Collaboration with Large Language Models: Centralized or Decentralized Systems?cited by 104
  11. 2023PentestGPT: An LLM-empowered Automatic Penetration Testing Toolcited by 94
  12. 2023MetaAgents: Large Language Model Based Agents for Decision-Making on Teamingcited by 85
  13. 2024Benchmark Self-Evolving: A Multi-Agent Framework for Dynamic LLM Evaluationcited by 73
  14. 2022Fairness Testing: A Comprehensive Survey and Analysis of Trendscited by 61
  15. 2023Experiential Co-Learning of Software-Developing Agentscited by 57
  16. 2023Large Language Model-Powered Smart Contract Vulnerability Detection: New Perspectivescited by 48
  17. 2024AgentFL: Scaling LLM-based Fault Localization to Project-Level Contextcited by 47
  18. 2023ChatGPT for Vulnerability Detection, Classification, and Repair: How Far Are We?cited by 44
  19. 2023When ChatGPT Meets Smart Contract Vulnerability Detection: How Far Are We?cited by 42
  20. 2023Explainable Automated Debugging via Large Language Model-driven Scientific Debuggingcited by 40
  21. 2023AXNav: Replaying Accessibility Tests from Natural Languagecited by 31
  22. 2024CodeS: Natural Language to Code Repository via Multi-Layer Sketchcited by 28
  23. 2023Flows: Building Blocks of Reasoning and Collaborating AIcited by 25
  24. 2024Hierarchical Auto-Organizing System for Open-Ended Multi-Agent Navigationcited by 24
  25. 2023Trustworthy and Synergistic Artificial Intelligence for Software Engineering: Vision and Roadmapscited by 19
  26. 2024Multi-role Consensus through LLMs Discussions for Vulnerability Detectioncited by 19
  27. 2017Agile Software Development Methods: Review and Analysiscited by 18
  28. 2023INTERVENOR: Prompting the Coding Ability of Large Language Models with the Interactive Chain of Repaircited by 18
  29. 2024Experimenting a New Programming Practice with LLMscited by 17
  30. 2024ACFIX: Guiding LLMs with Mined Common RBAC Practices for Context-Aware Repair of Access Control Vulnerabilities in Smart Contractscited by 15
  31. 2023ChatGPT: A Study on its Utility for Ubiquitous Software Engineering Taskscited by 12
  32. 2023Representation Learning for Stack Overflow Posts: How Far are We?cited by 6
  33. 2023CCBERT: Self-Supervised Code Change Representation Learningcited by 6
  34. 2021A Comparative Study of Vulnerability Reporting by Software Composition Analysis Toolscited by 4
  35. 2023APIDocBooster: An Extract-Then-Abstract Framework Leveraging Large Language Models for Augmenting API Documentationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 74; 2026: 76.

155 papers.

Sort:

By area:

  1. 2024Large Language Model-Based Agents for Software Engineering: A Surveycited by 173
  2. 2025On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHubcited by 53
  3. 2025SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarioscited by 49
  4. 2025Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AIcited by 39
  5. 2025Internet of Agents: Fundamentals, Applications, and Challengescited by 31
  6. 2025Speed at the Cost of Quality: How Cursor AI Increases Short-Term Velocity and Long-Term Complexity in Open-Source Projectscited by 23
  7. 2025The Coming Crisis of Multi-Agent Misalignment: AI Alignment Must Be a Dynamic and Social Processcited by 21
  8. 2025Let the Trial Begin: A Mock-Court Approach to Vulnerability Detection using LLM-Based Agentscited by 20
  9. 2025Exploring Autonomous Agents: A Closer Look at Why They Fail When Completing Taskscited by 20
  10. 2025Retrieval-Augmented Code Generation: A Survey with Focus on Repository-Level Approachescited by 20
  11. 2025Intrinsic Memory Agents: Heterogeneous Multi-Agent LLM Systems through Structured Contextual Memorycited by 19
  12. 2025Aegis: Automated Error Generation and Attribution for Multi-Agent Systemscited by 19
38 more
  1. 2025GNNs as Predictors of Agentic Workflow Performancescited by 17
  2. 2025From CVE Entries to Verifiable Exploits: An Automated Multi-Agent Framework for Reproducing CVEscited by 17
  3. 2025Process-Level Trajectory Evaluation for Environment Configuration in Software Engineering Agentscited by 17
  4. 2026Colosseum: Auditing Collusion in Cooperative Multi-Agent Systemscited by 17
  5. 2024Defining and Detecting the Defects of the Large Language Model-based Autonomous Agentscited by 14
  6. 2025A Survey on Inference Engines for Large Language Models: Perspectives on Optimization and Efficiencycited by 12
  7. 2025Enhancing LLM Performance Through Debate: An Empirical Study on Multi-Agent Debate for Coding Taskscited by 11
  8. 2025Towards Transparent and Incentive-Compatible Collaboration in Decentralized LLM Multi-Agent Systems: A Blockchain-Driven Approachcited by 11
  9. 2025The Trust Paradox in LLM-Based Multi-Agent Systems: When Collaboration Becomes a Security Vulnerabilitycited by 11
  10. 2025Stop Wasting Your Tokens: Towards Efficient Runtime Multi-Agent Systemscited by 11
  11. 2025Traceability and Accountability in Role-Specialized Multi-Agent LLM Pipelinescited by 10
  12. 2025Generative AI and Empirical Software Engineering: A Paradigm Shiftcited by 9
  13. 2025Multi-agent systems for chemical engineering: A review and perspectivecited by 9
  14. 2025Facilitating Trustworthy Human-Agent Collaboration in LLM-based Multi-Agent System oriented Software Engineeringcited by 7
  15. 2025TAI3: Testing Agent Integrity in Interpreting User Intentcited by 7
  16. 2026DiLLS: Interactive Diagnosis of LLM-based Multi-agent Systems via Layered Summary of Agent Behaviorscited by 7
  17. 2026Agentic Business Process Management: A Research Manifestocited by 7
  18. 2024Can We Trust AI Agents? A Case Study of an LLM-Based Multi-Agent System for Ethical AIcited by 6
  19. 2025Agentic Services Computingcited by 6
  20. 2025Metacognitive Self-Correction for Multi-Agent System via Prototype-Guided Next-Execution Reconstructioncited by 6
  21. 2025Practical challenges of control monitoring in frontier AI deploymentscited by 6
  22. 2025Human-AI Experience in Integrated Development Environments: A Systematic Literature Reviewcited by 5
  23. 2025Benchmarking LLM for Code Smells Detection: OpenAI GPT-4.0 vs DeepSeek-V3cited by 5
  24. 2025Dive into the Agent Matrix: A Realistic Evaluation of Self-Replication Risk in LLM Agentscited by 5
  25. 2025FreshBrew: A Benchmark for Evaluating AI Agents on Java Code Migrationcited by 5
  26. 2025SWEnergy: An Empirical Study on Energy Efficiency in Agentic Issue Resolution Frameworks with SLMscited by 5
  27. 2026ORCH: many analyses, one merge-a deterministic multi-agent orchestrator for discrete-choice reasoning with EMA-guided routingcited by 5
  28. 2024A Large Language Model-based multi-agent manufacturing system for intelligent shopfloorcited by 4
  29. 2025Teamwork makes the dream work: LLMs-Based Agents for GitHub README.MD Summarizationcited by 4
  30. 2025PAGENT: Learning to Patch Software Engineering Agentscited by 4
  31. 2025Chimera: Harnessing Multi-Agent LLMs for Automatic Insider Threat Simulationcited by 4
  32. 2025Red Teaming Program Repair Agents: When Correct Patches can Hide Vulnerabilitiescited by 4
  33. 2025ALMAS: an Autonomous LLM-based Multi-Agent Software Engineering Frameworkcited by 4
  34. 2025Towards Engineering Multi-Agent LLMs: A Protocol-Driven Approachcited by 4
  35. 2025Toward Agentic Software Engineering Beyond Code: Framing Vision, Values, and Vocabularycited by 4
  36. 2025Issue-Oriented Agent-Based Framework for Automated Review Comment Generationcited by 4
  37. 2025The Future of Generative AI in Software Engineering: A Vision from Industry and Academia in the European GENIUS Projectcited by 4
  38. 2026Optimizing Agentic Workflows using Meta-toolscited by 4

50 most cited of 155 All 155 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026LLM-Based Agentic Systems for Software Engineering: Challenges and Opportunities
  2. 2024Agents in Software Engineering: Survey, Landscape, and Vision
  3. 2025Designing LLM-based Multi-Agent Systems for Software Engineering Tasks: Quality Attributes, Design Patterns and Rationale
  4. 2024From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future
  5. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges
  6. 2025A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic System
  7. 2025LLMs Working in Harmony: A Survey on the Technological Aspects of Building Effective LLM-Based Multi Agent Systems
  8. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challenges