mlsanityData to 7 Oct 2026

CodePori: Large-Scale System for Autonomous Software Development Using Multi-Agent Technology

arXiv 2402.01411 ↗First posted 2 Feb 2024cs.SEcited by 21

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Context: LLM-based multi-agent systems enable automation and decision support in software development, yet existing studies rely on benchmark datasets offering only binary pass-or-fail results, limiting insight into real-world applicability. Objective: This study empirically investigates the potential and limitations of LLM-based agents in autonomous software development tasks. Method: A two-phase approach was employed: developing a multi-agent system, CodePori, for automated code generation, and conducting participant-based evaluation to assess practical performance. Results: Participant feedback reveals key strengths, challenges, and areas for improvement in LLM-based multi-agent systems, highlighting aspects missed by standard code-generation benchmarks. Conclusions: While LLM-based multi-agent systems show potential for large-scale software development, successful integration requires addressing challenges such as memory limitations, hallucinations, and code smells, alongside a practitioner-centric perspective.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 52 of its 122 references that are in the map. First posted on arXiv 2 Feb 2024.

Builds on

52 of 122 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Autonomous Agents in Software Development: A Vision Paper30 Nov
  2. 2023ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code16 Nov
  3. 2023ChatDev: Communicative Agents for Software Development16 Jul

Other references · 49

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2021Program Synthesis with Large Language Modelscited by 3,509
  9. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  10. 2023A Survey of Large Language Modelscited by 2,537
  11. 2022Emergent Abilities of Large Language Modelscited by 2,483
  12. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
37 more
  1. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  2. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  3. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  4. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  5. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  6. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  7. 2021Measuring Coding Challenge Competence With APPScited by 879
  8. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  9. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
  10. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  11. 2022Large Language Models Can Self-Improvecited by 656
  12. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  13. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 389
  14. 2022CodeT: Code Generation with Generated Testscited by 369
  15. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  16. 2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 355
  17. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 294
  18. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 257
  19. 2023A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPTcited by 221
  20. 2017Learning to Represent Programs with Graphscited by 219
  21. 2023Time Travel in LLMs: Tracing Data Contamination in Large Language Modelscited by 175
  22. 2023GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Modelscited by 174
  23. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  24. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospectscited by 130
  25. 2020Unit Test Case Generation with Transformers and Focal Contextcited by 118
  26. 2023Regulating ChatGPT and other Large Generative AI Modelscited by 102
  27. 2023ChatGPT is not all you need. A State of the Art Review of large Generative AI modelscited by 85
  28. 2023Towards an Understanding of Large Language Models in Software Engineering Taskscited by 67
  29. 2023A Survey of Large Language Models for Code: Evolution, Benchmarking, and Future Trendscited by 66
  30. 2024AI Hallucinations: A Misnomer Worth Clarifyingcited by 34
  31. 2023A/B Testing: A Systematic Literature Reviewcited by 16
  32. 2019Learning Software Configuration Spaces: A Systematic Literature Reviewcited by 14
  33. 2023Comparing Software Developers with ChatGPT: An Empirical Investigationcited by 11
  34. 2021Understanding Emails and Drafting Responses -- An Approach Using GPT-3cited by 10
  35. 2019Code Smell Detection using Multilabel Classification Approachcited by 6
  36. 2023Navigating Complexity in Software Engineering: A Prototype for Comparing GPT-n Solutionscited by 5
  37. 2022Statistical Dataset Evaluation: Reliability, Difficulty, and Validitycited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 9; 2026: 7.

21 papers.

Sort:

By area:

  1. 2024Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-stepcited by 123
  2. 2026Code as Agent Harnesscited by 40
  3. 2024FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Modelscited by 39
  4. 2025Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AIcited by 39
  5. 2024DreamFactory: Pioneering Multi-Scene Long Video Generation with a Multi-Agent Frameworkcited by 31
  6. 2025The FM Agentcited by 20
  7. 2025DeepCode: Open Agentic Codingcited by 14
  8. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 13
  9. 2024Large Language Model Evaluation Via Multi AI Agents: Preliminary resultscited by 10
  10. 2026DiLLS: Interactive Diagnosis of LLM-based Multi-agent Systems via Layered Summary of Agent Behaviorscited by 7
  11. 2024Experimenting with Multi-Agent Software Development: Towards a Unified Platformcited by 6
  12. 2026Security in the Age of AI Teammates: An Empirical Study of Agentic Pull Requests on GitHubcited by 6
9 more
  1. 2025Cognitive Agents Powered by Large Language Models for Agile Software Project Managementcited by 4
  2. 2025RA-Gen: A Controllable Code Generation Framework Using ReAct for Multi-Agent Task Executioncited by 2
  3. 2026LLM-Based Multi-Agent Systems for Code Generation: A Multi-Vocal Literature Reviewcited by 2
  4. 2025Autonomous Legacy Web Application Upgrades Using a Multi-Agent Systemcited by 1
  5. 2025A Research Roadmap for Augmenting Software Engineering Processes and Software Products with Generative AIcited by 1
  6. 2026Automated Summarization of Software Documents: An LLM-based Multi-Agent Approachcited by 1
  7. 2025Distributed Approach to Haskell Based Applications Refactoring with LLMs Based Multi-Agent Systemscited by 0
  8. 2026See, Plan, Snap: Evaluating Multimodal GUI Agents in Scratchcited by 0
  9. 2026Building an Internal Coding Agent at Zup: Lessons and Open Questionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Developing LLM-based Multi-Agent Systems in Software Engineering: A Mixed-Method Experience Report
  2. 2025ALMAS: an Autonomous LLM-based Multi-Agent Software Engineering Framework
  3. 2025A Survey on Code Generation with LLM-based Agents
  4. 2024LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead
  5. 2026LLM-Based Agentic Systems for Software Engineering: Challenges and Opportunities
  6. 2025Benchmarking and Studying the LLM-based Agent System in End-to-End Software Development
  7. 2024Agentless: Demystifying LLM-based Software Engineering Agents
  8. 2025Designing LLM-based Multi-Agent Systems for Software Engineering Tasks: Quality Attributes, Design Patterns and Rationale