mlsanityData to 7 Oct 2026

SOEN-101: Code Generation by Emulating Software Process Models Using Large Language Model Agents

arXiv 2403.15852 ↗First posted 23 Mar 2024cs.SE cs.AIcited by 34

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Software process models are essential to facilitate collaboration and communication among software teams to solve complex development tasks. Inspired by these software engineering practices, we present FlowGen - a code generation framework that emulates software process models based on multiple Large Language Model (LLM) agents. We emulate three process models, FlowGenWaterfall, FlowGenTDD, and FlowGenScrum, by assigning LLM agents to embody roles (i.e., requirement engineer, architect, developer, tester, and scrum master) that correspond to everyday development activities and organize their communication patterns. The agents work collaboratively using chain-of-thought and prompt composition with continuous self-refinement to improve the code quality. We use GPT3.5 as our underlying LLM and several baselines (RawGPT, CodeT, Reflexion) to evaluate code generation on four benchmarks: HumanEval, HumanEval-ET, MBPP, and MBPP-ET. Our findings show that FlowGenScrum excels compared to other process models, achieving a Pass@1 of 75.2, 65.5, 82.5, and 56.7 in HumanEval, HumanEval-ET, MBPP, and MBPP-ET, respectively (an average of 15% improvement over RawGPT). Compared with other state-of-the-art techniques, FlowGenScrum achieves a higher Pass@1 in MBPP compared to CodeT, with both outperforming Reflexion. Notably, integrating CodeT into FlowGenScrum resulted in statistically significant improvements, achieving the highest Pass@1 scores. Our analysis also reveals that the development activities impacted code smell and exception handling differently, with design and code review adding more exception handling and reducing code smells. Finally, FlowGen models maintain stable Pass@1 scores across GPT3.5 versions and temperature values, highlighting the effectiveness of software process models in enhancing the quality and stability of LLM-generated code.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 32 of its 52 references that are in the map. First posted on arXiv 23 Mar 2024.

Builds on

32 of 52 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023ChatDev: Communicative Agents for Software Development16 Jul
  2. 2023Self-collaboration Code Generation via ChatGPT15 Apr

Other references · 30

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  5. 2021Program Synthesis with Large Language Modelscited by 3,509
  6. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  7. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  8. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  9. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  10. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  11. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  12. 2022CodeT: Code Generation with Generated Testscited by 369
18 more
  1. 2023Character-LLM: A Trainable Agent for Role-Playingcited by 335
  2. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 307
  3. 2023How is ChatGPT's behavior changing over time?cited by 265
  4. 2023Auto-GPT for Online Decision Making: Benchmarks and Additional Opinionscited by 250
  5. 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 238
  6. 2023An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generationcited by 193
  7. 2023Structured Chain-of-Thought Prompting for Code Generationcited by 136
  8. 2022Large Language Models are Few-shot Testers: Exploring LLM-based General Bug Reproductioncited by 110
  9. 2023No More Manual Tests? Evaluating and Improving ChatGPT for Unit Test Generationcited by 82
  10. 2023Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPTcited by 71
  11. 2023CodeScore: Evaluating Code Generation by Learning Code Executioncited by 66
  12. 2023ChatGPT Prompt Patterns for Improving Code Quality, Refactoring, Requirements Elicitation, and Software Designcited by 66
  13. 2021Reordering Examples Helps during Priming-based Few-Shot Learningcited by 50
  14. 2023Evaluating Instruction-Tuned Large Language Models on Code Comprehension and Generationcited by 49
  15. 2023Log Parsing with Prompt-based Few-shot Learningcited by 28
  16. 2022Alleviating the Sample Selection Bias in Few-shot Learning by Removing Projection to the Centroidcited by 20
  17. 2024How to Refactor this Code? An Exploratory Study on Developer-ChatGPT Refactoring Conversationscited by 11
  18. 2024Assessing the Latent Automated Program Repair Capabilities of Large Language Models using Round-Trip Translationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 18; 2026: 11.

34 papers.

Sort:

By area:

  1. 2024LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Aheadcited by 155
  2. 2025Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scalingcited by 93
  3. 2025Large Language Model-based Data Science Agent: A Surveycited by 22
  4. 2024NLPerturbator: Studying the Robustness of Code LLMs to Natural Language Variationscited by 19
  5. 2024A Multi-Expert Large Language Model Architecture for Verilog Code Generationcited by 15
  6. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 13
  7. 2025HiBench: Benchmarking LLMs Capability on Hierarchical Structure Reasoningcited by 12
  8. 2025Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metricscited by 12
  9. 2025MANTRA: Enhancing Automated Method-Level Refactoring with Contextual RAG and Multi-Agent LLM Collaborationcited by 10
  10. 2024A Multi-Agent Approach to Fault Localization via Graph-Based Retrieval and Reflexioncited by 8
  11. 2025Enhancing LLM Code Generation: A Systematic Evaluation of Multi-Agent Collaboration and Runtime Debugging for Improved Accuracy, Reliability, and Latencycited by 8
  12. 2025Towards Advancing Code Generation with Large Language Models: A Research Roadmapcited by 5
22 more
  1. 2025RobuNFR: Evaluating the Robustness of Large Language Models on Non-Functional Requirements Aware Code Generationcited by 5
  2. 2025Cognitive Agents Powered by Large Language Models for Agile Software Project Managementcited by 4
  3. 2026Your Code Agent Can Grow Alongside You with Structured Memorycited by 4
  4. 2025Understanding and Bridging the Planner-Coder Gap: A Systematic Study on the Robustness of Multi-Agent Systems for Code Generationcited by 3
  5. 2025Towards Agentic OS: An LLM Agent Framework for Linux Schedulerscited by 2
  6. 2025Secure-Instruct: An Automated Pipeline for Synthesizing Instruction-Tuning Datasets Using LLMs for Secure Code Generationcited by 2
  7. 2025Evaluating Software Process Models for Multi-Agent Class-Level Code Generationcited by 2
  8. 2026LLM-Based Multi-Agent Systems for Code Generation: A Multi-Vocal Literature Reviewcited by 2
  9. 2026ASEval: Automated Trajectory-Level Security Testing for Autonomous Agentscited by 2
  10. 2024System Test Case Design from Requirements Specifications: Insights and Challenges of Using ChatGPTcited by 1
  11. 2025Towards Machine-Generated Code for the Resolution of User Intentionscited by 1
  12. 2025A Research Roadmap for Augmenting Software Engineering Processes and Software Products with Generative AIcited by 1
  13. 2026From LLMs to Agents in Programming: The Impact of Providing an LLM with a Compilercited by 1
  14. 2026Social Bias in LLM-Generated Code: Benchmark and Mitigationcited by 1
  15. 2026MANGO: Automated Multi-Agent Test Oracle Generation for Vision-Language-Action Modelscited by 1
  16. 2026Small, Free, and Effective: Orchestrating Open-Weight Small Language Models to Outperform Single LLM for Malware Analysiscited by 1
  17. 2026FlowScout: From Execution Feedback to Reliable Tool-Using Agent Workflowscited by 1
  18. 2025Wired for Reuse: Automating Context-Aware Code Adaptation in IDEs via LLM-Based Agentcited by 0
  19. 2025Knowledge-Guided Multi-Agent Framework for Application-Level Software Code Generationcited by 0
  20. 2026FVRuleLearner: Operator-Level Reasoning Tree (Op-Tree)-Based Rules Learning for Formal Verificationcited by 0
  21. 2026Rethinking Code Performance Benchmarks for LLMscited by 0
  22. 2026DOCSCHISEL: Adaptive Tool Documentation Optimization Framework for LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey on Code Generation with LLM-based Agents
  2. 2025Towards Realistic Project-Level Code Generation via Multi-Agent Collaboration and Semantic Architecture Modeling
  3. 2024Using a Feedback Loop for LLM-based Infrastructure as Code Generation
  4. 2026E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch
  5. 2025CodeVisionary: An Agent-based Framework for Evaluating Large Language Models in Code Generation
  6. 2025AdaCoder: An Adaptive Planning and Multi-Agent Framework for Function-Level Code Generation
  7. 2025Lifecycle-Aware code generation: Leveraging Software Engineering Phases in LLMs
  8. 2024Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvement