AgentForge: Execution-Grounded Multi-Agent LLM Framework for Autonomous Software Engineering
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Large language models generate plausible code but cannot verify correctness. Existing multi-agent systems simulate execution or leave verification optional. We introduce execution-grounded verification as a first-class principle: every code change must survive sandboxed execution before propagation. We instantiate this principle in AGENTFORGE, a multi-agent framework where Planner, Coder, Tester, Debugger, and Critic agents coordinate through shared memory and a mandatory Docker sandbox. We formalize software engineering with LLMs as an iterative decision process over repository states, where execution feedback provides a stronger supervision signal than next-token likelihood. AGENTFORGE achieves 40.0% resolution on SWE-BENCH Lite, outperforming single-agent baselines by 26--28 points. Ablations confirm that execution feedback and role decomposition each independently drive performance. The framework is open-source at https://github.com/raja21068/AutoCodeAI.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 26 of its 41 references that are in the map. First posted on arXiv 13 Apr 2026.
Builds on
26 of 41 references.
Closest ancestors · 8
Same direction, newest first
- 2026SEMAG: Self-Evolutionary Multi-Agent Code Generation16 Mar
- 2025AgentMesh: A Cooperative Multi-Agent Generative AI Framework for Software Development Automation26 Jul
- 2024RGD: Multi-LLM Based Agent Debugger via Refinement and Generation Guidance2 Oct
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agents23 Jul
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May
- 2024MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution26 Mar
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation16 Aug
- 2023ChatDev: Communicative Agents for Software Development16 Jul
Other references · 18
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2021Program Synthesis with Large Language Modelscited by 3,637
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2023Code Llama: Open Foundation Models for Codecited by 2,139
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
- 2022Competition-Level Code Generation with AlphaCodecited by 1,462
- 2023Teaching Large Language Models to Self-Debugcited by 988
6 more
- 2023StarCoder: may the source be with you!cited by 799
- 2024AutoCodeRover: Autonomous Program Improvementcited by 278
- 2023Is Self-Repair a Silver Bullet for Code Generation?cited by 197
- 2022Practical Program Repair in the Era of Large Pre-trained Language Modelscited by 133
- 2023InferFix: End-to-End Program Repair with LLMscited by 114
- 2025InspectCoder: Dynamic Analysis-Enabled Self Repair through interactive LLM-Debugger Collaborationcited by 4
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026Multi-agent Collaboration with State Managementcited by 6
- 2026Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Modelcited by 3
- 2026ZhuLong: Execution-Grounded LLM Agent for EDA Scripting with Offline API Self-Explorationcited by 1
- 2026AsynCodeBench: Benchmarking Collaboration of Asynchronous Multi-Agent Systems in Software Engineeringcited by 0
- 2026AECP: Artifact-Exclusive Communication Protocol for Multi-Agent Code Generationcited by 0
- 2026AECP: Artifact-Exclusive Communication Protocol for Multi-Agent Code Generation5 Oct 2026
- 2026AsynCodeBench: Benchmarking Collaboration of Asynchronous Multi-Agent Systems in Software Engineering26 Sep 2026
- 2026Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Model14 Aug 2026
- 2026ZhuLong: Execution-Grounded LLM Agent for EDA Scripting with Offline API Self-Exploration8 Aug 2026
- 2026Multi-agent Collaboration with State Management19 May 2026
Similar, not cited
Close in wording, no citation either way
- 2024Agentless: Demystifying LLM-based Software Engineering Agents
- 2025A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic System
- 2024CodePori: Large-Scale System for Autonomous Software Development Using Multi-Agent Technology
- 2025LoCoBench-Agent: An Interactive Benchmark for LLM Agents in Long-Context Software Engineering
- 2026LLM-Based Agentic Systems for Software Engineering: Challenges and Opportunities
- 2025Benchmarking and Studying the LLM-based Agent System in End-to-End Software Development
- 2025A Comprehensive Empirical Evaluation of Agent Frameworks on Code-centric Software Engineering Tasks
- 2026SWE-World: Building Software Engineering Agents in Docker-Free Environments