mlsanityData to 7 Oct 2026

ExecuGraph: A Multi-Agent, Execution-Grounded Framework for Reliable Backend Code Synthesis with Large Language Models

arXiv 2607.20499 ↗First posted 18 Jun 2026cs.AI cs.SEcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Large Language Models generate plausible backend code, but a single-pass paradigm provides no guarantee of correctness or runtime reliability. We present ExecuGraph, a multi-agent framework that places execution-based validation at the center of backend code synthesis. Six specialized agents (Planner, Code Generator, Logical Reviewer, Evaluator, Optimizer, and Explainer) are coordinated by a typed directed workflow with a bounded retry budget, implemented on LangGraph with locally hosted models (Ollama) and an optional retrieval layer for algorithmic technique recall. A subprocess-isolated sandbox with a wall-clock timeout guards every evaluation. We evaluate on a curated 30-problem DSA suite (internal-30), HumanEval (n=64), and an APPS-introductory subset, contrasting ExecuGraph against a single-agent one-shot baseline and a single-agent execution-retry baseline (a Reflexion-style ablation that isolates the contribution of multi-agent decomposition). On internal-30, the three conditions are statistically indistinguishable (n=30; paired Wilcoxon p=0.59 MF vs. SO, p=0.08 SR vs. SO); 95% bootstrap confidence intervals on all pairwise mean differences include zero. On HumanEval, multi-full edges ahead by +3.1 pp. The strongest signal is cross-model: with DeepSeekCoder V2 Lite, graph-category accuracy improves from 57.5% (oneshot) to 80.0% (multi-full), a +22.5 pp jump that supports a scaling hypothesis: the value of multi-agent decomposition grows with base-model capability. The framework's primary contribution is methodological: a single codebase that collapses by configuration into one-shot, execution-retry, and per-agent ablation conditions, enabling controlled measurement of each lever's marginal contribution. A per-agent ablation, retry-budget sweep, error-class taxonomy, and test-source audit are reported.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 23 of its 32 references that are in the map. First posted on arXiv 18 Jun 2026.

Builds on

23 of 32 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024MapCoder: Multi-Agent Code Generation for Competitive Problem Solving18 May

Other references · 22

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  9. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  10. 2021Program Synthesis with Large Language Modelscited by 3,509
  11. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  12. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
10 more
  1. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  2. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  3. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  4. 2023Teaching Large Language Models to Self-Debugcited by 963
  5. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  6. 2021Measuring Coding Challenge Competence With APPScited by 879
  7. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  8. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  9. 2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 325
  10. 2026JudgeSense: A Benchmark for Prompt Sensitivity in LLM-as-a-Judge Systemscited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026BackendForge: Benchmarking Agentic End-to-End Code Generation with Backend Services
  2. 2026E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch
  3. 2026Constraint Decay: The Fragility of LLM Agents in Backend Code Generation
  4. 2026ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development
  5. 2025SWE-Compass: Towards Unified Evaluation of Agentic Coding Abilities for Large Language Models
  6. 2025StackPilot: Autonomous Function Agents for Scalable and Environment-Free Code Execution
  7. 2025CodeCoR: An LLM-Based Self-Reflective Multi-Agent Framework for Code Generation
  8. 2026SAFEdit: Does Multi-Agent Decomposition Resolve the Reliability Challenges of Instructed Code Editing?