mlsanityData to 14 Oct 2026

How Generation Architecture Shapes Code Complexity in Multi-Agent LLM Systems: A Paired Study on HumanEval

arXiv 2606.00308 ↗First posted 29 May 2026cs.SE cs.AI cs.LGcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Large-language-model code generation has shifted from single-shot prompting to multi-agent orchestrations - analyst, coder, tester, and debugger pipelines - and is evaluated almost exclusively on functional correctness. Whether these architectures also affect the structural complexity of the code they produce, and which orchestration layers carry the cost, remains largely unexamined: prior work has documented prompt-level effects on code complexity, but the architecture-level question is open. We compare six widely-used multi-agent configurations (Basic, AC, ACT, Debugger, AC+Debugger, ACT+Debugger) under two models from the GPT-4o family across all 164 HumanEval tasks - 1,968 paired observations - using the five RADON complexity metrics (SLOC, cyclomatic complexity, and Halstead Volume, Difficulty, and Effort). We apply a paired non-parametric statistical pipeline (Friedman omnibus, Wilcoxon signed-rank post-hoc with Holm correction, Kendall's W and matched-pairs rank-biserial effect sizes) in both all-completions and passing-only conditions. The six architectures collapse into two indistinguishable complexity clusters separated by a 50-130% gap, the same partition in both models and under both conditions; among the architectural layers, the analyst-coder split inflates complexity, the runtime debugger does not - and on the analyst-coder background actively deflates it - and the tester re-inflates it. The heavy cluster's additional complexity buys no pass@1 advantage: the leanest architectures match or beat the heaviest on accuracy. Architectural elaboration in LLM code generation should therefore be justified by measured benefit on the dimensions that matter, not assumed.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 17 of its 30 references that are in the map. First posted on arXiv 29 May 2026.

Builds on

17 of 30 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Enhancing LLM Code Generation: A Systematic Evaluation of Multi-Agent Collaboration and Runtime Debugging for Improved Accuracy, Reliability, and Latency4 May
  2. 2025Why Do Multi-Agent LLM Systems Fail?17 Mar
  3. 2024MapCoder: Multi-Agent Code Generation for Competitive Problem Solving18 May
  4. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May
  5. 2024Executable Code Actions Elicit Better LLM Agents1 Feb
  6. 2023ChatDev: Communicative Agents for Software Development16 Jul

Other references · 11

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  6. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
  7. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  8. 2024Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineeringcited by 103
  9. 2025Enhancing LLM-Based Code Generation with Complexity Metrics: A Feedback-Driven Approachcited by 10
  10. 2025Do Prompt Patterns Affect Code Quality? A First Empirical Assessment of ChatGPT-Generated Codecited by 4
  11. 2024Creative and Correct: Requesting Diverse Code Solutions from AI Foundation Modelscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Taming Scylla: Understanding the multi-headed agentic daemon of the coding seas
  2. 2026Constraint Decay: The Fragility of LLM Agents in Backend Code Generation
  3. 2026AI-Generated Smells: An Analysis of Code and Architecture in LLM and Agent-Driven Development
  4. 2026Inside the Scaffold: A Source-Code Taxonomy of Coding Agent Architectures
  5. 2025Understanding and Bridging the Planner-Coder Gap: A Systematic Study on the Robustness of Multi-Agent Systems for Code Generation
  6. 2025Evaluating Software Process Models for Multi-Agent Class-Level Code Generation
  7. 2025CODESIM: Multi-Agent Code Generation and Problem Solving through Simulation-Driven Planning and Debugging
  8. 2026An Exact Generate - Transform Decomposition of Small-LLM Team Scaling Across Orchestration Architectures