The Specification Gap: Coordination Failure Under Partial Knowledge in Code Agents
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
When multiple LLM-based code agents independently implement parts of the same class, they must agree on shared internal representations, even when the specification leaves those choices implicit. We study this coordination problem across 51 class-generation tasks, progressively stripping specification detail from full docstrings (L0) to bare signatures (L3), and introducing opposing structural biases (lists vs. dictionaries) to stress-test integration. Three findings emerge. First, a persistent specification gap: two-agent integration accuracy drops from 58% to 25% as detail is removed, while a single-agent baseline degrades more gracefully (89% to 56%), leaving a 25--39 pp coordination gap that is consistent across two Claude models (Sonnet, Haiku) and three independent runs. Second, an AST-based conflict detector achieves 97% precision at the weakest specification level without additional LLM calls, yet a factorial recovery experiment shows that restoring the full specification alone recovers the single-agent ceiling (89%), while providing conflict reports adds no measurable benefit. Third, decomposing the gap into coordination cost (+16 pp) and information asymmetry (+11 pp) suggests that the two effects are independent and approximately additive. The gap is not merely a consequence of hidden information, but reflects the difficulty of producing compatible code without shared decisions. These results support a specification-first view of multi-agent code generation: richer specifications are both the primary coordination mechanism and the sufficient recovery instrument.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 11 of its 24 references that are in the map. First posted on arXiv 25 Mar 2026.
Builds on
11 of 24 references.
Closest ancestors · 4
Same direction, newest first
- 2025AdaCoder: An Adaptive Planning and Multi-Agent Framework for Function-Level Code Generation5 Apr
- 2025CODESIM: Multi-Agent Code Generation and Problem Solving through Simulation-Driven Planning and Debugging8 Feb
- 2024CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges14 Jan
- 2023ChatDev: Communicative Agents for Software Development16 Jul
Other references · 7
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2021Program Synthesis with Large Language Modelscited by 3,637
- 2023Code Llama: Open Foundation Models for Codecited by 2,139
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
- 2023StarCoder: may the source be with you!cited by 799
- 2023ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generationcited by 168
- 2025EvoGit: Decentralized Code Evolution via Git-Based Multi-Agent Collaborationcited by 9
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Grounding AI Agents in Contracts: An Empirical Evaluation of Spec-Driven Test Generation
- 2026SpecPath: Testing Coding Agents Across Contract-Equivalent Specification Histories
- 2026How Generation Architecture Shapes Code Complexity in Multi-Agent LLM Systems: A Paired Study on HumanEval
- 2026Who Holds the Pen? Let Specifications, Not Agents, Sign Off
- 2026When the Specification Emerges: Benchmarking Faithfulness Loss in Long-Horizon Coding Agents
- 2026Constraint Decay: The Fragility of LLM Agents in Backend Code Generation
- 2026LiveFMBench: Unveiling the Power and Limits of Agentic Workflows in Specification Generation
- 2026SpecFirst: Behavioral Specification Elicitation as a First-Class Step in Agent-Based Program Synthesis from Scratch