When Agents Go Quiet: Output Generation Capacity and Format-Cost Separation for LLM Document Synthesis
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
LLM-powered coding agents suffer from a poorly understood failure mode we term output stalling: the agent silently produces empty responses when attempting to generate large, format-heavy documents. We present a theoretical framework that explains and prevents this failure through three contributions. (1) We introduce Output Generation Capacity (OGC), a formal measure of an agent's effective ability to produce output given its current context state - distinct from and empirically smaller than the raw context window. (2) We prove a Format-Cost Separation Theorem showing that deferred template rendering is always at least as token-efficient as direct generation for any format with overhead multiplier μf > 1, and derive tight bounds on the savings. (3) We formalize Adaptive Strategy Selection, a decision framework that maps the ratio of estimated output cost to available OGC into an optimal generation strategy (direct, chunked, or deferred). We validate the theory through controlled experiments across three models (Claude 3.5 Sonnet, GPT-4o, Llama 3.1 70B), four document types, and an ablation study isolating each component's contribution. Deferred rendering reduces LLM generation tokens by 48-72% across all conditions and eliminates output stalling entirely. We instantiate the framework as GEN-PILOT, an open-source MCP server, demonstrating that the theory translates directly into a practical tool.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 26 of its 38 references that are in the map. First posted on arXiv 17 Apr 2026.
Builds on
26 of 38 references.
Closest ancestors · 2
Same direction, newest first
- 2026Resilient Write: A Six-Layer Durable Write Surface for LLM Coding Agents12 Apr
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May
Other references · 24
- 2017Attention Is All You Needcited by 60,361
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
- 2022Survey of Hallucination in Natural Language Generationcited by 2,075
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
12 more
- 2022Language Models (Mostly) Know What They Knowcited by 1,856
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
- 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
- 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
- 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
- 2023Efficient Guided Generation for Large Language Modelscited by 200
- 2022MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoningcited by 156
- 2023Retrieval meets Long Context Large Language Modelscited by 115
- 2022Prompting Is Programming: A Query Language for Large Language Modelscited by 110
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026An Empirical Cost Attribution of Context-Compression Gateways in Multi-Turn Coding Agents
- 2026CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Agents
- 2026When the Specification Emerges: Benchmarking Faithfulness Loss in Long-Horizon Coding Agents
- 2026Constraint Decay: The Fragility of LLM Agents in Backend Code Generation
- 2025More with Less: An Empirical Study of Turn-Control Strategies for Efficient Coding Agents
- 2026Beyond Token Savings: A Systematic Study of Context Compression in LLM Agents
- 2026Maximising Transparency in Context Compaction Improves Retention of Facts and Commitments for Lightweight Coding Agents
- 2026Agents Don't Paginate: First-Chunk Selection for LLM Tool Responses