An Exact Generate - Transform Decomposition of Small-LLM Team Scaling Across Orchestration Architectures
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Replacing one LLM agent with a collaborating team can raise accuracy, but whether scaling the team helps, and which architecture to scale, is unclear. Sweeping eight agent orchestration architectures across five instruction-tuned 7-9B models, five short-answer benchmarks, and an executable-code benchmark up to 30 calls, we find that the returns to team scaling are sharply task-dependent: from three to thirty calls accuracy rises by up to 17 points on the two arithmetic word-problem benchmarks (GSM8K, GSMHard) but by at most four on ARC, GPQA, and MMLU, for every architecture, a split the usual task-averaged number conceals. Proposer-Critic captures the arithmetic gains, scaling steepest and, in aggregate, surpassing every other architecture at the largest budget (item-clustered intervals exclude zero), though it ranks among the weakest elsewhere, and no architecture wins across tasks. We explain these trajectories with an exact generate-transform decomposition. Partitioning any workflow into proposal coverage and a downstream transform, any accuracy change splits exactly into an extensive coverage dividend and an intensive transformation change. The decomposition diagnoses each task: arithmetic offers coverage headroom that a critic-guided transform converts, whereas the multiple-choice benchmarks either saturate in coverage or fail to convert it, and on open-ended code generative recovery nearly vanishes so accuracy tracks coverage. At equal call budgets token cost still varies 2.1x. Extra calls therefore create candidate opportunity that only some architectures, on some tasks, convert. Team scaling is a task- and architecture-specific bet, not a uniform lever.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 18 of its 37 references that are in the map. First posted on arXiv 28 Sep 2026.
Builds on
18 of 37 references.
References · 18
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
- 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
- 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
- 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
- 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
6 more
- 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 420
- 2025MultiAgentBench: Evaluating the Collaboration and Competition of LLM agentscited by 169
- 2025Multi-agent Architecture Search via Agentic Supernetcited by 164
- 2025Towards a Science of Scaling Agent Systemscited by 125
- 2025LLMs Can Generate a Better Answer by Aggregating Their Own Responsescited by 27
- 2026When Agents Disagree: The Selection Bottleneck in Multi-Agent LLM Pipelinescited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026How Generation Architecture Shapes Code Complexity in Multi-Agent LLM Systems: A Paired Study on HumanEval
- 2026Multi-agent Scaling Across Disjunctive and Compensatory Tasks
- 2026Team of Thoughts: Efficient Test-time Scaling of Agentic Systems through Orchestrated Tool Calling
- 2026Scaling LLM-Driven Multi-Agent Systems: Design Principles and Architectural Scalability Analysis
- 2026Architectural Design, Not Only Model Intelligence, Governs Multi-Agent LLM Performance
- 2026PerspectiveGap: A Benchmark for Multi-Agent Orchestration Prompting
- 2026ClawArena-Team: Benchmarking Subagent Orchestration and Dynamic Workflows in Language-Model Agents
- 2026AdaptOrch: Task-Adaptive Multi-Agent Orchestration in the Era of LLM Performance Convergence