# Towards a Science of Scaling Agent Systems

arXiv: https://arxiv.org/abs/2512.08296 · HTML: https://mlsanity.com/paper/2512.08296/

## Key facts

- Cited by 138 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 53 of its 65 references that are in the map.
- First posted on arXiv 9 Dec 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 30 of 53)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [Large Language Model based Multi-Agents: A Survey of Progress and Challenges](https://mlsanity.com/paper/2402.01680/) (2024)
- [BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://mlsanity.com/paper/2504.12516/) (2025)
- [τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environment](https://mlsanity.com/paper/2506.07982/) (2025)
- [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025)
- [AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors](https://mlsanity.com/paper/2308.10848/) (2023)
- [Multi-Agent Collaboration Mechanisms: A Survey of LLMs](https://mlsanity.com/paper/2501.06322/) (2025)
- [Cognitive Architectures for Language Agents](https://mlsanity.com/paper/2309.02427/) (2023)
- [Accelerating scientific discovery with Co-Scientist](https://mlsanity.com/paper/2502.18864/) (2025)

## Built on by (most cited first, 30 of 138)

- [When Single-Agent with Skills Replace Multi-Agent Systems and When They Fail](https://mlsanity.com/paper/2601.04748/) (2026)
- [Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets](https://mlsanity.com/paper/2604.02460/) (2026)
- [Understanding Agent Scaling in LLM-Based Multi-Agent Systems via Diversity](https://mlsanity.com/paper/2602.03794/) (2026)
- [Rethinking the Value of Multi-Agent Workflow: A Strong Single Agent Baseline](https://mlsanity.com/paper/2601.12307/) (2026)
- [Structured Scaling of AI Discovery Across Diverse Scientific Domains](https://mlsanity.com/paper/2604.19341/) (2026)
- [Rethinking the Value of Agent-Generated Tests for LLM-Based Software Engineering Agents](https://mlsanity.com/paper/2602.07900/) (2026)
- [AgentArk: Distilling Multi-Agent Intelligence into a Single LLM Agent](https://mlsanity.com/paper/2602.03955/) (2026)
- [HEARTS: Benchmarking LLM Reasoning on Health Time Series](https://mlsanity.com/paper/2603.06638/) (2026)
- [AI for Auto-Research: Roadmap & User Guide](https://mlsanity.com/paper/2605.18661/) (2026)
- [Dual Latent Memory for Visual Multi-agent System](https://mlsanity.com/paper/2602.00471/) (2026)
- [Agents in the Wild: Safety, Society, and the Illusion of Sociality on Moltbook](https://mlsanity.com/paper/2602.13284/) (2026)
- [How Well Does Agent Development Reflect Real-World Work?](https://mlsanity.com/paper/2603.01203/) (2026)
- [Hidden Measurement Error in LLM Pipelines Distorts Annotation, Evaluation, and Benchmarking](https://mlsanity.com/paper/2604.11581/) (2026)
- [TeamBench: Evaluating Agent Coordination under Enforced Role Separation](https://mlsanity.com/paper/2605.07073/) (2026)
- [AutoScientists: Self-Organizing Agent Teams for Long-Running Scientific Experimentation](https://mlsanity.com/paper/2605.28655/) (2026)
- [The Illusion of Multi-Agent Advantage](https://mlsanity.com/paper/2606.13003/) (2026)
- [An Information Theoretic Perspective on Agentic System Design](https://mlsanity.com/paper/2512.21720/) (2025)
- [On the Reliability Limits of LLM-Based Multi-Agent Planning](https://mlsanity.com/paper/2603.26993/) (2026)
- [Proof of Time: A Benchmark for Evaluating Scientific Idea Judgments](https://mlsanity.com/paper/2601.07606/) (2026)
- [Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptation](https://mlsanity.com/paper/2607.08938/) (2026)
- [Token Economics for LLM Agents: A Dual-View Study from Computing and Economics](https://mlsanity.com/paper/2605.09104/) (2026)
- [Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2605.14892/) (2026)
- [EvoMAS: Evolutionary Generation of Multi-Agent Systems](https://mlsanity.com/paper/2602.06511/) (2026)
- [Table-as-Search: Formulate Long-Horizon Agentic Information Seeking as Table Completion](https://mlsanity.com/paper/2602.06724/) (2026)
- [Sovereign Agentic Loops: Decoupling AI Reasoning from Execution in Real-World Systems](https://mlsanity.com/paper/2604.22136/) (2026)
- [TacoMAS: Test-Time Co-Evolution of Topology and Capability in LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2605.09539/) (2026)
- [Economy of Minds: Emerging Multi-Agent Intelligence with Economic Interactions](https://mlsanity.com/paper/2606.02859/) (2026)
- [MAS-PromptBench: When Does Prompt Optimization Improve Multi-Agent LLM Systems?](https://mlsanity.com/paper/2606.23664/) (2026)
- [Phase Transition for Budgeted Multi-Agent Synergy](https://mlsanity.com/paper/2601.17311/) (2026)
- [El Agente Gr'afico: A Semantic Execution Runtime for Scientific Agents](https://mlsanity.com/paper/2602.17902/) (2026)

## Abstract (arXiv)

Agents, language model-based systems capable of reasoning, planning, and acting are widely adopted in real-world tasks, yet how their performance changes as these systems scale across key dimensions remains underexplored. We introduce quantitative scaling principles for agent systems as a predictive model, capturing how performance varies with coordination, model capability, and measurable system and task factors. Across 260 configurations spanning six agentic benchmarks, five canonical architectures (Single-Agent and four Multi-Agent: Independent, Centralized, Decentralized, Hybrid), and three LLM families, we perform controlled evaluations, standardizing tools, prompts, and compute to isolate architectural effects. The resulting model achieves a cross-validated R^2=0.373 across all six benchmarks (R^2=0.413 with a task-grounded capability metric). We identify a robust capability-saturation effect and additional patterns: (1) a coordination yields diminishing returns once single-agent baselines exceed certain performance; (2) tool-heavy tasks appear to incur multi-agent overhead; and (3) architectures without centralized verification tend to propagate errors more than those with centralized coordination. Relative performance change compared to single-agent baseline ranges from +80.8% on decomposable financial reasoning to -70.0% on sequential planning, demonstrating that architecture-task alignment determines collaborative success. The framework identifies the best-performing architecture for 87% of held-out configurations and shows consistent relative architecture preferences on unseen frontier models. Agent effectiveness depends on alignment between coordination and task structure, and that mismatched coordination degrades the performance.