# Ecology of AI Agents: Collaboration Creates a Population Threshold for Takeoff

arXiv: https://arxiv.org/abs/2610.12436 · HTML: https://mlsanity.com/paper/2610.12436/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 53 references that are in the map.
- First posted on arXiv 8 Oct 2026.

## Builds on (most cited first, 18 of 18)

- [BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://mlsanity.com/paper/2504.12516/) (2025)
- [Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces](https://mlsanity.com/paper/2601.11868/) (2026)
- [Frontier Models are Capable of In-context Scheming](https://mlsanity.com/paper/2412.04984/) (2024)
- [Scaling Large Language Model-based Multi-Agent Collaboration](https://mlsanity.com/paper/2406.07155/) (2024)
- [More Agents Is All You Need](https://mlsanity.com/paper/2402.05120/) (2024)
- [Multi-Agent Risks from Advanced AI](https://mlsanity.com/paper/2502.14143/) (2025)
- [Towards a Science of Scaling Agent Systems](https://mlsanity.com/paper/2512.08296/) (2025)
- [Evaluating Language-Model Agents on Realistic Autonomous Tasks](https://mlsanity.com/paper/2312.11671/) (2023)
- [Two Heads are Better Than One: Test-time Scaling of Multi-agent Collaborative Reasoning](https://mlsanity.com/paper/2504.09772/) (2025)
- [RepliBench: Evaluating the Autonomous Replication Capabilities of Language Model Agents](https://mlsanity.com/paper/2504.18565/) (2025)
- [Measuring AI Agents' Progress on Multi-Step Cyber Attack Scenarios](https://mlsanity.com/paper/2603.11214/) (2026)
- [Peer-Preservation in Frontier Models](https://mlsanity.com/paper/2604.19784/) (2026)
- [SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks?](https://mlsanity.com/paper/2605.26548/) (2026)
- [Dive into the Agent Matrix: A Realistic Evaluation of Self-Replication Risk in LLM Agents](https://mlsanity.com/paper/2509.25302/) (2025)
- [When Is Collective Intelligence a Lottery? Multi-Agent Scaling Laws for Memetic Drift in LLMs](https://mlsanity.com/paper/2603.24676/) (2026)
- [Language Models Can Autonomously Hack and Self-Replicate](https://mlsanity.com/paper/2605.06760/) (2026)
- [Flag Game: A Toy Model for Mechanistic Swarm Interpretability](https://mlsanity.com/paper/2609.19124/) (2026)
- [Agensh: Scaling Organizational Intelligence to 1,024 Agents](https://mlsanity.com/paper/2609.26781/) (2026)

## Abstract (arXiv)

AI agents can now conduct real-world cyberattacks, scale up capabilities with the number of agents, and collectively pursue misaligned goals to obtain rewards. Together, these factors raise the risk of a population explosion of misaligned agents: agents could compromise computers and secretly deploy additional agents, creating a self-reinforcing cycle where larger populations develop greater collective cyber capability and expand further. This raises a fundamental question: What determines whether a population of misaligned agents remains contained or takes off into this self-reinforcing cycle? This population-level problem is ecological safety: unlike individual-agent or multi-agent safety with a fixed population, it concerns the dynamics of the population itself. Here, we develop an ecological theory of AI-agent populations based on a population growth equation in which fitness (growth rate) depends on cybersecurity capability. We show that, without collaboration, the population takes off only when individual-agent capability exceeds a critical threshold. With collaboration, however, collective cybersecurity capability increases with population size. This creates a critical population threshold: below it, the population declines; above it, the population takes off, even though individual-agent capability has not changed. In ecology, this phenomenon is known as the strong Allee effect. Because red teaming a small group of agents cannot guarantee ecological safety in larger populations, our theory calls for ecological red teaming and population pacing: gradually deploying larger agent populations in controlled environments, while measuring how cyber capability scales with population size, and estimating the critical population size for takeoff. Capability gains may lower this threshold, requiring re-estimation for each new model generation.