# Multi-Agent Risks from Advanced AI

arXiv: https://arxiv.org/abs/2502.14143 · HTML: https://mlsanity.com/paper/2502.14143/

## Key facts

- Cited by 167 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Feb 2025.

## Built on by (most cited first, 30 of 167)

- [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025)
- [Survey on Evaluation of LLM-based Agents](https://mlsanity.com/paper/2503.16416/) (2025)
- [AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges](https://mlsanity.com/paper/2505.10468/) (2025)
- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment](https://mlsanity.com/paper/2504.15585/) (2025)
- [A Survey on Trustworthy LLM Agents: Threats and Countermeasures](https://mlsanity.com/paper/2503.09648/) (2025)
- [Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents](https://mlsanity.com/paper/2505.02077/) (2025)
- [Your Agent May Misevolve: Emergent Risks in Self-evolving LLM Agents](https://mlsanity.com/paper/2509.26354/) (2025)
- [AI4Research: A Survey of Artificial Intelligence for Scientific Research](https://mlsanity.com/paper/2507.01903/) (2025)
- [A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasures](https://mlsanity.com/paper/2506.19676/) (2025)
- [Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI](https://mlsanity.com/paper/2505.19443/) (2025)
- [Virtual Agent Economies](https://mlsanity.com/paper/2509.10147/) (2025)
- [How malicious AI swarms can threaten democracy: The fusion of agentic AI and LLMs marks a new frontier in information warfare](https://mlsanity.com/paper/2506.06299/) (2025)
- [Risk Analysis Techniques for Governed LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2508.05687/) (2025)
- [An Economy of AI Agents](https://mlsanity.com/paper/2509.01063/) (2025)
- [Magentic Marketplace: An Open-Source Environment for Studying Agentic Markets](https://mlsanity.com/paper/2510.25779/) (2025)
- [Institutional AI: Governing LLM Collusion in Multi-Agent Cournot Markets via Public Governance Graphs](https://mlsanity.com/paper/2601.11369/) (2026)
- [Intelligent AI Delegation](https://mlsanity.com/paper/2602.11865/) (2026)
- [Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems](https://mlsanity.com/paper/2602.15198/) (2026)
- [CoopEval: Benchmarking Cooperation-Sustaining Mechanisms and LLM Agents in Social Dilemmas](https://mlsanity.com/paper/2604.15267/) (2026)
- [Group size effects and collective misalignment in LLM multi-agent systems](https://mlsanity.com/paper/2510.22422/) (2025)
- [FAIRGAME: a Framework for AI Agents Bias Recognition using Game Theory](https://mlsanity.com/paper/2504.14325/) (2025)
- [Must Read: A Comprehensive Survey of Computational Persuasion](https://mlsanity.com/paper/2505.07775/) (2025)
- [Inherent and emergent liability issues in LLM-based agentic systems: a principal-agent perspective](https://mlsanity.com/paper/2504.03255/) (2025)
- [Seven Security Challenges in Cross-domain Multi-agent LLM Systems](https://mlsanity.com/paper/2505.23847/) (2025)
- [Evaluating LLM Agent Collusion in Double Auctions](https://mlsanity.com/paper/2507.01413/) (2025)
- [A Pragmatic View of AI Personhood](https://mlsanity.com/paper/2510.26396/) (2025)
- [Media and responsible AI governance: a game-theoretic and LLM analysis](https://mlsanity.com/paper/2503.09858/) (2025)
- [The Singapore Consensus on Global AI Safety Research Priorities](https://mlsanity.com/paper/2506.20702/) (2025)
- [Distributional AGI Safety](https://mlsanity.com/paper/2512.16856/) (2025)

## Abstract (arXiv)

The rapid development of advanced AI agents and the imminent deployment of many instances of these agents will give rise to multi-agent systems of unprecedented complexity. These systems pose novel and under-explored risks. In this report, we provide a structured taxonomy of these risks by identifying three key failure modes (miscoordination, conflict, and collusion) based on agents' incentives, as well as seven key risk factors (information asymmetries, network effects, selection pressures, destabilising dynamics, commitment problems, emergent agency, and multi-agent security) that can underpin them. We highlight several important instances of each risk, as well as promising directions to help mitigate them. By anchoring our analysis in a range of real-world examples and experimental evidence, we illustrate the distinct challenges posed by multi-agent systems and their implications for the safety, governance, and ethics of advanced AI.