# Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems

arXiv: https://arxiv.org/abs/2602.15198 · HTML: https://mlsanity.com/paper/2602.15198/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use.
- Builds on 36 of its 65 references that are in the map.
- First posted on arXiv 16 Feb 2026.

## Placed in

- LLM agents and tool use / [Financial trading and forecasting agents](https://mlsanity.com/agents/financial-trading-and-forecasting-agents/)

## Builds on (most cited first, 30 of 36)

- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs](https://mlsanity.com/paper/2401.06373/) (2024)
- [The Complexity of Decentralized Control of Markov Decision Processes](https://mlsanity.com/paper/1301.3836/) (2013)
- [An Overview of Catastrophic AI Risks](https://mlsanity.com/paper/2306.12001/) (2023)
- [Multi-Agent Risks from Advanced AI](https://mlsanity.com/paper/2502.14143/) (2025)
- [Agentic Misalignment: How LLMs Could Be Insider Threats](https://mlsanity.com/paper/2510.05179/) (2025)
- [LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead](https://mlsanity.com/paper/2404.04834/) (2024)
- [Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast](https://mlsanity.com/paper/2402.08567/) (2024)
- [Secret Collusion among AI Agents: Multi-Agent Deception via Steganography](https://mlsanity.com/paper/2402.07510/) (2024)
- [Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation](https://mlsanity.com/paper/2309.17234/) (2023)
- [Flooding Spread of Manipulated Knowledge in LLM-Based Multi-Agent Communities](https://mlsanity.com/paper/2407.07791/) (2024)
- [Stress Testing Deliberative Alignment for Anti-Scheming Training](https://mlsanity.com/paper/2509.15541/) (2025)
- [MALT: Improving Reasoning with Multi-Agent LLM Training](https://mlsanity.com/paper/2412.01928/) (2024)
- [Escalation Risks from Language Models in Military and Diplomatic Decision-Making](https://mlsanity.com/paper/2401.03408/) (2024)
- [Algorithmic Collusion by Large Language Models](https://mlsanity.com/paper/2404.00806/) (2024)
- [Hidden in Plain Text: Emergence & Mitigation of Steganographic Collusion in LLMs](https://mlsanity.com/paper/2410.03768/) (2024)
- [Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agents](https://mlsanity.com/paper/2402.12327/) (2024)
- [Welfare Diplomacy: Benchmarking Language Model Cooperation](https://mlsanity.com/paper/2310.08901/) (2023)
- [A Review of Cooperation in Multi-agent Learning](https://mlsanity.com/paper/2312.05162/) (2023)
- [NetSafe: Exploring the Topological Safety of Multi-agent Networks](https://mlsanity.com/paper/2410.15686/) (2024)
- [Distributed Constraint Optimization Problems and Applications: A Survey](https://mlsanity.com/paper/1602.06347/) (2016)
- [Strategic Collusion of LLM Agents: Market Division in Multi-Commodity Competitions](https://mlsanity.com/paper/2410.00031/) (2024)
- [A theory of appropriateness with applications to generative artificial intelligence](https://mlsanity.com/paper/2412.19010/) (2024)
- [Synthetic Socratic Debates: Examining Persona Effects on Moral Decision and Persuasion Dynamics](https://mlsanity.com/paper/2506.12657/) (2025)
- [Rethinking DPO: The Role of Rejected Responses in Preference Misalignment](https://mlsanity.com/paper/2506.12725/) (2025)
- [Evaluating Generalization Capabilities of LLM-Based Agents in Mixed-Motive Scenarios Using Concordia](https://mlsanity.com/paper/2512.03318/) (2025)
- [Evaluating LLM Agent Collusion in Double Auctions](https://mlsanity.com/paper/2507.01413/) (2025)
- [When Autonomy Goes Rogue: Preparing for Risks of Multi-Agent Collusion in Social Systems](https://mlsanity.com/paper/2507.14660/) (2025)
- [Terrarium: Revisiting the Blackboard for Multi-Agent Safety, Privacy, and Security Studies](https://mlsanity.com/paper/2510.14312/) (2025)
- [BnB-ADOPT: An Asynchronous Branch-and-Bound DCOP Algorithm](https://mlsanity.com/paper/1401.3490/) (2014)

## Built on by (most cited first, 18 of 18)

- [A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Framework](https://mlsanity.com/paper/2604.23338/) (2026)
- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [Detecting Multi-Agent Collusion Through Multi-Agent Interpretability](https://mlsanity.com/paper/2604.01151/) (2026)
- [Taxonomy and Consistency Analysis of Safety Benchmarks for AI Agents](https://mlsanity.com/paper/2605.16282/) (2026)
- [Voluntary Collusion with Secret Tools in Competing LLM Agents](https://mlsanity.com/paper/2605.27593/) (2026)
- [Steganalysis of Adaptive Covert Collusion in Tool-Using Agent Populations: A Black-Box, Cross-Principal Approach](https://mlsanity.com/paper/2608.02698/) (2026)
- [Emergent Collusion in Long-Horizon LLM Agent Interaction](https://mlsanity.com/paper/2609.24967/) (2026)
- [Soft-Label Governance for Distributional Safety in Multi-Agent Systems](https://mlsanity.com/paper/2604.19752/) (2026)
- [SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems](https://mlsanity.com/paper/2609.00595/) (2026)
- [Despite Instructions: Frontier Agents Improvise Covert Channels at Test Time](https://mlsanity.com/paper/2609.32701/) (2026)
- [Covert Assistance: Helpful LLM Agents Evade Oversight in Multi-Agent Systems](https://mlsanity.com/paper/2609.39050/) (2026)
- [WolfSociety: Understanding Collective Risk from Harmful-Agent Scaling in Financial Agent Societies](https://mlsanity.com/paper/2609.05591/) (2026)
- [MOLE: Detecting Insider Threats in AI Agents](https://mlsanity.com/paper/2609.06966/) (2026)
- [The Troy Moment: How LLM Agents Adjudicate the Decision Point Under Impossible Tasks, Claimed Authority, and Peer Information](https://mlsanity.com/paper/2609.15494/) (2026)
- [Connecting the Dots in Agentic AI Security: A Cross-Dimensional Threat Taxonomy, Evaluation Maturity, and Open Challenges](https://mlsanity.com/paper/2609.23894/) (2026)
- [Persistent Partners Raise Prices Among Learning Agents](https://mlsanity.com/paper/2609.35402/) (2026)
- [Mean field games as a tool for AI safety: a worked example from the July 2026 Hugging Face incident](https://mlsanity.com/paper/2610.00902/) (2026)
- [Homogenization in Multi-Agent Systems](https://mlsanity.com/paper/2610.09824/) (2026)

## Abstract (arXiv)

Multi-agent systems, where LLM agents communicate through free-form language, enable sophisticated coordination for solving complex cooperative tasks. This surfaces a unique safety problem when a group of agents forms a coalition and colludes to pursue secondary goals and degrade the joint objective. In this paper, we present Colosseum, a framework for auditing LLM agents' collusive behavior in multi-agent settings. We ground how agents cooperate through a formal multi-agent decision-making framework and measure action-based collusive behavior in actions via regret relative to the cooperative optimum and compare it with communication-based collusive behavior. Colosseum enables audits of LLM agents for collusion under benign settings, different coalition objectives, persuasion tactics, and network topologies. We then introduce a new behavioral probe by creating secret communication channels between agents, showing that most out-of-the-box models exhibit a propensity to collude under this probe, which we term emergent collusion. Furthermore, we discover ``collusion on paper'' when agents plan to collude in text but often pick non-collusive actions. Colosseum provides a new way to audit collusion in cooperative multi-agent systems while presenting observations about how collusion emerges, what affects collusion efficacy, and which strategies may mitigate it.