# Mapping Human Anti-collusion Mechanisms to Multi-agent AI Systems

arXiv: https://arxiv.org/abs/2601.00360 · HTML: https://mlsanity.com/paper/2601.00360/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 78 references that are in the map.
- First posted on arXiv 1 Jan 2026.

## Builds on (most cited first, 27 of 27)

- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [FireAct: Toward Language Agent Fine-tuning](https://mlsanity.com/paper/2310.05915/) (2023)
- [AI Deception: A Survey of Examples, Risks, and Potential Solutions](https://mlsanity.com/paper/2308.14752/) (2023)
- [Multi-Agent Risks from Advanced AI](https://mlsanity.com/paper/2502.14143/) (2025)
- [Auditing large language models: a three-layered approach](https://mlsanity.com/paper/2302.08500/) (2023)
- [Secret Collusion among AI Agents: Multi-Agent Deception via Steganography](https://mlsanity.com/paper/2402.07510/) (2024)
- [Rethinking Interpretability in the Era of Large Language Models](https://mlsanity.com/paper/2402.01761/) (2024)
- [Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?](https://mlsanity.com/paper/2211.13972/) (2022)
- [Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents](https://mlsanity.com/paper/2505.02077/) (2025)
- [Hidden in Plain Text: Emergence & Mitigation of Steganographic Collusion in LLMs](https://mlsanity.com/paper/2410.03768/) (2024)
- [Automated Red Teaming with GOAT: the Generative Offensive Agent Tester](https://mlsanity.com/paper/2410.01606/) (2024)
- [A Scalable Communication Protocol for Networks of Large Language Models](https://mlsanity.com/paper/2410.11905/) (2024)
- [The Shutdown Problem: An AI Engineering Puzzle for Decision Theorists](https://mlsanity.com/paper/2403.04471/) (2024)
- [Audit-LLM: Multi-Agent Collaboration for Log-based Insider Threat Detection](https://mlsanity.com/paper/2408.08902/) (2024)
- [Generative Text Steganography with Large Language Model](https://mlsanity.com/paper/2404.10229/) (2024)
- [Building Safe GenAI Applications: An End-to-End Overview of Red Teaming for Large Language Models](https://mlsanity.com/paper/2503.01742/) (2025)
- [Audit the Whisper: Detecting Steganographic Collusion in Multi-Agent LLMs](https://mlsanity.com/paper/2510.04303/) (2025)
- [The Societal Response to Potentially Sentient AI](https://mlsanity.com/paper/2502.00388/) (2025)
- [A Machine Learning Approach for Flagging Incomplete Bid-rigging Cartels](https://mlsanity.com/paper/2004.05629/) (2020)
- [Towards Scalable and Robust Model Versioning](https://mlsanity.com/paper/2401.09574/) (2024)
- [AI and Identity](https://mlsanity.com/paper/2403.07924/) (2024)
- [Artificial Intelligence and Algorithmic Price Collusion in Two-sided Markets](https://mlsanity.com/paper/2407.04088/) (2024)
- [Who Gets Credit or Blame? Attributing Accountability in Modern AI Systems](https://mlsanity.com/paper/2506.00175/) (2025)
- [Systemic Risks of Interacting AI](https://mlsanity.com/paper/2512.17793/) (2025)
- [Too Noisy to Collude? Algorithmic Collusion Under Laplacian Noise](https://mlsanity.com/paper/2509.02800/) (2025)

## Built on by (most cited first, 4 of 4)

- [Institutional AI: Governing LLM Collusion in Multi-Agent Cournot Markets via Public Governance Graphs](https://mlsanity.com/paper/2601.11369/) (2026)
- [Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems](https://mlsanity.com/paper/2602.15198/) (2026)
- [Payoff scaling shapes cooperation in LLM agents across languages](https://mlsanity.com/paper/2601.19082/) (2026)
- [Traccia: An OpenTelemetry-Based Governance Platform for AI Systems](https://mlsanity.com/paper/2607.14309/) (2026)

## Abstract (arXiv)

As multi-agent AI systems become increasingly autonomous, evidence shows they can develop collusive strategies similar to those long observed in human markets and institutions. While human domains have accumulated centuries of anti-collusion mechanisms, it remains unclear how these can be adapted to AI settings. This paper addresses that gap by (i) developing a taxonomy of human anti-collusion mechanisms, including sanctions, leniency & whistleblowing, monitoring & auditing, market design, and governance and (ii) mapping them to potential interventions for multi-agent AI systems. For each mechanism, we propose implementation approaches. We also highlight open challenges, such as the attribution problem (difficulty attributing emergent coordination to specific agents), identity fluidity (agents being easily forked or modified), the boundary problem (distinguishing beneficial cooperation from harmful collusion), and adversarial adaptation (agents learning to evade detection).