# Learning to Mitigate AI Collusion on Economic Platforms

arXiv: https://arxiv.org/abs/2202.07106 · HTML: https://mlsanity.com/paper/2202.07106/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 68 references that are in the map.
- First posted on arXiv 15 Feb 2022.

## Builds on (most cited first, 21 of 21)

- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [The AI Economist: Improving Equality and Productivity with AI-Driven Tax Policies](https://mlsanity.com/paper/2004.13332/) (2020)
- [Automated Mechanism Design via Neural Networks](https://mlsanity.com/paper/1805.03382/) (2018)
- [A Permutation-Equivariant Neural Network Architecture For Auction Design](https://mlsanity.com/paper/2003.01497/) (2020)
- [Why Should I Trust You, Bellman? The Bellman Error is a Poor Replacement for Value Error](https://mlsanity.com/paper/2201.12417/) (2022)
- [Auction learning as a two-player game](https://mlsanity.com/paper/2006.05684/) (2020)
- [M3RL: Mind-aware Multi-agent Management Reinforcement Learning](https://mlsanity.com/paper/1810.00147/) (2018)
- [PreferenceNet: Encoding Human Preferences in Auction Design with Deep Learning](https://mlsanity.com/paper/2106.03215/) (2021)
- [Certifying Strategyproof Auction Networks](https://mlsanity.com/paper/2006.08742/) (2020)
- [ProportionNet: Balancing Fairness and Revenue for Auction Design with Deep Learning](https://mlsanity.com/paper/2010.06398/) (2020)
- [Multi-agent Reinforcement Learning in Bayesian Stackelberg Markov Games for Adaptive Moving Target Defense](https://mlsanity.com/paper/2007.10457/) (2020)
- [Learning Revenue-Maximizing Auctions With Differentiable Matching](https://mlsanity.com/paper/2106.07877/) (2021)
- [The AI Economist: Optimal Economic Policy Design via Two-level Deep Reinforcement Learning](https://mlsanity.com/paper/2108.02755/) (2021)
- [Reinforcement Mechanism Design, with Applications to Dynamic Pricing in Sponsored Search Auctions](https://mlsanity.com/paper/1711.10279/) (2017)
- [Coordinating the Crowd: Inducing Desirable Equilibria in Non-Cooperative Systems](https://mlsanity.com/paper/1901.10923/) (2019)
- [Deep Learning-powered Iterative Combinatorial Auctions](https://mlsanity.com/paper/1907.05771/) (2019)
- [Robust Reinforcement Learning Under Minimax Regret for Green Security](https://mlsanity.com/paper/2106.08413/) (2021)
- [Coordinating Followers to Reach Better Equilibria: End-to-End Gradient Descent for Stackelberg Games](https://mlsanity.com/paper/2106.03278/) (2021)
- [Convex-Concave Min-Max Stackelberg Games](https://mlsanity.com/paper/2110.05192/) (2021)
- [Reinforcement Learning of Sequential Price Mechanisms](https://mlsanity.com/paper/2010.01180/) (2020)
- [Model-free Reinforcement Learning for Stochastic Stackelberg Security Games](https://mlsanity.com/paper/2005.11853/) (2020)

## Built on by (most cited first, 10 of 10)

- [Secret Collusion among AI Agents: Multi-Agent Deception via Steganography](https://mlsanity.com/paper/2402.07510/) (2024)
- [LLM Economist: Large Population Models and Mechanism Design in Multi-Agent Generative Simulacra](https://mlsanity.com/paper/2507.15815/) (2025)
- [Oracles & Followers: Stackelberg Equilibria in Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2210.11942/) (2022)
- [Seven Security Challenges in Cross-domain Multi-agent LLM Systems](https://mlsanity.com/paper/2505.23847/) (2025)
- [On the Fragility of AI Agent Collusion](https://mlsanity.com/paper/2603.20281/) (2026)
- [Artificial Intelligence and Algorithmic Price Collusion in Two-sided Markets](https://mlsanity.com/paper/2407.04088/) (2024)
- [Manipulation and Peer Mechanisms: A Survey](https://mlsanity.com/paper/2210.01984/) (2022)
- [Impact of Price Inflation on Algorithmic Collusion Through Reinforcement Learning Agents](https://mlsanity.com/paper/2504.05335/) (2025)
- [Mitigating Retaliatory Algorithmic Collusion in Repeated Games](https://mlsanity.com/paper/2609.20548/) (2026)
- [When and How Order-Flow Interventions Work After Dealers Adapt](https://mlsanity.com/paper/2610.18158/) (2026)

## Abstract (arXiv)

Algorithmic pricing on online e-commerce platforms raises the concern of tacit collusion, where reinforcement learning algorithms learn to set collusive prices in a decentralized manner and through nothing more than profit feedback. This raises the question as to whether collusive pricing can be prevented through the design of suitable "buy boxes," i.e., through the design of the rules that govern the elements of e-commerce sites that promote particular products and prices to consumers. In this paper, we demonstrate that reinforcement learning (RL) can also be used by platforms to learn buy box rules that are effective in preventing collusion by RL sellers. For this, we adopt the methodology of Stackelberg POMDPs, and demonstrate success in learning robust rules that continue to provide high consumer welfare together with sellers employing different behavior models or having out-of-distribution costs for goods.