# Learned Collusion

arXiv: https://arxiv.org/abs/2304.12647 · HTML: https://mlsanity.com/paper/2304.12647/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 39 references that are in the map.
- First posted on arXiv 25 Apr 2023.

## Builds on (most cited first, 3 of 3)

- [Artificial Intelligence and Auction Design](https://mlsanity.com/paper/2202.05947/) (2022)
- [Berk-Nash Equilibrium: A Framework for Modeling Agents with Misspecified Models](https://mlsanity.com/paper/1411.1152/) (2014)
- [Modeling the effects of environmental and perceptual uncertainty using deterministic reinforcement learning dynamics with partial observability](https://mlsanity.com/paper/2109.07259/) (2021)

## Abstract (arXiv)

Q-learning can be described as an all-purpose automaton that provides estimates (Q-values) of the continuation values associated with each available action and follows the naive policy of almost always choosing the action with highest Q-value. We consider a family of automata based on Q-values, whose policy may systematically favor some actions over others, for example through a bias that favors cooperation. We look for stable equilibrium biases, easily learned under converging logit/best-response dynamics over biases, not requiring any tacit agreement. These biases strongly foster collusion or cooperation across a rich array of payoff and monitoring structures, independently of initial Q-values.