# Memoryless Algorithmic Collusion: Sure to Fail, Slow to Fall

arXiv: https://arxiv.org/abs/2409.01147 · HTML: https://mlsanity.com/paper/2409.01147/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- First posted on arXiv 2 Sep 2024.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Built on by (most cited first, 3 of 3)

- [Memory-Induced Supra-Competitive Outcomes Between Deep Reinforcement Learning Agents in Optimal Trade Execution](https://mlsanity.com/paper/2605.20348/) (2026)
- [Wasserstein Equilibrium Decoding for Reliable Medical Visual Question Answering](https://mlsanity.com/paper/2605.18313/) (2026)
- [Persistent Partners Raise Prices Among Learning Agents](https://mlsanity.com/paper/2609.35402/) (2026)

## Abstract (arXiv)

This paper shows that, in a class of Bertrand-style competition games, memoryless Q-learning algorithms should adapt to Nash Equilibrium given sufficient explorations in the long-run. This is also verified through accelerating simulations, while the convergence time grows super-exponentially for high discount factors. The resilience of collusive outcomes in the short-run is due to exploration of unprofitable actions, making the system resemble random walk. A structural model is proposed to estimate the convergence time, which not only explains the role of discount factor, but also unveils the non-monotonic relation in learning rate, which is overlooked in the literature.