# Multi-Agent Reinforcement Learning for Market Making: Competition without Collusion

arXiv: https://arxiv.org/abs/2510.25929 · HTML: https://mlsanity.com/paper/2510.25929/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 22 references that are in the map.
- First posted on arXiv 29 Oct 2025.

## Builds on (most cited first, 6 of 6)

- [Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1703.02702/) (2017)
- [Reinforcement Learning for Market Making in a Multi-agent Dealer Market](https://mlsanity.com/paper/1911.05892/) (2019)
- [Robust Market Making via Adversarial Reinforcement Learning](https://mlsanity.com/paper/2003.01820/) (2020)
- [Optimizing Market Making using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1812.10252/) (2018)
- [Robust Market Making: To Quote, or not To Quote](https://mlsanity.com/paper/2508.16588/) (2025)
- [ARL-Based Multi-Action Market Making with Hawkes Processes and Variable Volatility](https://mlsanity.com/paper/2508.16589/) (2025)

## Built on by (most cited first, 2 of 2)

- [Predicting Decisions of AI Agents from Limited Interaction through Text-Tabular Modeling](https://mlsanity.com/paper/2605.12411/) (2026)
- [When and How Order-Flow Interventions Work After Dealers Adapt](https://mlsanity.com/paper/2610.18158/) (2026)

## Abstract (arXiv)

Algorithmic collusion has emerged as a central question in AI: Will the interaction between different AI agents deployed in markets lead to collusion? More generally, understanding how emergent behavior, be it a cartel or market dominance from more advanced bots, affects the market overall is an important research question. We propose a hierarchical multi-agent reinforcement learning framework to study algorithmic collusion in market making. The framework includes a self-interested market maker (Agent~A), which is trained in an uncertain environment shaped by an adversary, and three bottom-layer competitors: the self-interested Agent~B1 (whose objective is to maximize its own PnL), the competitive Agent~B2 (whose objective is to minimize the PnL of its opponent), and the hybrid Agent~B^star, which can modulate between the behavior of the other two. To analyze how these agents shape the behavior of each other and affect market outcomes, we propose interaction-level metrics that quantify behavioral asymmetry and system-level dynamics, while providing signals potentially indicative of emergent interaction patterns. Experimental results show that Agent~B2 secures dominant performance in a zero-sum setting against B1, aggressively capturing order flow while tightening average spreads, thus improving market execution efficiency. In contrast, Agent~B^star exhibits a self-interested inclination when co-existing with other profit-seeking agents, securing dominant market share through adaptive quoting, yet exerting a milder adverse impact on the rewards of Agents~A and B1 compared to B2. These findings suggest that adaptive incentive control supports more sustainable strategic co-existence in heterogeneous agent environments and offers a structured lens for evaluating behavioral design in algorithmic trading systems.