# Phantom -- A RL-driven multi-agent framework to model complex systems

arXiv: https://arxiv.org/abs/2210.06012 · HTML: https://mlsanity.com/paper/2210.06012/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 12 Oct 2022.

## Builds on (most cited first, 14 of 14)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [OpenSpiel: A Framework for Reinforcement Learning in Games](https://mlsanity.com/paper/1908.09453/) (2019)
- [Acme: A Research Framework for Distributed Reinforcement Learning](https://mlsanity.com/paper/2006.00979/) (2020)
- [Reinforcement Learning for Market Making in a Multi-agent Dealer Market](https://mlsanity.com/paper/1911.05892/) (2019)
- [ABIDES-Gym: Gym Environments for Multi-Agent Discrete Event Simulation and Application to Financial Markets](https://mlsanity.com/paper/2110.14771/) (2021)
- [ABIDES: Towards High-Fidelity Market Simulation for AI Research](https://mlsanity.com/paper/1904.12066/) (2019)
- [Reverb: A Framework For Experience Replay](https://mlsanity.com/paper/2102.04736/) (2021)
- [Mava: a research library for distributed multi-agent reinforcement learning in JAX](https://mlsanity.com/paper/2107.01460/) (2021)
- [Launchpad: A Programming Model for Distributed Machine Learning Research](https://mlsanity.com/paper/2106.04516/) (2021)
- [Calibration of Shared Equilibria in General Sum Partially Observable Markov Games](https://mlsanity.com/paper/2006.13085/) (2020)
- [WarpDrive: Extremely Fast End-to-End Deep Multi-Agent Reinforcement Learning on a GPU](https://mlsanity.com/paper/2108.13976/) (2021)
- [Towards a fully RL-based Market Simulator](https://mlsanity.com/paper/2110.06829/) (2021)

## Built on by (most cited first, 5 of 5)

- [Reinforcement Learning for Combining Search Methods in the Calibration of Economic ABMs](https://mlsanity.com/paper/2302.11835/) (2023)
- [Simulating the Economic Impact of Rationality through Reinforcement Learning and Agent-Based Modelling](https://mlsanity.com/paper/2405.02161/) (2024)
- [Learning and Calibrating Heterogeneous Bounded Rational Market Behaviour with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2402.00787/) (2024)
- [Heterogeneous RBCs via Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2510.12272/) (2025)
- [Natural-gas storage modelling by deep reinforcement learning](https://mlsanity.com/paper/2511.02646/) (2025)

## Abstract (arXiv)

Agent based modelling (ABM) is a computational approach to modelling complex systems by specifying the behaviour of autonomous decision-making components or agents in the system and allowing the system dynamics to emerge from their interactions. Recent advances in the field of Multi-agent reinforcement learning (MARL) have made it feasible to study the equilibrium of complex environments where multiple agents learn simultaneously. However, most ABM frameworks are not RL-native, in that they do not offer concepts and interfaces that are compatible with the use of MARL to learn agent behaviours. In this paper, we introduce a new open-source framework, Phantom, to bridge the gap between ABM and MARL. Phantom is an RL-driven framework for agent-based modelling of complex multi-agent systems including, but not limited to economic systems and markets. The framework aims to provide the tools to simplify the ABM specification in a MARL-compatible way - including features to encode dynamic partial observability, agent utility functions, heterogeneity in agent preferences or types, and constraints on the order in which agents can act (e.g. Stackelberg games, or more complex turn-taking environments). In this paper, we present these features, their design rationale and present two new environments leveraging the framework.