# ABIDES-Gym: Gym Environments for Multi-Agent Discrete Event Simulation and Application to Financial Markets

arXiv: https://arxiv.org/abs/2110.14771 · HTML: https://mlsanity.com/paper/2110.14771/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 24 references that are in the map.
- First posted on arXiv 27 Oct 2021.

## Builds on (most cited first, 9 of 9)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [Tune: A Research Platform for Distributed Model Selection and Training](https://mlsanity.com/paper/1807.05118/) (2018)
- [Challenges of Real-World Reinforcement Learning](https://mlsanity.com/paper/1904.12901/) (2019)
- [OpenSpiel: A Framework for Reinforcement Learning in Games](https://mlsanity.com/paper/1908.09453/) (2019)
- [Get Real: Realism Metrics for Robust Limit Order Book Market Simulations](https://mlsanity.com/paper/1912.04941/) (2019)
- [ABIDES: Towards High-Fidelity Market Simulation for AI Research](https://mlsanity.com/paper/1904.12066/) (2019)
- [How to Evaluate Trading Strategies: Single Agent Market Replay or Multiple Agent Interactive Simulation?](https://mlsanity.com/paper/1906.12010/) (2019)

## Built on by (most cited first, 30 of 36)

- [Alpha-GPT: Human-AI Interactive Alpha Mining for Quantitative Investment](https://mlsanity.com/paper/2308.00016/) (2023)
- [MarS: a Financial Market Simulation Engine Powered by Generative Foundation Model](https://mlsanity.com/paper/2409.07486/) (2024)
- [When AI Meets Finance (StockAgent): Large Language Model-based Stock Trading in Simulated Real-world Environments](https://mlsanity.com/paper/2407.18957/) (2024)
- [Dynamic Datasets and Market Environments for Financial Reinforcement Learning](https://mlsanity.com/paper/2304.13174/) (2023)
- [Scalable Primal-Dual Actor-Critic Method for Safe Multi-Agent RL with General Utilities](https://mlsanity.com/paper/2305.17568/) (2023)
- [JAX-LOB: A GPU-Accelerated limit order book simulator to unlock large scale reinforcement learning for trading](https://mlsanity.com/paper/2308.13289/) (2023)
- [Quant 4.0: Engineering Quantitative Investment with Automated, Explainable and Knowledge-driven Artificial Intelligence](https://mlsanity.com/paper/2301.04020/) (2022)
- [K-SHAP: Policy Clustering Algorithm for Anonymous Multi-Agent State-Action Pairs](https://mlsanity.com/paper/2302.11996/) (2023)
- [Phantom -- A RL-driven multi-agent framework to model complex systems](https://mlsanity.com/paper/2210.06012/) (2022)
- [Optimal Execution with Reinforcement Learning](https://mlsanity.com/paper/2411.06389/) (2024)
- [Deep Calibration of Market Simulations using Neural Density Estimators and Embedding Networks](https://mlsanity.com/paper/2311.11913/) (2023)
- [Model-based gym environments for limit order book trading](https://mlsanity.com/paper/2209.07823/) (2022)
- [Asynchronous Deep Double Duelling Q-Learning for Trading-Signal Execution in Limit Order Book Markets](https://mlsanity.com/paper/2301.08688/) (2023)
- [CNN-DRL with Shuffled Features in Finance](https://mlsanity.com/paper/2402.03338/) (2024)
- [Limited or Biased: Modeling Sub-Rational Human Investors in Financial Markets](https://mlsanity.com/paper/2210.08569/) (2022)
- [FinEvo: From Isolated Backtests to Ecological Market Games for Multi-Agent Financial Strategy Evolution](https://mlsanity.com/paper/2602.00948/) (2026)
- [OpenFinGym: A Verifiable Multi-Task Gym Environment for Evaluating Quant Agents](https://mlsanity.com/paper/2606.26350/) (2026)
- [Equitable Marketplace Mechanism Design](https://mlsanity.com/paper/2209.15418/) (2022)
- [INTAGS: Interactive Agent-Guided Simulation](https://mlsanity.com/paper/2309.01784/) (2023)
- [Empirical Equilibria in Agent-based Economic systems with Learning agents](https://mlsanity.com/paper/2408.12038/) (2024)
- [Finding Optimal Trading History in Reinforcement Learning for Stock Market Trading](https://mlsanity.com/paper/2502.12537/) (2025)
- [DiffVolume: Diffusion Models for Volume Generation in Limit Order Books](https://mlsanity.com/paper/2508.08698/) (2025)
- [Heterogeneous RBCs via Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2510.12272/) (2025)
- [CTMSTOU driven markets: simulated environment for regime-awareness in trading policies](https://mlsanity.com/paper/2202.00941/) (2022)
- [Revisiting Ensemble Methods for Stock Trading and Crypto Trading Tasks at ACM ICAIF FinRL Contest 2023-2024](https://mlsanity.com/paper/2501.10709/) (2025)
- [Tail-Safe Hedging: Explainable Risk-Sensitive Reinforcement Learning with a White-Box CBF--QP Safety Layer in Arbitrage-Free Markets](https://mlsanity.com/paper/2510.04555/) (2025)
- [Natural-gas storage modelling by deep reinforcement learning](https://mlsanity.com/paper/2511.02646/) (2025)
- [Diverse Approaches to Optimal Execution Schedule Generation](https://mlsanity.com/paper/2601.22113/) (2026)
- [Decoupling Identity from Utility: Privacy-by-Design Frameworks for Financial Ecosystems](https://mlsanity.com/paper/2604.14495/) (2026)
- [Algometrics: Forecasting Under Algorithmic Feedback](https://mlsanity.com/paper/2605.23978/) (2026)

## Abstract (arXiv)

Model-free Reinforcement Learning (RL) requires the ability to sample trajectories by taking actions in the original problem environment or a simulated version of it. Breakthroughs in the field of RL have been largely facilitated by the development of dedicated open source simulators with easy to use frameworks such as OpenAI Gym and its Atari environments. In this paper we propose to use the OpenAI Gym framework on discrete event time based Discrete Event Multi-Agent Simulation (DEMAS). We introduce a general technique to wrap a DEMAS simulator into the Gym framework. We expose the technique in detail and implement it using the simulator ABIDES as a base. We apply this work by specifically using the markets extension of ABIDES, ABIDES-Markets, and develop two benchmark financial markets OpenAI Gym environments for training daily investor and execution agents. As a result, these two environments describe classic financial problems with a complex interactive market behavior response to the experimental agent's action.