# MARS-Gym: A Gym framework to model, train, and evaluate Recommender Systems for Marketplaces

arXiv: https://arxiv.org/abs/2010.07035 · HTML: https://mlsanity.com/paper/2010.07035/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2020.

## Built on by (most cited first, 13 of 13)

- [Designing Reinforcement Learning Algorithms for Digital Interventions: Pre-implementation Guidelines](https://mlsanity.com/paper/2206.03944/) (2022)
- [A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directions](https://mlsanity.com/paper/2109.03540/) (2021)
- [Fair ranking: a critical review, challenges, and future directions](https://mlsanity.com/paper/2201.12662/) (2022)
- [Accelerating Offline Reinforcement Learning Application in Real-Time Bidding and Recommendation: Potential Use of Simulation](https://mlsanity.com/paper/2109.08331/) (2021)
- [Simulations in Recommender Systems: An industry perspective](https://mlsanity.com/paper/2109.06723/) (2021)
- [Advancing Retail Data Science: Comprehensive Evaluation of Synthetic Data](https://mlsanity.com/paper/2406.13130/) (2024)
- [Synthetic Data-Based Simulators for Recommender Systems: A Survey](https://mlsanity.com/paper/2206.11338/) (2022)
- [RetailSynth: Synthetic Data Generation for Retail AI Systems Evaluation](https://mlsanity.com/paper/2312.14095/) (2023)
- [Measuring What Matters: Connecting AI Ethics Evaluations to System Attributes, Hazards, and Harms](https://mlsanity.com/paper/2510.10339/) (2025)
- [Simulation-Based Benchmarking of Reinforcement Learning Agents for Personalized Retail Promotions](https://mlsanity.com/paper/2405.10469/) (2024)
- [Hybrid Model with Time Modeling for Sequential Recommender Systems](https://mlsanity.com/paper/2103.06138/) (2021)
- [Optimized Recommender Systems with Deep Reinforcement Learning](https://mlsanity.com/paper/2110.03039/) (2021)
- [Synthetic Data Generator for Adaptive Interventions in Global Health](https://mlsanity.com/paper/2303.01954/) (2023)

## Abstract (arXiv)

Recommender Systems are especially challenging for marketplaces since they must maximize user satisfaction while maintaining the healthiness and fairness of such ecosystems. In this context, we observed a lack of resources to design, train, and evaluate agents that learn by interacting within these environments. For this matter, we propose MARS-Gym, an open-source framework to empower researchers and engineers to quickly build and evaluate Reinforcement Learning agents for recommendations in marketplaces. MARS-Gym addresses the whole development pipeline: data processing, model design and optimization, and multi-sided evaluation. We also provide the implementation of a diverse set of baseline agents, with a metrics-driven analysis of them in the Trivago marketplace dataset, to illustrate how to conduct a holistic assessment using the available metrics of recommendation, off-policy estimation, and fairness. With MARS-Gym, we expect to bridge the gap between academic research and production systems, as well as to facilitate the design of new algorithms and applications.