# Simulation-Based Benchmarking of Reinforcement Learning Agents for Personalized Retail Promotions

arXiv: https://arxiv.org/abs/2405.10469 · HTML: https://mlsanity.com/paper/2405.10469/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 8 of its 19 references that are in the map.
- First posted on arXiv 16 May 2024.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 8 of 8)

- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro](https://mlsanity.com/paper/1912.11554/) (2019)
- [RecSim: A Configurable Simulation Platform for Recommender Systems](https://mlsanity.com/paper/1909.04847/) (2019)
- [MARS-Gym: A Gym framework to model, train, and evaluate Recommender Systems for Marketplaces](https://mlsanity.com/paper/2010.07035/) (2020)
- [Artificial Intelligence Index Report 2023](https://mlsanity.com/paper/2310.03715/) (2023)
- [Pearl: A Production-ready Reinforcement Learning Agent](https://mlsanity.com/paper/2312.03814/) (2023)
- [Simulations in Recommender Systems: An industry perspective](https://mlsanity.com/paper/2109.06723/) (2021)
- [RetailSynth: Synthetic Data Generation for Retail AI Systems Evaluation](https://mlsanity.com/paper/2312.14095/) (2023)

## Built on by (most cited first, 4 of 4)

- [Advancing Retail Data Science: Comprehensive Evaluation of Synthetic Data](https://mlsanity.com/paper/2406.13130/) (2024)
- [RetailBench: Benchmarking long horizon reasoning and coherent decision making of LLM agents in realistic retail environments](https://mlsanity.com/paper/2606.15862/) (2026)
- [RetailBench: Evaluating Long-Horizon Autonomous Decision-Making and Strategy Stability of LLM Agents in Realistic Retail Environments](https://mlsanity.com/paper/2603.16453/) (2026)
- [Scalable and Interpretable Contextual Bandits: A Literature Review and Retail Offer Prototype](https://mlsanity.com/paper/2505.16918/) (2025)

## Abstract (arXiv)

The development of open benchmarking platforms could greatly accelerate the adoption of AI agents in retail. This paper presents comprehensive simulations of customer shopping behaviors for the purpose of benchmarking reinforcement learning (RL) agents that optimize coupon targeting. The difficulty of this learning problem is largely driven by the sparsity of customer purchase events. We trained agents using offline batch data comprising summarized customer purchase histories to help mitigate this effect. Our experiments revealed that contextual bandit and deep RL methods that are less prone to over-fitting the sparse reward distributions significantly outperform static policies. This study offers a practical framework for simulating AI agents that optimize the entire retail customer journey. It aims to inspire the further development of simulation tools for retail AI systems.