# Deploying Deep Reinforcement Learning Systems: A Taxonomy of Challenges

arXiv: https://arxiv.org/abs/2308.12438 · HTML: https://mlsanity.com/paper/2308.12438/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 80 references that are in the map.
- First posted on arXiv 23 Aug 2023.

## Builds on (most cited first, 24 of 24)

- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020)
- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [Challenges of Real-World Reinforcement Learning](https://mlsanity.com/paper/1904.12901/) (2019)
- [TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems](https://mlsanity.com/paper/2010.08678/) (2020)
- [Deep Learning with Low Precision by Half-wave Gaussian Quantization](https://mlsanity.com/paper/1702.00953/) (2017)
- [TensorFlow-Serving: Flexible, High-Performance ML Serving](https://mlsanity.com/paper/1712.06139/) (2017)
- [Massively Parallel Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1507.04296/) (2015)
- [Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems](https://mlsanity.com/paper/1901.06455/) (2019)
- [A Survey on Methods and Theories of Quantized Neural Networks](https://mlsanity.com/paper/1808.04752/) (2018)
- [A Survey of Deep Reinforcement Learning in Video Games](https://mlsanity.com/paper/1912.10944/) (2019)
- [A First Look at Deep Learning Apps on Smartphones](https://mlsanity.com/paper/1812.05448/) (2018)
- [An Empirical Study towards Characterizing Deep Learning Development and Deployment across Different Frameworks and Platforms](https://mlsanity.com/paper/1909.06727/) (2019)
- [TensorFlow.js: Machine Learning for the Web and Beyond](https://mlsanity.com/paper/1901.05350/) (2019)
- [Inverse reinforcement learning for video games](https://mlsanity.com/paper/1810.10593/) (2018)
- [An Empirical Study on Deployment Faults of Deep Learning Based Mobile Applications](https://mlsanity.com/paper/2101.04930/) (2021)
- [Faults in Deep Reinforcement Learning Programs: A Taxonomy and A Detection Approach](https://mlsanity.com/paper/2101.00135/) (2021)
- [Moving Deep Learning into Web Browser: How Far Can We Go?](https://mlsanity.com/paper/1901.09388/) (2019)
- [An Empirical Study of Challenges in Converting Deep Learning Models](https://mlsanity.com/paper/2206.14322/) (2022)
- [Interpreting Cloud Computer Vision Pain-Points: A Mining Study of Stack Overflow](https://mlsanity.com/paper/2001.10130/) (2020)
- [Explaining Online Reinforcement Learning Decisions of Self-Adaptive Systems](https://mlsanity.com/paper/2210.05931/) (2022)
- [MR-iNet Gym: Framework for Edge Deployment of Deep Reinforcement Learning on Embedded Software Defined Radio](https://mlsanity.com/paper/2204.04507/) (2022)
- [GriddlyJS: A Web IDE for Reinforcement Learning](https://mlsanity.com/paper/2207.06105/) (2022)

## Built on by (most cited first, 2 of 2)

- [A Survey of Reinforcement Learning for Optimization in Automation](https://mlsanity.com/paper/2502.09417/) (2025)
- [Robust Remote Reinforcement Learning over Unreliable Communication Channels using Homomorphic State Encoding](https://mlsanity.com/paper/2508.07722/) (2025)

## Abstract (arXiv)

Deep reinforcement learning (DRL), leveraging Deep Learning (DL) in reinforcement learning, has shown significant potential in achieving human-level autonomy in a wide range of domains, including robotics, computer vision, and computer games. This potential justifies the enthusiasm and growing interest in DRL in both academia and industry. However, the community currently focuses mostly on the development phase of DRL systems, with little attention devoted to DRL deployment. In this paper, we propose an empirical study on Stack Overflow (SO), the most popular Q&A forum for developers, to uncover and understand the challenges practitioners faced when deploying DRL systems. Specifically, we categorized relevant SO posts by deployment platforms: server/cloud, mobile/embedded system, browser, and game engine. After filtering and manual analysis, we examined 357 SO posts about DRL deployment, investigated the current state, and identified the challenges related to deploying DRL systems. Then, we investigate the prevalence and difficulty of these challenges. Results show that the general interest in DRL deployment is growing, confirming the study's relevance and importance. Results also show that DRL deployment is more difficult than other DRL issues. Additionally, we built a taxonomy of 31 unique challenges in deploying DRL to different platforms. On all platforms, RL environment-related challenges are the most popular, and communication-related challenges are the most difficult among practitioners. We hope our study inspires future research and helps the community overcome the most common and difficult challenges practitioners face when deploying DRL systems.