# Common Challenges of Deep Reinforcement Learning Applications Development: An Empirical Study

arXiv: https://arxiv.org/abs/2310.09575 · HTML: https://mlsanity.com/paper/2310.09575/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Oct 2023.

## Built on by (most cited first, 5 of 5)

- [Real Faults in Model Context Protocol (MCP) Software: a Comprehensive Taxonomy](https://mlsanity.com/paper/2603.05637/) (2026)
- [A Reference Architecture of Reinforcement Learning Frameworks](https://mlsanity.com/paper/2603.06413/) (2026)
- [Unraveling Code Clone Dynamics in Deep Learning Frameworks](https://mlsanity.com/paper/2404.17046/) (2024)
- [SymbXRL: Symbolic Explainable Deep Reinforcement Learning for Mobile Networks](https://mlsanity.com/paper/2601.22024/) (2026)
- [An Empirical Study of OpenAI API Discussions on Stack Overflow](https://mlsanity.com/paper/2505.04084/) (2025)

## Abstract (arXiv)

Machine Learning (ML) is increasingly being adopted in different industries. Deep Reinforcement Learning (DRL) is a subdomain of ML used to produce intelligent agents. Despite recent developments in DRL technology, the main challenges that developers face in the development of DRL applications are still unknown. To fill this gap, in this paper, we conduct a large-scale empirical study of 927 DRL-related posts extracted from Stack Overflow, the most popular Q&A platform in the software community. Through the process of labeling and categorizing extracted posts, we created a taxonomy of common challenges encountered in the development of DRL applications, along with their corresponding popularity levels. This taxonomy has been validated through a survey involving 65 DRL developers. Results show that at least 45% of developers experienced 18 of the 21 challenges identified in the taxonomy. The most frequent source of difficulty during the development of DRL applications are Comprehension, API usage, and Design problems, while Parallel processing, and DRL libraries/frameworks are classified as the most difficult challenges to address, with respect to the time required to receive an accepted answer. We hope that the research community will leverage this taxonomy to develop efficient strategies to address the identified challenges and improve the quality of DRL applications.