mlsanityData to 14 Oct 2026

Faults in Deep Reinforcement Learning Programs: A Taxonomy and A Detection Approach

arXiv 2101.00135 ↗First posted 1 Jan 2021cs.SE cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

A growing demand is witnessed in both industry and academia for employing Deep Learning (DL) in various domains to solve real-world problems. Deep Reinforcement Learning (DRL) is the application of DL in the domain of Reinforcement Learning (RL). Like any software systems, DRL applications can fail because of faults in their programs. In this paper, we present the first attempt to categorize faults occurring in DRL programs. We manually analyzed 761 artifacts of DRL programs (from Stack Overflow posts and GitHub issues) developed using well-known DRL frameworks (OpenAI Gym, Dopamine, Keras-rl, Tensorforce) and identified faults reported by developers/users. We labeled and taxonomized the identified faults through several rounds of discussions. The resulting taxonomy is validated using an online survey with 19 developers/researchers. To allow for the automatic detection of faults in DRL programs, we have defined a meta-model of DRL programs and developed DRLinter, a model-based fault detection approach that leverages static analysis and graph transformations. The execution flow of DRLinter consists in parsing a DRL program to generate a model conforming to our meta-model and applying detection rules on the model to identify faults occurrences. The effectiveness of DRLinter is evaluated using 15 synthetic DRLprograms in which we injected faults observed in the analyzed artifacts of the taxonomy. The results show that DRLinter can successfully detect faults in all synthetic faulty programs.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 16 of its 49 references that are in the map. First posted on arXiv 1 Jan 2021.

Builds on

16 of 49 references.

References · 16

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  3. 2016OpenAI Gymcited by 1,736
  4. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  5. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  6. 2019Solving Rubik's Cube with a Robot Handcited by 734
  7. 2019Reinforcement Learning in Healthcare: A Surveycited by 310
  8. 2017DeepStack: Expert-Level Artificial Intelligence in No-Limit Pokercited by 233
  9. 2017Deep Reinforcement Learning framework for Autonomous Drivingcited by 225
  10. 2018An Introduction to Deep Reinforcement Learningcited by 206
  11. 2018Dopamine: A Research Framework for Deep Reinforcement Learningcited by 122
  12. 2019Taxonomy of Real Faults in Deep Learning Systemscited by 90
4 more
  1. 2019A Comprehensive Study on Deep Learning Bug Characteristicscited by 65
  2. 2017Learning to Fly by Crashingcited by 41
  3. 2020Repairing Deep Neural Networks: Fix Patterns and Challengescited by 33
  4. 2016Understanding the Factors that Impact the Popularity of GitHub Repositoriescited by 27

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 5; 2024: 4; 2025: 3; 2026: 3.

18 papers.

Sort:

By area:

  1. 2024Bugs in Large Language Models Generated Code: An Empirical Studycited by 60
  2. 2021Silent Bugs in Deep Learning Frameworks: An Empirical Study of Keras and TensorFlowcited by 25
  3. 2023A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitionerscited by 22
  4. 2023METAL: Metamorphic Testing Framework for Analyzing Large-Language Model Qualitiescited by 20
  5. 2022Bugs in Machine Learning-based Systems: A Faultload Benchmarkcited by 15
  6. 2022A Search-Based Testing Approach for Deep Reinforcement Learning Agentscited by 13
  7. 2023Bug Characterization in Machine Learning-based Systemscited by 10
  8. 2024A Survey on Failure Analysis and Fault Injection in AI Systemscited by 10
  9. 2025Smart-LLaMA-DPO: Reinforced Large Language Model for Explainable Smart Contract Vulnerability Detectioncited by 7
  10. 2023Mutation Testing of Deep Reinforcement Learning Based on Real Faultscited by 4
  11. 2023Deploying Deep Reinforcement Learning Systems: A Taxonomy of Challengescited by 2
  12. 2024Toward Debugging Deep Reinforcement Learning Programs with RLExplorercited by 1
6 more
  1. 2025Why Attention Fails: A Taxonomy of Faults in Attention-Based Neural Networkscited by 1
  2. 2026A Taxonomy of Runtime Faults in Model Context Protocol Serverscited by 1
  3. 2024Fuzzy Logic Guided Reward Function Variation: An Oracle for Testing Reinforcement Learning Programscited by 0
  4. 2025Applications and Challenges of Fairness APIs in Machine Learning Softwarecited by 0
  5. 2026A Comprehensive Study of Implementation Bugs in Multi-modal Agentscited by 0
  6. 2026Evaluating RL Explainability Methods by How Much They Help Fix Bugs in Agentscited by 0