mlsanityData to 7 Oct 2026

Benchmark for Out-of-Distribution Detection in Deep Reinforcement Learning

arXiv 2112.02694 ↗First posted 5 Dec 2021cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

Reinforcement Learning (RL) based solutions are being adopted in a variety of domains including robotics, health care and industrial automation. Most focus is given to when these solutions work well, but they fail when presented with out of distribution inputs. RL policies share the same faults as most machine learning models. Out of distribution detection for RL is generally not well covered in the literature, and there is a lack of benchmarks for this task. In this work we propose a benchmark to evaluate OOD detection methods in a Reinforcement Learning setting, by modifying the physical parameters of non-visual standard environments or corrupting the state observation for visual environments. We discuss ways to generate custom RL environments that can produce OOD data, and evaluate three uncertainty methods for the OOD detection task. Our results show that ensemble methods have the best OOD detection performance with a lower standard deviation across multiple environments.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 17 of its 31 references that are in the map. First posted on arXiv 5 Dec 2021.

Builds on

17 of 31 references.

References · 17

  1. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  3. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  4. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  5. 2016OpenAI Gymcited by 1,684
  6. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  7. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
  8. 2019Learning agile and dynamic motor skills for legged robotscited by 402
  9. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  10. 2019Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Comingcited by 332
  11. 2018Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batchescited by 133
  12. 2019Unsupervised Out-of-Distribution Detection by Maximum Classifier Discrepancycited by 112
5 more
  1. 2019DropConnect Is Effective in Modeling Uncertainty of Bayesian Deep Networkscited by 92
  2. 2018Safe Reinforcement Learning with Model Uncertainty Estimatescited by 70
  3. 2018Classification Uncertainty of Deep Neural Networks Based on Gradient Informationcited by 43
  4. 2018Open Category Detection with PAC Guaranteescited by 35
  5. 2019Uncertainty-Based Out-of-Distribution Detection in Deep Reinforcement Learningcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 0; 2024: 2; 2025: 3; 2026: 2.

9 papers.

  1. 2022Disentangling Epistemic and Aleatoric Uncertainty in Reinforcement Learningcited by 27
  2. 2024Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advancescited by 24
  3. 2024Rethinking Out-of-Distribution Detection for Reinforcement Learning: Advancing Methods for Evaluation and Detectioncited by 17
  4. 2025Guaranteeing Out-Of-Distribution Detection in Deep RL via Transition Estimationcited by 6
  5. 2022How to Enable Uncertainty Estimation in Proximal Policy Optimizationcited by 3
  6. 2025Safety Monitoring for Learning-Enabled Cyber-Physical Systems in Out-of-Distribution Scenarioscited by 3
  7. 2026RAPT: Model-Predictive Out-of-Distribution Detection and Failure Diagnosis for Sim-to-Real Humanoid Deploymentcited by 1
  8. 2026OOD-RL-Bench: A Benchmark Framework for Out-of-Distribution Detection in Reinforcement Learningcited by 1
  9. 2025DEEDEE: Fast and Scalable Out-of-Distribution Dynamics Detectioncited by 0