mlsanityData to 7 Oct 2026

Measuring the Reliability of Reinforcement Learning Algorithms

arXiv 1912.05663 ↗First posted 10 Dec 2019stat.ML cs.AI cs.LGcited by 49

Not on the map. Why?

Abstract (arXiv)

Lack of reliability is a well-known issue for reinforcement learning (RL) algorithms. This problem has gained increasing attention in recent years, and efforts to improve it have grown substantially. To aid RL researchers and production users with the evaluation and improvement of reliability, we propose a set of metrics that quantitatively measure different aspects of reliability. In this work, we focus on variability and risk, both during training and after learning (on a fixed policy). We designed these metrics to be general-purpose, and we also designed complementary statistical tests to enable rigorous comparisons on these metrics. In this paper, we first describe the desired properties of the metrics and their design, the aspects of reliability that they measure, and their applicability to different scenarios. We then describe the statistical tests and make additional practical recommendations for reporting results. The metrics and accompanying statistical tools have been made available as an open-source library at https://github.com/google-research/rl-reliability-metrics. We apply our metrics to a set of common RL algorithms and environments, compare them, and analyze the results.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 13; 2023: 8; 2024: 4; 2025: 10; 2026: 7.

49 papers.

Sort:

By area:

  1. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  2. 2025A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibilitycited by 89
  3. 2021Revisiting Design Choices in Offline Model-Based Reinforcement Learningcited by 50
  4. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 48
  5. 2021Is High Variance Unavoidable in RL? A Case Study in Continuous Controlcited by 26
  6. 2023For Pre-Trained Vision Models in Motor Control, Not All Policy Learning Methods are Created Equalcited by 22
  7. 2022Evaluate & Evaluation on the Hub: Better Best Practices for Data and Model Measurementscited by 21
  8. 2024On the consistency of hyper-parameter selection in value-based deep reinforcement learningcited by 21
  9. 2023LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarioscited by 20
  10. 2021Stackelberg Actor-Critic: Game-Theoretic Reinforcement Learning Algorithmscited by 18
  11. 2022Zipfian environments for Reinforcement Learningcited by 13
  12. 2023A Domain-Agnostic Approach for Characterization of Lifelong Learning Systemscited by 13
37 more
  1. 2025Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learningcited by 12
  2. 2023Policy Optimization in a Noisy Neighborhood: On Return Landscapes in Continuous Controlcited by 11
  3. 2022The Impact of Task Underspecification in Evaluating Deep Reinforcement Learningcited by 10
  4. 2021Continuous Control With Ensemble Deep Deterministic Policy Gradientscited by 8
  5. 2022L2Explorer: A Lifelong Reinforcement Learning Assessment Environmentcited by 6
  6. 2024REValueD: Regularised Ensemble Value-Decomposition for Factorisable Markov Decision Processescited by 6
  7. 2024Real-time system optimal traffic routing under uncertainties -- Can physics models boost reinforcement learning?cited by 6
  8. 2025Relative Entropy Pathwise Policy Optimizationcited by 6
  9. 2025Replicable Reinforcement Learning with Linear Function Approximationcited by 6
  10. 2021A Distance-based Anomaly Detection Framework for Deep Reinforcement Learningcited by 5
  11. 2022Deep Reinforcement Learning for Cryptocurrency Trading: Practical Approach to Address Backtest Overfittingcited by 5
  12. 2023PRUDEX-Compass: Towards Systematic Evaluation of Reinforcement Learning in Financial Marketscited by 5
  13. 2023Ensemble Value Functions for Efficient Exploration in Multi-Agent Reinforcement Learningcited by 5
  14. 2023Measuring and Mitigating Interference in Reinforcement Learningcited by 5
  15. 2024Assessing the Impact of Distribution Shift on Reinforcement Learning Performancecited by 4
  16. 2026SoK: The Pitfalls of Deep Reinforcement Learning for Cybersecuritycited by 4
  17. 2026Beyond Rewards in Reinforcement Learning for Cyber Defencecited by 3
  18. 2025A2Perf: Real-World Autonomous Agents Benchmarkcited by 2
  19. 2026On Effectiveness and Efficiency of Agentic Tool-calling and RL Trainingcited by 2
  20. 2021Multi-Agent Reinforcement Learning via Adaptive Kalman Temporal Difference and Successor Representationcited by 1
  21. 2022AKF-SR: Adaptive Kalman Filtering-based Successor Representationcited by 1
  22. 2022Continual Reinforcement Learning with TELLAcited by 1
  23. 2022Ensemble Reinforcement Learning in Continuous Spaces -- A Hierarchical Multi-Step Approach for Policy Trainingcited by 1
  24. 2022Off-Policy Reinforcement Learning with Loss Function Weighted by Temporal Difference Errorcited by 1
  25. 2023Benchmarking Robustness of Deep Reinforcement Learning approaches to Online Portfolio Managementcited by 1
  26. 2025Learning more with the same effort: how randomization improves the robustness of a robotic deep reinforcement learning agentcited by 1
  27. 2025Less is more? Rewards in RL for Cyber Defencecited by 1
  28. 2026Behavior-Consistent Deep Reinforcement Learningcited by 1
  29. 2022Measuring Interventional Robustness in Reinforcement Learningcited by 0
  30. 2022Hybrid Indoor Localization via Reinforcement Learning-based Information Fusioncited by 0
  31. 2022When is Cognitive Radar Beneficial?cited by 0
  32. 2025Revisiting Ensemble Methods for Stock Trading and Crypto Trading Tasks at ACM ICAIF FinRL Contest 2023-2024cited by 0
  33. 2025Domain-driven Metrics for Reinforcement Learning: A Case Study on Epidemic Control using Agent-based Simulationcited by 0
  34. 2025DEEDEE: Fast and Scalable Out-of-Distribution Dynamics Detectioncited by 0
  35. 2026Characterizing MARL for Energy Control: A Multi-KPI Benchmark on the CityLearn Environmentcited by 0
  36. 2026Boosting deep Reinforcement Learning using pretraining with Logical Optionscited by 0
  37. 2026Jaxolotl: A Unified High-Performance Benchmark Suite for LTL-Based Multi-Task RLcited by 0