# A Survey on Reproducibility by Evaluating Deep Reinforcement Learning Algorithms on Real-World Robots

arXiv: https://arxiv.org/abs/1909.03772 · HTML: https://mlsanity.com/paper/1909.03772/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Sep 2019.

## Built on by (most cited first, 12 of 12)

- [Deep Reinforcement Learning at the Edge of the Statistical Precipice](https://mlsanity.com/paper/2108.13264/) (2021)
- [CaRL: Learning Scalable Planning Policies with Simple Rewards](https://mlsanity.com/paper/2504.17838/) (2025)
- [Is High Variance Unavoidable in RL? A Case Study in Continuous Control](https://mlsanity.com/paper/2110.11222/) (2021)
- [Replicable Reinforcement Learning](https://mlsanity.com/paper/2305.15284/) (2023)
- [Open RL Benchmark: Comprehensive Tracked Experiments for Reinforcement Learning](https://mlsanity.com/paper/2402.03046/) (2024)
- [Reproducibility of Machine Learning: Terminology, Recommendations and Open Issues](https://mlsanity.com/paper/2302.12691/) (2023)
- [Constrained Reinforcement Learning using Distributional Representation for Trustworthy Quadrotor UAV Tracking Control](https://mlsanity.com/paper/2302.11694/) (2023)
- [Active Alignments of Lens Systems with Reinforcement Learning](https://mlsanity.com/paper/2503.02075/) (2025)
- [IterMask3D: Unsupervised Anomaly Detection and Segmentation with Test-Time Iterative Mask Refinement in 3D Brain MR](https://mlsanity.com/paper/2504.04911/) (2025)
- [Towards Augmented Microscopy with Reinforcement Learning-Enhanced Workflows](https://mlsanity.com/paper/2208.02865/) (2022)
- [Behavior-Consistent Deep Reinforcement Learning](https://mlsanity.com/paper/2605.21214/) (2026)
- [Do World Models Make Better Robots? A Survey of Evaluation Benchmarks for Predictive Embodied Intelligence](https://mlsanity.com/paper/2609.29669/) (2026)

## Abstract (arXiv)

As reinforcement learning (RL) achieves more success in solving complex tasks, more care is needed to ensure that RL research is reproducible and that algorithms herein can be compared easily and fairly with minimal bias. RL results are, however, notoriously hard to reproduce due to the algorithms' intrinsic variance, the environments' stochasticity, and numerous (potentially unreported) hyper-parameters. In this work we investigate the many issues leading to irreproducible research and how to manage those. We further show how to utilise a rigorous and standardised evaluation approach for easing the process of documentation, evaluation and fair comparison of different algorithms, where we emphasise the importance of choosing the right measurement metrics and conducting proper statistics on the results, for unbiased reporting of the results.