Data to 7 Oct 2026

Deep Reinforcement Learning that Matters

arXiv 1709.06560 ↗First posted 19 Sep 2017cs.LG stat.MLcited by 727

Not on the map. Why?

Cited by 58 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In recent years, significant progress has been made in solving challenging problems across various domains using deep reinforcement learning (RL). Reproducing existing work and accurately judging the improvements offered by novel methods is vital to sustaining this progress. Unfortunately, reproducing results for state-of-the-art deep RL methods is seldom straightforward. In particular, non-determinism in standard benchmark environments, combined with variance intrinsic to the methods, can make reported results tough to interpret. Without significance metrics and tighter standardization of experimental reporting, it is difficult to determine whether improvements over the prior state-of-the-art are meaningful. In this paper, we investigate challenges posed by reproducibility, proper experimental techniques, and reporting procedures. We illustrate the variability in reported metrics and results when comparing against common baselines and suggest guidelines to make future results in deep RL more reproducible. We aim to spur discussion about how to ensure continued progress in the field by minimizing wasted effort stemming from results that are non-reproducible and easily misinterpreted.

Key facts. Cited by 727 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Sep 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 4; 2019: 3; 2020: 11; 2021: 145; 2022: 123; 2023: 98; 2024: 82; 2025: 106; 2026: 155.

727 papers.

Sort:

By area:

2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 361
2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
2023Efficient Online Reinforcement Learning with Offline Datacited by 332
2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 278

38 more

2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 203
2024AI Agents That Mattercited by 199
2021Accounting for Variance in Machine Learning Benchmarkscited by 195
2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
2021Embodied Intelligence via Learning and Evolutioncited by 154
2021Well-tuned Simple Nets Excel on Tabular Datasetscited by 151
2021The Benchmark Lotterycited by 133
2022Understanding and Preventing Capacity Loss in Reinforcement Learningcited by 124
2023Revisiting the Minimalist Approach to Offline Reinforcement Learningcited by 121
2022CORL: Research-oriented Deep Offline Reinforcement Learning Librarycited by 117
2020Hyperparameter Selection for Offline Reinforcement Learningcited by 113
2021B-Pref: Benchmarking Preference-Based Reinforcement Learningcited by 106
2021XIRL: Cross-embodiment Inverse Reinforcement Learningcited by 101
2024ARGS: Alignment as Reward-Guided Searchcited by 99
2021MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Researchcited by 96
2025A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibilitycited by 89
2021Do Transformer Modifications Transfer Across Implementations and Applications?cited by 87
2021d3rlpy: An Offline Deep Reinforcement Learning Librarycited by 84
2024Better than classical? The subtle art of benchmarking quantum machine learning modelscited by 82
2021Federated Reinforcement Learning: Techniques, Applications, and Open Challengescited by 79
2022MetaMorph: Learning Universal Controllers with Transformerscited by 78
2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 73
2021What Matters for Adversarial Imitation Learning?cited by 70
2021A Survey on Interpretable Reinforcement Learningcited by 70
2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
2021Learning to Fly -- a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Controlcited by 68
2021Model Selection for Offline Reinforcement Learning: Practical Considerations for Healthcare Settingscited by 68
2020f-IRL: Inverse Reinforcement Learning via State Marginal Matchingcited by 67
2024REBEL: Reinforcement Learning via Regressing Relative Rewardscited by 66
2021Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspectivecited by 64
2021On the Importance of Hyperparameter Optimization for Model-based Reinforcement Learningcited by 59
2023POPGym: Benchmarking Partially Observable Reinforcement Learningcited by 59
2023Hyperparameters in Reinforcement Learning and How To Tune Themcited by 58
2023Elo Uncovered: Robustness and Best Practices in Language Model Evaluationcited by 58
2020RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learningcited by 57
2021Randomness In Neural Network Training: Characterizing The Impact of Toolingcited by 57
2021Cooperative Exploration for Multi-Agent Deep Reinforcement Learningcited by 57

50 most cited of 727 All 727 (JSON)