mlsanityData to 7 Oct 2026

Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Control

arXiv 1708.04133 ↗First posted 10 Aug 2017cs.LGcited by 92

Not on the map. Why?

Abstract (arXiv)

Policy gradient methods in reinforcement learning have become increasingly prevalent for state-of-the-art performance in continuous control tasks. Novel methods typically benchmark against a few key algorithms such as deep deterministic policy gradients and trust region policy optimization. As such, it is important to present and use consistent baselines experiments. However, this can be difficult due to general variance in the algorithms, hyper-parameter tuning, and environment stochasticity. We investigate and discuss: the significance of hyper-parameters in policy gradients for continuous control, general variance in the algorithms, and reproducibility of reported results. We provide guidelines on reporting novel results as comparisons against baseline methods such that future researchers can make informed decisions when investigating novel methods.

Key facts. Cited by 92 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Aug 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 2; 2021: 21; 2022: 17; 2023: 16; 2024: 11; 2025: 15; 2026: 9.

92 papers.

Sort:

By area:

  1. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  2. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
  3. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 278
  4. 2021What Matters for Adversarial Imitation Learning?cited by 70
  5. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  6. 2022Leakage and the Reproducibility Crisis in ML-based Sciencecited by 63
  7. 2021On the Importance of Hyperparameter Optimization for Model-based Reinforcement Learningcited by 59
  8. 2023Empirical Design in Reinforcement Learningcited by 54
  9. 2022VRL3: A Data-Driven Framework for Visual Deep Reinforcement Learningcited by 51
  10. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 48
  11. 2021Nondeterminism and Instability in Neural Network Optimizationcited by 47
  12. 2022Reproducibility in Learningcited by 44
38 more
  1. 2025Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoidscited by 40
  2. 2023Small batch deep reinforcement learningcited by 35
  3. 2025CaRL: Learning Scalable Planning Policies with Simple Rewardscited by 34
  4. 2021On Proximal Policy Optimization's Heavy-tailed Gradientscited by 30
  5. 2021Is High Variance Unavoidable in RL? A Case Study in Continuous Controlcited by 26
  6. 2022Hyperbolic Deep Reinforcement Learningcited by 25
  7. 2023Replicable Reinforcement Learningcited by 21
  8. 2024On the consistency of hyper-parameter selection in value-based deep reinforcement learningcited by 21
  9. 2021Learning Pessimism for Robust and Efficient Off-Policy Reinforcement Learningcited by 18
  10. 2021Co-Adaptation of Algorithmic and Implementational Innovations in Inference-based Deep Reinforcement Learningcited by 15
  11. 2024Position: Benchmarking is Limited in Reinforcement Learning Researchcited by 15
  12. 2022Offline Reinforcement Learning with Closed-Form Policy Improvement Operatorscited by 14
  13. 2023Aligning Language Models with Offline Learning from Human Feedbackcited by 14
  14. 2023REFORMS: Reporting Standards for Machine Learning Based Sciencecited by 13
  15. 2026stable-worldmodel: A Platform for Reproducible World Modeling Research and Evaluationcited by 13
  16. 2023Off-Policy RL Algorithms Can be Sample-Efficient for Continuous Control via Sample Multiple Reusecited by 12
  17. 2024What is Reproducibility in Artificial Intelligence and Machine Learning Research?cited by 11
  18. 2021Learning Temporally-Consistent Representations for Data-Efficient Reinforcement Learningcited by 10
  19. 2021Simplifying Deep Reinforcement Learning via Self-Supervisioncited by 9
  20. 2024A Preference-driven Paradigm for Enhanced Translation with Large Language Modelscited by 9
  21. 2021Measuring Progress in Deep Reinforcement Learning Sample Efficiencycited by 8
  22. 2021Tuning Mixed Input Hyperparameters on the Fly for Efficient Population Based AutoRLcited by 8
  23. 2023Reproducibility of Machine Learning: Terminology, Recommendations and Open Issuescited by 8
  24. 2023AutoRL Hyperparameter Landscapescited by 8
  25. 2023Maximum diffusion reinforcement learningcited by 8
  26. 2021Aggressive Q-Learning with Ensembles: Achieving Both High Sample Efficiency and High Asymptotic Performancecited by 7
  27. 2022Hyperparameter Tuning for Deep Reinforcement Learning Applicationscited by 7
  28. 2022Multi-Horizon Representations with Hierarchical Forward Models for Reinforcement Learningcited by 7
  29. 2019Randomized Adversarial Imitation Learning for Autonomous Drivingcited by 6
  30. 2020DERAIL: Diagnostic Environments for Reward And Imitation Learningcited by 6
  31. 2022CACTO: Continuous Actor-Critic with Trajectory Optimization -- Towards global optimalitycited by 6
  32. 2023Improving and Benchmarking Offline Reinforcement Learning Algorithmscited by 6
  33. 2025Replicable Reinforcement Learning with Linear Function Approximationcited by 6
  34. 2023PASTA: Pretrained Action-State Transformer Agentscited by 5
  35. 2024Combining Automated Optimisation of Hyperparameters and Reward Shapecited by 5
  36. 2024ACL-QL: Adaptive Conservative Level in Q-Learning for Offline Reinforcement Learningcited by 5
  37. 2022Deep Learning Reproducibility and Explainable AI (XAI)cited by 4
  38. 2022Hyperparameters in Contextual RL are Highly Situationalcited by 4

50 most cited of 92 All 92 (JSON)