mlsanityData to 14 Oct 2026

A Hitchhiker's Guide to Statistical Comparisons of Reinforcement Learning Algorithms

arXiv 1904.06979 ↗First posted 15 Apr 2019stat.ME cs.LGcited by 27

Not on the map. Why?

Abstract (arXiv)

Consistently checking the statistical significance of experimental results is the first mandatory step towards reproducible science. This paper presents a hitchhiker's guide to rigorous comparisons of reinforcement learning algorithms. After introducing the concepts of statistical testing, we review the relevant statistical tests and compare them empirically in terms of false positive rate and statistical power as a function of the sample size (number of seeds) and effect size. We further investigate the robustness of these tests to violations of the most common hypotheses (normal distributions, same distributions, equal variances). Beside simulations, we compare empirical distributions obtained by running Soft-Actor Critic and Twin-Delayed Deep Deterministic Policy Gradient on Half-Cheetah. We conclude by providing guidelines and code to perform rigorous comparisons of RL algorithm performances.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). First posted on arXiv 15 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 7; 2022: 6; 2023: 9; 2024: 3; 2025: 0; 2026: 2.

27 papers.

Sort:

By area:

  1. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  2. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 49
  3. 2021Is High Variance Unavoidable in RL? A Case Study in Continuous Controlcited by 28
  4. 2023Universal Morphology Control via Contextual Modulationcited by 23
  5. 2023Semantic HELM: A Human-Readable Memory for Reinforcement Learningcited by 17
  6. 2023A Domain-Agnostic Approach for Characterization of Lifelong Learning Systemscited by 13
  7. 2021Curriculum Learning for Safe Mapless Navigationcited by 11
  8. 2022Constrained Reinforcement Learning for Robotics via Scenario-Based Programmingcited by 11
  9. 2022Learn the Time to Learn: Replay Scheduling in Continual Learningcited by 11
  10. 2023Policy Optimization in a Noisy Neighborhood: On Return Landscapes in Continuous Controlcited by 11
  11. 2023Assessing SATNet's Ability to Solve the Symbol Grounding Problemcited by 10
  12. 2021Grounding Spatio-Temporal Language with Transformerscited by 7
15 more
  1. 2023Improving Deep Policy Gradients with Value Function Searchcited by 7
  2. 2022Reinforcement Learning in Practice: Opportunities and Challengescited by 6
  3. 2024Inferring Behavior-Specific Context Improves Zero-Shot Generalization in Reinforcement Learningcited by 5
  4. 2024Improving Policy Optimization via varepsilon-Retraincited by 5
  5. 2021Improved Cooperation by Exploiting a Common Signalcited by 3
  6. 2021AI-driven Prices for Externalities and Sustainability in Production Marketscited by 3
  7. 2023AdaStop: adaptive statistical testing for sound comparisons of Deep RL agentscited by 3
  8. 2022Divide & Conquer Imitation Learningcited by 2
  9. 2023Learning An Active Inference Model of Driver Perception and Control: Application to Vehicle Car-Followingcited by 2
  10. 2022Reinforcement Teachingcited by 1
  11. 2023Efficient Epistemic Uncertainty Estimation in Regression Ensemble Models Using Pairwise-Distance Estimatorscited by 1
  12. 2026AtomComposer: Discovering Chemical Space from First Principles with Reinforcement Learningcited by 1
  13. 2021Blockwise Sequential Model Learning for Partially Observable Reinforcement Learningcited by 0
  14. 2024Boosting Robustness in Preference-Based Reinforcement Learning with Dynamic Sparsitycited by 0
  15. 2026Multi-action Tangled Program Graphs for Multi-task Reinforcement Learning with Continuous Controlcited by 0