mlsanityData to 7 Oct 2026

How to Choose a Reinforcement-Learning Algorithm

arXiv 2407.20917 ↗First posted 30 Jul 2024cs.LG cs.AI cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

The field of reinforcement learning offers a large variety of concepts and methods to tackle sequential decision-making problems. This variety has become so large that choosing an algorithm for a task at hand can be challenging. In this work, we streamline the process of choosing reinforcement-learning algorithms and action-distribution families. We provide a structured overview of existing methods and their properties, as well as guidelines for when to choose which methods. An interactive version of these guidelines is available online at https://rl-picker.github.io/.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 68 of its 112 references that are in the map. First posted on arXiv 30 Jul 2024.

Builds on

68 of 112 references.

References · 68

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2015Continuous control with deep reinforcement learningcited by 3,279
  5. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  6. 2015Trust Region Policy Optimizationcited by 2,503
  7. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  8. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  9. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  10. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  11. 2016Generative Adversarial Imitation Learningcited by 1,576
  12. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
38 more
  1. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  2. 2015Prioritized Experience Replaycited by 1,070
  3. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  4. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  5. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  6. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  7. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  8. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  9. 2017Deep Reinforcement Learning that Matterscited by 727
  10. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  11. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  12. 2016Deep Exploration via Bootstrapped DQNcited by 518
  13. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 464
  14. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
  15. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 361
  16. 2018Maximum a Posteriori Policy Optimisationcited by 318
  17. 2019Reinforcement Learning in Healthcare: A Surveycited by 308
  18. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
  19. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
  20. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 278
  21. 2017Noisy Networks for Explorationcited by 262
  22. 2018Distributed Prioritized Experience Replaycited by 260
  23. 2018Distributed Distributional Deterministic Policy Gradientscited by 195
  24. 2019Imitation Learning via Off-Policy Distribution Matchingcited by 182
  25. 2017Bridging the Gap Between Value and Policy Based Reinforcement Learningcited by 178
  26. 2017Equivalence Between Policy Gradients and Soft Q-Learningcited by 172
  27. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
  28. 2018Applications of Deep Reinforcement Learning in Communications and Networking: A Surveycited by 163
  29. 2019Benchmarking Model-Based Reinforcement Learningcited by 158
  30. 2017Deep Exploration via Randomized Value Functionscited by 155
  31. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
  32. 2017Stochastic Neural Networks for Hierarchical Reinforcement Learningcited by 124
  33. 2017Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximationcited by 119
  34. 2017Backpropagation through the Void: Optimizing control variates for black-box gradient estimationcited by 104
  35. 2019Recent Advances in Imitation Learning from Observationcited by 92
  36. 2018Implicit Reparameterization Gradientscited by 84
  37. 2020Distributional Soft Actor-Critic: Off-Policy Reinforcement Learning for Addressing Value Estimation Errorscited by 80
  38. 2019A survey on intrinsic motivation in reinforcement learningcited by 79

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Statistical Multicriteria Evaluation of LLM-Generated Textcited by 9
  2. 2025Reasoning in visual navigation of end-to-end trained agents: a dynamical systems approachcited by 4
  3. 2026Performative Learning Theorycited by 0