mlsanityData to 7 Oct 2026

Sample-Efficient Automated Deep Reinforcement Learning

arXiv 2009.01555 ↗First posted 3 Sep 2020cs.LG stat.MLcited by 31

Not on the map. Why?

Abstract (arXiv)

Despite significant progress in challenging problems across various domains, applying state-of-the-art deep reinforcement learning (RL) algorithms remains challenging due to their sensitivity to the choice of hyperparameters. This sensitivity can partly be attributed to the non-stationarity of the RL problem, potentially requiring different hyperparameter settings at various stages of the learning process. Additionally, in the RL setting, hyperparameter optimization (HPO) requires a large number of environment interactions, hindering the transfer of the successes in RL to real-world applications. In this work, we tackle the issues of sample-efficient and dynamic HPO in RL. We propose a population-based automated RL (AutoRL) framework to meta-optimize arbitrary off-policy RL algorithms. In this framework, we optimize the hyperparameters and also the neural architecture while simultaneously training the agent. By sharing the collected experience across the population, we substantially increase the sample efficiency of the meta-optimization. We demonstrate the capabilities of our sample-efficient AutoRL approach in a case study with the popular TD3 algorithm in the MuJoCo benchmark suite, where we reduce the number of environment interactions needed for meta-optimization by up to an order of magnitude compared to population-based training.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 8; 2023: 4; 2024: 8; 2025: 4; 2026: 1.

31 papers.

Sort:

By area:

  1. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  2. 2023Hyperparameters in Reinforcement Learning and How To Tune Themcited by 58
  3. 2021Evolving Reinforcement Learning Algorithmscited by 48
  4. 2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 41
  5. 2024B-STaR: Monitoring and Balancing Exploration and Exploitation in Self-Taught Reasonerscited by 26
  6. 2023Evolutionary Reinforcement Learning: A Surveycited by 24
  7. 2024Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithmscited by 23
  8. 2022Combining Evolution and Deep Reinforcement Learning for Policy Search: a Surveycited by 20
  9. 2022Bayesian Generational Population-Based Trainingcited by 20
  10. 2023AutoCost: Evolving Intrinsic Cost for Zero-violation Reinforcement Learningcited by 16
  11. 2024A Method for Evaluating Hyperparameter Sensitivity in Reinforcement Learningcited by 15
  12. 2024Multi-conditioned Graph Diffusion for Neural Architecture Searchcited by 13
19 more
  1. 2022Reinforcement Learning with Automated Auxiliary Loss Searchcited by 12
  2. 2022Automated Reinforcement Learning: An Overviewcited by 9
  3. 2021Tuning Mixed Input Hyperparameters on the Fly for Efficient Population Based AutoRLcited by 8
  4. 2025Eau De Q-Network: Adaptive Distillation of Neural Networks in Deep Reinforcement Learningcited by 6
  5. 2021Towards Automatic Actor-Critic Solutions to Continuous Controlcited by 4
  6. 2022Hyperparameters in Contextual RL are Highly Situationalcited by 4
  7. 2024Adaptive Q-Network: On-the-fly Target Selection for Deep Reinforcement Learningcited by 3
  8. 2024ARLBench: Flexible and Efficient Benchmarking for Hyperparameter Optimization in Reinforcement Learningcited by 3
  9. 2021Towards robust and domain agnostic reinforcement learning competitionscited by 2
  10. 2022ARLO: A Framework for Automated Reinforcement Learningcited by 2
  11. 2024Decomposing Control Lyapunov Functions for Efficient Reinforcement Learningcited by 2
  12. 2023Shrink-Perturb Improves Architecture Mixing during Population Based Training for Neural Architecture Searchcited by 1
  13. 2025Multiple-Frequencies Population-Based Trainingcited by 1
  14. 2021REIN-2: Giving Birth to Prepared Reinforcement Learning Agents Using Reinforcement Learning Agentscited by 0
  15. 2021Automatic tuning of hyper-parameters of reinforcement learning algorithms using Bayesian optimization with behavioral cloningcited by 0
  16. 2024Skill-aware Mutual Information Optimisation for Generalisation in Reinforcement Learningcited by 0
  17. 2025Scaling DRL for Decision Making: A Survey on Data, Network, and Training Budget Strategiescited by 0
  18. 2025Iterated Population Based Training with Task-Agnostic Restartscited by 0
  19. 2026Distorted Distributional Policy Evaluation for Offline Reinforcement Learningcited by 0