mlsanityData to 7 Oct 2026

ARLBench: Flexible and Efficient Benchmarking for Hyperparameter Optimization in Reinforcement Learning

arXiv 2409.18827 ↗First posted 27 Sep 2024cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Hyperparameters are a critical factor in reliably training well-performing reinforcement learning (RL) agents. Unfortunately, developing and evaluating automated approaches for tuning such hyperparameters is both costly and time-consuming. As a result, such approaches are often only evaluated on a single domain or algorithm, making comparisons difficult and limiting insights into their generalizability. We propose ARLBench, a benchmark for hyperparameter optimization (HPO) in RL that allows comparisons of diverse HPO approaches while being highly efficient in evaluation. To enable research into HPO in RL, even in settings with low compute resources, we select a representative subset of HPO tasks spanning a variety of algorithm and environment combinations. This selection allows for generating a performance profile of an automated RL (AutoRL) method using only a fraction of the compute previously necessary, enabling a broader range of researchers to work on HPO in RL. With the extensive and large-scale dataset on hyperparameter landscapes that our selection is based on, ARLBench is an efficient, flexible, and future-oriented foundation for research on AutoRL. Both the benchmark and the dataset are available at https://github.com/automl/arlbench.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 49 of its 75 references that are in the map. First posted on arXiv 27 Sep 2024.

Builds on

49 of 75 references.

References · 49

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  4. 2017Deep Reinforcement Learning that Matterscited by 727
  5. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  6. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 466
  7. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 412
  8. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 375
  9. 2017Population Based Training of Neural Networkscited by 351
  10. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 340
  11. 2021SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimizationcited by 229
  12. 2020Prioritized Level Replaycited by 190
37 more
  1. 2022Evolving Curricula with Regret-Based Environment Designcited by 166
  2. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  3. 2018Meta-Gradient Reinforcement Learningcited by 156
  4. 2021Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020cited by 146
  5. 2021HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPOcited by 111
  6. 2020Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning Researchcited by 78
  7. 2021HPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenMLcited by 75
  8. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  9. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
  10. 2020Surrogate NAS Benchmarks: Going Beyond the Limited Search Spaces of Tabular NAS Benchmarkscited by 67
  11. 2020Provably Efficient Online Hyperparameter Optimization with Population-Based Banditscited by 61
  12. 2021YAHPO Gym -- An Efficient Multi-Objective Multi-Fidelity Benchmark for Hyperparameter Optimizationcited by 61
  13. 2021On the Importance of Hyperparameter Optimization for Model-based Reinforcement Learningcited by 59
  14. 2022EnvPool: A Highly Parallel Reinforcement Learning Environment Execution Enginecited by 58
  15. 2023Hyperparameters in Reinforcement Learning and How To Tune Themcited by 58
  16. 2019Tabular Benchmarks for Joint Architecture and Hyperparameter Optimizationcited by 57
  17. 2023XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAXcited by 54
  18. 2021Evolving Reinforcement Learning Algorithmscited by 48
  19. 2021Bootstrapped Meta-Learningcited by 45
  20. 2022Atari-5: Distilling the Arcade Learning Environment down to Five Gamescited by 44
  21. 2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 41
  22. 2022NAS-Bench-Suite: NAS Evaluation is (Now) Surprisingly Easycited by 36
  23. 2023Small batch deep reinforcement learningcited by 35
  24. 2020Sample-Efficient Automated Deep Reinforcement Learningcited by 31
  25. 2022A Survey of Methods for Automated Algorithm Configurationcited by 29
  26. 2024On the consistency of hyper-parameter selection in value-based deep reinforcement learningcited by 21
  27. 2021When should agents explore?cited by 20
  28. 2022Bayesian Generational Population-Based Trainingcited by 20
  29. 2024Discovering Temporally-Aware Reinforcement Learning Algorithmscited by 18
  30. 2019Meta-Surrogate Benchmarking for Hyperparameter Optimizationcited by 16
  31. 2023Structure in Deep Reinforcement Learning: A Survey and Open Problemscited by 16
  32. 2022DeepCAVE: An Interactive Analysis Tool for Automated Machine Learningcited by 14
  33. 2020Deep Reinforcement Learning using Cyclical Learning Ratescited by 6
  34. 2021Decaying Clipping Range in Proximal Policy Optimizationcited by 6
  35. 2024Combining Automated Optimisation of Hyperparameters and Reward Shapecited by 5
  36. 2024Instance Selection for Dynamic Algorithm Configuration with Reinforcement Learning: Improving Generalizationcited by 5
  37. 2024Adaptive Q-Network: On-the-fly Target Selection for Deep Reinforcement Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025ULTHO: Ultra-Lightweight yet Efficient Hyperparameter Optimization in Deep Reinforcement Learningcited by 1
  2. 2025Deep Reinforcement Learning for Dynamic Algorithm Configuration: A Case Study on Optimizing OneMax with the (1+(λ,λ))-GAcited by 1
  3. 2026Efficient Heteroscedastic Bayesian Optimization for Risk-Aware AutoRLcited by 0