# Hyperparameter Tuning for Deep Reinforcement Learning Applications

arXiv: https://arxiv.org/abs/2201.11182 · HTML: https://mlsanity.com/paper/2201.11182/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 26 Jan 2022.

## Builds on (most cited first, 15 of 15)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization](https://mlsanity.com/paper/1603.06560/) (2016)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [BOHB: Robust and Efficient Hyperparameter Optimization at Scale](https://mlsanity.com/paper/1807.01774/) (2018)
- [Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Control](https://mlsanity.com/paper/1708.04133/) (2017)
- [Provably Efficient Online Hyperparameter Optimization with Population-Based Bandits](https://mlsanity.com/paper/2002.02518/) (2020)
- [Learning to Perform Physics Experiments via Deep Reinforcement Learning](https://mlsanity.com/paper/1611.01843/) (2016)
- [Hyp-RL : Hyperparameter Optimization by Reinforcement Learning](https://mlsanity.com/paper/1906.11527/) (2019)
- [Loss Function Search for Face Recognition](https://mlsanity.com/paper/2007.06542/) (2020)
- [Deep Reinforcement Learning for Process Control: A Primer for Beginners](https://mlsanity.com/paper/2004.05490/) (2020)
- [Comparison of Reinforcement Learning algorithms applied to the Cart Pole problem](https://mlsanity.com/paper/1810.01940/) (2018)
- [Efficient Hyperparameter Optimization in Deep Learning Using a Variable Length Genetic Algorithm](https://mlsanity.com/paper/2006.12703/) (2020)
- [Do optimization methods in deep learning applications matter?](https://mlsanity.com/paper/2002.12642/) (2020)

## Built on by (most cited first, 7 of 7)

- [Hyperparameters in Reinforcement Learning and How To Tune Them](https://mlsanity.com/paper/2306.01324/) (2023)
- [B-STaR: Monitoring and Balancing Exploration and Exploitation in Self-Taught Reasoners](https://mlsanity.com/paper/2412.17256/) (2024)
- [Automatic Environment Shaping is the Next Frontier in RL](https://mlsanity.com/paper/2407.16186/) (2024)
- [AnalogVNN: A fully modular framework for modeling and optimizing photonic neural networks](https://mlsanity.com/paper/2210.10048/) (2022)
- [Asynchronous Evolution of Deep Neural Network Architectures](https://mlsanity.com/paper/2308.04102/) (2023)
- [QF-tuner: Breaking Tradition in Reinforcement Learning](https://mlsanity.com/paper/2402.16562/) (2024)
- [Parameter Estimation using Reinforcement Learning Causal Curiosity: Limits and Challenges](https://mlsanity.com/paper/2505.08453/) (2025)

## Abstract (arXiv)

Reinforcement learning (RL) applications, where an agent can simply learn optimal behaviors by interacting with the environment, are quickly gaining tremendous success in a wide variety of applications from controlling simple pendulums to complex data centers. However, setting the right hyperparameters can have a huge impact on the deployed solution performance and reliability in the inference models, produced via RL, used for decision-making. Hyperparameter search itself is a laborious process that requires many iterations and computationally expensive to find the best settings that produce the best neural network architectures. In comparison to other neural network architectures, deep RL has not witnessed much hyperparameter tuning, due to its algorithm complexity and simulation platforms needed. In this paper, we propose a distributed variable-length genetic algorithm framework to systematically tune hyperparameters for various RL applications, improving training time and robustness of the architecture, via evolution. We demonstrate the scalability of our approach on many RL problems (from simple gyms to complex applications) and compared with Bayesian approach. Our results show that with more generations, optimal solutions that require fewer training episodes and are computationally cheap while being more robust for deployment. Our results are imperative to advance deep reinforcement learning controllers for real-world problems.