mlsanityData to 14 Oct 2026

Hyper: Hyperparameter Robust Efficient Exploration in Reinforcement Learning

arXiv 2412.03767 ↗First posted 4 Dec 2024cs.LG stat.MLcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

The exploration & exploitation dilemma poses significant challenges in reinforcement learning (RL). Recently, curiosity-based exploration methods achieved great success in tackling hard-exploration problems. However, they necessitate extensive hyperparameter tuning on different environments, which heavily limits the applicability and accessibility of this line of methods. In this paper, we characterize this problem via analysis of the agent behavior, concluding the fundamental difficulty of choosing a proper hyperparameter. We then identify the difficulty and the instability of the optimization when the agent learns with curiosity. We propose our method, hyperparameter robust exploration (Hyper), which extensively mitigates the problem by effectively regularizing the visitation of the exploration and decoupling the exploitation to ensure stable training. We theoretically justify that Hyper is provably efficient under function approximation setting and empirically demonstrate its appealing performance and robustness in various environments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 16 of its 40 references that are in the map. First posted on arXiv 4 Dec 2024.

Builds on

16 of 40 references.

References · 16

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  3. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  4. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  5. 2018Exploration by Random Network Distillationcited by 993
  6. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 520
  7. 2017Noisy Networks for Explorationcited by 266
  8. 2019Self-Supervised Exploration via Disagreementcited by 265
  9. 2017Count-Based Exploration with Neural Density Modelscited by 257
  10. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 212
  11. 2019Reinforcement Learning in Feature Space: Matrix Bandit, Kernels, and Regret Boundcited by 201
  12. 2020Temporally-Extended ε-Greedy Explorationcited by 29
4 more
  1. 2018DORA The Explorer: Directed Outreaching Reinforcement Action-Selectioncited by 27
  2. 2022Redeeming Intrinsic Rewards via Constrained Optimizationcited by 24
  3. 2021Decoupled Exploration and Exploitation Policies for Sample-Efficient Reinforcement Learningcited by 14
  4. 2023LESSON: Learning to Integrate Exploration Strategies for Reinforcement Learning via an Option Frameworkcited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Emergence of Exploration in Policy Gradient Reinforcement Learning via Retrying
  2. 2024On the consistency of hyper-parameter selection in value-based deep reinforcement learning
  3. 2024A Method for Evaluating Hyperparameter Sensitivity in Reinforcement Learning
  4. 2024Combining Automated Optimisation of Hyperparameters and Reward Shape
  5. 2024Q-Star Meets Scalable Posterior Sampling: Bridging Theory and Practice via HyperAgent
  6. 2026Solving Parameter-Robust Avoid Problems with Unknown Feasibility using Reinforcement Learning
  7. 2025Learning to Explore in Diverse Reward Settings via Temporal-Difference-Error Maximization
  8. 2025Exploration by Random Reward Perturbation