mlsanityData to 14 Oct 2026

Reward Bonuses with Gain Scheduling Inspired by Iterative Deepening Search

arXiv 2212.10765 ↗First posted 21 Dec 2022cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

This paper introduces a novel method of adding intrinsic bonuses to task-oriented reward function in order to efficiently facilitate reinforcement learning search. While various bonuses have been designed to date, they are analogous to the depth-first and breadth-first search algorithms in graph theory. This paper, therefore, first designs two bonuses for each of them. Then, a heuristic gain scheduling is applied to the designed bonuses, inspired by the iterative deepening search, which is known to inherit the advantages of the two search algorithms. The proposed method is expected to allow agent to efficiently reach the best solution in deeper states by gradually exploring unknown states. In three locomotion tasks with dense rewards and three simple tasks with sparse rewards, it is shown that the two types of bonuses contribute to the performance improvement of the different tasks complementarily. In addition, by combining them with the proposed gain scheduling, all tasks can be accomplished with high performance.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 26 of its 48 references that are in the map. First posted on arXiv 21 Dec 2022.

Builds on

26 of 48 references.

References · 26

  1. 2016Layer Normalizationcited by 5,669
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2018World Modelscited by 1,513
  4. 2017Hindsight Experience Replaycited by 1,136
  5. 2015Prioritized Experience Replaycited by 1,099
  6. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  7. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  8. 2016Deep Exploration via Bootstrapped DQNcited by 527
  9. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  10. 2020dm_control: Software and Tasks for Continuous Controlcited by 375
  11. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 319
  12. 2019Self-Supervised Exploration via Disagreementcited by 265
14 more
  1. 2020Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Surveycited by 254
  2. 2018Randomized Prior Functions for Deep Reinforcement Learningcited by 219
  3. 2018Self-Imitation Learningcited by 138
  4. 2018Sim-to-Real Reinforcement Learning for Deformable Object Manipulationcited by 131
  5. 2019A survey on intrinsic motivation in reinforcement learningcited by 81
  6. 2019Improving Exploration in Soft-Actor-Critic with Normalizing Flows Policiescited by 38
  7. 2022L2C2: Locally Lipschitz Continuous Constraint towards Stable and Smooth Reinforcement Learningcited by 27
  8. 2020PlaNet of the Bayesians: Reconsidering and Improving Deep Planning Network by Incorporating Bayesian Inferencecited by 23
  9. 2021Squareplus: A Softplus-Like Algebraic Rectifiercited by 19
  10. 2020Temporal Difference Uncertainties as a Signal for Explorationcited by 15
  11. 2018TD-Regularized Actor-Critic Methodscited by 14
  12. 2021Optimistic Reinforcement Learning by Forward Kullback-Leibler Divergence Optimizationcited by 8
  13. 2022Consolidated Adaptive T-soft Update for Deep Reinforcement Learningcited by 5
  14. 2022Proximal Policy Optimization with Adaptive Threshold for Symmetric Relative Density Ratiocited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 0; 2026: 4.

5 papers.

  1. 2023Intentionally-underestimated Value Function at Terminal State for Temporal-difference Learning with Mis-designed Rewardcited by 5
  2. 2026Flexible Empowerment at Reasoning with Extended Best-of-N Samplingcited by 3
  3. 2026Pseudo-Quantized Actor-Critic Algorithm for Robustness to Noisy Temporal Difference Errorcited by 1
  4. 2026Learning to Tune Pure Pursuit in Autonomous Racing: Joint Lookahead and Steering-Gain Control with PPOcited by 0
  5. 2026Dynamic Lookahead Distance via Reinforcement Learning-Based Pure Pursuit for Autonomous Racingcited by 0