HG2P: Hippocampus-inspired High-reward Graph and Model-Free Q-Gradient Penalty for Path Planning and Motion Control
Not on the map. Why?
Abstract (arXiv)
Goal-conditioned hierarchical reinforcement learning (HRL) decomposes complex reaching tasks into a sequence of simple subgoal-conditioned tasks, showing significant promise for addressing long-horizon planning in large-scale environments. This paper bridges the goal-conditioned HRL based on graph-based planning to brain mechanisms, proposing a hippocampus-striatum-like dual-controller hypothesis. Inspired by the brain mechanisms of organisms (i.e., the high-reward preferences observed in hippocampal replay) and instance-based theory, we propose a high-return sampling strategy for constructing memory graphs, improving sample efficiency. Additionally, we derive a model-free lower-level Q-function gradient penalty to resolve the model dependency issues present in prior work, improving the generalization of Lipschitz constraints in applications. Finally, we integrate these two extensions, High-reward Graph and model-free Gradient Penalty (HG2P), into the state-of-the-art framework ACLG, proposing a novel goal-conditioned HRL framework, HG2P+ACLG. Experimentally, the results demonstrate that our method outperforms state-of-the-art goal-conditioned HRL algorithms on a variety of long-horizon navigation tasks and robotic manipulation tasks.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 27 of its 48 references that are in the map. First posted on arXiv 12 Oct 2024.
Builds on
27 of 48 references.
References · 27
- 2015Continuous control with deep reinforcement learningcited by 3,279
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
- 2017Improved Training of Wasserstein GANscited by 2,223
- 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
- 2018Exploration by Random Network Distillationcited by 966
- 2017Neural Episodic Controlcited by 148
- 2014Model-based Reinforcement Learning and the Eluder Dimensioncited by 116
- 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 105
- 2016Model-Free Episodic Controlcited by 100
- 2020World Model as a Graph: Learning Latent Landmarks for Planningcited by 67
- 2020Generating Adjacency-Constrained Subgoals in Hierarchical Reinforcement Learningcited by 55
- 2020Regularizing Action Policies for Smooth Control with Reinforcement Learningcited by 47
15 more
- 2021Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learningcited by 46
- 2019Mapping State Space using Landmarks for Universal Goal Reachingcited by 44
- 2018Episodic Memory Deep Q-Networkscited by 41
- 2022L2C2: Locally Lipschitz Continuous Constraint towards Stable and Smooth Reinforcement Learningcited by 22
- 2023Harnessing Mixed Offline Reinforcement Learning Datasets via Trajectory Weightingcited by 22
- 2023Reparameterized Policy Learning for Multimodal Trajectory Optimizationcited by 20
- 2022DHRL: A Graph-Based Approach for Long-Horizon and Sparse Hierarchical Reinforcement Learningcited by 18
- 2021Active Hierarchical Exploration with Stable Subgoal Representation Learningcited by 17
- 2020Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learningcited by 15
- 2023Imitating Graph-Based Planning with Goal-Conditioned Policiescited by 13
- 2021Adjacency constraint for efficient hierarchical reinforcement learningcited by 10
- 2019Learning to Control in Metric Space with Optimal Regretcited by 9
- 2018Temporal Regularization in Markov Decision Processcited by 3
- 2022Robust Offline Reinforcement Learning with Gradient Penalty and Constraint Relaxationcited by 2
- 2023Guided Cooperation in Hierarchical Reinforcement Learning via Model-based Rolloutcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.