Goal Reaching with Eikonal-Constrained Hierarchical Quasimetric Reinforcement Learning

arXiv 2512.12046 ↗First posted 12 Dec 2025cs.LG cs.RO cs.SYcited by 6

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Goal-Conditioned Reinforcement Learning (GCRL) mitigates the difficulty of reward design by framing tasks as goal reaching rather than maximizing hand-crafted reward signals. In this setting, the optimal goal-conditioned value function naturally forms a quasimetric, motivating Quasimetric RL (QRL), which constrains value learning to quasimetric mappings and enforces local consistency through discrete, trajectory-based constraints. We propose Eikonal-Constrained Quasimetric RL (Eik-QRL), a continuous-time reformulation of QRL based on the Eikonal Partial Differential Equation (PDE). This PDE-based structure makes Eik-QRL trajectory-free, requiring only sampled states and goals, while improving out-of-distribution generalization. We provide theoretical guarantees for Eik-QRL and identify limitations that arise under complex dynamics. To address these challenges, we introduce Eik-Hierarchical QRL (Eik-HiQRL), which integrates Eik-QRL into a hierarchical decomposition. Empirically, Eik-HiQRL achieves state-of-the-art performance in offline goal-conditioned navigation and yields consistent gains over QRL in manipulation tasks, matching temporal-difference methods.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 40 of its 63 references that are in the map. First posted on arXiv 12 Dec 2025.

Builds on

40 of 63 references.

References · 40

2013Playing Atari with Deep Reinforcement Learningcited by 3,274
2015Trust Region Policy Optimizationcited by 2,580
2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
2016Concrete Problems in AI Safetycited by 1,642
2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
2017Hindsight Experience Replaycited by 1,136
2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 840
2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337

28 more

2022The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Modelscited by 328
2020Critic Regularized Regressioncited by 275
2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 260
2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 238
2018Sorting out Lipschitz function approximationcited by 213
2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 176
2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 141
2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
2021Goal Misgeneralization in Deep Reinforcement Learningcited by 123
2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 121
2019Reward Tampering Problems and Solutions in Reinforcement Learning: A Causal Influence Diagram Perspectivecited by 108
2020DeepReach: A Deep Learning Approach to High-Dimensional Reachabilitycited by 91
2022Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RLcited by 89
2020Chip Placement with Deep Reinforcement Learningcited by 83
2017Reinforcement Learning with a Corrupted Reward Channelcited by 70
2022How Far I'll Go: Offline Goal-Conditioned Reinforcement Learning via f-Advantage Regressioncited by 66
2021Adversarial Intrinsic Motivation for Reinforcement Learningcited by 40
2023What is Essential for Unseen Goal Generalization of Offline Goal-conditioned RL?cited by 36
2020Hamilton-Jacobi Deep Q-Learning for Deterministic Continuous-Time Systems with Lipschitz Continuous Controlscited by 31
2023Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shapingcited by 25
2020An Inductive Bias for Distances: Neural Nets that Respect the Triangle Inequalitycited by 24
2022Improved Representation of Asymmetrical Distances with Interval Quasimetric Embeddingscited by 24
2023SMORE: Score Models for Offline Goal-Conditioned Reinforcement Learningcited by 21
2025Physics-informed Value Learner for Offline Goal-Conditioned Reinforcement Learningcited by 15
2024Reinforcement Learning-based Receding Horizon Control using Adaptive Control Barrier Functions for Safety-Critical Systemscited by 7
2025Hierarchical Multi-Agent Reinforcement Learning with Control Barrier Functions for Safety-Critical Autonomous Systemscited by 5
2021Online Baum-Welch algorithm for Hierarchical Imitation Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

2026Mollified Value Learningcited by 1
2026Latent Representation Alignment for Offline Goal-Conditioned Reinforcement Learningcited by 1
2026Physics-informed Goal-Conditioned Reinforcement Learning under Hybrid Contact Dynamicscited by 0
2026Dual Advantage Fieldscited by 0
2026Direction-Conditioned Policies via Compositional Subgoal Scoring for Online Goal-Conditioned Reinforcement Learningcited by 0
2026Learning Goal-Reaching Quasimetric Geometry From Finite-Time Reachabilitycited by 0

Similar, not cited

Close in wording, no citation either way