mlsanityData to 14 Oct 2026

Offline Goal-Conditioned Reinforcement Learning with Projective Quasimetric Planning

arXiv 2506.18847 ↗First posted 23 Jun 2025cs.LGcited by 5

On the map: Offline RL and imitation learning from data → Diffusion and flow policies for offline RL

Abstract (arXiv)

Offline Goal-Conditioned Reinforcement Learning seeks to train agents to reach specified goals from previously collected trajectories. Scaling that promises to long-horizon tasks remains challenging, notably due to compounding value-estimation errors. Principled geometric offers a potential solution to address these issues. Following this insight, we introduce Projective Quasimetric Planning (ProQ), a compositional framework that learns an asymmetric distance and then repurposes it, firstly as a repulsive energy forcing a sparse set of keypoints to uniformly spread over the learned latent space, and secondly as a structured directional cost guiding towards proximal sub-goals. In particular, ProQ couples this geometry with a Lagrangian out-of-distribution detector to ensure the learned keypoints stay within reachable areas. By unifying metric learning, keypoint coverage, and goal-conditioned control, our approach produces meaningful sub-goals and robustly drives long-horizon goal-reaching on diverse a navigation benchmarks.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Diffusion and flow policies for offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 50 of its 70 references that are in the map. First posted on arXiv 23 Jun 2025.

Builds on

50 of 70 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024OGBench: Benchmarking Offline Goal-Conditioned RL26 Oct
  2. 2022Hierarchical Planning Through Goal-Conditioned Offline Reinforcement Learning24 May
  3. 2022Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RL9 Feb

Other references · 47

  1. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  2. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  3. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  4. 2017Hindsight Experience Replaycited by 1,136
  5. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  6. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  7. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
  8. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  9. 2017A Brief Survey of Deep Reinforcement Learningcited by 483
  10. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  11. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 415
  12. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
35 more
  1. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 286
  2. 2023Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuningcited by 237
  3. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 223
  4. 2018Semi-parametric Topological Memory for Navigationcited by 209
  5. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 207
  6. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 194
  7. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 166
  8. 2019Goal-conditioned Imitation Learningcited by 162
  9. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 158
  10. 2019Planning with Goal-Conditioned Policiescited by 149
  11. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 141
  12. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 134
  13. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  14. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 121
  15. 2018Near-Optimal Representation Learning for Hierarchical Reinforcement Learningcited by 110
  16. 2024Foundation Policies with Hilbert Representationscited by 94
  17. 2022Latent Plans for Task-Agnostic Offline Reinforcement Learningcited by 81
  18. 2020World Model as a Graph: Learning Latent Landmarks for Planningcited by 69
  19. 2022A Policy-Guided Imitation Approach for Offline Reinforcement Learningcited by 59
  20. 2020Generating Adjacency-Constrained Subgoals in Hierarchical Reinforcement Learningcited by 58
  21. 2022Efficient Planning in a Compact Latent Action Spacecited by 57
  22. 2021Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learningcited by 47
  23. 2019Mapping State Space using Landmarks for Universal Goal Reachingcited by 44
  24. 2017DDCO: Discovery of Deep Continuous Options for Robot Learning from Demonstrationscited by 43
  25. 2019Learning to Reach Goals via Iterated Supervised Learningcited by 26
  26. 2020An Inductive Bias for Distances: Neural Nets that Respect the Triangle Inequalitycited by 24
  27. 2022Improved Representation of Asymmetrical Distances with Interval Quasimetric Embeddingscited by 24
  28. 2021Successor Feature Landmarks for Long-Horizon Goal-Conditioned Reinforcement Learningcited by 23
  29. 2020Plan2Vec: Unsupervised Representation Learning by Latent Planscited by 21
  30. 2019Self-supervised Learning of Distance Functions for Goal-Conditioned Reinforcement Learningcited by 18
  31. 2023Imitating Graph-Based Planning with Goal-Conditioned Policiescited by 13
  32. 2022Value Memory Graph: A Graph-Structured World Model for Offline Reinforcement Learningcited by 12
  33. 2024Bisimulation metric for Model Predictive Controlcited by 6
  34. 2021Graph augmented Deep Reinforcement Learning in the GameRLand3D environmentcited by 5
  35. 2024Hierarchical Subspaces of Policies for Continual Offline Reinforcement Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2026Temporal-Distance JEPA: Plan-Aware Representation Learning for Latent World Model Predictive Controlcited by 8
  2. 2026Diffusion Subgoal Planning for Long-Horizon Offline Goal-Conditioned Reinforcement Learningcited by 3
  3. 2025Aligning LLMs on a Budget: Inference-Time Alignment with Heuristic Reward Modelscited by 0
  4. 2026Offline Reinforcement Learning of High-Quality Behaviors Under Robust Style Alignmentcited by 0
  5. 2026Beyond Success Rates: Trainability and Extractability for Offline GCRLcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Offline Goal-conditioned Reinforcement Learning with Quasimetric Representations
  2. 2026Learning Goal-Reaching Quasimetric Geometry From Finite-Time Reachability
  3. 2025Multistep Quasimetric Learning for Scalable Goal-conditioned Reinforcement Learning
  4. 2025Goal Reaching with Eikonal-Constrained Hierarchical Quasimetric Reinforcement Learning
  5. 2026Direction-Conditioned Policies via Compositional Subgoal Scoring for Online Goal-Conditioned Reinforcement Learning
  6. 2025Test-Time Graph Search for Goal-Conditioned Reinforcement Learning
  7. 2022Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Space
  8. 2026Multi-step Proximal Policy Improvement in Offline Reinforcement Learning