mlsanityData to 7 Oct 2026

Latent Representation Alignment for Offline Goal-Conditioned Reinforcement Learning

arXiv 2605.25740 ↗First posted 25 May 2026cs.LGcited by 1

On the map: Offline RL and imitation learning from data → Diffusion and flow policies for offline RL

Abstract (arXiv)

Offline goal-conditioned reinforcement learning (GCRL) provides a practical framework for obtaining goal-reaching policies from fixed datasets. However, learning a reliable goal-conditioned value function in long-horizon tasks remains challenging. In this paper, we identify erroneous generalization in goal-conditioned value functions as a fundamental bottleneck, and demonstrate that appropriate inductive bias in the value function is crucial for addressing the bottleneck. Building on these findings, we propose Latent-Aligned Value Learning (LAVL), an offline GCRL algorithm that integrates latent-representation-based value generalization with hierarchical planning in a unified framework. Extensive experiments on OGBench demonstrate that LAVL consistently outperforms existing offline GCRL methods, achieving the highest performance on 20 out of 22 datasets. Notably, LAVL exhibits strong performance in long-horizon tasks and trajectory stitching datasets, where prior methods suffer significant performance degradation. Our code is available at https://github.com/oh-lab/LAVL.git.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Diffusion and flow policies for offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 36 of its 44 references that are in the map. First posted on arXiv 25 May 2026.

Builds on

36 of 44 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Physics-informed Value Learner for Offline Goal-Conditioned Reinforcement Learning8 Sep
  2. 2025Option-aware Temporally Abstracted Value for Offline Goal-Conditioned Reinforcement Learning19 May
  3. 2025Flow Q-Learning4 Feb
  4. 2024OGBench: Benchmarking Offline Goal-Conditioned RL26 Oct
  5. 2022Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RL9 Feb

Other references · 31

  1. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  3. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  4. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  5. 2017Hindsight Experience Replaycited by 1,096
  6. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  7. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  8. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  9. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
  10. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
  11. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
  12. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
19 more
  1. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 188
  2. 2019Planning with Goal-Conditioned Policiescited by 144
  3. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 133
  4. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
  5. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  6. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 116
  7. 2020C-Learning: Learning to Achieve Goals via Recursive Classificationcited by 99
  8. 2024Foundation Policies with Hilbert Representationscited by 89
  9. 2019Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generationcited by 83
  10. 2020World Model as a Graph: Learning Latent Landmarks for Planningcited by 67
  11. 2024Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Makingcited by 53
  12. 2021Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learningcited by 46
  13. 2019Mapping State Space using Landmarks for Universal Goal Reachingcited by 44
  14. 2021Adversarial Intrinsic Motivation for Reinforcement Learningcited by 40
  15. 20251000 Layer Networks for Self-Supervised RL: Scaling Depth Can Enable New Goal-Reaching Capabilitiescited by 39
  16. 2022Improved Representation of Asymmetrical Distances with Interval Quasimetric Embeddingscited by 22
  17. 2021Hierarchical Reinforcement Learning with Timed Subgoalscited by 20
  18. 2025A Clean Slate for Offline Reinforcement Learningcited by 10
  19. 2025Goal Reaching with Eikonal-Constrained Hierarchical Quasimetric Reinforcement Learningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026NFTR: From Provable Mode-Averaging to Geodesic Subgoal Selection in Offline Goal-Conditioned RLcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Recursive Value Learning for Long-Horizon Offline Goal-Conditioned RL
  2. 2021Offline Reinforcement Learning with Value-based Episodic Memory
  3. 2026GTRL: Grounding Divide-and-Conquer Value Learning with Temporal Differences
  4. 2022How Far I'll Go: Offline Goal-Conditioned Reinforcement Learning via f-Advantage Regression
  5. 2023Provably Efficient Offline Goal-Conditioned Reinforcement Learning with General Function Approximation and Single-Policy Concentrability
  6. 2026Beyond Success Rates: Trainability and Extractability for Offline GCRL
  7. 2026Efficient Hierarchical Implicit Flow Q-learning for Offline Goal-conditioned Reinforcement Learning
  8. 2026Unifying Value Alignment and Assignment in Cross-Domain Offline Reinforcement Learning with Heterogeneous Datasets