mlsanityData to 14 Oct 2026

Dual Goal Representations

arXiv 2510.06714 ↗First posted 8 Oct 2025cs.LG cs.AIcited by 16

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

In this work, we introduce dual goal representations for goal-conditioned reinforcement learning (GCRL). A dual goal representation characterizes a state by "the set of temporal distances from all other states"; in other words, it encodes a state through its relations to every other state, measured by temporal distance. This representation provides several appealing theoretical properties. First, it depends only on the intrinsic dynamics of the environment and is invariant to the original state representation. Second, it contains provably sufficient information to recover an optimal goal-reaching policy, while being able to filter out exogenous noise. Based on this concept, we develop a practical goal representation learning method that can be combined with any existing GCRL algorithm. Through diverse experiments on the OGBench task suite, we empirically show that dual goal representations consistently improve offline goal-reaching performance across 20 state- and pixel-based tasks.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 44 of its 58 references that are in the map. First posted on arXiv 8 Oct 2025.

Builds on

44 of 58 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025A Survey of State Representation Learning for Deep Reinforcement Learning20 Jun
  2. 2025Horizon Reduction Makes RL Scalable4 Jun
  3. 2024Unsupervised-to-Online Reinforcement Learning27 Aug
  4. 2024Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Making24 Jun
  5. 2024Foundation Policies with Hilbert Representations23 Feb

Other references · 39

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  3. 2016Layer Normalizationcited by 5,669
  4. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  6. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  7. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  8. 2016Deep Variational Information Bottleneckcited by 930
  9. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  10. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  11. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
  12. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 395
27 more
  1. 2016Successor Features for Transfer in Reinforcement Learningcited by 340
  2. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
  3. 2019Learning Latent Plans from Playcited by 316
  4. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  5. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 238
  6. 2018Semi-parametric Topological Memory for Navigationcited by 209
  7. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 194
  8. 2025Flow Q-Learningcited by 184
  9. 2019Provably efficient RL with Rich Observations via Latent State Decodingcited by 169
  10. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 141
  11. 2019Scalable methods for computing state similarity in deterministic Markov Decision Processescited by 133
  12. 2021Rapid Exploration for Open-World Navigation with Latent Goal Modelscited by 121
  13. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 121
  14. 2021Learning One Representation to Optimize All Rewardscited by 115
  15. 2018The Laplacian in RL: Learning Representations with Efficient Approximationscited by 86
  16. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 86
  17. 2022Does Zero-Shot Reinforcement Learning Exist?cited by 83
  18. 2023Reinforcement Learning from Passive Data via Latent Intentionscited by 68
  19. 2022How Far I'll Go: Offline Goal-Conditioned Reinforcement Learning via f-Advantage Regressioncited by 66
  20. 2023Dual RL: Unification and New Methods for Reinforcement and Imitation Learningcited by 48
  21. 2022Guaranteed Discovery of Control-Endogenous Latent States with Multi-Step Inverse Modelscited by 45
  22. 2023Contrastive Difference Predictive Codingcited by 36
  23. 2023GOPlan: Goal-conditioned Offline Reinforcement Learning by Planning with Learned Modelscited by 20
  24. 2025Temporal Representation Alignment: Successor Features Enable Emergent Compositionality in Robot Instruction Followingcited by 14
  25. 2025Self-Predictive Representations for Combinatorial Generalization in Behavioral Cloningcited by 12
  26. 2022State Representation Learning for Goal-Conditioned Reinforcement Learningcited by 11
  27. 2018Early Fusion for Goal Directed Robotic Visioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 16.

16 papers.

Sort:

By area:

  1. 2026DeFlow: Decoupling Manifold Modeling and Value Maximization for Offline Policy Extractioncited by 5
  2. 2026Refining Compositional Diffusion for Reliable Long-Horizon Planningcited by 2
  3. 2026Unifying Goal-Conditioned RL and Unsupervised Skill Learning via Control-Maximizationcited by 2
  4. 2026Abstraction for Offline Goal-Conditioned Reinforcement Learningcited by 2
  5. 2026Action-Sufficient Goal Representationscited by 1
  6. 2026Mollified Value Learningcited by 1
  7. 2026MSPR: Multi-scale Predictive Representations for Goal-conditioned Reinforcement Learningcited by 1
  8. 2026Compositional Transduction with Latent Analogies for Offline Goal-Conditioned Reinforcement Learningcited by 1
  9. 2026The Geometry of Empowermentcited by 1
  10. 2026Beyond Success Rates: Trainability and Extractability for Offline GCRLcited by 0
  11. 2026Exploiting Local Dynamics Regularity for Reusable Skills in Offline Hierarchical RLcited by 0
  12. 2026Dual Advantage Fieldscited by 0
4 more
  1. 2026Direction-Conditioned Policies via Compositional Subgoal Scoring for Online Goal-Conditioned Reinforcement Learningcited by 0
  2. 2026Do Better Goal Representations Improve Goal-Conditioned Reinforcement Learning?cited by 0
  3. 2026Direction-Conditioned Policies for Online Goal-Conditioned Reinforcement Learningcited by 0
  4. 2026Imagining Goals with Geometric Horizon Models for Policy Generalizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024TLDR: Unsupervised Goal-Conditioned RL via Temporal Distance-Aware Representations
  2. 2025Offline Goal-conditioned Reinforcement Learning with Quasimetric Representations
  3. 2026DAGR: State-Conditioned Goal Representations via Difference-Aware Goal Cross-Attention
  4. 2026Learning Multiple Timescales for Goal-Conditioned Reinforcement Learning
  5. 2025Incorporating Spatial Information into Goal-Conditioned Hierarchical Reinforcement Learning via Graph Representations
  6. 2026Latent Representation Alignment for Offline Goal-Conditioned Reinforcement Learning
  7. 2023Goal-Conditioned Predictive Coding for Offline Reinforcement Learning
  8. 2025First-Order Representation Languages for Goal-Conditioned RL