mlsanityData to 14 Oct 2026

Abstraction for Offline Goal-Conditioned Reinforcement Learning

arXiv 2605.22711 ↗First posted 21 May 2026cs.LG cs.AIcited by 2

On the map: Offline RL and imitation learning from data → Diffusion and flow policies for offline RL

Abstract (arXiv)

Markov Decision Processes (MDPs) often exhibit significant redundancy due to symmetries and shared structure across state-goal pairs in real-world Goal-Conditioned Reinforcement Learning (GCRL). While hierarchical policies have been motivated for horizon reduction via temporal abstraction in offline GCRL, we demonstrate that hierarchy also enables absolute abstraction. By introducing relativised options as well as distinct representations for different levels of the hierarchy, we demonstrate how an agent can reuse experience across similar contexts of the state-space. Based on this framework, we introduce two simple algorithms for learning relativised options and abstracting from the absolute frame of reference. Our experiments show that such inductive biases significantly improve performance in offline GCRL.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Diffusion and flow policies for offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 36 of its 61 references that are in the map. First posted on arXiv 21 May 2026.

Builds on

36 of 61 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024OGBench: Benchmarking Offline Goal-Conditioned RL26 Oct
  2. 2022Hierarchical Planning Through Goal-Conditioned Offline Reinforcement Learning24 May

Other references · 34

  1. 2020Scaling Laws for Neural Language Modelscited by 6,586
  2. 2016Layer Normalizationcited by 5,669
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  4. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  5. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  6. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  7. 2017Hindsight Experience Replaycited by 1,136
  8. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 899
  9. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  10. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
  11. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  12. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
22 more
  1. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 302
  2. 2024Flow Matching Guide and Codecited by 301
  3. 2019Challenges of Real-World Reinforcement Learningcited by 291
  4. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  5. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 231
  6. 2025Real-Time Execution of Action Chunking Flow Policiescited by 216
  7. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 141
  8. 2023Revisiting the Minimalist Approach to Offline Reinforcement Learningcited by 126
  9. 2020EMaQ: Expected-Max Q-Learning Operator for Simple Yet Effective Offline and Online RLcited by 115
  10. 2018Near-Optimal Representation Learning for Hierarchical Reinforcement Learningcited by 110
  11. 2024Is Value Learning Really the Main Bottleneck in Offline RL?cited by 89
  12. 2023Reinforcement Learning from Passive Data via Latent Intentionscited by 68
  13. 2022A Policy-Guided Imitation Approach for Offline Reinforcement Learningcited by 59
  14. 2025Horizon Reduction Makes RL Scalablecited by 57
  15. 20251000 Layer Networks for Self-Supervised RL: Scaling Depth Can Enable New Goal-Reaching Capabilitiescited by 45
  16. 2022Addressing Optimism Bias in Sequence Modeling for Reinforcement Learningcited by 22
  17. 2025A Survey of State Representation Learning for Deep Reinforcement Learningcited by 17
  18. 2025Dual Goal Representationscited by 16
  19. 2025Scalable Offline Model-Based RL with Action Chunkscited by 13
  20. 2025Graph-Assisted Stitching for Offline Hierarchical Reinforcement Learningcited by 12
  21. 2025Transitive RL: Value Learning via Divide and Conquercited by 12
  22. 2025A Clean Slate for Offline Reinforcement Learningcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Learning Multiple Timescales for Goal-Conditioned Reinforcement Learningcited by 0
  2. 2026World-Model Policy Arbiter for Goal-Conditioned Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Adaptive Coarse-to-Fine Subgoal Refinement for Long-Horizon Offline Goal-Conditioned Reinforcement Learning
  2. 2025Realizable Abstractions: Near-Optimal Hierarchical Reinforcement Learning
  3. 2023Provably Efficient Offline Goal-Conditioned Reinforcement Learning with General Function Approximation and Single-Policy Concentrability
  4. 2025Option-aware Temporally Abstracted Value for Offline Goal-Conditioned Reinforcement Learning
  5. 2024Temporal Abstraction in Reinforcement Learning with Offline Data
  6. 2026Q-Shaped Options for Hierarchical Reinforcement Learning
  7. 2024Reconciling Spatial and Temporal Abstractions for Goal Representation
  8. 2025Flattening Hierarchies with Policy Bootstrapping