mlsanityData to 7 Oct 2026

Exploiting Local Dynamics Regularity for Reusable Skills in Offline Hierarchical RL

arXiv 2605.26371 ↗First posted 25 May 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Hierarchical Reinforcement Learning (HRL) promises to solve long-horizon Reinforcement Learning (RL) tasks more efficiently than non-hierarchical counterparts by discovering and reusing temporally-extended skills. However, obtaining skills that are actually reusable remains an open challenge. Towards this end, we focus on abstractions that exploit the intuition of local dynamics: local transitions in different global contexts require similar kinds of action sequences. By aligning these contexts with the action sequences they require, we are able to learn which skills to reuse and where to reuse them. In principle, this information should benefit many HRL algorithms, where high-level policies have to reason about the low-level skills they use. The resulting algorithm CARL (Contrastive Action-based Representations for Reusable Local Control) shows both qualitative clustering of meaningful skills in complex humanoid environments and improved downstream performance on the OGBench benchmark when integrated with HIQL.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 29 references that are in the map. First posted on arXiv 25 May 2026.

Builds on

20 of 29 references.

References · 20

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  4. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  5. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  6. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  7. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  8. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 188
  9. 2019Scalable methods for computing state similarity in deterministic Markov Decision Processescited by 133
  10. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 133
  11. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  12. 2022Deep Hierarchical Planning from Pixelscited by 96
8 more
  1. 2020Generating Adjacency-Constrained Subgoals in Hierarchical Reinforcement Learningcited by 55
  2. 2022Bisimulation Makes Analogies in Goal-Conditioned Reinforcement Learningcited by 38
  3. 2018Hierarchical Reinforcement Learning with Hindsightcited by 26
  4. 2022Agent-Controller Representations: Principled Offline RL with Rich Exogenous Informationcited by 15
  5. 2025Dual Goal Representationscited by 13
  6. 2022Planning with Spatial-Temporal Abstraction from Point Clouds for Deformable Object Manipulationcited by 10
  7. 2024Learning Action-based Representations Using Invariancecited by 5
  8. 2025Hierarchical Reinforcement Learning with Uncertainty-Guided Diffusional Subgoalscited by 5

Built on by

Not cited yet.