mlsanityData to 14 Oct 2026

Dynamic Contrastive Skill Learning with State-Transition Based Skill Clustering and Dynamic Length Adjustment

arXiv 2504.14805 ↗First posted 21 Apr 2025cs.LG cs.AI cs.ROcited by 3

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Reinforcement learning (RL) has made significant progress in various domains, but scaling it to long-horizon tasks with complex decision-making remains challenging. Skill learning attempts to address this by abstracting actions into higher-level behaviors. However, current approaches often fail to recognize semantically similar behaviors as the same skill and use fixed skill lengths, limiting flexibility and generalization. To address this, we propose Dynamic Contrastive Skill Learning (DCSL), a novel framework that redefines skill representation and learning. DCSL introduces three key ideas: state-transition based skill representation, skill similarity function learning, and dynamic skill length adjustment. By focusing on state transitions and leveraging contrastive learning, DCSL effectively captures the semantic context of behaviors and adapts skill lengths to match the appropriate temporal extent of behaviors. Our approach enables more flexible and adaptive skill extraction, particularly in complex or noisy datasets, and demonstrates competitive performance compared to existing methods in task completion and efficiency.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 29 of its 35 references that are in the map. First posted on arXiv 21 Apr 2025.

Builds on

29 of 35 references.

References · 29

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  5. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  6. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  7. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  8. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  9. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  10. 2016The Option-Critic Architecturecited by 448
  11. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 378
  12. 2019Learning Latent Plans from Playcited by 316
17 more
  1. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
  2. 2019Challenges of Real-World Reinforcement Learningcited by 291
  3. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  4. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 223
  5. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 166
  6. 2018Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiencycited by 105
  7. 2020C-Learning: Learning to Achieve Goals via Recursive Classificationcited by 103
  8. 2020Learning Robot Skills with Temporal Variational Inferencecited by 54
  9. 2022Skill-based Model-based Reinforcement Learningcited by 50
  10. 2020Emergent Real-World Robotic Skills via Unsupervised Off-Policy Reinforcement Learningcited by 31
  11. 2023Behavior Contrastive Learning for Unsupervised Skill Discoverycited by 26
  12. 2019The Termination Criticcited by 24
  13. 2023Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Datacited by 23
  14. 2022Learning Options via Compressioncited by 22
  15. 2022Cross-Domain Transfer via Semantic Skill Imitationcited by 18
  16. 2023Skill-Critic: Refining Learned Skills for Hierarchical Reinforcement Learningcited by 6
  17. 2024Skills Regularized Task Decomposition for Multi-task Offline Reinforcement Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Chain-of-Goals Hierarchical Policy for Long-Horizon Offline Goal-Conditioned RLcited by 5
  2. 2026How to Mitigate the Distribution Shift Problem in Robotics Control: A Robust and Adaptive Approach Based on Offline to Online Imitation Learningcited by 0
  3. 2026Improving Online Reinforcement Learning via Bidirectional Behavior Prior Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026DSD: Learning Diverse and Reusable Motor Skills via Diffusion Skill Discovery
  2. 2024Robust Policy Learning via Offline Skill Diffusion
  3. 2025Unsupervised Skill Discovery through Skill Regions Differentiation
  4. 2025Goal-Oriented Skill Abstraction for Offline Multi-Task Reinforcement Learning
  5. 2026Leveraging Human Feedback for Semantically-Relevant Skill Discovery
  6. 2022Bayesian Nonparametrics for Offline Skill Discovery
  7. 2024EXTRACT: Efficient Policy Learning by Extracting Transferable Robot Skills from Offline Data
  8. 2026SkillCAT: Contrastive, Assessment-Augmented and Topology-AwareSkill Self-Evolution for LLM Agents