mlsanityData to 7 Oct 2026

Periodic Skill Discovery

arXiv 2511.03187 ↗First posted 5 Nov 2025cs.LG cs.ROcited by 1

Not on the map. Why?

Abstract (arXiv)

Unsupervised skill discovery in reinforcement learning (RL) aims to learn diverse behaviors without relying on external rewards. However, current methods often overlook the periodic nature of learned skills, focusing instead on increasing the mutual dependence between states and skills or maximizing the distance traveled in latent space. Considering that many robotic tasks - particularly those involving locomotion - require periodic behaviors across varying timescales, the ability to discover diverse periodic skills is essential. Motivated by this, we propose Periodic Skill Discovery (PSD), a framework that discovers periodic behaviors in an unsupervised manner. The key idea of PSD is to train an encoder that maps states to a circular latent space, thereby naturally encoding periodicity in the latent representation. By capturing temporal distance, PSD can effectively learn skills with diverse periods in complex robotic tasks, even with pixel-based observations. We further show that these learned skills achieve high performance on downstream tasks such as hurdling. Moreover, integrating PSD with an existing skill discovery method offers more diverse behaviors, thus broadening the agent's repertoire. Our code and demos are available at https://jonghaepark.github.io/psd/

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 73 of its 102 references that are in the map. First posted on arXiv 5 Nov 2025.

Builds on

73 of 102 references.

References · 73

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2021Convex optimizationcited by 2,479
  6. 2018Exploration by Random Network Distillationcited by 966
  7. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  8. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  9. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  10. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
  11. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
  12. 2015Illuminating search spaces by mapping elitescited by 406
38 more
  1. 2020Planning to Explore via Self-Supervised World Modelscited by 365
  2. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  3. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 307
  4. 2017Count-Based Exploration with Neural Density Modelscited by 255
  5. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  6. 2018Provably Efficient Maximum Entropy Explorationcited by 240
  7. 2023Perpetual Humanoid Control for Real-time Simulated Avatarscited by 198
  8. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 188
  9. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 171
  10. 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 131
  11. 2021Discovering and Achieving Goals via World Modelscited by 131
  12. 2018Variational Option Discovery Algorithmscited by 121
  13. 2020Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skillscited by 120
  14. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 116
  15. 2017Quality and Diversity Optimization: A Unifying Modular Frameworkcited by 111
  16. 2018Model-Based Active Explorationcited by 105
  17. 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 100
  18. 2020C-Learning: Learning to Achieve Goals via Recursive Classificationcited by 99
  19. 2018Temporal Difference Models: Model-Free Deep RL for Model-Based Controlcited by 89
  20. 2024Foundation Policies with Hilbert Representationscited by 89
  21. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 80
  22. 2020Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Compositioncited by 71
  23. 2022CIC: Contrastive Intrinsic Control for Unsupervised Skill Discoverycited by 66
  24. 2019Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discoverycited by 65
  25. 2022Lipschitz-constrained Unsupervised Skill Discoverycited by 60
  26. 2020Quality-Diversity Optimization: a novel branch of stochastic optimizationcited by 59
  27. 2019Autonomous skill discovery with Quality-Diversity and Unsupervised Descriptorscited by 57
  28. 2020Diversity Policy Gradient for Sample Efficient Quality-Diversity Optimizationcited by 53
  29. 2024Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Makingcited by 53
  30. 2023Controllability-Aware Unsupervised Skill Discoverycited by 49
  31. 2017EX2: Exploration with Exemplar Models for Deep Reinforcement Learningcited by 42
  32. 2021Learning more skills through optimistic explorationcited by 37
  33. 2023Contrastive Difference Predictive Codingcited by 35
  34. 2023Not Only Rewards But Also Constraints: Applications on Legged Robot Locomotioncited by 33
  35. 2022Choreographer: Learning and Adapting Skills in Imaginationcited by 31
  36. 2021Unsupervised Behaviour Discovery with Quality-Diversity Optimisationcited by 30
  37. 2021Robust Predictable Controlcited by 29
  38. 2021Unsupervised Skill Discovery with Bottleneck Option Learningcited by 28

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Learning Generalizable Skill Policy with Data-Efficient Unsupervised RLcited by 0