mlsanityData to 7 Oct 2026

Lipschitz-constrained Unsupervised Skill Discovery

arXiv 2202.00914 ↗First posted 2 Feb 2022cs.LG cs.AI cs.ROcited by 60

Not on the map. Why?

Abstract (arXiv)

We study the problem of unsupervised skill discovery, whose goal is to learn a set of diverse and useful skills with no external reward. There have been a number of skill discovery methods based on maximizing the mutual information (MI) between skills and states. However, we point out that their MI objectives usually prefer static skills to dynamic ones, which may hinder the application for downstream tasks. To address this issue, we propose Lipschitz-constrained Skill Discovery (LSD), which encourages the agent to discover more diverse, dynamic, and far-reaching skills. Another benefit of LSD is that its learned representation function can be utilized for solving goal-following downstream tasks even in a zero-shot manner - i.e., without further training or complex planning. Through experiments on various MuJoCo robotic locomotion and manipulation environments, we demonstrate that LSD outperforms previous approaches in terms of skill diversity, state space coverage, and performance on seven downstream tasks including the challenging task of following multiple goals on Humanoid. Our code and videos are available at https://shpark.me/projects/lsd/.

Key facts. Cited by 60 papers in the map (data through 7 Oct 2026). Builds on 29 of its 41 references that are in the map. First posted on arXiv 2 Feb 2022.

Builds on

29 of 41 references.

References · 29

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  4. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  6. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  7. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  8. 2018Learning Dexterous In-Hand Manipulationcited by 643
  9. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
  10. 2016Successor Features for Transfer in Reinforcement Learningcited by 331
  11. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 309
  12. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
17 more
  1. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
  2. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  3. 2017Inverse Reward Designcited by 246
  4. 2019Efficient Exploration via State Marginal Matchingcited by 186
  5. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  6. 2015Variational Information Maximisation for Intrinsically Motivated Reinforcement Learningcited by 142
  7. 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 131
  8. 2020Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skillscited by 120
  9. 2016Robust Large Margin Deep Neural Networkscited by 117
  10. 2021APS: Active Pretraining with Successor Featurescited by 109
  11. 2017A Laplacian Framework for Option Discovery in Reinforcement Learningcited by 108
  12. 2017Eigenoption Discovery through the Deep Successor Representationcited by 85
  13. 2021Hierarchical Reinforcement Learning By Discovering Intrinsic Optionscited by 57
  14. 2021Unsupervised Skill Discovery with Bottleneck Option Learningcited by 28
  15. 2019Composing Task-Agnostic Policies with Deep Reinforcement Learningcited by 20
  16. 2021Mutual Information State Intrinsic Controlcited by 18
  17. 2021Braxlines: Fast and Interactive Toolkit for RL-driven Behavior Engineering beyond Reward Maximizationcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 13; 2024: 16; 2025: 16; 2026: 13.

60 papers.

Sort:

By area:

  1. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 80
  2. 2023Controllability-Aware Unsupervised Skill Discoverycited by 49
  3. 2022Choreographer: Learning and Adapting Skills in Imaginationcited by 31
  4. 2023Deep Laplacian-based Options for Temporally-Extended Explorationcited by 28
  5. 2023Behavior Contrastive Learning for Unsupervised Skill Discoverycited by 26
  6. 2022Pretraining in Deep Reinforcement Learning: A Surveycited by 18
  7. 2024Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learningcited by 18
  8. 2024TLDR: Unsupervised Goal-Conditioned RL via Temporal Distance-Aware Representationscited by 17
  9. 2024Language Guided Skill Discoverycited by 16
  10. 2023LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teacherscited by 14
  11. 2024Learning to Assist Humans without Inferring Rewardscited by 12
  12. 2025Automated Skill Discovery for Language Agents through Exploration and Iterative Feedbackcited by 11
38 more
  1. 2024PEAC: Unsupervised Pre-training for Cross-Embodiment Reinforcement Learningcited by 9
  2. 2024Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learningcited by 9
  3. 2024SkiLD: Unsupervised Skill Discovery Guided by Factor Interactionscited by 9
  4. 2025Multi-agent Coordination via Flow Matchingcited by 9
  5. 2023Learning to Discover Skills through Guidancecited by 8
  6. 2024Do's and Don'ts: Learning Desirable Skills with Instruction Videoscited by 8
  7. 2023Predictable MDP Abstraction for Unsupervised Model-Based RLcited by 7
  8. 2023Hierarchical Empowerment: Towards Tractable Empowerment-Based Skill Learningcited by 7
  9. 2023Learning Diverse Skills for Local Navigation under Multi-constraint Optimalitycited by 7
  10. 2024Constrained Ensemble Exploration for Unsupervised Skill Discoverycited by 7
  11. 2024Dr. Strategy: Model-Based Generalist Agents with Strategic Dreamingcited by 6
  12. 2024Constrained Intrinsic Motivation for Reinforcement Learningcited by 6
  13. 2025Self-Improving Skill Learning for Robust Skill-based Meta-Reinforcement Learningcited by 5
  14. 2025Task Adaptation from Skills: Information Geometry, Disentanglement, and New Objectives for Unsupervised Reinforcement Learningcited by 5
  15. 2025Unsupervised Skill Discovery as Exploration for Learning Agile Locomotioncited by 5
  16. 2023Offline Diversity Maximization Under Imitation Constraintscited by 4
  17. 2024S-EPOA: Overcoming the Indistinguishability of Segments with Skill-Driven Preference-Based Reinforcement Learningcited by 4
  18. 2025Skill Learning via Policy Diversity Yields Identifiable Representations for Reinforcement Learningcited by 4
  19. 2026SCALAR: Learning and Composing Skills through LLM Guided Symbolic Planning and Deep RL Groundingcited by 4
  20. 2023Subwords as Skills: Tokenization for Sparse-Reward Reinforcement Learningcited by 3
  21. 2024SLIM: Skill Learning with Multiple Criticscited by 3
  22. 2024Unsupervised Skill Discovery for Robotic Manipulation through Automatic Task Generationcited by 3
  23. 2025Human-Aligned Skill Discovery: Balancing Behaviour Exploration and Alignmentcited by 3
  24. 2025Efficient Skill Discovery via Regret-Aware Optimizationcited by 3
  25. 2024Exploration by Learning Diverse Skills through Successor State Measurescited by 2
  26. 2025Policy Compatible Skill Incremental Learning via Lazy Learning Interfacecited by 2
  27. 2025Reference Grounded Skill Discoverycited by 2
  28. 2025Learning Skills from Action-Free Videoscited by 2
  29. 2024Latent-Predictive Empowerment: Measuring Empowerment without a Simulatorcited by 1
  30. 2025Deep Reinforcement Learning with Hybrid Intrinsic Reward Modelcited by 1
  31. 2025RedRFT: A Light-Weight Benchmark for Reinforcement Fine-Tuning-Based Red Teamingcited by 1
  32. 2025Periodic Skill Discoverycited by 1
  33. 2026Task-Aware Exploration via a Predictive Bisimulation Metriccited by 1
  34. 2026Saliency-Guided Representation with Consistency Policy Learning for Visual Unsupervised Reinforcement Learningcited by 1
  35. 2026COLLIE: Guiding Skill Discovery in Semantically Coherent Latent Spacecited by 1
  36. 2023ComSD: Balancing Behavioral Quality and Diversity in Unsupervised Skill Discoverycited by 0
  37. 2023A Central Motor System Inspired Pre-training Reinforcement Learning for Robotic Controlcited by 0
  38. 2025AMPED: Adaptive Multi-objective Projection for balancing Exploration and skill Diversificationcited by 0

50 most cited of 60 All 60 (JSON)