mlsanityData to 7 Oct 2026

Controllability-Aware Unsupervised Skill Discovery

arXiv 2302.05103 ↗First posted 10 Feb 2023cs.RO cs.AI cs.LGcited by 49

Not on the map. Why?

Abstract (arXiv)

One of the key capabilities of intelligent agents is the ability to discover useful skills without external supervision. However, the current unsupervised skill discovery methods are often limited to acquiring simple, easy-to-learn skills due to the lack of incentives to discover more complex, challenging behaviors. We introduce a novel unsupervised skill discovery method, Controllability-aware Skill Discovery (CSD), which actively seeks complex, hard-to-control skills without supervision. The key component of CSD is a controllability-aware distance function, which assigns larger values to state transitions that are harder to achieve with the current skills. Combined with distance-maximizing skill discovery, CSD progressively learns more challenging skills over the course of training as our jointly trained distance function reduces rewards for easy-to-achieve skills. Our experimental results in six robotic manipulation and locomotion environments demonstrate that CSD can discover diverse complex skills including object manipulation and locomotion skills with no supervision, significantly outperforming prior unsupervised skill discovery methods. Videos and code are available at https://seohong.me/projects/csd/

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 46 of its 64 references that are in the map. First posted on arXiv 10 Feb 2023.

Builds on

46 of 64 references.

References · 46

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2021Convex optimizationcited by 2,479
  5. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  7. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  8. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  9. 2018Exploration by Random Network Distillationcited by 966
  10. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  11. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  12. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 380
34 more
  1. 2020Planning to Explore via Self-Supervised World Modelscited by 365
  2. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  3. 2019Self-Supervised Exploration via Disagreementcited by 263
  4. 2016Variational Intrinsic Controlcited by 262
  5. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  6. 2019Efficient Exploration via State Marginal Matchingcited by 186
  7. 2021Reinforcement Learning with Prototypical Representationscited by 179
  8. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  9. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 171
  10. 2022Masked World Models for Visual Controlcited by 169
  11. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 139
  12. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
  13. 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 131
  14. 2021Discovering and Achieving Goals via World Modelscited by 131
  15. 2018Variational Option Discovery Algorithmscited by 121
  16. 2020Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skillscited by 120
  17. 2021Learning One Representation to Optimize All Rewardscited by 110
  18. 2021APS: Active Pretraining with Successor Featurescited by 109
  19. 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102
  20. 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 100
  21. 2022Does Zero-Shot Reinforcement Learning Exist?cited by 79
  22. 2021Asymmetric self-play for automatic goal discovery in robotic manipulationcited by 71
  23. 2022Lipschitz-constrained Unsupervised Skill Discoverycited by 60
  24. 2018Self-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddingscited by 52
  25. 2021Learning more skills through optimistic explorationcited by 37
  26. 2021Unsupervised Skill Discovery with Bottleneck Option Learningcited by 28
  27. 2019Composing Task-Agnostic Policies with Deep Reinforcement Learningcited by 20
  28. 2021Direct then Diffuse: Incremental Unsupervised Skill Discovery for State Covering and Goal Reachingcited by 19
  29. 2021Mutual Information State Intrinsic Controlcited by 18
  30. 2022It Takes Four to Tango: Multiagent Selfplay for Automatic Curriculum Generationcited by 14
  31. 2021Braxlines: Fast and Interactive Toolkit for RL-driven Behavior Engineering beyond Reward Maximizationcited by 11
  32. 2022One After Another: Learning Incremental Skills for a Changing Worldcited by 10
  33. 2022A Mixture of Surprises for Unsupervised Reinforcement Learningcited by 9
  34. 2022Skill-Based Reinforcement Learning with Intrinsic Reward Matchingcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 20; 2025: 11; 2026: 8.

49 papers.

Sort:

By area:

  1. 2023Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidancecited by 85
  2. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 80
  3. 2024Learning Multimodal Behaviors from Scratch with Diffusion Policy Gradientcited by 49
  4. 2023Language Reward Modulation for Pretraining Reinforcement Learningcited by 42
  5. 2024Highly Efficient Self-Adaptive Reward Shaping for Reinforcement Learningcited by 21
  6. 2024Incremental Learning of Retrievable Skills For Efficient Continual Task Adaptationcited by 19
  7. 2024Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learningcited by 18
  8. 2024TLDR: Unsupervised Goal-Conditioned RL via Temporal Distance-Aware Representationscited by 17
  9. 2024Language Guided Skill Discoverycited by 16
  10. 2024PEAC: Unsupervised Pre-training for Cross-Embodiment Reinforcement Learningcited by 9
  11. 2024Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learningcited by 9
  12. 2024SkiLD: Unsupervised Skill Discovery Guided by Factor Interactionscited by 9
37 more
  1. 2023Learning to Discover Skills through Guidancecited by 8
  2. 2024Do's and Don'ts: Learning Desirable Skills with Instruction Videoscited by 8
  3. 2023Learning Diverse Skills for Local Navigation under Multi-constraint Optimalitycited by 7
  4. 2024Constrained Ensemble Exploration for Unsupervised Skill Discoverycited by 7
  5. 2024Constrained Intrinsic Motivation for Reinforcement Learningcited by 6
  6. 2025Average-Reward Soft Actor-Criticcited by 6
  7. 2024Agentic Skill Discoverycited by 5
  8. 2025Self-Improving Skill Learning for Robust Skill-based Meta-Reinforcement Learningcited by 5
  9. 2025Task Adaptation from Skills: Information Geometry, Disentanglement, and New Objectives for Unsupervised Reinforcement Learningcited by 5
  10. 2023Offline Diversity Maximization Under Imitation Constraintscited by 4
  11. 2026SCALAR: Learning and Composing Skills through LLM Guided Symbolic Planning and Deep RL Groundingcited by 4
  12. 2023Subwords as Skills: Tokenization for Sparse-Reward Reinforcement Learningcited by 3
  13. 2024SLIM: Skill Learning with Multiple Criticscited by 3
  14. 2024Rethinking Mutual Information for Language Conditioned Skill Discovery on Imitation Learningcited by 3
  15. 2024Unsupervised Skill Discovery for Robotic Manipulation through Automatic Task Generationcited by 3
  16. 2023Exploration with Principles for Diverse AI Supervisioncited by 2
  17. 2024Exploration by Learning Diverse Skills through Successor State Measurescited by 2
  18. 2025Policy Compatible Skill Incremental Learning via Lazy Learning Interfacecited by 2
  19. 2025Reference Grounded Skill Discoverycited by 2
  20. 2025Learning Skills from Action-Free Videoscited by 2
  21. 2026Behavioral Mode Discovery for Fine-tuning Multimodal Generative Policiescited by 2
  22. 2024MENTOR: Guiding Hierarchical Reinforcement Learning with Human Feedback and Dynamic Distance Constraintcited by 1
  23. 2024Latent-Predictive Empowerment: Measuring Empowerment without a Simulatorcited by 1
  24. 2025Information-Theoretic Policy Pre-Training with Empowermentcited by 1
  25. 2025Periodic Skill Discoverycited by 1
  26. 2026COLLIE: Guiding Skill Discovery in Semantically Coherent Latent Spacecited by 1
  27. 2023Leveraging Prior Knowledge in Reinforcement Learning via Double-Sided Bounds on the Value Functioncited by 0
  28. 2023ComSD: Balancing Behavioral Quality and Diversity in Unsupervised Skill Discoverycited by 0
  29. 2024Exploration by Running Away from the Pastcited by 0
  30. 2025AMPED: Adaptive Multi-objective Projection for balancing Exploration and skill Diversificationcited by 0
  31. 2025From Tabula Rasa to Emergent Abilities: Discovering Robot Skills via Real-World Unsupervised Quality-Diversitycited by 0
  32. 2025Focused Skill Discovery: Learning to Control Specific State Variables while Minimizing Side Effectscited by 0
  33. 2026Group-Invariant Unsupervised Skill Discovery: Symmetry-aware Skill Representations for Generalizable Behaviorcited by 0
  34. 2026SUSD: Structured Unsupervised Skill Discovery through State Factorizationcited by 0
  35. 2026Leveraging Human Feedback for Semantically-Relevant Skill Discoverycited by 0
  36. 2026Learning Generalizable Skill Policy with Data-Efficient Unsupervised RLcited by 0
  37. 2026Learning Transferable Skills using Goal-Conditioned Bisimulationcited by 0