mlsanityData to 7 Oct 2026

Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skills

arXiv 2002.03647 ↗First posted 10 Feb 2020cs.LG cs.AI stat.MLcited by 120

Not on the map. Why?

Abstract (arXiv)

Acquiring abilities in the absence of a task-oriented reward function is at the frontier of reinforcement learning research. This problem has been studied through the lens of empowerment, which draws a connection between option discovery and information theory. Information-theoretic skill discovery methods have garnered much interest from the community, but little research has been conducted in understanding their limitations. Through theoretical analysis and empirical evidence, we show that existing algorithms suffer from a common limitation -- they discover options that provide a poor coverage of the state space. In light of this, we propose 'Explore, Discover and Learn' (EDL), an alternative approach to information-theoretic skill discovery. Crucially, EDL optimizes the same information-theoretic objective derived from the empowerment literature, but addresses the optimization problem using different machinery. We perform an extensive evaluation of skill discovery methods on controlled environments and show that EDL offers significant advantages, such as overcoming the coverage problem, reducing the dependence of learned skills on the initial state, and allowing the user to define a prior over which behaviors should be learned. Code is publicly available at https://github.com/victorcampos7/edl.

Key facts. Cited by 120 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 26; 2022: 24; 2023: 24; 2024: 22; 2025: 14; 2026: 10.

120 papers.

Sort:

By area:

  1. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  2. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  3. 2021Discovering and Achieving Goals via World Modelscited by 131
  4. 2021APS: Active Pretraining with Successor Featurescited by 109
  5. 2021Celebrating Diversity in Shared Multi-Agent Reinforcement Learningcited by 96
  6. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 80
  7. 2022CIC: Contrastive Intrinsic Control for Unsupervised Skill Discoverycited by 66
  8. 2022Lipschitz-constrained Unsupervised Skill Discoverycited by 60
  9. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  10. 2024Proposer-Agent-Evaluator(PAE): Autonomous Skill Discovery For Foundation Model Internet Agentscited by 50
  11. 2023Controllability-Aware Unsupervised Skill Discoverycited by 49
  12. 2024Learning Multimodal Behaviors from Scratch with Diffusion Policy Gradientcited by 49
38 more
  1. 2021Hierarchical Skills for Efficient Explorationcited by 37
  2. 2021The Information Geometry of Unsupervised Reinforcement Learningcited by 36
  3. 2023Beyond Reward: Offline Preference-guided Policy Optimizationcited by 35
  4. 2023Contrastive Difference Predictive Codingcited by 35
  5. 2021Autonomous Reinforcement Learning: Formalism and Benchmarkingcited by 33
  6. 2022Choreographer: Learning and Adapting Skills in Imaginationcited by 31
  7. 2022Challenging Common Assumptions in Convex Reinforcement Learningcited by 29
  8. 2021Unsupervised Skill Discovery with Bottleneck Option Learningcited by 28
  9. 2023Planning Goals for Explorationcited by 28
  10. 2023Behavior Contrastive Learning for Unsupervised Skill Discoverycited by 26
  11. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 25
  12. 2021Specializing Versatile Skill Libraries using Local Mixture of Expertscited by 24
  13. 2022Neuroevolution is a Competitive Alternative to Reinforcement Learning for Skill Discoverycited by 23
  14. 2021Beyond Fine-Tuning: Transferring Behavior in Reinforcement Learningcited by 21
  15. 2021Wasserstein Unsupervised Reinforcement Learningcited by 21
  16. 2022A State-Distribution Matching Approach to Non-Episodic Reinforcement Learningcited by 21
  17. 2022Mastering the Unsupervised Reinforcement Learning Benchmark from Pixelscited by 20
  18. 2023Reparameterized Policy Learning for Multimodal Trajectory Optimizationcited by 20
  19. 2021Direct then Diffuse: Incremental Unsupervised Skill Discovery for State Covering and Goal Reachingcited by 19
  20. 2021Mutual Information State Intrinsic Controlcited by 18
  21. 2022Pretraining in Deep Reinforcement Learning: A Surveycited by 18
  22. 2024Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learningcited by 18
  23. 2021Active Hierarchical Exploration with Stable Subgoal Representation Learningcited by 17
  24. 2023A Definition of Open-Ended Learning Problems for Goal-Conditioned Agentscited by 16
  25. 2021Policy Information Capacity: Information-Theoretic Measure for Task Complexity in Deep Reinforcement Learningcited by 15
  26. 2021Learn Goal-Conditioned Policy with Intrinsic Motivation for Deep Reinforcement Learningcited by 15
  27. 2021Variational Empowerment as Representation Learning for Goal-Based Reinforcement Learningcited by 15
  28. 2021Unsupervised Domain Adaptation with Dynamics-Aware Rewards in Reinforcement Learningcited by 15
  29. 2024Global Reinforcement Learning: Beyond Linear and Convex Rewards via Submodular Semi-gradient Methodscited by 15
  30. 2024Acquiring Diverse Skills using Curriculum Reinforcement Learning with Mixture of Expertscited by 14
  31. 2021Sparse Reward Exploration via Novelty Search and Emitterscited by 13
  32. 2022Versatile Skill Control via Self-supervised Adversarial Imitation of Unlabeled Mixed Motionscited by 13
  33. 2024Leveraging Skills from Unlabeled Prior Data for Efficient Online Explorationcited by 13
  34. 2023CLUE: Calibrated Latent Guidance for Offline Reinforcement Learningcited by 11
  35. 2025Plasticity as the Mirror of Empowermentcited by 11
  36. 2022One After Another: Learning Incremental Skills for a Changing Worldcited by 10
  37. 2022A Mixture of Surprises for Unsupervised Reinforcement Learningcited by 9
  38. 2024Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learningcited by 9

50 most cited of 120 All 120 (JSON)