mlsanityData to 7 Oct 2026

Learning to Discover Skills through Guidance

arXiv 2310.20178 ↗First posted 31 Oct 2023cs.LG cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

In the field of unsupervised skill discovery (USD), a major challenge is limited exploration, primarily due to substantial penalties when skills deviate from their initial trajectories. To enhance exploration, recent methodologies employ auxiliary rewards to maximize the epistemic uncertainty or entropy of states. However, we have identified that the effectiveness of these rewards declines as the environmental complexity rises. Therefore, we present a novel USD algorithm, skill discovery with guidance (DISCO-DANCE), which (1) selects the guide skill that possesses the highest potential to reach unexplored states, (2) guides other skills to follow guide skill, then (3) the guided skills are dispersed to maximize their discriminability in unexplored states. Empirical evaluation demonstrates that DISCO-DANCE outperforms other USD baselines in challenging environments, including two navigation benchmarks and a continuous control benchmark. Qualitative visualizations and code of DISCO-DANCE are available at https://mynsng.github.io/discodance.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 56 of its 59 references that are in the map. First posted on arXiv 31 Oct 2023.

Builds on

56 of 59 references.

References · 56

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  4. 2017Hindsight Experience Replaycited by 1,096
  5. 2015Prioritized Experience Replaycited by 1,070
  6. 2018Exploration by Random Network Distillationcited by 966
  7. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  8. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  9. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  10. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  11. 2020dm_control: Software and Tasks for Continuous Controlcited by 368
  12. 2020Planning to Explore via Self-Supervised World Modelscited by 365
38 more
  1. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
  2. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
  3. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  4. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  5. 2019Self-Supervised Exploration via Disagreementcited by 263
  6. 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
  7. 2016Variational Intrinsic Controlcited by 262
  8. 2020First return, then explorecited by 256
  9. 2022Masked Visual Pre-training for Motor Controlcited by 246
  10. 2018Provably Efficient Maximum Entropy Explorationcited by 240
  11. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 220
  12. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 207
  13. 2019Efficient Exploration via State Marginal Matchingcited by 186
  14. 2021Reinforcement Learning with Prototypical Representationscited by 179
  15. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  16. 2019Planning with Goal-Conditioned Policiescited by 144
  17. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
  18. 2020Parrot: Data-Driven Behavioral Priors for Reinforcement Learningcited by 136
  19. 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 131
  20. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
  21. 2018Variational Option Discovery Algorithmscited by 121
  22. 2020Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skillscited by 120
  23. 2021APS: Active Pretraining with Successor Featurescited by 109
  24. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 107
  25. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 107
  26. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
  27. 2021Demonstration-Guided Reinforcement Learning with Learned Skillscited by 73
  28. 2022CIC: Contrastive Intrinsic Control for Unsupervised Skill Discoverycited by 66
  29. 2022Lipschitz-constrained Unsupervised Skill Discoverycited by 60
  30. 2021Hierarchical Reinforcement Learning By Discovering Intrinsic Optionscited by 57
  31. 2022Skill-based Model-based Reinforcement Learningcited by 50
  32. 2023Controllability-Aware Unsupervised Skill Discoverycited by 49
  33. 2021TRAIL: Near-Optimal Imitation Learning with Suboptimal Datacited by 39
  34. 2022Mask-based Latent Reconstruction for Reinforcement Learningcited by 39
  35. 2021Hierarchical Few-Shot Imitation with Skill Transition Modelscited by 37
  36. 2021Learning more skills through optimistic explorationcited by 37
  37. 2022Skill-based Meta-Reinforcement Learningcited by 36
  38. 2020Masked Contrastive Representation Learning for Reinforcement Learningcited by 32

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 2.

8 papers.

  1. 2024SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learningcited by 70
  2. 2024Do's and Don'ts: Learning Desirable Skills with Instruction Videoscited by 8
  3. 2025The impact of intrinsic rewards on exploration in Reinforcement Learningcited by 8
  4. 2024Dr. Strategy: Model-Based Generalist Agents with Strategic Dreamingcited by 6
  5. 2025Efficient Skill Discovery via Regret-Aware Optimizationcited by 3
  6. 2025Periodic Skill Discoverycited by 1
  7. 2026SUSD: Structured Unsupervised Skill Discovery through State Factorizationcited by 0
  8. 2026Offline Multi-agent Continual Cooperation via Skill Partition and Reusecited by 0