Data to 7 Oct 2026

Dynamics-Aware Unsupervised Discovery of Skills

arXiv 1907.01657 ↗First posted 2 Jul 2019cs.LG cs.RO stat.MLcited by 308

Not on the map. Why?

Cited by 33 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Conventionally, model-based reinforcement learning (MBRL) aims to learn a global model for the dynamics of the environment. A good model can potentially enable planning algorithms to generate a large variety of behaviors and solve diverse tasks. However, learning an accurate model for complex dynamical systems is difficult, and even then, the model might not generalize well outside the distribution of states on which it was trained. In this work, we combine model-based learning with model-free learning of primitives that make model-based planning easy. To that end, we aim to answer the question: how can we discover skills whose outcomes are easy to predict? We propose an unsupervised learning algorithm, Dynamics-Aware Discovery of Skills (DADS), which simultaneously discovers predictable behaviors and learns their dynamics. Our method can leverage continuous skill spaces, theoretically, allowing us to learn infinitely many behaviors even for high-dimensional state-spaces. We demonstrate that zero-shot planning in the learned latent space significantly outperforms standard MBRL and model-free goal-conditioned RL, can handle sparse-reward tasks, and substantially improves over prior hierarchical RL methods for unsupervised skill discovery.

Key facts. Cited by 308 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 2; 2021: 56; 2022: 65; 2023: 64; 2024: 41; 2025: 45; 2026: 35.

308 papers.

Sort:

By area:

2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 298
2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 212
2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 164
2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
2021APS: Active Pretraining with Successor Featurescited by 109
2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 105
2021Celebrating Diversity in Shared Multi-Agent Reinforcement Learningcited by 96

38 more

2022Deep Hierarchical Planning from Pixelscited by 96
2024Foundation Policies with Hilbert Representationscited by 89
2023Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidancecited by 85
2021Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulationcited by 82
2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 80
2021Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Taskscited by 75
2023XSkill: Cross Embodiment Skill Discoverycited by 74
2021Demonstration-Guided Reinforcement Learning with Learned Skillscited by 73
2023SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Executioncited by 69
2022CIC: Contrastive Intrinsic Control for Unsupervised Skill Discoverycited by 66
2021Accelerating Robotic Reinforcement Learning via Parameterized Action Primitivescited by 63
2023LOTUS: Continual Imitation Learning for Robot Manipulation Through Unsupervised Skill Discoverycited by 63
2022Lipschitz-constrained Unsupervised Skill Discoverycited by 60
2021Hierarchical Reinforcement Learning By Discovering Intrinsic Optionscited by 57
2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
2022Diversify and Disambiguate: Learning From Underspecified Datacited by 55
2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Modelscited by 51
2022Skill-based Model-based Reinforcement Learningcited by 50
2024Proposer-Agent-Evaluator(PAE): Autonomous Skill Discovery For Foundation Model Internet Agentscited by 50
2023Controllability-Aware Unsupervised Skill Discoverycited by 49
2023PlayFusion: Skill Acquisition via Diffusion from Language-Annotated Playcited by 48
2023Language Reward Modulation for Pretraining Reinforcement Learningcited by 42
2023Combinatorial Optimization with Policy Adaptation using Latent Space Searchcited by 41
2024CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agentscited by 41
2021TRAIL: Near-Optimal Imitation Learning with Suboptimal Datacited by 39
2021Hierarchical Skills for Efficient Explorationcited by 37
2021The Information Geometry of Unsupervised Reinforcement Learningcited by 36
2022Skill-based Meta-Reinforcement Learningcited by 36
2021Discovering Diverse Multi-Agent Strategic Behavior via Reward Randomizationcited by 35
2022Discovering Policies with DOMiNO: Diversity Optimization Maintaining Near Optimalitycited by 35
2022Spectral Decomposition Representation for Reinforcement Learningcited by 35
2023Beyond Reward: Offline Preference-guided Policy Optimizationcited by 35
2024PolyNet: Learning Diverse Solution Strategies for Neural Combinatorial Optimizationcited by 35
2021Autonomous Reinforcement Learning: Formalism and Benchmarkingcited by 33
2022Choreographer: Learning and Adapting Skills in Imaginationcited by 31
2024Unsupervised Zero-Shot Reinforcement Learning via Functional Reward Encodingscited by 31
2024BUMBLE: Unifying Reasoning and Acting with Vision-Language Models for Building-wide Mobile Manipulationcited by 31
2022LISA: Learning Interpretable Skill Abstractions from Languagecited by 30

50 most cited of 308 All 308 (JSON)