mlsanityData to 7 Oct 2026

Diversity is All You Need: Learning Skills without a Reward Function

arXiv 1802.06070 ↗First posted 16 Feb 2018cs.AI cs.ROcited by 734

Not on the map. Why?

Cited by 58 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Intelligent creatures can explore their environments and learn useful skills without supervision. In this paper, we propose DIAYN ('Diversity is All You Need'), a method for learning useful skills without a reward function. Our proposed method learns skills by maximizing an information theoretic objective using a maximum entropy policy. On a variety of simulated robotic tasks, we show that this simple objective results in the unsupervised emergence of diverse skills, such as walking and jumping. In a number of reinforcement learning benchmark environments, our method is able to learn a skill that solves the benchmark task despite never receiving the true task reward. We show how pretrained skills can provide a good parameter initialization for downstream tasks, and can be composed hierarchically to solve complex, sparse reward tasks. Our results suggest that unsupervised discovery of skills can serve as an effective pretraining mechanism for overcoming challenges of exploration and data efficiency in reinforcement learning.

Key facts. Cited by 734 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 2; 2019: 1; 2020: 2; 2021: 105; 2022: 147; 2023: 140; 2024: 109; 2025: 120; 2026: 108.

734 papers.

Sort:

By area:

  1. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  2. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  3. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 386
  4. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  5. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 298
  6. 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 294
  7. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  8. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  9. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 164
  10. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  11. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 153
  12. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
38 more
  1. 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 140
  2. 2021Collaborating with Humans without Human Datacited by 139
  3. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
  4. 2021Discovering and Achieving Goals via World Modelscited by 131
  5. 2022Learning GFlowNets from partial episodes for improved convergence and stabilitycited by 129
  6. 2022Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behaviorcited by 113
  7. 2021APS: Active Pretraining with Successor Featurescited by 109
  8. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 109
  9. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
  10. 2023Policy Expansion for Bridging Offline-to-Online Reinforcement Learningcited by 103
  11. 2025Maximizing Confidence Alone Improves Reasoningcited by 97
  12. 2021Celebrating Diversity in Shared Multi-Agent Reinforcement Learningcited by 96
  13. 2022Deep Hierarchical Planning from Pixelscited by 96
  14. 2022Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learningcited by 93
  15. 2024Foundation Policies with Hilbert Representationscited by 89
  16. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  17. 2023Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidancecited by 85
  18. 2021Reward is enough for convex MDPscited by 82
  19. 2021Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulationcited by 82
  20. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 80
  21. 2022Does Zero-Shot Reinforcement Learning Exist?cited by 79
  22. 2021Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Taskscited by 75
  23. 2023XSkill: Cross Embodiment Skill Discoverycited by 74
  24. 2021Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordinationcited by 69
  25. 2023SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Executioncited by 69
  26. 2022CIC: Contrastive Intrinsic Control for Unsupervised Skill Discoverycited by 66
  27. 2022Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigationcited by 66
  28. 2023OMNI: Open-endedness via Models of human Notions of Interestingnesscited by 66
  29. 2021Accelerating Robotic Reinforcement Learning via Parameterized Action Primitivescited by 63
  30. 2023LOTUS: Continual Imitation Learning for Robot Manipulation Through Unsupervised Skill Discoverycited by 63
  31. 2022Lipschitz-constrained Unsupervised Skill Discoverycited by 60
  32. 2022Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learningcited by 58
  33. 2021Hierarchical Reinforcement Learning By Discovering Intrinsic Optionscited by 57
  34. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  35. 2021No One Representation to Rule Them All: Overlapping Features of Training Methodscited by 57
  36. 2022Winner Takes It All: Training Performant RL Populations for Combinatorial Optimizationcited by 57
  37. 2023A Survey of Temporal Credit Assignment in Deep Reinforcement Learningcited by 56
  38. 2022Diversify and Disambiguate: Learning From Underspecified Datacited by 55

50 most cited of 734 All 734 (JSON)