mlsanityData to 14 Oct 2026

A Unified Algorithm Framework for Unsupervised Discovery of Skills based on Determinantal Point Process

arXiv 2212.00211 ↗First posted 1 Dec 2022cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Learning rich skills under the option framework without supervision of external rewards is at the frontier of reinforcement learning research. Existing works mainly fall into two distinctive categories: variational option discovery that maximizes the diversity of the options through a mutual information loss (while ignoring coverage) and Laplacian-based methods that focus on improving the coverage of options by increasing connectivity of the state space (while ignoring diversity). In this paper, we show that diversity and coverage in unsupervised option discovery can indeed be unified under the same mathematical framework. To be specific, we explicitly quantify the diversity and coverage of the learned options through a novel use of Determinantal Point Process (DPP) and optimize these objectives to discover options with both superior diversity and coverage. Our proposed algorithm, ODPP, has undergone extensive evaluation on challenging tasks created with Mujoco and Atari. The results demonstrate that our algorithm outperforms state-of-the-art baselines in both diversity- and coverage-driven categories.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 33 of its 61 references that are in the map. First posted on arXiv 1 Dec 2022.

Builds on

33 of 61 references.

References · 33

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2013Auto-Encoding Variational Bayescited by 4,053
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  6. 2016Deep Variational Information Bottleneckcited by 930
  7. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  8. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
  9. 2016Variational Intrinsic Controlcited by 267
  10. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 230
  11. 2019Efficient Exploration via State Marginal Matchingcited by 188
  12. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 166
21 more
  1. 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 133
  2. 2018Variational Option Discovery Algorithmscited by 122
  3. 2020Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skillscited by 122
  4. 2020Effective Diversity in Population Based Reinforcement Learningcited by 119
  5. 2021APS: Active Pretraining with Successor Featurescited by 112
  6. 2019Diverse Trajectory Forecasting with Determinantal Point Processescited by 80
  7. 2019DeepPool: Distributed Model-free Algorithm for Ride-sharing using Deep Reinforcement Learningcited by 54
  8. 2019DAC: The Double Actor-Critic Architecture for Learning Optionscited by 43
  9. 2021Modelling Behavioural Diversity for Learning in Open-Ended Gamescited by 39
  10. 2021Learning more skills through optimistic explorationcited by 38
  11. 2020Multi-Agent Determinantal Q-Learningcited by 36
  12. 2020Relative Variational Intrinsic Controlcited by 34
  13. 2019Discovering Options for Exploration by Minimizing Cover Timecited by 31
  14. 2018GDPP: Learning Diverse Generations Using Determinantal Point Processcited by 26
  15. 2021Towards Better Laplacian Representation in Reinforcement Learning with Generalized Graph Drawingcited by 23
  16. 2016Playing Atari Games with Deep Reinforcement Learning and Human Checkpoint Replaycited by 22
  17. 2021Adversarial Option-Aware Hierarchical Imitation Learningcited by 19
  18. 2021Direct then Diffuse: Incremental Unsupervised Skill Discovery for State Covering and Goal Reachingcited by 19
  19. 2022Option-Aware Adversarial Inverse Reinforcement Learning for Robotic Controlcited by 16
  20. 2020Variational Intrinsic Control Revisitedcited by 11
  21. 2022Learning Multi-agent Skills for Tabular Reinforcement Learning using Factor Graphscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 2.

7 papers.

  1. 2024Bayes Adaptive Monte Carlo Tree Search for Offline Model-based Reinforcement Learningcited by 11
  2. 2024Variational Offline Multi-agent Skill Discoverycited by 6
  3. 2026Continual Policy Consolidation for Lifelong Robot Learningcited by 3
  4. 2025Policy-Driven World Model Adaptation for Robust Offline Model-based Reinforcement Learningcited by 2
  5. 2025VendiRL: A Framework for Self-Supervised Reinforcement Learning of Diversely Diverse Skillscited by 2
  6. 2025Diverse Mini-Batch Selection in Reinforcement Learning for Efficient Chemical Exploration in de novo Drug Designcited by 1
  7. 2026Offline Discovery of Interpretable Skills from Multi-Task Trajectoriescited by 1