mlsanityData to 14 Oct 2026

Unsupervised Skill Discovery with Bottleneck Option Learning

arXiv 2106.14305 ↗First posted 27 Jun 2021cs.LG cs.AI cs.ROcited by 28

Not on the map. Why?

Abstract (arXiv)

Having the ability to acquire inherent skills from environments without any external rewards or supervision like humans is an important problem. We propose a novel unsupervised skill discovery method named Information Bottleneck Option Learning (IBOL). On top of the linearization of environments that promotes more various and distant state transitions, IBOL enables the discovery of diverse skills. It provides the abstraction of the skills learned with the information bottleneck framework for the options with improved stability and encouraged disentanglement. We empirically demonstrate that IBOL outperforms multiple state-of-the-art unsupervised skill discovery methods on the information-theoretic evaluations and downstream tasks in MuJoCo environments, including Ant, HalfCheetah, Hopper and D'Kitty.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 33 of its 46 references that are in the map. First posted on arXiv 27 Jun 2021.

Builds on

33 of 46 references.

References · 33

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2013Auto-Encoding Variational Bayescited by 4,053
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  6. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  7. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  8. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 941
  9. 2016Deep Variational Information Bottleneckcited by 930
  10. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  11. 2018Disentangling by Factorisingcited by 616
  12. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
21 more
  1. 2015Adversarial Autoencoderscited by 364
  2. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
  3. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
  4. 2019Challenges of Real-World Reinforcement Learningcited by 291
  5. 2016Variational Intrinsic Controlcited by 267
  6. 2017Inverse Reward Designcited by 249
  7. 2017Emergence of Invariance and Disentanglement in Deep Representationscited by 215
  8. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 166
  9. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 141
  10. 2016Information Dropout: Learning Optimal Representations Through Noisy Computationcited by 132
  11. 2017Stochastic Neural Networks for Hierarchical Reinforcement Learningcited by 124
  12. 2018Variational Option Discovery Algorithmscited by 122
  13. 2020Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skillscited by 122
  14. 2018Near-Optimal Representation Learning for Hierarchical Reinforcement Learningcited by 110
  15. 2018Model-Based Active Explorationcited by 105
  16. 2017Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigationcited by 77
  17. 2019ROBEL: Robotics Benchmarks for Learning with Low-Cost Robotscited by 72
  18. 2021Hierarchical Reinforcement Learning By Discovering Intrinsic Optionscited by 58
  19. 2019Theory and Evaluation Metrics for Learning Disentangled Representationscited by 57
  20. 2018Self-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddingscited by 52
  21. 2020Emergent Real-World Robotic Skills via Unsupervised Off-Policy Reinforcement Learningcited by 31

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 6; 2023: 10; 2024: 3; 2025: 6; 2026: 2.

28 papers.

Sort:

By area:

  1. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 86
  2. 2022Lipschitz-constrained Unsupervised Skill Discoverycited by 62
  3. 2023Controllability-Aware Unsupervised Skill Discoverycited by 49
  4. 2022Choreographer: Learning and Adapting Skills in Imaginationcited by 31
  5. 2023Deep Laplacian-based Options for Temporally-Extended Explorationcited by 29
  6. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 28
  7. 2021Braxlines: Fast and Interactive Toolkit for RL-driven Behavior Engineering beyond Reward Maximizationcited by 11
  8. 2024PEAC: Unsupervised Pre-training for Cross-Embodiment Reinforcement Learningcited by 9
  9. 2024Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learningcited by 9
  10. 2024SkiLD: Unsupervised Skill Discovery Guided by Factor Interactionscited by 9
  11. 2023Predictable MDP Abstraction for Unsupervised Model-Based RLcited by 8
  12. 2023Hierarchical Empowerment: Towards Tractable Empowerment-Based Skill Learningcited by 7
16 more
  1. 2023Controlled Diversity with Preference : Towards Learning a Diverse Set of Desired Skillscited by 6
  2. 2023Granger Causal Interaction Skill Chainscited by 6
  3. 2022Flexible Attention-Based Multi-Policy Fusion for Efficient Deep Reinforcement Learningcited by 5
  4. 2025Task Adaptation from Skills: Information Geometry, Disentanglement, and New Objectives for Unsupervised Reinforcement Learningcited by 5
  5. 2023Offline Diversity Maximization Under Imitation Constraintscited by 4
  6. 2023Diversify & Conquer: Outcome-directed Curriculum RL via Out-of-Distribution Disagreementcited by 4
  7. 2022Beyond Rewards: a Hierarchical Perspective on Offline Multiagent Behavioral Analysiscited by 3
  8. 2025Efficient Skill Discovery via Regret-Aware Optimizationcited by 3
  9. 2025Dual-Force: Enhanced Offline Diversity Maximization under Imitation Constraintscited by 2
  10. 2026Behavioral Mode Discovery for Fine-tuning Multimodal Generative Policiescited by 2
  11. 2022A Survey on Deep Reinforcement Learning-based Approaches for Adaptation and Generalizationcited by 1
  12. 2025Periodic Skill Discoverycited by 1
  13. 2026Learning Generalizable Skill Policy with Data-Efficient Unsupervised RLcited by 1
  14. 2023Unsupervised Discovery of Continuous Skills on a Spherecited by 0
  15. 2025Unsupervised Data Generation for Offline Reinforcement Learning: A Perspective from Modelcited by 0
  16. 2025Focused Skill Discovery: Learning to Control Specific State Variables while Minimizing Side Effectscited by 0