# Unsupervised Skill Discovery with Bottleneck Option Learning

arXiv: https://arxiv.org/abs/2106.14305 · HTML: https://mlsanity.com/paper/2106.14305/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 46 references that are in the map.
- First posted on arXiv 27 Jun 2021.

## Builds on (most cited first, 30 of 33)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Diversity is All You Need: Learning Skills without a Reward Function](https://mlsanity.com/paper/1802.06070/) (2018)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Data-Efficient Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1805.08296/) (2018)
- [Adversarial Autoencoders](https://mlsanity.com/paper/1511.05644/) (2015)
- [FeUdal Networks for Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1703.01161/) (2017)
- [Dynamics-Aware Unsupervised Discovery of Skills](https://mlsanity.com/paper/1907.01657/) (2019)
- [Challenges of Real-World Reinforcement Learning](https://mlsanity.com/paper/1904.12901/) (2019)
- [Variational Intrinsic Control](https://mlsanity.com/paper/1611.07507/) (2016)
- [Inverse Reward Design](https://mlsanity.com/paper/1711.02827/) (2017)
- [Emergence of Invariance and Disentanglement in Deep Representations](https://mlsanity.com/paper/1706.01350/) (2017)
- [OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learning](https://mlsanity.com/paper/2010.13611/) (2020)
- [Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play](https://mlsanity.com/paper/1703.05407/) (2017)
- [Information Dropout: Learning Optimal Representations Through Noisy Computation](https://mlsanity.com/paper/1611.01353/) (2016)
- [Stochastic Neural Networks for Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1704.03012/) (2017)
- [Variational Option Discovery Algorithms](https://mlsanity.com/paper/1807.10299/) (2018)
- [Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skills](https://mlsanity.com/paper/2002.03647/) (2020)
- [Near-Optimal Representation Learning for Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1810.01257/) (2018)
- [Model-Based Active Exploration](https://mlsanity.com/paper/1810.12162/) (2018)
- [Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation](https://mlsanity.com/paper/1709.10489/) (2017)
- [ROBEL: Robotics Benchmarks for Learning with Low-Cost Robots](https://mlsanity.com/paper/1909.11639/) (2019)
- [Hierarchical Reinforcement Learning By Discovering Intrinsic Options](https://mlsanity.com/paper/2101.06521/) (2021)

## Built on by (most cited first, 28 of 28)

- [METRA: Scalable Unsupervised RL with Metric-Aware Abstraction](https://mlsanity.com/paper/2310.08887/) (2023)
- [Lipschitz-constrained Unsupervised Skill Discovery](https://mlsanity.com/paper/2202.00914/) (2022)
- [Controllability-Aware Unsupervised Skill Discovery](https://mlsanity.com/paper/2302.05103/) (2023)
- [Choreographer: Learning and Adapting Skills in Imagination](https://mlsanity.com/paper/2211.13350/) (2022)
- [Deep Laplacian-based Options for Temporally-Extended Exploration](https://mlsanity.com/paper/2301.11181/) (2023)
- [An information-theoretic perspective on intrinsic motivation in reinforcement learning: a survey](https://mlsanity.com/paper/2209.08890/) (2022)
- [Braxlines: Fast and Interactive Toolkit for RL-driven Behavior Engineering beyond Reward Maximization](https://mlsanity.com/paper/2110.04686/) (2021)
- [PEAC: Unsupervised Pre-training for Cross-Embodiment Reinforcement Learning](https://mlsanity.com/paper/2405.14073/) (2024)
- [Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2410.11251/) (2024)
- [SkiLD: Unsupervised Skill Discovery Guided by Factor Interactions](https://mlsanity.com/paper/2410.18416/) (2024)
- [Predictable MDP Abstraction for Unsupervised Model-Based RL](https://mlsanity.com/paper/2302.03921/) (2023)
- [Hierarchical Empowerment: Towards Tractable Empowerment-Based Skill Learning](https://mlsanity.com/paper/2307.02728/) (2023)
- [Controlled Diversity with Preference : Towards Learning a Diverse Set of Desired Skills](https://mlsanity.com/paper/2303.04592/) (2023)
- [Granger Causal Interaction Skill Chains](https://mlsanity.com/paper/2306.09509/) (2023)
- [Flexible Attention-Based Multi-Policy Fusion for Efficient Deep Reinforcement Learning](https://mlsanity.com/paper/2210.03729/) (2022)
- [Task Adaptation from Skills: Information Geometry, Disentanglement, and New Objectives for Unsupervised Reinforcement Learning](https://mlsanity.com/paper/2506.10629/) (2025)
- [Offline Diversity Maximization Under Imitation Constraints](https://mlsanity.com/paper/2307.11373/) (2023)
- [Diversify & Conquer: Outcome-directed Curriculum RL via Out-of-Distribution Disagreement](https://mlsanity.com/paper/2310.19261/) (2023)
- [Beyond Rewards: a Hierarchical Perspective on Offline Multiagent Behavioral Analysis](https://mlsanity.com/paper/2206.09046/) (2022)
- [Efficient Skill Discovery via Regret-Aware Optimization](https://mlsanity.com/paper/2506.21044/) (2025)
- [Dual-Force: Enhanced Offline Diversity Maximization under Imitation Constraints](https://mlsanity.com/paper/2501.04426/) (2025)
- [Behavioral Mode Discovery for Fine-tuning Multimodal Generative Policies](https://mlsanity.com/paper/2605.11387/) (2026)
- [A Survey on Deep Reinforcement Learning-based Approaches for Adaptation and Generalization](https://mlsanity.com/paper/2202.08444/) (2022)
- [Periodic Skill Discovery](https://mlsanity.com/paper/2511.03187/) (2025)
- [Learning Generalizable Skill Policy with Data-Efficient Unsupervised RL](https://mlsanity.com/paper/2607.00392/) (2026)
- [Unsupervised Discovery of Continuous Skills on a Sphere](https://mlsanity.com/paper/2305.14377/) (2023)
- [Unsupervised Data Generation for Offline Reinforcement Learning: A Perspective from Model](https://mlsanity.com/paper/2506.19643/) (2025)
- [Focused Skill Discovery: Learning to Control Specific State Variables while Minimizing Side Effects](https://mlsanity.com/paper/2510.04901/) (2025)

## Abstract (arXiv)

Having the ability to acquire inherent skills from environments without any external rewards or supervision like humans is an important problem. We propose a novel unsupervised skill discovery method named Information Bottleneck Option Learning (IBOL). On top of the linearization of environments that promotes more various and distant state transitions, IBOL enables the discovery of diverse skills. It provides the abstraction of the skills learned with the information bottleneck framework for the options with improved stability and encouraged disentanglement. We empirically demonstrate that IBOL outperforms multiple state-of-the-art unsupervised skill discovery methods on the information-theoretic evaluations and downstream tasks in MuJoCo environments, including Ant, HalfCheetah, Hopper and D'Kitty.