Latent-Predictive Empowerment: Measuring Empowerment without a Simulator
Not on the map. Why?
Abstract (arXiv)
Empowerment has the potential to help agents learn large skillsets, but is not yet a scalable solution for training general-purpose agents. Recent empowerment methods learn diverse skillsets by maximizing the mutual information between skills and states; however, these approaches require a model of the transition dynamics, which can be challenging to learn in realistic settings with high-dimensional and stochastic observations. We present Latent-Predictive Empowerment (LPE), an algorithm that can compute empowerment in a more practical manner. LPE learns large skillsets by maximizing an objective that is a principled replacement for the mutual information between skills and states and that only requires a simpler latent-predictive model rather than a full simulator of the environment. We show empirically in a variety of settings--including ones with high-dimensional observations and highly stochastic transition dynamics--that our empowerment objective (i) learns similar-sized skillsets as the leading empowerment algorithm that assumes access to a model of the transition dynamics and (ii) outperforms other model-based approaches to empowerment.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 36 of its 51 references that are in the map. First posted on arXiv 15 Oct 2024.
Builds on
36 of 51 references.
References · 36
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
- 2018World Modelscited by 1,513
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
- 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
- 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
- 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
- 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
- 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
- 2016Variational Intrinsic Controlcited by 267
- 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
- 2018Visual Reinforcement Learning with Imagined Goalscited by 251
- 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 212
24 more
- 2019Efficient Exploration via State Marginal Matchingcited by 188
- 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 174
- 2015Variational Information Maximisation for Intrinsically Motivated Reinforcement Learningcited by 144
- 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 133
- 2021Discovering and Achieving Goals via World Modelscited by 131
- 2018Variational Option Discovery Algorithmscited by 122
- 2020Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skillscited by 122
- 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 102
- 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 86
- 2022BYOL-Explore: Exploration by Bootstrapped Predictioncited by 73
- 2013Empowerment -- an Introductioncited by 68
- 2019Shaping Belief States with Generative Environment Models for RLcited by 66
- 2022Lipschitz-constrained Unsupervised Skill Discoverycited by 62
- 2023Controllability-Aware Unsupervised Skill Discoverycited by 49
- 2019Hierarchical Reinforcement Learning with Advantage-Based Auxiliary Rewardscited by 40
- 2021Learning more skills through optimistic explorationcited by 38
- 2022Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objectivecited by 28
- 2020AvE: Assistance via Empowermentcited by 27
- 2022INFOrmation Prioritization through EmPOWERment in Visual Model-Based RLcited by 27
- 2017Unsupervised Real-Time Control through Variational Empowermentcited by 18
- 2021Braxlines: Fast and Interactive Toolkit for RL-driven Behavior Engineering beyond Reward Maximizationcited by 11
- 2023Hierarchical Empowerment: Towards Tractable Empowerment-Based Skill Learningcited by 7
- 2015Efficient Empowermentcited by 4
- 2021HAC Explore: Accelerating Exploration with Hierarchical Reinforcement Learningcited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.