# Unsupervised Real-Time Control through Variational Empowerment

arXiv: https://arxiv.org/abs/1710.05101 · HTML: https://mlsanity.com/paper/1710.05101/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2017.

## Built on by (most cited first, 18 of 18)

- [Emergent Bartering Behaviour in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2205.06760/) (2022)
- [Variational Empowerment as Representation Learning for Goal-Based Reinforcement Learning](https://mlsanity.com/paper/2106.01404/) (2021)
- [Mind the Gap when Conditioning Amortised Inference in Sequential Latent-Variable Models](https://mlsanity.com/paper/2101.07046/) (2021)
- [Information is Power: Intrinsic Control via Information Capture](https://mlsanity.com/paper/2112.03899/) (2021)
- [Understanding the Origin of Information-Seeking Exploration in Probabilistic Objectives for Control](https://mlsanity.com/paper/2103.06859/) (2021)
- [Intrinsic Motivation in Dynamical Control Systems](https://mlsanity.com/paper/2301.00005/) (2022)
- [Information-Theoretic Policy Pre-Training with Empowerment](https://mlsanity.com/paper/2510.05996/) (2025)
- [Latent Matters: Learning Deep State-Space Models](https://mlsanity.com/paper/2602.23050/) (2026)
- [Temporal Representations for Exploration: Learning Complex Exploratory Behavior without Extrinsic Rewards](https://mlsanity.com/paper/2603.02008/) (2026)
- [Filter-Aware Model-Predictive Control](https://mlsanity.com/paper/2304.10246/) (2023)
- [Latent-Predictive Empowerment: Measuring Empowerment without a Simulator](https://mlsanity.com/paper/2410.11155/) (2024)
- [Emergence of Physical Intelligence via Controllable Information Production](https://mlsanity.com/paper/2601.22449/) (2026)
- [Reliably Re-Acting to Partner's Actions with the Social Intrinsic Motivation of Transfer Empowerment](https://mlsanity.com/paper/2203.03355/) (2022)
- [Ball-and-socket joint pose estimation using magnetic field](https://mlsanity.com/paper/2210.03984/) (2022)
- [PRISM: Probabilistic Real-Time Inference in Spatial World Models](https://mlsanity.com/paper/2212.02988/) (2022)
- [Generative Intrinsic Optimization: Intrinsic Control with Model Learning](https://mlsanity.com/paper/2310.08100/) (2023)
- [Bellman Meets Lyapunov: Unsupervised Reinforcement Learning via Mastering Chaos](https://mlsanity.com/paper/2610.02012/) (2026)
- [Scaling Empowered Agents](https://mlsanity.com/paper/2610.13002/) (2026)

## Abstract (arXiv)

We introduce a methodology for efficiently computing a lower bound to empowerment, allowing it to be used as an unsupervised cost function for policy learning in real-time control. Empowerment, being the channel capacity between actions and states, maximises the influence of an agent on its near future. It has been shown to be a good model of biological behaviour in the absence of an extrinsic goal. But empowerment is also prohibitively hard to compute, especially in nonlinear continuous spaces. We introduce an efficient, amortised method for learning empowerment-maximising policies. We demonstrate that our algorithm can reliably handle continuous dynamical systems using system dynamics learned from raw data. The resulting policies consistently drive the agents into states where they can use their full potential.