# Intrinsic Motivation in Dynamical Control Systems

arXiv: https://arxiv.org/abs/2301.00005 · HTML: https://mlsanity.com/paper/2301.00005/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 44 references that are in the map.
- First posted on arXiv 29 Dec 2022.

## Builds on (most cited first, 16 of 16)

- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Diversity is All You Need: Learning Skills without a Reward Function](https://mlsanity.com/paper/1802.06070/) (2018)
- [Dynamics-Aware Unsupervised Discovery of Skills](https://mlsanity.com/paper/1907.01657/) (2019)
- [Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation](https://mlsanity.com/paper/1604.06057/) (2016)
- [Large-Scale Study of Curiosity-Driven Learning](https://mlsanity.com/paper/1808.04355/) (2018)
- [Variational Intrinsic Control](https://mlsanity.com/paper/1611.07507/) (2016)
- [Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning](https://mlsanity.com/paper/1509.08731/) (2015)
- [Variational Option Discovery Algorithms](https://mlsanity.com/paper/1807.10299/) (2018)
- [Empowerment -- an Introduction](https://mlsanity.com/paper/1310.1863/) (2013)
- [Relative Variational Intrinsic Control](https://mlsanity.com/paper/2012.07827/) (2020)
- [Emergent Real-World Robotic Skills via Unsupervised Off-Policy Reinforcement Learning](https://mlsanity.com/paper/2004.12974/) (2020)
- [AvE: Assistance via Empowerment](https://mlsanity.com/paper/2006.14796/) (2020)
- [Unsupervised Real-Time Control through Variational Empowerment](https://mlsanity.com/paper/1710.05101/) (2017)
- [Variational Intrinsic Control Revisited](https://mlsanity.com/paper/2010.03281/) (2020)
- [Efficient Empowerment Estimation for Unsupervised Stabilization](https://mlsanity.com/paper/2007.07356/) (2020)
- [Intrinsically motivated collective motion](https://mlsanity.com/paper/1907.08097/) (2019)

## Built on by (most cited first, 8 of 8)

- [Deep Reinforcement Learning in Applied Control: Challenges, Analysis, and Insights](https://mlsanity.com/paper/2507.08196/) (2025)
- [Learning Task-Sufficient World Models by Synergizing Agentic Exploration and Structured Modeling](https://mlsanity.com/paper/2607.04409/) (2026)
- [SuPLE: Robot Learning with Lyapunov Rewards](https://mlsanity.com/paper/2411.13613/) (2024)
- [Decentralized Traffic Flow Optimization Through Intrinsic Motivation](https://mlsanity.com/paper/2505.11520/) (2025)
- [Emergence of Physical Intelligence via Controllable Information Production](https://mlsanity.com/paper/2601.22449/) (2026)
- [Goals and the Structure of Experience](https://mlsanity.com/paper/2508.15013/) (2025)
- [Multi-Agent Empowerment and Emergence of Complex Behavior in Groups](https://mlsanity.com/paper/2604.21155/) (2026)
- [Bellman Meets Lyapunov: Unsupervised Reinforcement Learning via Mastering Chaos](https://mlsanity.com/paper/2610.02012/) (2026)

## Abstract (arXiv)

Biological systems often choose actions without an explicit reward signal, a phenomenon known as intrinsic motivation. The computational principles underlying this behavior remain poorly understood. In this study, we investigate an information-theoretic approach to intrinsic motivation, based on maximizing an agent's empowerment (the mutual information between its past actions and future states). We show that this approach generalizes previous attempts to formalize intrinsic motivation, and we provide a computationally efficient algorithm for computing the necessary quantities. We test our approach on several benchmark control problems, and we explain its success in guiding intrinsically motivated behaviors by relating our information-theoretic control function to fundamental properties of the dynamical system representing the combined agent-environment system. This opens the door for designing practical artificial, intrinsically motivated controllers and for linking animal behaviors to their dynamical properties.