mlsanityData to 14 Oct 2026

Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning

arXiv 1509.08731 ↗First posted 29 Sep 2015stat.ML cs.AI cs.LGcited by 144

Not on the map. Why?

Cited by 36 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The mutual information is a core statistical quantity that has applications in all areas of machine learning, whether this is in training of density models over multiple data modalities, in maximising the efficiency of noisy transmission channels, or when learning behaviour policies for exploration by artificial agents. Most learning algorithms that involve optimisation of the mutual information rely on the Blahut-Arimoto algorithm --- an enumerative algorithm with exponential complexity that is not suitable for modern machine learning applications. This paper provides a new approach for scalable optimisation of the mutual information by merging techniques from variational inference and deep learning. We develop our approach by focusing on the problem of intrinsically-motivated learning, where the mutual information forms the definition of a well-known internal drive known as empowerment. Using a variational lower bound on the mutual information, combined with convolutional networks for handling visual input streams, we develop a stochastic optimisation algorithm that allows for scalable information maximisation and empowerment-based reasoning directly from pixels to actions.

Key facts. Cited by 144 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Sep 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 2; 2019: 0; 2020: 0; 2021: 31; 2022: 27; 2023: 23; 2024: 19; 2025: 20; 2026: 22.

144 papers.

Sort:

By area:

  1. 2018Reward learning from human preferences and demonstrations in Ataricited by 316
  2. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 174
  3. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 166
  4. 2021APS: Active Pretraining with Successor Featurescited by 112
  5. 2021Invariant Information Bottleneck for Domain Generalizationcited by 86
  6. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 86
  7. 2021Causal Influence Detection for Improving Efficiency in Reinforcement Learningcited by 71
  8. 2022Lipschitz-constrained Unsupervised Skill Discoverycited by 62
  9. 2021Episodic Multi-agent Reinforcement Learning with Curiosity-Driven Explorationcited by 49
  10. 2022Guaranteed Discovery of Control-Endogenous Latent States with Multi-Step Inverse Modelscited by 45
  11. 2021MADE: Exploration via Maximizing Deviation from Explored Regionscited by 40
  12. 2021The Information Geometry of Unsupervised Reinforcement Learningcited by 37
38 more
  1. 2018Adversarial Imitation via Variational Inverse Reinforcement Learningcited by 36
  2. 2022Curious Exploration via Structured World Models Yields Zero-Shot Object Manipulationcited by 31
  3. 2021Which Mutual-Information Representation Learning Objectives are Sufficient for Control?cited by 29
  4. 2021How to Stay Curious while Avoiding Noisy TVs using Aleatoric Uncertainty Estimationcited by 27
  5. 2022INFOrmation Prioritization through EmPOWERment in Visual Model-Based RLcited by 27
  6. 2021Task-Agnostic Morphology Evolutioncited by 24
  7. 2022Any-Play: An Intrinsic Augmentation for Zero-Shot Coordinationcited by 23
  8. 2023Learning How to Propagate Messages in Graph Neural Networkscited by 23
  9. 2021Beyond Fine-Tuning: Transferring Behavior in Reinforcement Learningcited by 21
  10. 2022Versatile Skill Control via Self-supervised Adversarial Imitation of Unlabeled Mixed Motionscited by 21
  11. 2024Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learningcited by 21
  12. 2023Maximize to Explore: One Objective Function Fusing Estimation, Planning, and Explorationcited by 20
  13. 2021Direct then Diffuse: Incremental Unsupervised Skill Discovery for State Covering and Goal Reachingcited by 19
  14. 2023Theory of Mind as Intrinsic Motivation for Multi-Agent Reinforcement Learningcited by 19
  15. 2021Mutual Information State Intrinsic Controlcited by 18
  16. 2021Active Hierarchical Exploration with Stable Subgoal Representation Learningcited by 17
  17. 2021Co-GAIL: Learning Diverse Strategies for Human-Robot Collaborationcited by 17
  18. 2023Curious Replay for Model-based Adaptationcited by 16
  19. 2021Policy Information Capacity: Information-Theoretic Measure for Task Complexity in Deep Reinforcement Learningcited by 15
  20. 2021Variational Empowerment as Representation Learning for Goal-Based Reinforcement Learningcited by 15
  21. 2021Is Curiosity All You Need? On the Utility of Emergent Behaviours from Curious Explorationcited by 14
  22. 2022Representation Learning in Deep RL via Discrete Information Bottleneckcited by 13
  23. 2024Learning to Assist Humans without Inferring Rewardscited by 13
  24. 2021Exploration and preference satisfaction trade-off in reward-free learningcited by 12
  25. 2021Backprop-Free Reinforcement Learning with Active Neural Generative Codingcited by 12
  26. 2022Learning General World Models in a Handful of Reward-Free Deploymentscited by 12
  27. 2024Potential-Based Reward Shaping For Intrinsic Motivationcited by 12
  28. 2025Plasticity as the Mirror of Empowermentcited by 12
  29. 2026InfoPO: Information-Driven Policy Optimization for User-Centric Agentscited by 12
  30. 2021Braxlines: Fast and Interactive Toolkit for RL-driven Behavior Engineering beyond Reward Maximizationcited by 11
  31. 2021Information is Power: Intrinsic Control via Information Capturecited by 11
  32. 2026Why AI systems don't learn and what to do about it: Lessons on autonomous learning from cognitive sciencecited by 11
  33. 2021Understanding the Origin of Information-Seeking Exploration in Probabilistic Objectives for Controlcited by 10
  34. 2022DGPO: Discovering Multiple Strategies with Diversity-Guided Policy Optimizationcited by 10
  35. 2022Efficient Exploration in Resource-Restricted Reinforcement Learningcited by 10
  36. 2023Situation-Dependent Causal Influence-Based Cooperative Multi-agent Reinforcement Learningcited by 10
  37. 2025On Rollouts in Model-Based Reinforcement Learningcited by 10
  38. 2022A Mixture of Surprises for Unsupervised Reinforcement Learningcited by 9

50 most cited of 144 All 144 (JSON)