mlsanityData to 14 Oct 2026

Information is Power: Intrinsic Control via Information Capture

arXiv 2112.03899 ↗First posted 7 Dec 2021cs.LG cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

Humans and animals explore their environment and acquire useful skills even in the absence of clear goals, exhibiting intrinsic motivation. The study of intrinsic motivation in artificial agents is concerned with the following question: what is a good general-purpose objective for an agent? We study this question in dynamic partially-observed environments, and argue that a compact and general learning objective is to minimize the entropy of the agent's state visitation estimated using a latent state-space model. This objective induces an agent to both gather information about its environment, corresponding to reducing uncertainty, and to gain control over its environment, corresponding to reducing the unpredictability of future world states. We instantiate this approach as a deep reinforcement learning agent equipped with a deep variational Bayes filter. We find that our agent learns to discover, represent, and exercise control of dynamic objects in a variety of partially-observed environments sensed with visual observations without extrinsic reward.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 44 of its 88 references that are in the map. First posted on arXiv 7 Dec 2021.

Builds on

44 of 88 references.

References · 44

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  4. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  5. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
  6. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  7. 2020Mastering Atari with Discrete World Modelscited by 1,007
  8. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 772
  9. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  10. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  11. 2020Planning to Explore via Self-Supervised World Modelscited by 368
  12. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
32 more
  1. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 309
  2. 2018Large-Scale Study of Curiosity-Driven Learningcited by 280
  3. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 274
  4. 2016Variational Intrinsic Controlcited by 267
  5. 2019Self-Supervised Exploration via Disagreementcited by 265
  6. 2018Visual Reinforcement Learning with Imagined Goalscited by 251
  7. 2018Provably Efficient Maximum Entropy Explorationcited by 242
  8. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 231
  9. 2016ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learningcited by 179
  10. 2015Deep Kalman Filterscited by 166
  11. 2016Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Datacited by 156
  12. 2015Incentivizing Exploration In Reinforcement Learning With Deep Predictive Modelscited by 154
  13. 2015Deep Spatial Autoencoders for Visuomotor Learningcited by 153
  14. 2015Variational Information Maximisation for Intrinsically Motivated Reinforcement Learningcited by 144
  15. 2018Model-Based Active Explorationcited by 105
  16. 2020Offline Reinforcement Learning from Images with Latent Space Modelscited by 95
  17. 2017Filtering Variational Objectivescited by 91
  18. 2017Surprise-Based Intrinsic Motivation for Deep Reinforcement Learningcited by 83
  19. 2020Deep active inference agents using Monte-Carlo methodscited by 72
  20. 2018Unsupervised Predictive Memory in a Goal-Directed Agentcited by 55
  21. 2017Deep Active Inferencecited by 44
  22. 2017EX2: Exploration with Exemplar Models for Deep Reinforcement Learningcited by 42
  23. 2020Continual Learning of Control Primitives: Skill Discovery via Reset-Gamescited by 29
  24. 2017Unsupervised Real-Time Control through Variational Empowermentcited by 18
  25. 2018SOLAR: Deep Structured Representations for Model-Based Reinforcement Learningcited by 14
  26. 2019World Discovery Modelscited by 10
  27. 2020Efficient Empowerment Estimation for Unsupervised Stabilizationcited by 9
  28. 2018Approximate Bayesian inference in spatial environmentscited by 8
  29. 2019Learning latent state representation for speeding up explorationcited by 8
  30. 2020Variational State-Space Models for Localisation and Dense 3D Mapping in 6 DoFcited by 6
  31. 2015Efficient Empowermentcited by 4
  32. 2019Beta DVBF: Learning State-Space Models for Control from High Dimensional Observationscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 3; 2025: 2; 2026: 1.

11 papers.

  1. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 86
  2. 2023On efficient computation in active inferencecited by 15
  3. 2022A Mixture of Surprises for Unsupervised Reinforcement Learningcited by 9
  4. 2023A Unified View on Solving Objective Mismatch in Model-Based Reinforcement Learningcited by 9
  5. 2023Predictable MDP Abstraction for Unsupervised Model-Based RLcited by 8
  6. 2024Surprise-Adaptive Intrinsic Motivation for Unsupervised Reinforcement Learningcited by 4
  7. 2024BAMDP Shaping: a Unified Framework for Intrinsic Motivation and Reward Shapingcited by 4
  8. 2025Beyond Noisy-TVs: Noise-Robust Exploration Via Learning Progress Monitoringcited by 2
  9. 2025Information-Theoretic Policy Pre-Training with Empowermentcited by 2
  10. 2026Temporal Representations for Exploration: Learning Complex Exploratory Behavior without Extrinsic Rewardscited by 2
  11. 2024When in Doubt, Think Slow: Iterative Reasoning with Latent Imaginationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Intrinsic Rewards for Exploration without Harm from Observational Noise: A Simulation Study Based on the Free Energy Principle
  2. 2025An Information-Geometric Approach to Artificial Curiosity
  3. 2025The Missing Reward: Active Inference in the Era of Experience
  4. 2025From Curiosity to Competence: How World Models Interact with the Dynamics of Exploration
  5. 2026Intrinsic Motivation in Reinforcement Learning: A Research Agenda for Adaptive Self-Organisation
  6. 2024ELEMENT: Episodic and Lifelong Exploration via Maximum Entropy
  7. 2026Emergence of Physical Intelligence via Controllable Information Production
  8. 2018Learning to Play with Intrinsically-Motivated Self-Aware Agents