mlsanityData to 7 Oct 2026

An information-theoretic perspective on intrinsic motivation in reinforcement learning: a survey

arXiv 2209.08890 ↗First posted 19 Sep 2022cs.LGcited by 25

Not on the map. Why?

Abstract (arXiv)

The reinforcement learning (RL) research area is very active, with an important number of new contributions; especially considering the emergent field of deep RL (DRL). However a number of scientific and technical challenges still need to be resolved, amongst which we can mention the ability to abstract actions or the difficulty to explore the environment in sparse-reward settings which can be addressed by intrinsic motivation (IM). We propose to survey these research works through a new taxonomy based on information theory: we computationally revisit the notions of surprise, novelty and skill learning. This allows us to identify advantages and disadvantages of methods and exhibit current outlooks of research. Our analysis suggests that novelty and surprise can assist the building of a hierarchy of transferable skills that further abstracts the environment and makes the exploration process more robust.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 109 of its 217 references that are in the map. First posted on arXiv 19 Sep 2022.

Builds on

109 of 217 references.

References · 109

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2022Generative Adversarial Networkscited by 5,603
  3. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2013Auto-Encoding Variational Bayescited by 3,980
  6. 2015Continuous control with deep reinforcement learningcited by 3,279
  7. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  8. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  9. 2016Concrete Problems in AI Safetycited by 1,582
  10. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  11. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  12. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
38 more
  1. 2017Hindsight Experience Replaycited by 1,096
  2. 2018Exploration by Random Network Distillationcited by 966
  3. 2016Deep Variational Information Bottleneckcited by 915
  4. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  5. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  6. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  7. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  8. 2015Weight Uncertainty in Neural Networkscited by 554
  9. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  10. 2016The Option-Critic Architecturecited by 433
  11. 2020Planning to Explore via Self-Supervised World Modelscited by 365
  12. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  13. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 307
  14. 2018Large-Scale Study of Curiosity-Driven Learningcited by 277
  15. 2020Inductive Biases for Deep Learning of Higher-Level Cognitioncited by 273
  16. 2019Self-Supervised Exploration via Disagreementcited by 263
  17. 2016Variational Intrinsic Controlcited by 262
  18. 2017Noisy Networks for Explorationcited by 262
  19. 2020First return, then explorecited by 256
  20. 2017Count-Based Exploration with Neural Density Modelscited by 255
  21. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  22. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  23. 2018Provably Efficient Maximum Entropy Explorationcited by 240
  24. 2020Never Give Up: Learning Directed Exploration Strategiescited by 210
  25. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 207
  26. 2018An Introduction to Deep Reinforcement Learningcited by 201
  27. 2020Representation Learning via Invariant Causal Mechanismscited by 197
  28. 2019Efficient Exploration via State Marginal Matchingcited by 186
  29. 2021Reinforcement Learning with Prototypical Representationscited by 179
  30. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  31. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 171
  32. 2017Parameter Space Noise for Explorationcited by 158
  33. 2015Incentivizing Exploration In Reinforcement Learning With Deep Predictive Modelscited by 154
  34. 2020RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environmentscited by 135
  35. 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 131
  36. 2021Discovering and Achieving Goals via World Modelscited by 131
  37. 2017Stochastic Neural Networks for Hierarchical Reinforcement Learningcited by 124
  38. 2018Variational Option Discovery Algorithmscited by 121

50 most cited of 109 All 109 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 6; 2025: 10; 2026: 5.

25 papers.

Sort:

By area:

  1. 2024MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximizationcited by 30
  2. 2024RLeXplore: Accelerating Research in Intrinsically-Motivated Reinforcement Learningcited by 10
  3. 2023Hieros: Hierarchical Imagination on Structured State Space Sequence World Modelscited by 8
  4. 2025The impact of intrinsic rewards on exploration in Reinforcement Learningcited by 8
  5. 2026Why AI systems don't learn and what to do about it: Lessons on autonomous learning from cognitive sciencecited by 8
  6. 2025SOMBRL: Scalable and Optimistic Model-Based RLcited by 7
  7. 2024Beyond Optimism: Exploration With Partially Observable Rewardscited by 6
  8. 2025Intrinsically-Motivated Humans and Agents in Open-World Explorationcited by 6
  9. 2025From Curiosity to Competence: How World Models Interact with the Dynamics of Explorationcited by 5
  10. 2025How Should We Meta-Learn Reinforcement Learning Algorithms?cited by 5
  11. 2024BAMDP Shaping: a Unified Framework for Intrinsic Motivation and Reward Shapingcited by 4
  12. 2023Intrinsic Motivation in Model-based Reinforcement Learning: A Brief Reviewcited by 3
13 more
  1. 2024Image-Based Deep Reinforcement Learning with Intrinsically Motivated Stimuli: On the Execution of Complex Robotic Taskscited by 3
  2. 2023Self-supervised network distillation: an effective approach to exploration in sparse reward environmentscited by 2
  3. 2024Diversity Progress for Goal Selection in Discriminability-Motivated RLcited by 2
  4. 2025InDRiVE: Intrinsic Disagreement based Reinforcement for Vehicle Exploration through Curiosity Driven Generalized World Modelcited by 2
  5. 2025Imagine, Verify, Execute: Memory-guided Agentic Exploration with Vision-Language Modelscited by 2
  6. 2025VendiRL: A Framework for Self-Supervised Reinforcement Learning of Diversely Diverse Skillscited by 2
  7. 2023A general Markov decision process formalism for action-state entropy-regularized reward maximizationcited by 1
  8. 2025Emergence of Goal-Directed Behaviors via Active Inference with Self-Priorcited by 1
  9. 2025Towards a Formal Theory of the Need for Competence via Computational Intrinsic Motivationcited by 0
  10. 2026Can In-Context Learning Support Intrinsic Curiosity?cited by 0
  11. 2026Principled Direction-Free Intrinsic Motivation through Model-Free Epistemic Free-Energy Estimatorscited by 0
  12. 2026Policy Complexity, Reaction Time, and Bounded Rationality in Reinforcement Learningcited by 0
  13. 2026Tactile Curiosity Drives Robot Interactioncited by 0