mlsanityData to 7 Oct 2026

Intrinsic Motivation in Model-based Reinforcement Learning: A Brief Review

arXiv 2301.10067 ↗First posted 24 Jan 2023cs.LG cs.AIcited by 3

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

The reinforcement learning research area contains a wide range of methods for solving the problems of intelligent agent control. Despite the progress that has been made, the task of creating a highly autonomous agent is still a significant challenge. One potential solution to this problem is intrinsic motivation, a concept derived from developmental psychology. This review considers the existing methods for determining intrinsic motivation based on the world model obtained by the agent. We propose a systematic approach to current research in this field, which consists of three categories of methods, distinguished by the way they utilize a world model in the agent's components: complementary intrinsic reward, exploration policy, and intrinsically motivated goals. The proposed unified framework describes the architecture of agents using a world model and intrinsic motivation to improve learning. The potential for developing new techniques in this area of research is also examined.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 24 of its 42 references that are in the map. First posted on arXiv 24 Jan 2023.

Builds on

24 of 42 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Curious Exploration via Structured World Models Yields Zero-Shot Object Manipulation22 Jun
  2. 2020Planning to Explore via Self-Supervised World Models12 May

Other references · 22

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  3. 2018Exploration by Random Network Distillationcited by 966
  4. 2019Self-Supervised Exploration via Disagreementcited by 263
  5. 2021Discovering and Achieving Goals via World Modelscited by 131
  6. 2018Episodic Curiosity through Reachabilitycited by 109
  7. 2018Model-Based Active Explorationcited by 105
  8. 2022Deep Hierarchical Planning from Pixelscited by 96
  9. 2017Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learningcited by 79
  10. 2019A survey on intrinsic motivation in reinforcement learningcited by 79
  11. 2020Self-supervised Visual Reinforcement Learning with Object-centric Representationscited by 43
  12. 2019Contextual Imagined Goals for Self-Supervised Robotic Learningcited by 40
10 more
  1. 2020Model-based Reinforcement Learning: A Surveycited by 39
  2. 2020Active World Model Learning with Progress Curiositycited by 27
  3. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 25
  4. 2021Sample Efficient Reinforcement Learning via Model-Ensemble Exploration and Exploitationcited by 17
  5. 2021Self-supervised Reinforcement Learning with Independently Controllable Subgoalscited by 16
  6. 2021Is Curiosity All You Need? On the Utility of Emergent Behaviours from Curious Explorationcited by 14
  7. 2022Language and Culture Internalisation for Human-Like Autotelic AIcited by 14
  8. 2018Deep Intrinsically Motivated Continuous Actor-Critic for Efficient Robotic Visuomotor Skill Learningcited by 5
  9. 2022Walk the Random Walk: Learning to Discover and Reach Goals Without Supervisioncited by 3
  10. 2021Long-Term Exploration in Persistent MDPscited by 2

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1; 2025: 1.

3 papers.

  1. 2023Approximate Shielding of Atari Agents for Safe Explorationcited by 2
  2. 2025Learning Soft Robotic Dynamics with Active Explorationcited by 2
  3. 2024Bounded Exploration with World Model Uncertainty in Soft Actor-Critic Reinforcement Learning Algorithmcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Intrinsic Motivation in Reinforcement Learning: A Research Agenda for Adaptive Self-Organisation
  2. 2024The intrinsic motivation of reinforcement and imitation learning for sequential tasks
  3. 2025Minding Motivation: The Effect of Intrinsic Motivation on Agent Behaviors
  4. 2025Learning To Explore With Predictive World Model Via Self-Supervised Learning
  5. 2022Towards Improving Exploration in Self-Imitation Learning using Intrinsic Motivation
  6. 2023Motif: Intrinsic Motivation from Artificial Intelligence Feedback
  7. 2018Learning to Play with Intrinsically-Motivated Self-Aware Agents
  8. 2025Reward Models in Deep Reinforcement Learning: A Survey