mlsanityData to 14 Oct 2026

Go Beyond Imagination: Maximizing Episodic Reachability with World Models

arXiv 2308.13661 ↗First posted 25 Aug 2023cs.LGcited by 3

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

Efficient exploration is a challenging topic in reinforcement learning, especially for sparse reward tasks. To deal with the reward sparsity, people commonly apply intrinsic rewards to motivate agents to explore the state space efficiently. In this paper, we introduce a new intrinsic reward design called GoBI - Go Beyond Imagination, which combines the traditional lifelong novelty motivation with an episodic intrinsic reward that is designed to maximize the stepwise reachability expansion. More specifically, we apply learned world models to generate predicted future states with random actions. States with more unique predictions that are not in episodic memory are assigned high intrinsic rewards. Our method greatly outperforms previous state-of-the-art methods on 12 of the most challenging Minigrid navigation tasks and improves the sample efficiency on locomotion tasks from DeepMind Control Suite.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 39 of its 51 references that are in the map. First posted on arXiv 25 Aug 2023.

Builds on

39 of 51 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Mastering Diverse Domains through World Models10 Jan
  2. 2020Planning to Explore via Self-Supervised World Models12 May
  3. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec
  4. 2019Learning to Predict Without Looking Ahead: World Models Without Forward Prediction29 Oct
  5. 2018Recurrent World Models Facilitate Policy Evolution4 Sep
  6. 2018World Models27 Mar

Other references · 33

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  4. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  5. 2018Exploration by Random Network Distillationcited by 993
  6. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  7. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  8. 2020Reinforcement Learning with Augmented Datacited by 468
  9. 2020dm_control: Software and Tasks for Continuous Controlcited by 375
  10. 2018Quantifying Generalization in Reinforcement Learningcited by 352
  11. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
  12. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 309
21 more
  1. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
  2. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  3. 2018Provably Efficient Maximum Entropy Explorationcited by 242
  4. 2020Never Give Up: Learning Directed Exploration Strategiescited by 212
  5. 2019Efficient Exploration via State Marginal Matchingcited by 188
  6. 2016ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learningcited by 179
  7. 2021Reinforcement Learning with Prototypical Representationscited by 179
  8. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 174
  9. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 167
  10. 2019Benchmarking Model-Based Reinforcement Learningcited by 162
  11. 2015Incentivizing Exploration In Reinforcement Learning With Deep Predictive Modelscited by 154
  12. 2017Neural Episodic Controlcited by 151
  13. 2020RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environmentscited by 135
  14. 2018Episodic Curiosity through Reachabilitycited by 109
  15. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 108
  16. 2016Model-Free Episodic Controlcited by 102
  17. 2017Surprise-Based Intrinsic Motivation for Deep Reinforcement Learningcited by 83
  18. 2021Adversarially Guided Actor-Criticcited by 47
  19. 2021Interesting Object, Curious Agent: Learning Task-Agnostic Explorationcited by 38
  20. 2021Rank the Episodes: A Simple Approach for Exploration in Procedurally-Generated Environmentscited by 28
  21. 2020PlaNet of the Bayesians: Reconsidering and Improving Deep Planning Network by Incorporating Bayesian Inferencecited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Episodic Novelty Through Temporal Distancecited by 16
  2. 2026Remember to be Curious: Episodic Context and Persistent Worlds for 3D Explorationcited by 0
  3. 2026Explore Beyond the Boundary Using Entropic Informationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Endogenous Exploration in Reinforcement Learning with Intrinsic Curiosity
  2. 2024Fostering Intrinsic Motivation in Reinforcement Learning with Pretrained Foundation Models
  3. 2024ELEMENT: Episodic and Lifelong Exploration via Maximum Entropy
  4. 2026Task-Aware Exploration via a Predictive Bisimulation Metric
  5. 2024Open-World Reinforcement Learning over Long Short-Term Imagination
  6. 2025Autonomous state-space segmentation for Deep-RL sparse reward scenarios
  7. 2024MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximization
  8. 2025Curiosity-Driven Imagination: Discovering Plan Operators and Learning Associated Policies for Open-World Adaptation