mlsanityData to 14 Oct 2026

Dr. Strategy: Model-Based Generalist Agents with Strategic Dreaming

arXiv 2402.18866 ↗First posted 29 Feb 2024cs.LGcited by 6

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Model-based reinforcement learning (MBRL) has been a primary approach to ameliorating the sample efficiency issue as well as to make a generalist agent. However, there has not been much effort toward enhancing the strategy of dreaming itself. Therefore, it is a question whether and how an agent can "dream better" in a more structured and strategic way. In this paper, inspired by the observation from cognitive science suggesting that humans use a spatial divide-and-conquer strategy in planning, we propose a new MBRL agent, called Dr. Strategy, which is equipped with a novel Dreaming Strategy. The proposed agent realizes a version of divide-and-conquer-like strategy in dreaming. This is achieved by learning a set of latent landmarks and then utilizing these to learn a landmark-conditioned highway policy. With the highway policy, the agent can first learn in the dream to move to a landmark, and from there it tackles the exploration and achievement task in a more focused way. In experiments, we show that the proposed model outperforms prior pixel-based MBRL methods in various visually complex and partially observable navigation tasks.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 33 of its 46 references that are in the map. First posted on arXiv 29 Feb 2024.

Builds on

33 of 46 references.

References · 33

  1. 2017Neural Discrete Representation Learningcited by 4,712
  2. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  4. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  5. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  6. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  7. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  8. 2017Hindsight Experience Replaycited by 1,136
  9. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  10. 2020Mastering Atari with Discrete World Modelscited by 1,007
  11. 2018Exploration by Random Network Distillationcited by 993
  12. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
21 more
  1. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  2. 2020Planning to Explore via Self-Supervised World Modelscited by 368
  3. 2020First return, then explorecited by 266
  4. 2019Self-Supervised Exploration via Disagreementcited by 265
  5. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 215
  6. 2021Reinforcement Learning with Prototypical Representationscited by 179
  7. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 174
  8. 2021Discovering and Achieving Goals via World Modelscited by 131
  9. 2020Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skillscited by 122
  10. 2018Fast Decoding in Sequence Models using Discrete Latent Variablescited by 104
  11. 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 102
  12. 2022Deep Hierarchical Planning from Pixelscited by 101
  13. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 86
  14. 2020Long-Horizon Visual Planning with Goal-Conditioned Hierarchical Predictorscited by 62
  15. 2022Lipschitz-constrained Unsupervised Skill Discoverycited by 62
  16. 2022Evaluating Long-Term Memory in 3D Mazescited by 43
  17. 2022Choreographer: Learning and Adapting Skills in Imaginationcited by 31
  18. 2023Planning Goals for Explorationcited by 28
  19. 2021Curiosity-Driven Exploration via Latent Bayesian Surprisecited by 22
  20. 2023Unlocking the Power of Representations in Long-term Novelty-based Explorationcited by 9
  21. 2023Learning to Discover Skills through Guidancecited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 1.

6 papers.

  1. 2025Monte Carlo Tree Diffusion for System 2 Planningcited by 33
  2. 2024Open-World Reinforcement Learning over Long Short-Term Imaginationcited by 13
  3. 2024Exploring the limits of Hierarchical World Models in Reinforcement Learningcited by 11
  4. 2024Navigation with QPHIL: Quantizing Planner for Hierarchical Implicit Q-Learningcited by 2
  5. 2025Learn A Flexible Exploration Model for Parameterized Action Markov Decision Processescited by 0
  6. 2026Neural Nonmyopic Bayesian Optimization in Dynamic Cost Settingscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Bridging Imagination and Reality for Model-Based Deep Reinforcement Learning
  2. 2024World Models with Hints of Large Language Models for Goal Achieving
  3. 2020Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstruction
  4. 2024Do Agents Dream of Electric Sheep?: Improving Generalization in Reinforcement Learning through Generative Learning
  5. 2021DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representations
  6. 2025Multimodal Dreaming: A Global Workspace Approach to World Model-Based Reinforcement Learning
  7. 2022Multi-View Dreaming: Multi-View World Model with Contrastive Learning
  8. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigation