mlsanityData to 14 Oct 2026

Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstruction

arXiv 2007.14535 ↗First posted 29 Jul 2020cs.LG cs.AI cs.SYcited by 66

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

In the present paper, we propose a decoder-free extension of Dreamer, a leading model-based reinforcement learning (MBRL) method from pixels. Dreamer is a sample- and cost-efficient solution to robot learning, as it is used to train latent state-space models based on a variational autoencoder and to conduct policy optimization by latent trajectory imagination. However, this autoencoding based approach often causes object vanishing, in which the autoencoder fails to perceives key objects for solving control tasks, and thus significantly limiting Dreamer's potential. This work aims to relieve this Dreamer's bottleneck and enhance its performance by means of removing the decoder. For this purpose, we firstly derive a likelihood-free and InfoMax objective of contrastive learning from the evidence lower bound of Dreamer. Secondly, we incorporate two components, (i) independent linear dynamics and (ii) the random crop data augmentation, to the learning scheme so as to improve the training performance. In comparison to Dreamer and other recent model-free reinforcement learning methods, our newly devised Dreamer with InfoMax and without generative decoder (Dreaming) achieves the best scores on 5 difficult simulated robotics tasks, in which Dreamer suffers from object vanishing.

Key facts. Cited by 66 papers in the map (data through 14 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 5 of its 37 references that are in the map. First posted on arXiv 29 Jul 2020.

Builds on

5 of 37 references.

Closest ancestors · 4

Same direction, newest first

  1. 2020Planning to Explore via Self-Supervised World Models12 May
  2. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec
  3. 2018Recurrent World Models Facilitate Policy Evolution4 Sep
  4. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models30 May

Other references · 1

  1. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 8; 2022: 17; 2023: 14; 2024: 10; 2025: 8; 2026: 9.

66 papers.

Sort:

By area:

  1. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  2. 2022Masked World Models for Visual Controlcited by 170
  3. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  4. 2021DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representationscited by 74
  5. 2025A Comprehensive Survey on World Models for Embodied AIcited by 68
  6. 2022Challenges and Opportunities in Offline Reinforcement Learning from Visual Observationscited by 63
  7. 2024Bridging State and History Representations: Understanding Self-Predictive RLcited by 54
  8. 2026World Action Models: The Next Frontier in Embodied AIcited by 50
  9. 2021Towards Robust Bisimulation Metric Learningcited by 49
  10. 2022Visuo-Tactile Transformers for Manipulationcited by 47
  11. 2023Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learningcited by 45
  12. 2025Latent Action Learning Requires Supervision in the Presence of Distractorscited by 44
38 more
  1. 2023World Models and Predictive Coding for Cognitive and Developmental Robotics: Frontiers and Challengescited by 36
  2. 2021INVIGORATE: Interactive Visual Grounding and Grasping in Cluttercited by 32
  3. 2023Facing Off World Model Backbones: RNNs, Transformers, and S4cited by 31
  4. 2022Joint Embedding Predictive Architectures Focus on Slow Featurescited by 30
  5. 2022DreamingV2: Reinforcement Learning with Discrete World Models without Reconstructioncited by 29
  6. 2022Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objectivecited by 28
  7. 2022Augmenting Reinforcement Learning with Transformer-based Scene Representation Learning for Decision-making of Autonomous Drivingcited by 24
  8. 2023HarmonyDream: Task Harmonization Inside World Modelscited by 22
  9. 2024Learning Latent Dynamic Robust Representations for World Modelscited by 21
  10. 2023Simplified Temporal Consistency Reinforcement Learningcited by 17
  11. 2022Flow-based Recurrent Belief State Learning for POMDPscited by 16
  12. 2022Time to augment self-supervised visual representation learningcited by 15
  13. 2024Generative Emergent Communication: Large Language Model is a Collective World Modelcited by 15
  14. 2023Robust Representation Learning by Clustering with Bisimulation Metrics for Visual Reinforcement Learning with Distractionscited by 14
  15. 2024MuDreamer: Learning Predictive World Models without Reconstructioncited by 13
  16. 2023Mastering Strategy Card Game (Legends of Code and Magic) via End-to-End Policy and Optimistic Smooth Fictitious Playcited by 10
  17. 2024R-AIF: Solving Sparse-Reward Robotic Tasks from Pixels with Active Inference and World Modelscited by 10
  18. 2022S2P: State-conditioned Image Synthesis for Data Augmentation in Offline Reinforcement Learningcited by 9
  19. 2023Combining Reconstruction and Contrastive Methods for Multimodal Representations in RLcited by 9
  20. 2021Measuring Progress in Deep Reinforcement Learning Sample Efficiencycited by 8
  21. 2023Hieros: Hierarchical Imagination on Structured State Space Sequence World Modelscited by 8
  22. 2023DreamSmooth: Improving Model-based Reinforcement Learning via Reward Smoothingcited by 8
  23. 2022Multi-View Dreaming: Multi-View World Model with Contrastive Learningcited by 6
  24. 2022Transformer in Transformer as Backbone for Deep Reinforcement Learningcited by 6
  25. 2024Learning to Play Atari in a World of Tokenscited by 6
  26. 2021Planning from Pixels in Environments with Combinatorially Hard Search Spacescited by 5
  27. 2021Maximum Entropy Model-based Reinforcement Learningcited by 5
  28. 2023Task Aware Dreamer for Task Generalization in Reinforcement Learningcited by 5
  29. 2025Towards Empowerment Gain through Causal Structure Learning in Model-Based RLcited by 5
  30. 2021A Whole Brain Probabilistic Generative Model: Toward Realizing Cognitive Architectures for Developmental Robotscited by 4
  31. 2026Next Embedding Prediction Makes World Models Strongercited by 4
  32. 2022Tactile-Sensitive NewtonianVAE for High-Accuracy Industrial Connector Insertioncited by 3
  33. 2022Cost-Driven Representation Learning for Linear Quadratic Gaussian Control: Part Icited by 3
  34. 2023Sample-efficient Real-time Planning with Curiosity Cross-Entropy Method and Contrastive Learningcited by 3
  35. 2023ConBaT: Control Barrier Transformer for Safe Policy Learningcited by 3
  36. 2025Edge General Intelligence Through World Models and Agentic AI: Fundamentals, Solutions, and Challengescited by 3
  37. 2025Dream to Recall: Imagination-Guided Experience Retrieval for Memory-Persistent Vision-and-Language Navigationcited by 3
  38. 2026World Models as Group Actionscited by 3

50 most cited of 66 All 66 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2018Learning Real-World Robot Policies by Dreaming
  2. 2026R2-Dreamer: Redundancy-Reduced World Models without Decoders or Augmentation
  3. 2023Model-Based Reinforcement Learning with Isolated Imaginations
  4. 2026Dreamer-CDP: Improving Reconstruction-free World Models Via Continuous Deterministic Representation Prediction
  5. 2020Bridging Imagination and Reality for Model-Based Deep Reinforcement Learning
  6. 2022Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Models
  7. 2026Dream-MPC: Gradient-Based Model Predictive Control with Latent Imagination
  8. 2026DreamFormer: Dream Imitation with a Transformer World Model for Language-Conditioned Robotic Manipulation